2023-10-18T10:28:32.051 INFO:root:teuthology version: 0.0.1.dev201+g202b180 2023-10-18T10:28:32.052 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/dparmar-2023-10-18_10:19:04-fs:functional-wip-63105-distro-default-smithi/7431296 --owner scheduled_dparmar@teuthology --name dparmar-2023-10-18_10:19:04-fs:functional-wip-63105-distro-default-smithi --verbose --description fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-recovery} -- /home/teuthworker/archive/dparmar-2023-10-18_10:19:04-fs:functional-wip-63105-distro-default-smithi/7431296/orig.config.yaml 2023-10-18T10:28:32.183 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-18T10:28:32.276 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/dparmar-2023-10-18_10:19:04-fs:functional-wip-63105-distro-default-smithi/7431296 branch: wip-63105 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-recovery} email: null first_in_suite: false job_id: '7431296' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: dparmar-2023-10-18_10:19:04-fs:functional-wip-63105-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-63105 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - evicting unresponsive client - but it is still running - slow request - MDS_CLIENT_LATE_RELEASE - t responding to mclientcaps - file system flag refuse_client_session is set - Degraded data redundancy - MDS_CLIENTS_LAGGY - Reduced data availability sha1: 6f3e6b48891b9b50415c0822b6b870797fcda16d ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 6f3e6b48891b9b50415c0822b6b870797fcda16d kclient: syntax: v1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-63105 sha1: 22c76d63886ee29a8c38dcd6634331fbd621f64a owner: scheduled_dparmar@teuthology priority: 55 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7825 sha1: 6f3e6b48891b9b50415c0822b6b870797fcda16d sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-63105 suite_path: /home/teuthworker/src/github.com_dparmar18_ceph_22c76d63886ee29a8c38dcd6634331fbd621f64a/qa suite_relpath: qa suite_repo: https://github.com/dparmar18/ceph suite_sha1: 22c76d63886ee29a8c38dcd6634331fbd621f64a syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi089.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK1OYlDgi4O1/6ng/uXhZ8MUwlMXFIH1rfYalkqOVAKHxilpUJbwZU6muYWI8DFrjmTYl+hk901BCq77/ZnsJAE= smithi154.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD206tYTbUO+0hlBk/fd4mL1UBAKK4y+NSWI2uFM6AlUPiOt/tJXuysZRQtUD6n/+FEL5LTD7erHsfj2X4n3pZ4= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_client_recovery teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 202b180cb047e798fb131047314a862593f45403 timestamp: 2023-10-18_10:19:04 tube: smithi user: dparmar verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-18T10:28:32.277 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_dparmar18_ceph_22c76d63886ee29a8c38dcd6634331fbd621f64a/qa; will attempt to use it 2023-10-18T10:28:32.277 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_dparmar18_ceph_22c76d63886ee29a8c38dcd6634331fbd621f64a/qa/tasks 2023-10-18T10:28:32.278 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-18T10:28:32.280 INFO:teuthology.task.internal:Checking packages... 2023-10-18T10:28:32.305 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '6f3e6b48891b9b50415c0822b6b870797fcda16d' 2023-10-18T10:28:32.305 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-18T10:28:32.306 INFO:teuthology.packaging:ref: None 2023-10-18T10:28:32.306 INFO:teuthology.packaging:tag: None 2023-10-18T10:28:32.306 INFO:teuthology.packaging:branch: wip-63105 2023-10-18T10:28:32.307 INFO:teuthology.packaging:sha1: 6f3e6b48891b9b50415c0822b6b870797fcda16d 2023-10-18T10:28:32.307 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-63105 2023-10-18T10:28:32.605 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-5610-g6f3e6b48-1jammy 2023-10-18T10:28:32.606 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-18T10:28:32.656 INFO:teuthology.task.internal:no buildpackages task found 2023-10-18T10:28:32.657 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-18T10:28:32.675 INFO:teuthology.task.internal:Saving configuration 2023-10-18T10:28:32.702 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-18T10:28:32.710 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-18T10:28:32.734 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi089.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2023-10-18_10:19:04-fs:functional-wip-63105-distro-default-smithi/7431296', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-18 10:22:32.967283', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINclsEyzxFSvEIHXSi8iirPHTtnNyIzjvvsydUOpHqGU'} 2023-10-18T10:28:32.753 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2023-10-18_10:19:04-fs:functional-wip-63105-distro-default-smithi/7431296', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-18 10:22:32.969279', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINXrkGG48pwvvBEqirETNa+4KMVcDxiuEfZllnVo/008'} 2023-10-18T10:28:32.753 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-18T10:28:32.767 INFO:teuthology.task.internal:roles: ubuntu@smithi089.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-18T10:28:32.767 INFO:teuthology.task.internal:roles: ubuntu@smithi154.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-18T10:28:32.768 INFO:teuthology.run_tasks:Running task console_log... 2023-10-18T10:28:32.840 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3c36a61af0>, signals=[15]) 2023-10-18T10:28:32.841 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-18T10:28:32.881 INFO:teuthology.task.internal:Opening connections... 2023-10-18T10:28:32.882 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi089.front.sepia.ceph.com 2023-10-18T10:28:32.884 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T10:28:32.962 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi154.front.sepia.ceph.com 2023-10-18T10:28:32.963 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T10:28:33.042 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-18T10:28:33.060 DEBUG:teuthology.orchestra.run.smithi089:> uname -m 2023-10-18T10:28:33.065 INFO:teuthology.orchestra.run.smithi089.stdout:x86_64 2023-10-18T10:28:33.066 DEBUG:teuthology.orchestra.run.smithi089:> cat /etc/os-release 2023-10-18T10:28:33.112 INFO:teuthology.orchestra.run.smithi089.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-10-18T10:28:33.112 INFO:teuthology.orchestra.run.smithi089.stdout:NAME="Ubuntu" 2023-10-18T10:28:33.112 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_ID="22.04" 2023-10-18T10:28:33.112 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-10-18T10:28:33.113 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_CODENAME=jammy 2023-10-18T10:28:33.113 INFO:teuthology.orchestra.run.smithi089.stdout:ID=ubuntu 2023-10-18T10:28:33.113 INFO:teuthology.orchestra.run.smithi089.stdout:ID_LIKE=debian 2023-10-18T10:28:33.113 INFO:teuthology.orchestra.run.smithi089.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-18T10:28:33.113 INFO:teuthology.orchestra.run.smithi089.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-18T10:28:33.114 INFO:teuthology.orchestra.run.smithi089.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-18T10:28:33.114 INFO:teuthology.orchestra.run.smithi089.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-18T10:28:33.114 INFO:teuthology.orchestra.run.smithi089.stdout:UBUNTU_CODENAME=jammy 2023-10-18T10:28:33.115 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server 2023-10-18T10:28:33.140 DEBUG:teuthology.orchestra.run.smithi154:> uname -m 2023-10-18T10:28:33.146 INFO:teuthology.orchestra.run.smithi154.stdout:x86_64 2023-10-18T10:28:33.146 DEBUG:teuthology.orchestra.run.smithi154:> cat /etc/os-release 2023-10-18T10:28:33.195 INFO:teuthology.orchestra.run.smithi154.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-10-18T10:28:33.195 INFO:teuthology.orchestra.run.smithi154.stdout:NAME="Ubuntu" 2023-10-18T10:28:33.195 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_ID="22.04" 2023-10-18T10:28:33.195 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-10-18T10:28:33.196 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_CODENAME=jammy 2023-10-18T10:28:33.196 INFO:teuthology.orchestra.run.smithi154.stdout:ID=ubuntu 2023-10-18T10:28:33.196 INFO:teuthology.orchestra.run.smithi154.stdout:ID_LIKE=debian 2023-10-18T10:28:33.196 INFO:teuthology.orchestra.run.smithi154.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-18T10:28:33.197 INFO:teuthology.orchestra.run.smithi154.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-18T10:28:33.197 INFO:teuthology.orchestra.run.smithi154.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-18T10:28:33.198 INFO:teuthology.orchestra.run.smithi154.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-18T10:28:33.198 INFO:teuthology.orchestra.run.smithi154.stdout:UBUNTU_CODENAME=jammy 2023-10-18T10:28:33.199 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server 2023-10-18T10:28:33.240 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-18T10:28:33.258 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-18T10:28:33.339 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-18T10:28:33.339 DEBUG:teuthology.orchestra.run.smithi089:> test '!' -e /home/ubuntu/cephtest 2023-10-18T10:28:33.342 DEBUG:teuthology.orchestra.run.smithi154:> test '!' -e /home/ubuntu/cephtest 2023-10-18T10:28:33.347 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-18T10:28:33.354 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-18T10:28:33.355 DEBUG:teuthology.orchestra.run.smithi089:> test -z $(ls -A /var/lib/ceph) 2023-10-18T10:28:33.390 DEBUG:teuthology.orchestra.run.smithi154:> test -z $(ls -A /var/lib/ceph) 2023-10-18T10:28:33.455 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-18T10:28:33.511 INFO:teuthology.run_tasks:Running task kernel... 2023-10-18T10:28:33.529 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-10-18T10:28:33.530 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2023-10-18T10:28:33.531 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2023-10-18T10:28:33.531 DEBUG:teuthology.orchestra.run.smithi089:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-18T10:28:33.532 DEBUG:teuthology.orchestra.run.smithi154:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-18T10:28:33.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:28:33.537 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2023-10-18T10:28:33.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:28:33.541 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2023-10-18T10:28:33.698 DEBUG:teuthology.task.kernel:sha1 for client.0 is 47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-18T10:28:33.698 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "47672119149b103ec7cec2cc65c063ccbb001ac7"... 2023-10-18T10:28:33.698 DEBUG:teuthology.orchestra.run.smithi089:> uname -r 2023-10-18T10:28:33.704 INFO:teuthology.orchestra.run.smithi089.stdout:5.15.0-56-generic 2023-10-18T10:28:33.704 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-18T10:28:33.705 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-18T10:28:33.705 INFO:teuthology.task.kernel:Installing linux-firmware on client.0... 2023-10-18T10:28:33.705 DEBUG:teuthology.orchestra.run.smithi089:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2023-10-18T10:28:33.772 DEBUG:teuthology.task.kernel:sha1 for client.1 is 47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-18T10:28:33.772 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "47672119149b103ec7cec2cc65c063ccbb001ac7"... 2023-10-18T10:28:33.772 DEBUG:teuthology.orchestra.run.smithi154:> uname -r 2023-10-18T10:28:33.773 INFO:teuthology.orchestra.run.smithi089.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2023-10-18T10:28:33.778 INFO:teuthology.orchestra.run.smithi154.stdout:5.15.0-56-generic 2023-10-18T10:28:33.779 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-18T10:28:33.779 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-18T10:28:33.779 INFO:teuthology.task.kernel:Installing linux-firmware on client.1... 2023-10-18T10:28:33.780 DEBUG:teuthology.orchestra.run.smithi154:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2023-10-18T10:28:33.849 INFO:teuthology.orchestra.run.smithi154.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2023-10-18T10:28:33.919 INFO:teuthology.orchestra.run.smithi089.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2023-10-18T10:28:33.919 INFO:teuthology.orchestra.run.smithi089.stderr:hint: is subject to change. To configure the initial branch name to use in all 2023-10-18T10:28:33.920 INFO:teuthology.orchestra.run.smithi089.stderr:hint: of your new repositories, which will suppress this warning, call: 2023-10-18T10:28:33.920 INFO:teuthology.orchestra.run.smithi089.stderr:hint: 2023-10-18T10:28:33.920 INFO:teuthology.orchestra.run.smithi089.stderr:hint: git config --global init.defaultBranch 2023-10-18T10:28:33.920 INFO:teuthology.orchestra.run.smithi089.stderr:hint: 2023-10-18T10:28:33.920 INFO:teuthology.orchestra.run.smithi089.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2023-10-18T10:28:33.921 INFO:teuthology.orchestra.run.smithi089.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2023-10-18T10:28:33.921 INFO:teuthology.orchestra.run.smithi089.stderr:hint: 2023-10-18T10:28:33.921 INFO:teuthology.orchestra.run.smithi089.stderr:hint: git branch -m 2023-10-18T10:28:33.921 INFO:teuthology.orchestra.run.smithi089.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2023-10-18T10:28:33.923 DEBUG:teuthology.orchestra.run.smithi089:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2023-10-18T10:28:33.975 INFO:teuthology.orchestra.run.smithi154.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2023-10-18T10:28:33.976 INFO:teuthology.orchestra.run.smithi154.stderr:hint: is subject to change. To configure the initial branch name to use in all 2023-10-18T10:28:33.976 INFO:teuthology.orchestra.run.smithi154.stderr:hint: of your new repositories, which will suppress this warning, call: 2023-10-18T10:28:33.976 INFO:teuthology.orchestra.run.smithi154.stderr:hint: 2023-10-18T10:28:33.977 INFO:teuthology.orchestra.run.smithi154.stderr:hint: git config --global init.defaultBranch 2023-10-18T10:28:33.977 INFO:teuthology.orchestra.run.smithi154.stderr:hint: 2023-10-18T10:28:33.977 INFO:teuthology.orchestra.run.smithi154.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2023-10-18T10:28:33.977 INFO:teuthology.orchestra.run.smithi154.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2023-10-18T10:28:33.977 INFO:teuthology.orchestra.run.smithi154.stderr:hint: 2023-10-18T10:28:33.978 INFO:teuthology.orchestra.run.smithi154.stderr:hint: git branch -m 2023-10-18T10:28:33.978 INFO:teuthology.orchestra.run.smithi154.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2023-10-18T10:28:33.979 DEBUG:teuthology.orchestra.run.smithi154:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2023-10-18T10:28:33.988 DEBUG:teuthology.orchestra.run.smithi089:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2023-10-18T10:28:34.046 DEBUG:teuthology.orchestra.run.smithi089:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2023-10-18T10:28:34.049 DEBUG:teuthology.orchestra.run.smithi154:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2023-10-18T10:28:34.101 DEBUG:teuthology.orchestra.run.smithi154:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2023-10-18T10:30:09.958 INFO:teuthology.orchestra.run.smithi089.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2023-10-18T10:30:09.959 INFO:teuthology.orchestra.run.smithi089.stderr: * [new branch] main -> origin/main 2023-10-18T10:30:09.988 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20190312 -> 20190312 2023-10-18T10:30:09.988 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20190416 -> 20190416 2023-10-18T10:30:09.988 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20190514 -> 20190514 2023-10-18T10:30:09.989 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20190618 -> 20190618 2023-10-18T10:30:09.989 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20190717 -> 20190717 2023-10-18T10:30:09.989 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20190815 -> 20190815 2023-10-18T10:30:09.989 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20190923 -> 20190923 2023-10-18T10:30:09.990 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20191022 -> 20191022 2023-10-18T10:30:09.990 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20191215 -> 20191215 2023-10-18T10:30:09.993 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20200122 -> 20200122 2023-10-18T10:30:09.994 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20200316 -> 20200316 2023-10-18T10:30:09.994 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20200421 -> 20200421 2023-10-18T10:30:09.994 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20200519 -> 20200519 2023-10-18T10:30:09.994 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20200619 -> 20200619 2023-10-18T10:30:09.995 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20200721 -> 20200721 2023-10-18T10:30:09.995 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20200817 -> 20200817 2023-10-18T10:30:09.995 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20200918 -> 20200918 2023-10-18T10:30:09.995 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20201022 -> 20201022 2023-10-18T10:30:09.995 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20201118 -> 20201118 2023-10-18T10:30:09.995 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20201218 -> 20201218 2023-10-18T10:30:09.996 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20210208 -> 20210208 2023-10-18T10:30:09.996 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20210315 -> 20210315 2023-10-18T10:30:09.996 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20210511 -> 20210511 2023-10-18T10:30:09.996 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20210716 -> 20210716 2023-10-18T10:30:09.996 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20210818 -> 20210818 2023-10-18T10:30:09.997 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20210919 -> 20210919 2023-10-18T10:30:09.997 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20211027 -> 20211027 2023-10-18T10:30:09.997 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20211216 -> 20211216 2023-10-18T10:30:09.997 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20220209 -> 20220209 2023-10-18T10:30:09.997 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20220310 -> 20220310 2023-10-18T10:30:09.998 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20220411 -> 20220411 2023-10-18T10:30:09.998 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20220509 -> 20220509 2023-10-18T10:30:09.998 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20220610 -> 20220610 2023-10-18T10:30:09.998 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20220708 -> 20220708 2023-10-18T10:30:09.999 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20220815 -> 20220815 2023-10-18T10:30:09.999 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20220913 -> 20220913 2023-10-18T10:30:09.999 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20221012 -> 20221012 2023-10-18T10:30:09.999 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20221109 -> 20221109 2023-10-18T10:30:09.999 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20221214 -> 20221214 2023-10-18T10:30:09.999 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20230117 -> 20230117 2023-10-18T10:30:10.000 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20230210 -> 20230210 2023-10-18T10:30:10.000 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20230310 -> 20230310 2023-10-18T10:30:10.000 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20230404 -> 20230404 2023-10-18T10:30:10.000 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20230515 -> 20230515 2023-10-18T10:30:10.000 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20230625 -> 20230625 2023-10-18T10:30:10.001 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20230804 -> 20230804 2023-10-18T10:30:10.001 INFO:teuthology.orchestra.run.smithi089.stderr: * [new tag] 20230919 -> 20230919 2023-10-18T10:30:13.192 INFO:teuthology.orchestra.run.smithi154.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2023-10-18T10:30:13.192 INFO:teuthology.orchestra.run.smithi154.stderr: * [new branch] main -> origin/main 2023-10-18T10:30:13.224 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20190312 -> 20190312 2023-10-18T10:30:13.224 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20190416 -> 20190416 2023-10-18T10:30:13.224 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20190514 -> 20190514 2023-10-18T10:30:13.224 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20190618 -> 20190618 2023-10-18T10:30:13.225 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20190717 -> 20190717 2023-10-18T10:30:13.225 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20190815 -> 20190815 2023-10-18T10:30:13.225 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20190923 -> 20190923 2023-10-18T10:30:13.225 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20191022 -> 20191022 2023-10-18T10:30:13.225 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20191215 -> 20191215 2023-10-18T10:30:13.226 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20200122 -> 20200122 2023-10-18T10:30:13.229 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20200316 -> 20200316 2023-10-18T10:30:13.229 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20200421 -> 20200421 2023-10-18T10:30:13.230 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20200519 -> 20200519 2023-10-18T10:30:13.230 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20200619 -> 20200619 2023-10-18T10:30:13.230 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20200721 -> 20200721 2023-10-18T10:30:13.230 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20200817 -> 20200817 2023-10-18T10:30:13.231 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20200918 -> 20200918 2023-10-18T10:30:13.231 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20201022 -> 20201022 2023-10-18T10:30:13.231 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20201118 -> 20201118 2023-10-18T10:30:13.231 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20201218 -> 20201218 2023-10-18T10:30:13.231 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20210208 -> 20210208 2023-10-18T10:30:13.232 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20210315 -> 20210315 2023-10-18T10:30:13.232 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20210511 -> 20210511 2023-10-18T10:30:13.232 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20210716 -> 20210716 2023-10-18T10:30:13.232 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20210818 -> 20210818 2023-10-18T10:30:13.232 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20210919 -> 20210919 2023-10-18T10:30:13.233 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20211027 -> 20211027 2023-10-18T10:30:13.233 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20211216 -> 20211216 2023-10-18T10:30:13.233 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20220209 -> 20220209 2023-10-18T10:30:13.233 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20220310 -> 20220310 2023-10-18T10:30:13.233 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20220411 -> 20220411 2023-10-18T10:30:13.233 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20220509 -> 20220509 2023-10-18T10:30:13.234 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20220610 -> 20220610 2023-10-18T10:30:13.234 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20220708 -> 20220708 2023-10-18T10:30:13.234 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20220815 -> 20220815 2023-10-18T10:30:13.234 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20220913 -> 20220913 2023-10-18T10:30:13.234 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20221012 -> 20221012 2023-10-18T10:30:13.235 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20221109 -> 20221109 2023-10-18T10:30:13.235 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20221214 -> 20221214 2023-10-18T10:30:13.235 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20230117 -> 20230117 2023-10-18T10:30:13.235 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20230210 -> 20230210 2023-10-18T10:30:13.235 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20230310 -> 20230310 2023-10-18T10:30:13.236 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20230404 -> 20230404 2023-10-18T10:30:13.236 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20230515 -> 20230515 2023-10-18T10:30:13.236 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20230625 -> 20230625 2023-10-18T10:30:13.236 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20230804 -> 20230804 2023-10-18T10:30:13.236 INFO:teuthology.orchestra.run.smithi154.stderr: * [new tag] 20230919 -> 20230919 2023-10-18T10:30:13.713 INFO:teuthology.orchestra.run.smithi089.stderr:Updating files: 51% (1624/3161) Updating files: 52% (1644/3161) Updating files: 53% (1676/3161) Updating files: 54% (1707/3161) Updating files: 55% (1739/3161) Updating files: 56% (1771/3161) Updating files: 57% (1802/3161) Updating files: 58% (1834/3161) Updating files: 59% (1865/3161) Updating files: 60% (1897/3161) Updating files: 61% (1929/3161) Updating files: 62% (1960/3161) Updating files: 63% (1992/3161) Updating files: 64% (2024/3161) Updating files: 65% (2055/3161) Updating files: 66% (2087/3161) Updating files: 66% (2106/3161) Updating files: 67% (2118/3161) Updating files: 68% (2150/3161) Updating files: 69% (2182/3161) Updating files: 70% (2213/3161) Updating files: 71% (2245/3161) Updating files: 72% (2276/3161) Updating files: 73% (2308/3161) Updating files: 74% (2340/3161) Updating files: 75% (2371/3161) Updating files: 76% (2403/3161) Updating files: 77% (2434/3161) Updating files: 78% (2466/3161) Updating files: 79% (2498/3161) Updating files: 80% (2529/3161) Updating files: 81% (2561/3161) Updating files: 81% (2581/3161) Updating files: 82% (2593/3161) Updating files: 83% (2624/3161) Updating files: 84% (2656/3161) Updating files: 85% (2687/3161) Updating files: 86% (2719/3161) Updating files: 87% (2751/3161) Updating files: 88% (2782/3161) Updating files: 89% (2814/3161) Updating files: 90% (2845/3161) Updating files: 91% (2877/3161) Updating files: 92% (2909/3161) Updating files: 93% (2940/3161) Updating files: 94% (2972/3161) Updating files: 95% (3003/3161) Updating files: 96% (3035/3161) Updating files: 97% (3067/3161) Updating files: 98% (3098/3161) Updating files: 99% (3130/3161) Updating files: 100% (3161/3161) Updating files: 100% (3161/3161), done. 2023-10-18T10:30:13.718 INFO:teuthology.orchestra.run.smithi089.stdout:HEAD is now at 8ff933de Merge branch 'robot/pr-0-1697570762' into 'main' 2023-10-18T10:30:13.738 INFO:teuthology.task.kernel:Downloading kernel 47672119149b103ec7cec2cc65c063ccbb001ac7 on client.0... 2023-10-18T10:30:13.738 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-18T10:30:13.964 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-rc5-g47672119149b/ 2023-10-18T10:30:13.964 DEBUG:teuthology.orchestra.run.smithi089:> rm -f /tmp/linux-image.deb && echo linux-image-6.6.0-rc5-g47672119149b_6.6.0-rc5-g47672119149b-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-rc5-g47672119149b/ --input-file=- 2023-10-18T10:30:13.967 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi089.front.sepia.ceph.com to complete... 2023-10-18T10:30:15.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18 10:30:15 URL:https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-rc5-g47672119149b/linux-image-6.6.0-rc5-g47672119149b_6.6.0-rc5-g47672119149b-1_amd64.deb [38909314/38909314] -> "/tmp/linux-image.deb" [1] 2023-10-18T10:30:15.483 INFO:teuthology.orchestra.run.smithi089.stderr:FINISHED --2023-10-18 10:30:15-- 2023-10-18T10:30:15.483 INFO:teuthology.orchestra.run.smithi089.stderr:Total wall clock time: 1.5s 2023-10-18T10:30:15.483 INFO:teuthology.orchestra.run.smithi089.stderr:Downloaded: 1 files, 37M in 1.4s (27.4 MB/s) 2023-10-18T10:30:15.485 INFO:teuthology.task.kernel:Installing kernel 47672119149b103ec7cec2cc65c063ccbb001ac7 on client.0... 2023-10-18T10:30:15.486 DEBUG:teuthology.orchestra.run.smithi089:> sudo dpkg -i /tmp/linux-image.deb 2023-10-18T10:30:15.710 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package linux-image-6.6.0-rc5-g47672119149b. 2023-10-18T10:30:16.933 INFO:teuthology.orchestra.run.smithi154.stderr:Updating files: 51% (1631/3161) Updating files: 52% (1644/3161) Updating files: 53% (1676/3161) Updating files: 54% (1707/3161) Updating files: 55% (1739/3161) Updating files: 56% (1771/3161) Updating files: 57% (1802/3161) Updating files: 58% (1834/3161) Updating files: 59% (1865/3161) Updating files: 60% (1897/3161) Updating files: 61% (1929/3161) Updating files: 62% (1960/3161) Updating files: 63% (1992/3161) Updating files: 64% (2024/3161) Updating files: 65% (2055/3161) Updating files: 66% (2087/3161) Updating files: 66% (2106/3161) Updating files: 67% (2118/3161) Updating files: 68% (2150/3161) Updating files: 69% (2182/3161) Updating files: 70% (2213/3161) Updating files: 71% (2245/3161) Updating files: 72% (2276/3161) Updating files: 73% (2308/3161) Updating files: 74% (2340/3161) Updating files: 75% (2371/3161) Updating files: 76% (2403/3161) Updating files: 77% (2434/3161) Updating files: 78% (2466/3161) Updating files: 79% (2498/3161) Updating files: 80% (2529/3161) Updating files: 81% (2561/3161) Updating files: 81% (2589/3161) Updating files: 82% (2593/3161) Updating files: 83% (2624/3161) Updating files: 84% (2656/3161) Updating files: 85% (2687/3161) Updating files: 86% (2719/3161) Updating files: 87% (2751/3161) Updating files: 88% (2782/3161) Updating files: 89% (2814/3161) Updating files: 90% (2845/3161) Updating files: 91% (2877/3161) Updating files: 92% (2909/3161) Updating files: 93% (2940/3161) Updating files: 94% (2972/3161) Updating files: 95% (3003/3161) Updating files: 96% (3035/3161) Updating files: 97% (3067/3161) Updating files: 98% (3098/3161) Updating files: 99% (3130/3161) Updating files: 100% (3161/3161) Updating files: 100% (3161/3161), done. 2023-10-18T10:30:16.938 INFO:teuthology.orchestra.run.smithi154.stdout:HEAD is now at 8ff933de Merge branch 'robot/pr-0-1697570762' into 'main' 2023-10-18T10:30:16.957 INFO:teuthology.task.kernel:Downloading kernel 47672119149b103ec7cec2cc65c063ccbb001ac7 on client.1... 2023-10-18T10:30:16.958 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-18T10:30:17.172 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-rc5-g47672119149b/ 2023-10-18T10:30:17.172 DEBUG:teuthology.orchestra.run.smithi154:> rm -f /tmp/linux-image.deb && echo linux-image-6.6.0-rc5-g47672119149b_6.6.0-rc5-g47672119149b-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-rc5-g47672119149b/ --input-file=- 2023-10-18T10:30:17.174 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi154.front.sepia.ceph.com to complete... 2023-10-18T10:30:17.727 INFO:teuthology.orchestra.run.smithi089.stdout:(Reading database ... 129542 files and directories currently installed.) 2023-10-18T10:30:17.732 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack /tmp/linux-image.deb ... 2023-10-18T10:30:17.863 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking linux-image-6.6.0-rc5-g47672119149b (6.6.0-rc5-g47672119149b-1) ... 2023-10-18T10:30:18.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18 10:30:18 URL:https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-rc5-g47672119149b/linux-image-6.6.0-rc5-g47672119149b_6.6.0-rc5-g47672119149b-1_amd64.deb [38909314/38909314] -> "/tmp/linux-image.deb" [1] 2023-10-18T10:30:18.761 INFO:teuthology.orchestra.run.smithi154.stderr:FINISHED --2023-10-18 10:30:18-- 2023-10-18T10:30:18.761 INFO:teuthology.orchestra.run.smithi154.stderr:Total wall clock time: 1.5s 2023-10-18T10:30:18.761 INFO:teuthology.orchestra.run.smithi154.stderr:Downloaded: 1 files, 37M in 1.4s (27.4 MB/s) 2023-10-18T10:30:18.763 INFO:teuthology.task.kernel:Installing kernel 47672119149b103ec7cec2cc65c063ccbb001ac7 on client.1... 2023-10-18T10:30:18.764 DEBUG:teuthology.orchestra.run.smithi154:> sudo dpkg -i /tmp/linux-image.deb 2023-10-18T10:30:19.058 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package linux-image-6.6.0-rc5-g47672119149b. 2023-10-18T10:30:21.096 INFO:teuthology.orchestra.run.smithi154.stdout:(Reading database ... 129542 files and directories currently installed.) 2023-10-18T10:30:21.100 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack /tmp/linux-image.deb ... 2023-10-18T10:30:21.199 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking linux-image-6.6.0-rc5-g47672119149b (6.6.0-rc5-g47672119149b-1) ... 2023-10-18T10:30:21.395 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up linux-image-6.6.0-rc5-g47672119149b (6.6.0-rc5-g47672119149b-1) ... 2023-10-18T10:30:21.501 INFO:teuthology.orchestra.run.smithi089.stderr:update-initramfs: Generating /boot/initrd.img-6.6.0-rc5-g47672119149b 2023-10-18T10:30:24.393 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up linux-image-6.6.0-rc5-g47672119149b (6.6.0-rc5-g47672119149b-1) ... 2023-10-18T10:30:24.500 INFO:teuthology.orchestra.run.smithi154.stderr:update-initramfs: Generating /boot/initrd.img-6.6.0-rc5-g47672119149b 2023-10-18T10:30:42.817 INFO:teuthology.orchestra.run.smithi089.stderr:Sourcing file `/etc/default/grub' 2023-10-18T10:30:42.832 INFO:teuthology.orchestra.run.smithi089.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-18T10:30:42.852 INFO:teuthology.orchestra.run.smithi089.stderr:Generating grub configuration file ... 2023-10-18T10:30:43.184 INFO:teuthology.orchestra.run.smithi089.stderr:Found linux image: /boot/vmlinuz-6.6.0-rc5-g47672119149b 2023-10-18T10:30:43.198 INFO:teuthology.orchestra.run.smithi089.stderr:Found initrd image: /boot/initrd.img-6.6.0-rc5-g47672119149b 2023-10-18T10:30:43.499 INFO:teuthology.orchestra.run.smithi089.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-10-18T10:30:43.516 INFO:teuthology.orchestra.run.smithi089.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-10-18T10:30:43.578 INFO:teuthology.orchestra.run.smithi089.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-10-18T10:30:43.584 INFO:teuthology.orchestra.run.smithi089.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-10-18T10:30:43.763 INFO:teuthology.orchestra.run.smithi089.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-10-18T10:30:43.764 INFO:teuthology.orchestra.run.smithi089.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-10-18T10:30:43.764 INFO:teuthology.orchestra.run.smithi089.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-10-18T10:30:43.804 INFO:teuthology.orchestra.run.smithi089.stderr:done 2023-10-18T10:30:44.011 DEBUG:teuthology.orchestra.run.smithi089:> dpkg-deb -c /tmp/linux-image.deb 2023-10-18T10:30:44.019 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./ 2023-10-18T10:30:44.019 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./boot/ 2023-10-18T10:30:44.019 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 4415097 2023-10-10 19:56 ./boot/System.map-6.6.0-rc5-g47672119149b 2023-10-18T10:30:44.031 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 169298 2023-10-10 19:56 ./boot/config-6.6.0-rc5-g47672119149b 2023-10-18T10:30:44.032 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11352928 2023-10-10 19:56 ./boot/vmlinuz-6.6.0-rc5-g47672119149b 2023-10-18T10:30:44.050 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/ 2023-10-18T10:30:44.050 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/kernel/ 2023-10-18T10:30:44.050 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/kernel/postinst.d/ 2023-10-18T10:30:44.050 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/kernel/postrm.d/ 2023-10-18T10:30:44.051 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/kernel/preinst.d/ 2023-10-18T10:30:44.051 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/kernel/prerm.d/ 2023-10-18T10:30:44.051 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/ 2023-10-18T10:30:44.051 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/ 2023-10-18T10:30:44.052 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/ 2023-10-18T10:30:44.052 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/ 2023-10-18T10:30:44.052 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/ 2023-10-18T10:30:44.052 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/ 2023-10-18T10:30:44.053 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/ 2023-10-18T10:30:44.053 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 490233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/aesni-intel.ko 2023-10-18T10:30:44.055 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 25329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-18T10:30:44.056 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 29529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-18T10:30:44.056 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 30313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-18T10:30:44.056 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 93113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-18T10:30:44.056 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 106729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-18T10:30:44.056 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 152241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-18T10:30:44.057 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-18T10:30:44.057 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-18T10:30:44.057 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 83889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-18T10:30:44.057 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-18T10:30:44.058 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 47225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-18T10:30:44.059 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 47209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-18T10:30:44.059 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 51801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-18T10:30:44.059 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 38937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-18T10:30:44.059 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 40529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-18T10:30:44.059 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 58585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-18T10:30:44.060 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 50497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-18T10:30:44.060 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 32065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-18T10:30:44.060 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-18T10:30:44.060 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/ 2023-10-18T10:30:44.060 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/ 2023-10-18T10:30:44.061 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/mce/ 2023-10-18T10:30:44.061 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 40113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-18T10:30:44.061 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpuid.ko 2023-10-18T10:30:44.061 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/msr.ko 2023-10-18T10:30:44.061 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/ 2023-10-18T10:30:44.062 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 353761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-18T10:30:44.062 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 881713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-18T10:30:44.062 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 2276817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm.ko 2023-10-18T10:30:44.100 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ 2023-10-18T10:30:44.100 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 53761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/af_alg.ko 2023-10-18T10:30:44.100 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 23097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/algif_hash.ko 2023-10-18T10:30:44.101 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/algif_skcipher.ko 2023-10-18T10:30:44.101 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ansi_cprng.ko 2023-10-18T10:30:44.101 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/anubis.ko 2023-10-18T10:30:44.101 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/arc4.ko 2023-10-18T10:30:44.101 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/asymmetric_keys/ 2023-10-18T10:30:44.102 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2023-10-18T10:30:44.102 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/ 2023-10-18T10:30:44.102 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-18T10:30:44.102 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_pq.ko 2023-10-18T10:30:44.102 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-18T10:30:44.103 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_tx.ko 2023-10-18T10:30:44.103 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_xor.ko 2023-10-18T10:30:44.103 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/raid6test.ko 2023-10-18T10:30:44.103 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/authenc.ko 2023-10-18T10:30:44.103 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 22641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/authencesn.ko 2023-10-18T10:30:44.104 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blake2b_generic.ko 2023-10-18T10:30:44.104 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blowfish_common.ko 2023-10-18T10:30:44.104 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blowfish_generic.ko 2023-10-18T10:30:44.104 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 38873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/camellia_generic.ko 2023-10-18T10:30:44.104 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 27321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast5_generic.ko 2023-10-18T10:30:44.105 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast6_generic.ko 2023-10-18T10:30:44.105 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast_common.ko 2023-10-18T10:30:44.105 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 35097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ccm.ko 2023-10-18T10:30:44.105 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cmac.ko 2023-10-18T10:30:44.105 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crc32_generic.ko 2023-10-18T10:30:44.106 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 47505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cryptd.ko 2023-10-18T10:30:44.106 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 39505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_engine.ko 2023-10-18T10:30:44.106 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_simd.ko 2023-10-18T10:30:44.106 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 24737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_user.ko 2023-10-18T10:30:44.106 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cts.ko 2023-10-18T10:30:44.107 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/des_generic.ko 2023-10-18T10:30:44.107 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/dh_generic.ko 2023-10-18T10:30:44.107 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/echainiv.ko 2023-10-18T10:30:44.107 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 25321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/essiv.ko 2023-10-18T10:30:44.107 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/fcrypt.ko 2023-10-18T10:30:44.107 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/khazad.ko 2023-10-18T10:30:44.108 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lrw.ko 2023-10-18T10:30:44.108 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lz4.ko 2023-10-18T10:30:44.108 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lz4hc.ko 2023-10-18T10:30:44.108 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/md4.ko 2023-10-18T10:30:44.108 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/michael_mic.ko 2023-10-18T10:30:44.109 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/pcbc.ko 2023-10-18T10:30:44.109 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/pcrypt.ko 2023-10-18T10:30:44.109 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/rmd160.ko 2023-10-18T10:30:44.109 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 29393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/seed.ko 2023-10-18T10:30:44.109 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 27081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/serpent_generic.ko 2023-10-18T10:30:44.110 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 149193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/tcrypt.ko 2023-10-18T10:30:44.110 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/tea.ko 2023-10-18T10:30:44.110 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 28777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/twofish_common.ko 2023-10-18T10:30:44.110 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/twofish_generic.ko 2023-10-18T10:30:44.110 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/vmac.ko 2023-10-18T10:30:44.111 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 36185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/wp512.ko 2023-10-18T10:30:44.111 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xcbc.ko 2023-10-18T10:30:44.111 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xor.ko 2023-10-18T10:30:44.111 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xts.ko 2023-10-18T10:30:44.111 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xxhash_generic.ko 2023-10-18T10:30:44.111 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ 2023-10-18T10:30:44.112 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/ 2023-10-18T10:30:44.112 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_extlog.ko 2023-10-18T10:30:44.112 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-18T10:30:44.112 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 28793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_pad.ko 2023-10-18T10:30:44.112 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/apei/ 2023-10-18T10:30:44.113 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 31241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/apei/einj.ko 2023-10-18T10:30:44.113 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/ec_sys.ko 2023-10-18T10:30:44.113 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/platform_profile.ko 2023-10-18T10:30:44.113 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 24401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/sbs.ko 2023-10-18T10:30:44.113 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/sbshc.ko 2023-10-18T10:30:44.114 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 103025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/video.ko 2023-10-18T10:30:44.114 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ 2023-10-18T10:30:44.114 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/acard-ahci.ko 2023-10-18T10:30:44.114 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 88865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ahci.ko 2023-10-18T10:30:44.115 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ahci_platform.ko 2023-10-18T10:30:44.115 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 88289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/libahci.ko 2023-10-18T10:30:44.115 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 34985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/libahci_platform.ko 2023-10-18T10:30:44.115 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_acpi.ko 2023-10-18T10:30:44.115 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 30497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ali.ko 2023-10-18T10:30:44.115 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 33569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_amd.ko 2023-10-18T10:30:44.116 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_artop.ko 2023-10-18T10:30:44.116 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_atiixp.ko 2023-10-18T10:30:44.116 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 24945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_atp867x.ko 2023-10-18T10:30:44.116 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cmd640.ko 2023-10-18T10:30:44.116 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 25249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-18T10:30:44.117 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cypress.ko 2023-10-18T10:30:44.117 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_efar.ko 2023-10-18T10:30:44.117 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt366.ko 2023-10-18T10:30:44.117 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 33977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-18T10:30:44.117 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 23129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-18T10:30:44.118 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-18T10:30:44.118 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_it8213.ko 2023-10-18T10:30:44.118 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 35273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_it821x.ko 2023-10-18T10:30:44.118 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_jmicron.ko 2023-10-18T10:30:44.119 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 51481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_legacy.ko 2023-10-18T10:30:44.119 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_marvell.ko 2023-10-18T10:30:44.119 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_mpiix.ko 2023-10-18T10:30:44.119 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_netcell.ko 2023-10-18T10:30:44.119 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ninja32.ko 2023-10-18T10:30:44.120 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ns87410.ko 2023-10-18T10:30:44.120 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ns87415.ko 2023-10-18T10:30:44.120 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-18T10:30:44.120 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_opti.ko 2023-10-18T10:30:44.120 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_optidma.ko 2023-10-18T10:30:44.121 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 32585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-18T10:30:44.121 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 37793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-18T10:30:44.121 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-18T10:30:44.121 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_piccolo.ko 2023-10-18T10:30:44.121 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_radisys.ko 2023-10-18T10:30:44.122 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_rdc.ko 2023-10-18T10:30:44.122 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_rz1000.ko 2023-10-18T10:30:44.122 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sch.ko 2023-10-18T10:30:44.122 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 23097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_serverworks.ko 2023-10-18T10:30:44.122 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 23457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sil680.ko 2023-10-18T10:30:44.123 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-18T10:30:44.123 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_triflex.ko 2023-10-18T10:30:44.123 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 30497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_via.ko 2023-10-18T10:30:44.123 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 22793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pdc_adma.ko 2023-10-18T10:30:44.123 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 29121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_inic162x.ko 2023-10-18T10:30:44.124 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 83345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_mv.ko 2023-10-18T10:30:44.124 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 70393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_nv.ko 2023-10-18T10:30:44.124 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 37377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_promise.ko 2023-10-18T10:30:44.124 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 23041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_qstor.ko 2023-10-18T10:30:44.124 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 29273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sil.ko 2023-10-18T10:30:44.125 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 38337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sil24.ko 2023-10-18T10:30:44.125 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sis.ko 2023-10-18T10:30:44.125 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_svw.ko 2023-10-18T10:30:44.125 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 44793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sx4.ko 2023-10-18T10:30:44.125 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_uli.ko 2023-10-18T10:30:44.126 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 35065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_via.ko 2023-10-18T10:30:44.126 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_vsc.ko 2023-10-18T10:30:44.126 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/auxdisplay/ 2023-10-18T10:30:44.126 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-18T10:30:44.126 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/bcma/ 2023-10-18T10:30:44.127 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 124473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/bcma/bcma.ko 2023-10-18T10:30:44.127 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/ 2023-10-18T10:30:44.127 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/aoe/ 2023-10-18T10:30:44.127 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 113665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/aoe/aoe.ko 2023-10-18T10:30:44.127 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/drbd/ 2023-10-18T10:30:44.128 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 965585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/drbd/drbd.ko 2023-10-18T10:30:44.128 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 173225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/floppy.ko 2023-10-18T10:30:44.128 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/mtip32xx/ 2023-10-18T10:30:44.128 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 94297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-18T10:30:44.128 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 115793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/nbd.ko 2023-10-18T10:30:44.129 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/null_blk/ 2023-10-18T10:30:44.129 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 110449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-18T10:30:44.129 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 98193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/pktcdvd.ko 2023-10-18T10:30:44.129 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 263769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/rbd.ko 2023-10-18T10:30:44.129 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkback/ 2023-10-18T10:30:44.130 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 98145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-18T10:30:44.130 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/zram/ 2023-10-18T10:30:44.130 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 60217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/zram/zram.ko 2023-10-18T10:30:44.130 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ 2023-10-18T10:30:44.130 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 30705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/applicom.ko 2023-10-18T10:30:44.131 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hangcheck-timer.ko 2023-10-18T10:30:44.131 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/ 2023-10-18T10:30:44.131 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-18T10:30:44.131 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-18T10:30:44.131 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-18T10:30:44.132 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-18T10:30:44.132 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-18T10:30:44.132 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ 2023-10-18T10:30:44.132 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 30713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-18T10:30:44.132 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 148601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-18T10:30:44.132 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-18T10:30:44.133 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 139313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-18T10:30:44.133 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 70921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-18T10:30:44.133 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 53513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-18T10:30:44.133 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 36321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/lp.ko 2023-10-18T10:30:44.133 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/mwave/ 2023-10-18T10:30:44.134 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 84097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/mwave/mwave.ko 2023-10-18T10:30:44.134 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 25873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/nvram.ko 2023-10-18T10:30:44.134 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 32169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ppdev.ko 2023-10-18T10:30:44.134 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 51113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tlclk.ko 2023-10-18T10:30:44.134 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/ 2023-10-18T10:30:44.135 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-18T10:30:44.135 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2023-10-18T10:30:44.135 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 23473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2023-10-18T10:30:44.135 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 32985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2023-10-18T10:30:44.135 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 30121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-18T10:30:44.136 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 22697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-18T10:30:44.136 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 23561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/xen-tpmfront.ko 2023-10-18T10:30:44.136 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/xillybus/ 2023-10-18T10:30:44.136 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/xillybus/xillybus_class.ko 2023-10-18T10:30:44.136 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 50945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/xillybus/xillybus_core.ko 2023-10-18T10:30:44.136 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/xillybus/xillybus_pcie.ko 2023-10-18T10:30:44.137 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/clk/ 2023-10-18T10:30:44.137 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/clk/clk-palmas.ko 2023-10-18T10:30:44.137 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 47849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/clk/clk-si5351.ko 2023-10-18T10:30:44.137 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/clk/clk-twl6040.ko 2023-10-18T10:30:44.137 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/clk/clk-wm831x.ko 2023-10-18T10:30:44.138 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/ 2023-10-18T10:30:44.138 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-18T10:30:44.138 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-18T10:30:44.138 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 25297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-18T10:30:44.139 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ 2023-10-18T10:30:44.139 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ 2023-10-18T10:30:44.139 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 70801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-18T10:30:44.139 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 243025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-18T10:30:44.139 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/ 2023-10-18T10:30:44.139 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/ 2023-10-18T10:30:44.140 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_common/ 2023-10-18T10:30:44.140 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 438761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-18T10:30:44.140 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2023-10-18T10:30:44.140 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 24081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-18T10:30:44.140 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/padlock-aes.ko 2023-10-18T10:30:44.141 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/padlock-sha.ko 2023-10-18T10:30:44.141 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/virtio/ 2023-10-18T10:30:44.141 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 68857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-18T10:30:44.141 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dca/ 2023-10-18T10:30:44.141 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 29785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dca/dca.ko 2023-10-18T10:30:44.142 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ 2023-10-18T10:30:44.142 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/dw/ 2023-10-18T10:30:44.142 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-18T10:30:44.142 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ioat/ 2023-10-18T10:30:44.142 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 122569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-18T10:30:44.143 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/ 2023-10-18T10:30:44.143 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 79953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/amd64_edac.ko 2023-10-18T10:30:44.143 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 35769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/e752x_edac.ko 2023-10-18T10:30:44.143 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 63617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-18T10:30:44.143 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i3000_edac.ko 2023-10-18T10:30:44.144 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i3200_edac.ko 2023-10-18T10:30:44.144 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 22537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i5100_edac.ko 2023-10-18T10:30:44.144 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 25657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i5400_edac.ko 2023-10-18T10:30:44.144 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 24913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i7300_edac.ko 2023-10-18T10:30:44.144 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 52041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i7core_edac.ko 2023-10-18T10:30:44.145 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i82975x_edac.ko 2023-10-18T10:30:44.145 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/ie31200_edac.ko 2023-10-18T10:30:44.145 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 56825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/sb_edac.ko 2023-10-18T10:30:44.145 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/x38_edac.ko 2023-10-18T10:30:44.145 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/ 2023-10-18T10:30:44.146 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-gpio.ko 2023-10-18T10:30:44.146 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 27185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-max14577.ko 2023-10-18T10:30:44.146 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 29569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-max77693.ko 2023-10-18T10:30:44.146 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 24745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-max8997.ko 2023-10-18T10:30:44.146 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-palmas.ko 2023-10-18T10:30:44.147 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 23713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-rt8973a.ko 2023-10-18T10:30:44.147 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 27785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-sm5502.ko 2023-10-18T10:30:44.147 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/ 2023-10-18T10:30:44.147 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 34977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/dmi-sysfs.ko 2023-10-18T10:30:44.147 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/efi/ 2023-10-18T10:30:44.148 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/efi/efi-pstore.ko 2023-10-18T10:30:44.148 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-18T10:30:44.148 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/ 2023-10-18T10:30:44.148 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-adp5520.ko 2023-10-18T10:30:44.148 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-amd8111.ko 2023-10-18T10:30:44.148 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-arizona.ko 2023-10-18T10:30:44.149 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-da9052.ko 2023-10-18T10:30:44.149 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-da9055.ko 2023-10-18T10:30:44.149 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 22065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-dln2.ko 2023-10-18T10:30:44.149 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 77849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-f7188x.ko 2023-10-18T10:30:44.150 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 28937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-generic.ko 2023-10-18T10:30:44.150 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-ich.ko 2023-10-18T10:30:44.150 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-janz-ttl.ko 2023-10-18T10:30:44.150 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-kempld.ko 2023-10-18T10:30:44.150 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-lp3943.ko 2023-10-18T10:30:44.151 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-max7300.ko 2023-10-18T10:30:44.151 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-max730x.ko 2023-10-18T10:30:44.151 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 22073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-max732x.ko 2023-10-18T10:30:44.151 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 25145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-ml-ioh.ko 2023-10-18T10:30:44.151 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 47073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-pca953x.ko 2023-10-18T10:30:44.151 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-pcf857x.ko 2023-10-18T10:30:44.152 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-rdc321x.ko 2023-10-18T10:30:44.152 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-sch.ko 2023-10-18T10:30:44.152 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-sch311x.ko 2023-10-18T10:30:44.152 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-tps65912.ko 2023-10-18T10:30:44.152 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-twl4030.ko 2023-10-18T10:30:44.153 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-twl6040.ko 2023-10-18T10:30:44.153 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-18T10:30:44.153 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-vx855.ko 2023-10-18T10:30:44.153 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-wm831x.ko 2023-10-18T10:30:44.153 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-wm8350.ko 2023-10-18T10:30:44.154 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-wm8994.ko 2023-10-18T10:30:44.154 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/ 2023-10-18T10:30:44.154 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-a4tech.ko 2023-10-18T10:30:44.154 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 39169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-apple.ko 2023-10-18T10:30:44.154 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-appleir.ko 2023-10-18T10:30:44.155 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-aureal.ko 2023-10-18T10:30:44.155 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-axff.ko 2023-10-18T10:30:44.155 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-belkin.ko 2023-10-18T10:30:44.155 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-cherry.ko 2023-10-18T10:30:44.155 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-chicony.ko 2023-10-18T10:30:44.156 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 60649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-cp2112.ko 2023-10-18T10:30:44.156 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-cypress.ko 2023-10-18T10:30:44.156 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-dr.ko 2023-10-18T10:30:44.156 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-elecom.ko 2023-10-18T10:30:44.156 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-elo.ko 2023-10-18T10:30:44.156 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-emsff.ko 2023-10-18T10:30:44.157 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-ezkey.ko 2023-10-18T10:30:44.157 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gaff.ko 2023-10-18T10:30:44.157 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-generic.ko 2023-10-18T10:30:44.157 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gt683r.ko 2023-10-18T10:30:44.157 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gyration.ko 2023-10-18T10:30:44.158 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-18T10:30:44.158 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-18T10:30:44.158 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtekff.ko 2023-10-18T10:30:44.158 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-hyperv.ko 2023-10-18T10:30:44.158 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-icade.ko 2023-10-18T10:30:44.159 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-kensington.ko 2023-10-18T10:30:44.159 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-keytouch.ko 2023-10-18T10:30:44.159 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-kye.ko 2023-10-18T10:30:44.159 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lcpower.ko 2023-10-18T10:30:44.159 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-led.ko 2023-10-18T10:30:44.160 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 42361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lenovo.ko 2023-10-18T10:30:44.160 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 30761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lg-g15.ko 2023-10-18T10:30:44.160 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 58185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-18T10:30:44.160 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 101369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-18T10:30:44.160 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 80993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech.ko 2023-10-18T10:30:44.160 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 30801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-magicmouse.ko 2023-10-18T10:30:44.161 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-microsoft.ko 2023-10-18T10:30:44.161 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-monterey.ko 2023-10-18T10:30:44.161 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 50161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-multitouch.ko 2023-10-18T10:30:44.161 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 32593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-ntrig.ko 2023-10-18T10:30:44.161 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-ortek.ko 2023-10-18T10:30:44.162 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-penmount.ko 2023-10-18T10:30:44.162 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-petalynx.ko 2023-10-18T10:30:44.162 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 72937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-picolcd.ko 2023-10-18T10:30:44.162 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-pl.ko 2023-10-18T10:30:44.162 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-plantronics.ko 2023-10-18T10:30:44.163 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-primax.ko 2023-10-18T10:30:44.163 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 35457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-rmi.ko 2023-10-18T10:30:44.163 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-18T10:30:44.163 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-18T10:30:44.163 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 31425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-18T10:30:44.164 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 29729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-18T10:30:44.164 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 31129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-18T10:30:44.164 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-18T10:30:44.164 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 29489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-18T10:30:44.164 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-18T10:30:44.165 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 29313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-18T10:30:44.165 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 27937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-18T10:30:44.165 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-18T10:30:44.165 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 23337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat.ko 2023-10-18T10:30:44.165 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-saitek.ko 2023-10-18T10:30:44.166 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-samsung.ko 2023-10-18T10:30:44.166 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 36817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-18T10:30:44.166 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sjoy.ko 2023-10-18T10:30:44.166 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 51249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sony.ko 2023-10-18T10:30:44.166 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-speedlink.ko 2023-10-18T10:30:44.167 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 25297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-steelseries.ko 2023-10-18T10:30:44.167 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sunplus.ko 2023-10-18T10:30:44.167 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-18T10:30:44.167 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-tivo.ko 2023-10-18T10:30:44.167 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-tmff.ko 2023-10-18T10:30:44.168 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-topseed.ko 2023-10-18T10:30:44.168 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-twinhan.ko 2023-10-18T10:30:44.168 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 71073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-uclogic.ko 2023-10-18T10:30:44.168 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-waltop.ko 2023-10-18T10:30:44.168 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 143473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-wiimote.ko 2023-10-18T10:30:44.169 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-xinmo.ko 2023-10-18T10:30:44.169 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-zpff.ko 2023-10-18T10:30:44.169 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-zydacron.ko 2023-10-18T10:30:44.169 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 267825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid.ko 2023-10-18T10:30:44.169 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 32321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/uhid.ko 2023-10-18T10:30:44.170 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/usbhid/ 2023-10-18T10:30:44.170 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 125721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/usbhid/usbhid.ko 2023-10-18T10:30:44.170 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/usbhid/usbkbd.ko 2023-10-18T10:30:44.170 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/usbhid/usbmouse.ko 2023-10-18T10:30:44.170 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 217873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/wacom.ko 2023-10-18T10:30:44.171 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/ 2023-10-18T10:30:44.171 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 64201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_balloon.ko 2023-10-18T10:30:44.171 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 89625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_utils.ko 2023-10-18T10:30:44.171 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 305465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_vmbus.ko 2023-10-18T10:30:44.171 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ 2023-10-18T10:30:44.171 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 66521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/abituguru.ko 2023-10-18T10:30:44.172 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 61393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/abituguru3.ko 2023-10-18T10:30:44.172 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 34745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-18T10:30:44.172 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ad7414.ko 2023-10-18T10:30:44.172 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ad7418.ko 2023-10-18T10:30:44.172 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 25257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adc128d818.ko 2023-10-18T10:30:44.173 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 31729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1025.ko 2023-10-18T10:30:44.173 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 97977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1026.ko 2023-10-18T10:30:44.173 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1029.ko 2023-10-18T10:30:44.173 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 45553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1031.ko 2023-10-18T10:30:44.173 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm9240.ko 2023-10-18T10:30:44.174 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ads7828.ko 2023-10-18T10:30:44.174 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7410.ko 2023-10-18T10:30:44.174 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 22049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7411.ko 2023-10-18T10:30:44.174 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 74433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7462.ko 2023-10-18T10:30:44.174 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 39585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7470.ko 2023-10-18T10:30:44.175 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 67929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7475.ko 2023-10-18T10:30:44.175 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7x10.ko 2023-10-18T10:30:44.175 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 39073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/amc6821.ko 2023-10-18T10:30:44.175 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 55265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/applesmc.ko 2023-10-18T10:30:44.175 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 49913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asb100.ko 2023-10-18T10:30:44.176 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 58617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asc7621.ko 2023-10-18T10:30:44.176 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 40889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-18T10:30:44.176 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/atxp1.ko 2023-10-18T10:30:44.176 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 28281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/coretemp.ko 2023-10-18T10:30:44.176 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 24657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/da9052-hwmon.ko 2023-10-18T10:30:44.177 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/da9055-hwmon.ko 2023-10-18T10:30:44.177 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 44465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-18T10:30:44.177 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 99601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/dme1737.ko 2023-10-18T10:30:44.177 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ds1621.ko 2023-10-18T10:30:44.177 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ds620.ko 2023-10-18T10:30:44.177 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 29777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc1403.ko 2023-10-18T10:30:44.178 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 33185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc2103.ko 2023-10-18T10:30:44.178 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 30321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc6w201.ko 2023-10-18T10:30:44.178 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 66985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f71805f.ko 2023-10-18T10:30:44.178 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 94457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f71882fg.ko 2023-10-18T10:30:44.178 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 41401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f75375s.ko 2023-10-18T10:30:44.179 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-18T10:30:44.179 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 55449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/fschmd.ko 2023-10-18T10:30:44.179 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/g760a.ko 2023-10-18T10:30:44.179 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 24017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/g762.ko 2023-10-18T10:30:44.180 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 46801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/gl518sm.ko 2023-10-18T10:30:44.180 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 45529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/gl520sm.ko 2023-10-18T10:30:44.180 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/hih6130.ko 2023-10-18T10:30:44.180 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-18T10:30:44.180 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-18T10:30:44.181 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 22601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-18T10:30:44.181 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 34529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ibmaem.ko 2023-10-18T10:30:44.181 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ibmpex.ko 2023-10-18T10:30:44.181 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 24873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ina209.ko 2023-10-18T10:30:44.181 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ina2xx.ko 2023-10-18T10:30:44.182 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 130449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/it87.ko 2023-10-18T10:30:44.182 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/jc42.ko 2023-10-18T10:30:44.182 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 24329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/k10temp.ko 2023-10-18T10:30:44.182 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/k8temp.ko 2023-10-18T10:30:44.182 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-18T10:30:44.183 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 48601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm63.ko 2023-10-18T10:30:44.183 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm73.ko 2023-10-18T10:30:44.183 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 36777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm75.ko 2023-10-18T10:30:44.183 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm77.ko 2023-10-18T10:30:44.183 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 34657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm78.ko 2023-10-18T10:30:44.184 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 31745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm80.ko 2023-10-18T10:30:44.184 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm83.ko 2023-10-18T10:30:44.184 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 72833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm85.ko 2023-10-18T10:30:44.184 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 46153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm87.ko 2023-10-18T10:30:44.184 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 37913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm90.ko 2023-10-18T10:30:44.185 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm92.ko 2023-10-18T10:30:44.185 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 108841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm93.ko 2023-10-18T10:30:44.185 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 33393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95234.ko 2023-10-18T10:30:44.185 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95241.ko 2023-10-18T10:30:44.185 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95245.ko 2023-10-18T10:30:44.186 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 24017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc2945.ko 2023-10-18T10:30:44.186 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4151.ko 2023-10-18T10:30:44.186 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4215.ko 2023-10-18T10:30:44.186 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4222.ko 2023-10-18T10:30:44.186 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4245.ko 2023-10-18T10:30:44.187 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4260.ko 2023-10-18T10:30:44.187 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4261.ko 2023-10-18T10:30:44.187 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 33489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max16065.ko 2023-10-18T10:30:44.187 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max1619.ko 2023-10-18T10:30:44.187 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 24353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max1668.ko 2023-10-18T10:30:44.188 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max197.ko 2023-10-18T10:30:44.188 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 27129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6639.ko 2023-10-18T10:30:44.188 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 23321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6650.ko 2023-10-18T10:30:44.188 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 35777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6697.ko 2023-10-18T10:30:44.188 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/mc13783-adc.ko 2023-10-18T10:30:44.189 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/mcp3021.ko 2023-10-18T10:30:44.189 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2023-10-18T10:30:44.189 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 44129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6683.ko 2023-10-18T10:30:44.189 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 119817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-18T10:30:44.189 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 60705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6775.ko 2023-10-18T10:30:44.190 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 59737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct7802.ko 2023-10-18T10:30:44.190 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 82337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pc87360.ko 2023-10-18T10:30:44.190 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 54433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pc87427.ko 2023-10-18T10:30:44.190 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pcf8591.ko 2023-10-18T10:30:44.190 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ 2023-10-18T10:30:44.191 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-18T10:30:44.191 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-18T10:30:44.191 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 28329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-18T10:30:44.191 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-18T10:30:44.191 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-18T10:30:44.192 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-18T10:30:44.192 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-18T10:30:44.192 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 82577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-18T10:30:44.192 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-18T10:30:44.192 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 30729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-18T10:30:44.193 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-18T10:30:44.193 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-18T10:30:44.193 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/powr1220.ko 2023-10-18T10:30:44.193 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch5627.ko 2023-10-18T10:30:44.193 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 28817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch5636.ko 2023-10-18T10:30:44.194 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 27145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-18T10:30:44.194 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 32137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sht15.ko 2023-10-18T10:30:44.194 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sht21.ko 2023-10-18T10:30:44.194 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/shtc1.ko 2023-10-18T10:30:44.194 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 40105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sis5595.ko 2023-10-18T10:30:44.195 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-18T10:30:44.195 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 39193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-18T10:30:44.195 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 37385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-18T10:30:44.195 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/thmc50.ko 2023-10-18T10:30:44.195 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp102.ko 2023-10-18T10:30:44.196 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp103.ko 2023-10-18T10:30:44.196 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp401.ko 2023-10-18T10:30:44.196 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp421.ko 2023-10-18T10:30:44.196 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-18T10:30:44.196 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 41945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/via686a.ko 2023-10-18T10:30:44.197 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 52457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/vt1211.ko 2023-10-18T10:30:44.197 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 51545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/vt8231.ko 2023-10-18T10:30:44.197 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 57513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-18T10:30:44.197 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 78873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83627hf.ko 2023-10-18T10:30:44.197 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 71345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83781d.ko 2023-10-18T10:30:44.198 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 63289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83791d.ko 2023-10-18T10:30:44.198 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 71321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83792d.ko 2023-10-18T10:30:44.198 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 101473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83793.ko 2023-10-18T10:30:44.198 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 119505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83795.ko 2023-10-18T10:30:44.198 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-18T10:30:44.199 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 35625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-18T10:30:44.199 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/wm831x-hwmon.ko 2023-10-18T10:30:44.199 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/wm8350-hwmon.ko 2023-10-18T10:30:44.199 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/ 2023-10-18T10:30:44.199 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/ 2023-10-18T10:30:44.200 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-18T10:30:44.200 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-18T10:30:44.200 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/ 2023-10-18T10:30:44.200 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 27969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ali1535.ko 2023-10-18T10:30:44.200 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 28801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ali1563.ko 2023-10-18T10:30:44.201 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 27777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2023-10-18T10:30:44.201 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-18T10:30:44.201 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-18T10:30:44.201 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-18T10:30:44.201 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2023-10-18T10:30:44.202 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 6929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-18T10:30:44.202 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2023-10-18T10:30:44.202 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 50201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-18T10:30:44.202 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 23905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-18T10:30:44.202 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-18T10:30:44.203 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-18T10:30:44.203 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-dln2.ko 2023-10-18T10:30:44.203 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-gpio.ko 2023-10-18T10:30:44.203 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 62817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-18T10:30:44.203 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 22433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-18T10:30:44.204 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 47889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-18T10:30:44.204 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-kempld.ko 2023-10-18T10:30:44.204 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-18T10:30:44.204 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 22337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-18T10:30:44.204 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ocores.ko 2023-10-18T10:30:44.205 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 25145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-18T10:30:44.205 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-18T10:30:44.205 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 47241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-18T10:30:44.205 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2023-10-18T10:30:44.206 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-18T10:30:44.206 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-18T10:30:44.206 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 22313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-sis5595.ko 2023-10-18T10:30:44.206 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 29361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-sis630.ko 2023-10-18T10:30:44.206 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-18T10:30:44.207 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2023-10-18T10:30:44.207 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-18T10:30:44.207 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-18T10:30:44.207 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-18T10:30:44.207 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-18T10:30:44.208 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 47657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-xiic.ko 2023-10-18T10:30:44.208 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 22449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-mux.ko 2023-10-18T10:30:44.208 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-18T10:30:44.208 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 27073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-stub.ko 2023-10-18T10:30:44.209 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/muxes/ 2023-10-18T10:30:44.209 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2023-10-18T10:30:44.209 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2023-10-18T10:30:44.209 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 29801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2023-10-18T10:30:44.209 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ 2023-10-18T10:30:44.210 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ 2023-10-18T10:30:44.210 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 273609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-18T10:30:44.210 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 879969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-18T10:30:44.210 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 69337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-18T10:30:44.211 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 324217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-18T10:30:44.211 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 97985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-18T10:30:44.211 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 282273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-18T10:30:44.211 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 66201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-18T10:30:44.211 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/ 2023-10-18T10:30:44.212 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/cxgb4/ 2023-10-18T10:30:44.212 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 493073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-18T10:30:44.212 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx4/ 2023-10-18T10:30:44.212 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 419737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-18T10:30:44.213 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx5/ 2023-10-18T10:30:44.213 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 739025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-18T10:30:44.340 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mthca/ 2023-10-18T10:30:44.340 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 296625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-18T10:30:44.341 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/ocrdma/ 2023-10-18T10:30:44.341 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 153697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2023-10-18T10:30:44.341 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/usnic/ 2023-10-18T10:30:44.341 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 168969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2023-10-18T10:30:44.341 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ 2023-10-18T10:30:44.342 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ipoib/ 2023-10-18T10:30:44.342 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 263081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-18T10:30:44.342 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/iser/ 2023-10-18T10:30:44.342 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 100241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-18T10:30:44.343 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/isert/ 2023-10-18T10:30:44.343 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 111425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-18T10:30:44.343 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srp/ 2023-10-18T10:30:44.343 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 128753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-18T10:30:44.343 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srpt/ 2023-10-18T10:30:44.343 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 129529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-18T10:30:44.344 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/ 2023-10-18T10:30:44.344 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/evbug.ko 2023-10-18T10:30:44.344 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 24129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/ff-memless.ko 2023-10-18T10:30:44.344 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/ 2023-10-18T10:30:44.344 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-18T10:30:44.345 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-18T10:30:44.345 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 41241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/gameport.ko 2023-10-18T10:30:44.345 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 41801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/joydev.ko 2023-10-18T10:30:44.345 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/ 2023-10-18T10:30:44.345 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/adp5520-keys.ko 2023-10-18T10:30:44.346 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/adp5588-keys.ko 2023-10-18T10:30:44.346 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/adp5589-keys.ko 2023-10-18T10:30:44.346 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 24761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2023-10-18T10:30:44.346 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 36665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-18T10:30:44.346 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2023-10-18T10:30:44.347 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/lkkbd.ko 2023-10-18T10:30:44.347 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/lm8323.ko 2023-10-18T10:30:44.347 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/lm8333.ko 2023-10-18T10:30:44.347 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 24265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/matrix_keypad.ko 2023-10-18T10:30:44.347 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/max7359_keypad.ko 2023-10-18T10:30:44.348 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/mcs_touchkey.ko 2023-10-18T10:30:44.348 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2023-10-18T10:30:44.348 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/newtonkbd.ko 2023-10-18T10:30:44.348 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/opencores-kbd.ko 2023-10-18T10:30:44.348 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/qt1070.ko 2023-10-18T10:30:44.348 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/qt2160.ko 2023-10-18T10:30:44.349 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 23817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/samsung-keypad.ko 2023-10-18T10:30:44.349 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/stowaway.ko 2023-10-18T10:30:44.349 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/sunkbd.ko 2023-10-18T10:30:44.349 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/tca6416-keypad.ko 2023-10-18T10:30:44.349 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/tca8418_keypad.ko 2023-10-18T10:30:44.350 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/twl4030_keypad.ko 2023-10-18T10:30:44.350 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/xtkbd.ko 2023-10-18T10:30:44.350 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/matrix-keymap.ko 2023-10-18T10:30:44.350 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/ 2023-10-18T10:30:44.350 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 34329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/appletouch.ko 2023-10-18T10:30:44.351 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 34177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-18T10:30:44.351 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 123681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-18T10:30:44.351 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 89449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-18T10:30:44.351 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/gpio_mouse.ko 2023-10-18T10:30:44.351 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 335977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/psmouse.ko 2023-10-18T10:30:44.352 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/sermouse.ko 2023-10-18T10:30:44.352 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 23881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-18T10:30:44.352 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 22569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-18T10:30:44.352 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-18T10:30:44.352 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/rmi4/ 2023-10-18T10:30:44.353 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 109873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-18T10:30:44.353 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/ 2023-10-18T10:30:44.353 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-18T10:30:44.353 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-18T10:30:44.353 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/ct82c710.ko 2023-10-18T10:30:44.354 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-18T10:30:44.354 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/parkbd.ko 2023-10-18T10:30:44.354 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/pcips2.ko 2023-10-18T10:30:44.354 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/ps2mult.ko 2023-10-18T10:30:44.354 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 24153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/serio_raw.ko 2023-10-18T10:30:44.355 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/serport.ko 2023-10-18T10:30:44.355 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/sparse-keymap.ko 2023-10-18T10:30:44.355 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/ 2023-10-18T10:30:44.355 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/amd/ 2023-10-18T10:30:44.355 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 39289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-18T10:30:44.356 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/ 2023-10-18T10:30:44.356 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-88pm860x.ko 2023-10-18T10:30:44.356 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-adp5520.ko 2023-10-18T10:30:44.356 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 59753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-bd2802.ko 2023-10-18T10:30:44.356 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 24785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-blinkm.ko 2023-10-18T10:30:44.357 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-da903x.ko 2023-10-18T10:30:44.357 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-da9052.ko 2023-10-18T10:30:44.357 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-gpio.ko 2023-10-18T10:30:44.357 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm3530.ko 2023-10-18T10:30:44.357 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 30505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm3533.ko 2023-10-18T10:30:44.357 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm355x.ko 2023-10-18T10:30:44.358 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm3642.ko 2023-10-18T10:30:44.358 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lp3944.ko 2023-10-18T10:30:44.358 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lp8788.ko 2023-10-18T10:30:44.358 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-max8997.ko 2023-10-18T10:30:44.358 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-mc13783.ko 2023-10-18T10:30:44.359 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-menf21bmc.ko 2023-10-18T10:30:44.359 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-pca9532.ko 2023-10-18T10:30:44.359 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-pca955x.ko 2023-10-18T10:30:44.359 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-pca963x.ko 2023-10-18T10:30:44.359 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 34433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-ss4200.ko 2023-10-18T10:30:44.360 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 24865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-tca6507.ko 2023-10-18T10:30:44.360 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-wm831x-status.ko 2023-10-18T10:30:44.360 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-wm8350.ko 2023-10-18T10:30:44.360 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ 2023-10-18T10:30:44.360 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-18T10:30:44.361 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-18T10:30:44.361 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-18T10:30:44.361 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 7617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-18T10:30:44.361 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-18T10:30:44.361 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-18T10:30:44.362 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-18T10:30:44.362 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-18T10:30:44.362 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/ 2023-10-18T10:30:44.362 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/bcache/ 2023-10-18T10:30:44.362 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 654969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/bcache/bcache.ko 2023-10-18T10:30:44.362 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 40577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-bio-prison.ko 2023-10-18T10:30:44.363 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 89601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-bufio.ko 2023-10-18T10:30:44.363 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 47529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-cache-smq.ko 2023-10-18T10:30:44.363 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 156009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-cache.ko 2023-10-18T10:30:44.363 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 106897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-crypt.ko 2023-10-18T10:30:44.363 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-delay.ko 2023-10-18T10:30:44.364 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 53857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-era.ko 2023-10-18T10:30:44.364 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 28465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-flakey.ko 2023-10-18T10:30:44.364 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 41593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-log-userspace.ko 2023-10-18T10:30:44.364 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 32737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-log.ko 2023-10-18T10:30:44.365 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 49545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-mirror.ko 2023-10-18T10:30:44.365 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 78033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-multipath.ko 2023-10-18T10:30:44.365 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-queue-length.ko 2023-10-18T10:30:44.365 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 70825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-raid.ko 2023-10-18T10:30:44.365 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 36297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-region-hash.ko 2023-10-18T10:30:44.366 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-round-robin.ko 2023-10-18T10:30:44.366 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-service-time.ko 2023-10-18T10:30:44.366 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 109593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-snapshot.ko 2023-10-18T10:30:44.366 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-switch.ko 2023-10-18T10:30:44.366 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 181865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-thin-pool.ko 2023-10-18T10:30:44.367 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 51737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-verity.ko 2023-10-18T10:30:44.367 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-zero.ko 2023-10-18T10:30:44.367 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/faulty.ko 2023-10-18T10:30:44.367 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/linear.ko 2023-10-18T10:30:44.367 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 32417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/multipath.ko 2023-10-18T10:30:44.368 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/persistent-data/ 2023-10-18T10:30:44.368 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 199937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-18T10:30:44.368 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 37337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid0.ko 2023-10-18T10:30:44.368 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 113073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid1.ko 2023-10-18T10:30:44.368 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 158681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid10.ko 2023-10-18T10:30:44.369 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 395721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid456.ko 2023-10-18T10:30:44.369 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/ 2023-10-18T10:30:44.369 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/ 2023-10-18T10:30:44.369 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 178137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptbase.ko 2023-10-18T10:30:44.369 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 63385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptctl.ko 2023-10-18T10:30:44.370 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 44665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptfc.ko 2023-10-18T10:30:44.370 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 37225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptlan.ko 2023-10-18T10:30:44.370 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 123321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptsas.ko 2023-10-18T10:30:44.370 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 79041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-18T10:30:44.370 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 53081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptspi.ko 2023-10-18T10:30:44.371 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/ 2023-10-18T10:30:44.371 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/88pm800.ko 2023-10-18T10:30:44.371 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/88pm805.ko 2023-10-18T10:30:44.371 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/88pm80x.ko 2023-10-18T10:30:44.371 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/arizona-i2c.ko 2023-10-18T10:30:44.372 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 123721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/arizona.ko 2023-10-18T10:30:44.372 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/bcm590xx.ko 2023-10-18T10:30:44.372 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/cros_ec_dev.ko 2023-10-18T10:30:44.372 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 34961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/dln2.ko 2023-10-18T10:30:44.372 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/janz-cmodio.ko 2023-10-18T10:30:44.372 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 53585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/kempld-core.ko 2023-10-18T10:30:44.373 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 27529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lm3533-core.ko 2023-10-18T10:30:44.373 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lm3533-ctrlbank.ko 2023-10-18T10:30:44.373 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lp3943.ko 2023-10-18T10:30:44.373 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 54569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lpc_ich.ko 2023-10-18T10:30:44.373 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lpc_sch.ko 2023-10-18T10:30:44.374 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/max8907.ko 2023-10-18T10:30:44.374 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 29857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/mc13xxx-core.ko 2023-10-18T10:30:44.374 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/mc13xxx-i2c.ko 2023-10-18T10:30:44.374 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/menf21bmc.ko 2023-10-18T10:30:44.374 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/pcf50633-adc.ko 2023-10-18T10:30:44.375 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/pcf50633-gpio.ko 2023-10-18T10:30:44.375 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 34873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/pcf50633.ko 2023-10-18T10:30:44.375 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/rdc321x-southbridge.ko 2023-10-18T10:30:44.375 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/retu-mfd.ko 2023-10-18T10:30:44.375 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 80425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/si476x-core.ko 2023-10-18T10:30:44.376 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 48457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/sm501.ko 2023-10-18T10:30:44.376 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/ti_am335x_tscadc.ko 2023-10-18T10:30:44.376 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 51137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/tps65010.ko 2023-10-18T10:30:44.376 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/tps6507x.ko 2023-10-18T10:30:44.376 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/viperboard.ko 2023-10-18T10:30:44.377 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/vx855.ko 2023-10-18T10:30:44.377 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/wl1273-core.ko 2023-10-18T10:30:44.377 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 44345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/wm8994.ko 2023-10-18T10:30:44.377 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ 2023-10-18T10:30:44.377 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ad525x_dpot-i2c.ko 2023-10-18T10:30:44.377 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 48489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ad525x_dpot.ko 2023-10-18T10:30:44.378 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/altera-stapl/ 2023-10-18T10:30:44.378 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 46585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-18T10:30:44.378 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/apds9802als.ko 2023-10-18T10:30:44.378 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 36729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/apds990x.ko 2023-10-18T10:30:44.378 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 45593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/bh1770glc.ko 2023-10-18T10:30:44.379 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/c2port/ 2023-10-18T10:30:44.379 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2023-10-18T10:30:44.379 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 35281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/c2port/core.ko 2023-10-18T10:30:44.379 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/cb710/ 2023-10-18T10:30:44.379 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 25161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/cb710/cb710.ko 2023-10-18T10:30:44.380 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ds1682.ko 2023-10-18T10:30:44.380 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/dummy-irq.ko 2023-10-18T10:30:44.380 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/echo/ 2023-10-18T10:30:44.380 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/echo/echo.ko 2023-10-18T10:30:44.380 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/ 2023-10-18T10:30:44.381 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 37849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/at24.ko 2023-10-18T10:30:44.381 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-18T10:30:44.381 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-18T10:30:44.381 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-18T10:30:44.381 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 33009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/enclosure.ko 2023-10-18T10:30:44.382 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/genwqe/ 2023-10-18T10:30:44.382 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 152201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/genwqe/genwqe_card.ko 2023-10-18T10:30:44.382 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/hmc6352.ko 2023-10-18T10:30:44.382 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 31321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/hpilo.ko 2023-10-18T10:30:44.382 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ibmasm/ 2023-10-18T10:30:44.382 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 72129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ibmasm/ibmasm.ko 2023-10-18T10:30:44.383 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ics932s401.ko 2023-10-18T10:30:44.383 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/isl29003.ko 2023-10-18T10:30:44.383 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/isl29020.ko 2023-10-18T10:30:44.383 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/ 2023-10-18T10:30:44.383 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 42513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-18T10:30:44.384 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-18T10:30:44.384 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/ 2023-10-18T10:30:44.384 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 125209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei-me.ko 2023-10-18T10:30:44.384 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 56521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-18T10:30:44.384 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 310113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei.ko 2023-10-18T10:30:44.385 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 31329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/phantom.ko 2023-10-18T10:30:44.385 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ti-st/ 2023-10-18T10:30:44.385 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 74265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ti-st/st_drv.ko 2023-10-18T10:30:44.385 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 24305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-18T10:30:44.385 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 28849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tifm_core.ko 2023-10-18T10:30:44.386 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tsl2550.ko 2023-10-18T10:30:44.386 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 48697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_balloon.ko 2023-10-18T10:30:44.386 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_vmci/ 2023-10-18T10:30:44.386 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 183609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-18T10:30:44.386 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ 2023-10-18T10:30:44.387 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/bonding/ 2023-10-18T10:30:44.387 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 490129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/bonding/bonding.ko 2023-10-18T10:30:44.387 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/ 2023-10-18T10:30:44.387 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/b53/ 2023-10-18T10:30:44.387 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 106033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-18T10:30:44.387 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 40297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-18T10:30:44.388 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-18T10:30:44.388 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6xxx/ 2023-10-18T10:30:44.388 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 453737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-18T10:30:44.388 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dummy.ko 2023-10-18T10:30:44.388 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/eql.ko 2023-10-18T10:30:44.389 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ 2023-10-18T10:30:44.389 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/ 2023-10-18T10:30:44.389 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 45785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-18T10:30:44.389 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 42105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-18T10:30:44.389 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 130553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-18T10:30:44.390 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 68849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-18T10:30:44.390 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/ 2023-10-18T10:30:44.390 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 37649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-18T10:30:44.390 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 59689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-18T10:30:44.390 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 27201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-18T10:30:44.391 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 99113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-18T10:30:44.391 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/adaptec/ 2023-10-18T10:30:44.391 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 54521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-18T10:30:44.391 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/agere/ 2023-10-18T10:30:44.391 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 69201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-18T10:30:44.392 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/alteon/ 2023-10-18T10:30:44.392 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 59585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-18T10:30:44.392 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/altera/ 2023-10-18T10:30:44.392 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 41409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-18T10:30:44.392 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/ 2023-10-18T10:30:44.393 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 54609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-18T10:30:44.393 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 49289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-18T10:30:44.393 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 87297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-18T10:30:44.393 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/ 2023-10-18T10:30:44.393 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/alx/ 2023-10-18T10:30:44.393 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 97337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-18T10:30:44.394 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1c/ 2023-10-18T10:30:44.394 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 104673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-18T10:30:44.394 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1e/ 2023-10-18T10:30:44.394 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 90257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-18T10:30:44.394 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/ 2023-10-18T10:30:44.395 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 81569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-18T10:30:44.395 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 69729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-18T10:30:44.395 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/ 2023-10-18T10:30:44.395 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 78401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-18T10:30:44.395 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 176761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-18T10:30:44.396 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2023-10-18T10:30:44.396 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 1421497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-18T10:30:44.396 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 128569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-18T10:30:44.396 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/genet/ 2023-10-18T10:30:44.396 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 132089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-18T10:30:44.396 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 312081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-18T10:30:44.397 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/ 2023-10-18T10:30:44.397 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/bna/ 2023-10-18T10:30:44.397 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 323513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-18T10:30:44.397 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/ 2023-10-18T10:30:44.397 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb/ 2023-10-18T10:30:44.398 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 193497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-18T10:30:44.398 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2023-10-18T10:30:44.398 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 347417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-18T10:30:44.398 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2023-10-18T10:30:44.398 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 1044433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-18T10:30:44.398 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2023-10-18T10:30:44.399 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 166193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-18T10:30:44.399 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2023-10-18T10:30:44.399 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 34881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-18T10:30:44.399 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/ 2023-10-18T10:30:44.399 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/enic/ 2023-10-18T10:30:44.400 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 204345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-18T10:30:44.400 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/ 2023-10-18T10:30:44.400 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/ 2023-10-18T10:30:44.400 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 69449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-18T10:30:44.400 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 61489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-18T10:30:44.400 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 141225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-18T10:30:44.401 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 51497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-18T10:30:44.401 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 62817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-18T10:30:44.401 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 31369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-18T10:30:44.401 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/ 2023-10-18T10:30:44.401 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 60873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-18T10:30:44.402 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 62833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-18T10:30:44.402 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 45721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dnet.ko 2023-10-18T10:30:44.402 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 25185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ec_bhf.ko 2023-10-18T10:30:44.402 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/ 2023-10-18T10:30:44.402 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/benet/ 2023-10-18T10:30:44.403 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 281137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-18T10:30:44.403 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 49457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-18T10:30:44.403 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 51737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-18T10:30:44.403 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/fujitsu/ 2023-10-18T10:30:44.403 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 45345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2023-10-18T10:30:44.403 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ 2023-10-18T10:30:44.404 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 90465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-18T10:30:44.404 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000/ 2023-10-18T10:30:44.404 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 295577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-18T10:30:44.404 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000e/ 2023-10-18T10:30:44.404 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 584497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-18T10:30:44.405 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/fm10k/ 2023-10-18T10:30:44.405 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 246209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-18T10:30:44.405 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/i40e/ 2023-10-18T10:30:44.405 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 925697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-18T10:30:44.405 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/iavf/ 2023-10-18T10:30:44.406 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 302849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-18T10:30:44.406 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igb/ 2023-10-18T10:30:44.406 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 514681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-18T10:30:44.406 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igbvf/ 2023-10-18T10:30:44.406 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 93593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-18T10:30:44.406 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbe/ 2023-10-18T10:30:44.407 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 734305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-18T10:30:44.407 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbevf/ 2023-10-18T10:30:44.407 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 151769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-18T10:30:44.407 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 86921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/jme.ko 2023-10-18T10:30:44.408 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/ 2023-10-18T10:30:44.408 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-18T10:30:44.408 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 102289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-18T10:30:44.408 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 122649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-18T10:30:44.408 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/ 2023-10-18T10:30:44.409 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/ 2023-10-18T10:30:44.409 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 727601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-18T10:30:44.409 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 278937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-18T10:30:44.409 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/ 2023-10-18T10:30:44.409 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2023-10-18T10:30:44.410 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 1113993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-18T10:30:44.410 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ 2023-10-18T10:30:44.410 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 53105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ks8842.ko 2023-10-18T10:30:44.410 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 40993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2023-10-18T10:30:44.410 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2023-10-18T10:30:44.411 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 95665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-18T10:30:44.411 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/ 2023-10-18T10:30:44.411 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/myri10ge/ 2023-10-18T10:30:44.411 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 108649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-18T10:30:44.411 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/ 2023-10-18T10:30:44.412 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 73441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-18T10:30:44.412 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 47777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-18T10:30:44.412 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/neterion/ 2023-10-18T10:30:44.412 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 150905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-18T10:30:44.412 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/nvidia/ 2023-10-18T10:30:44.413 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 136721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-18T10:30:44.413 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/ 2023-10-18T10:30:44.413 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 50481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-18T10:30:44.413 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 43145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-18T10:30:44.413 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/ 2023-10-18T10:30:44.413 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/netxen/ 2023-10-18T10:30:44.414 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 227897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-18T10:30:44.414 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 87081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-18T10:30:44.414 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2023-10-18T10:30:44.414 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 672049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-18T10:30:44.414 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rdc/ 2023-10-18T10:30:44.415 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 43017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-18T10:30:44.415 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/ 2023-10-18T10:30:44.415 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 67385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-18T10:30:44.415 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 83985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-18T10:30:44.415 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 29321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-18T10:30:44.416 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 189801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-18T10:30:44.416 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/samsung/ 2023-10-18T10:30:44.416 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/samsung/sxgbe/ 2023-10-18T10:30:44.416 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 104241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2023-10-18T10:30:44.416 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sfc/ 2023-10-18T10:30:44.417 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 1053921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-18T10:30:44.417 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/silan/ 2023-10-18T10:30:44.417 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 47097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-18T10:30:44.417 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/ 2023-10-18T10:30:44.417 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 54817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-18T10:30:44.418 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 63337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-18T10:30:44.418 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/ 2023-10-18T10:30:44.418 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 61601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-18T10:30:44.418 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 67193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-18T10:30:44.418 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 73409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-18T10:30:44.419 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 68177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-18T10:30:44.419 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/ 2023-10-18T10:30:44.419 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/ 2023-10-18T10:30:44.419 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2023-10-18T10:30:44.419 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 39905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-18T10:30:44.419 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2023-10-18T10:30:44.420 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 486961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-18T10:30:44.420 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/ 2023-10-18T10:30:44.420 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 103337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-18T10:30:44.420 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 166161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-18T10:30:44.420 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 66329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-18T10:30:44.421 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 78801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-18T10:30:44.421 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/tehuti/ 2023-10-18T10:30:44.421 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 50057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-18T10:30:44.421 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ti/ 2023-10-18T10:30:44.421 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 86529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-18T10:30:44.422 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/ 2023-10-18T10:30:44.422 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 80273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-18T10:30:44.422 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 83633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-18T10:30:44.422 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/ 2023-10-18T10:30:44.422 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 46241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-18T10:30:44.423 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 30849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-18T10:30:44.423 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/xircom/ 2023-10-18T10:30:44.423 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 60057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-18T10:30:44.423 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/fddi/ 2023-10-18T10:30:44.423 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 41065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/fddi/defxx.ko 2023-10-18T10:30:44.424 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/fddi/skfp/ 2023-10-18T10:30:44.424 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 216985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/fddi/skfp/skfp.ko 2023-10-18T10:30:44.424 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 77345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/geneve.ko 2023-10-18T10:30:44.424 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/hyperv/ 2023-10-18T10:30:44.424 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 235521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-18T10:30:44.425 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ieee802154/ 2023-10-18T10:30:44.425 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-18T10:30:44.425 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 25065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ifb.ko 2023-10-18T10:30:44.425 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ipvlan/ 2023-10-18T10:30:44.425 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 83769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-18T10:30:44.425 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 62209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/macvlan.ko 2023-10-18T10:30:44.426 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/macvtap.ko 2023-10-18T10:30:44.426 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/ 2023-10-18T10:30:44.426 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-18T10:30:44.426 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-18T10:30:44.426 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-gpio.ko 2023-10-18T10:30:44.427 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-18T10:30:44.427 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio.ko 2023-10-18T10:30:44.427 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mii.ko 2023-10-18T10:30:44.427 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 43953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/netconsole.ko 2023-10-18T10:30:44.427 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/nlmon.ko 2023-10-18T10:30:44.428 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/ 2023-10-18T10:30:44.428 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-18T10:30:44.428 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 38977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-18T10:30:44.428 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/ 2023-10-18T10:30:44.428 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/amd.ko 2023-10-18T10:30:44.428 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 50097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-18T10:30:44.429 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 44505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-18T10:30:44.429 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-18T10:30:44.429 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 42241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/broadcom.ko 2023-10-18T10:30:44.429 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/cicada.ko 2023-10-18T10:30:44.429 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/davicom.ko 2023-10-18T10:30:44.430 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/et1011c.ko 2023-10-18T10:30:44.430 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 24969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/icplus.ko 2023-10-18T10:30:44.430 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/lxt.ko 2023-10-18T10:30:44.430 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 82049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/marvell.ko 2023-10-18T10:30:44.430 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 76177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/micrel.ko 2023-10-18T10:30:44.431 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/national.ko 2023-10-18T10:30:44.431 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 97857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/phylink.ko 2023-10-18T10:30:44.431 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/qsemi.ko 2023-10-18T10:30:44.431 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 50065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/realtek.ko 2023-10-18T10:30:44.431 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 29985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/smsc.ko 2023-10-18T10:30:44.432 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-18T10:30:44.432 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 24289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/vitesse.ko 2023-10-18T10:30:44.432 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/plip/ 2023-10-18T10:30:44.432 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 43569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/plip/plip.ko 2023-10-18T10:30:44.432 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ 2023-10-18T10:30:44.432 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-18T10:30:44.433 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 29713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-18T10:30:44.433 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-18T10:30:44.433 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-18T10:30:44.433 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 28729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-18T10:30:44.433 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 42545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pppoe.ko 2023-10-18T10:30:44.434 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pppox.ko 2023-10-18T10:30:44.434 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 32689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pptp.ko 2023-10-18T10:30:44.434 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 28289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/sb1000.ko 2023-10-18T10:30:44.434 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip/ 2023-10-18T10:30:44.434 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 42465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip/slip.ko 2023-10-18T10:30:44.435 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/sungem_phy.ko 2023-10-18T10:30:44.435 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 64385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/tap.ko 2023-10-18T10:30:44.435 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/ 2023-10-18T10:30:44.435 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 106169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team.ko 2023-10-18T10:30:44.435 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-18T10:30:44.437 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-18T10:30:44.437 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 28881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-18T10:30:44.437 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_random.ko 2023-10-18T10:30:44.438 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-18T10:30:44.438 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 87017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/veth.ko 2023-10-18T10:30:44.438 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vmxnet3/ 2023-10-18T10:30:44.438 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 144297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-18T10:30:44.438 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vxlan/ 2023-10-18T10:30:44.439 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 297329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-18T10:30:44.439 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netback/ 2023-10-18T10:30:44.439 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 144649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-18T10:30:44.439 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ntb/ 2023-10-18T10:30:44.439 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ntb/ntb.ko 2023-10-18T10:30:44.440 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/ 2023-10-18T10:30:44.440 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/ 2023-10-18T10:30:44.440 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 264937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-18T10:30:44.440 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 102209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/nvme.ko 2023-10-18T10:30:44.440 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/ 2023-10-18T10:30:44.441 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 133841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport.ko 2023-10-18T10:30:44.441 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_cs.ko 2023-10-18T10:30:44.441 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 84465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_pc.ko 2023-10-18T10:30:44.441 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_serial.ko 2023-10-18T10:30:44.441 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/ 2023-10-18T10:30:44.441 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/ 2023-10-18T10:30:44.442 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 25009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-18T10:30:44.442 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2023-10-18T10:30:44.442 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2023-10-18T10:30:44.442 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/pci-stub.ko 2023-10-18T10:30:44.442 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 46529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/xen-pcifront.ko 2023-10-18T10:30:44.443 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/ 2023-10-18T10:30:44.443 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/i82092.ko 2023-10-18T10:30:44.443 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 145809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pcmcia.ko 2023-10-18T10:30:44.443 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 62137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pcmcia_core.ko 2023-10-18T10:30:44.443 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 34345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2023-10-18T10:30:44.444 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 34265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pd6729.ko 2023-10-18T10:30:44.444 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 112889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-18T10:30:44.444 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/phy/ 2023-10-18T10:30:44.444 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/phy/broadcom/ 2023-10-18T10:30:44.444 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2023-10-18T10:30:44.445 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/phy/samsung/ 2023-10-18T10:30:44.445 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2023-10-18T10:30:44.445 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/ 2023-10-18T10:30:44.445 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/intel/ 2023-10-18T10:30:44.445 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 70153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-18T10:30:44.446 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/ 2023-10-18T10:30:44.446 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/ 2023-10-18T10:30:44.446 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 50889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2023-10-18T10:30:44.446 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_ec.ko 2023-10-18T10:30:44.446 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2023-10-18T10:30:44.447 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 25417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2023-10-18T10:30:44.447 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2023-10-18T10:30:44.447 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2023-10-18T10:30:44.447 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2023-10-18T10:30:44.447 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/ 2023-10-18T10:30:44.448 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 67097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-18T10:30:44.448 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 39145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-18T10:30:44.448 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 36913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-18T10:30:44.448 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 59665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-18T10:30:44.448 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 31097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-18T10:30:44.449 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 127569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-18T10:30:44.449 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 36913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-18T10:30:44.449 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 49121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-18T10:30:44.449 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-18T10:30:44.449 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 40265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-18T10:30:44.450 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 22521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-18T10:30:44.450 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 36657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/hdaps.ko 2023-10-18T10:30:44.450 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/ibm_rtl.ko 2023-10-18T10:30:44.450 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel/ 2023-10-18T10:30:44.451 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-18T10:30:44.451 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2023-10-18T10:30:44.451 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 43497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-18T10:30:44.451 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-18T10:30:44.451 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-18T10:30:44.451 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 34801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-18T10:30:44.452 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 42329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-18T10:30:44.452 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-18T10:30:44.452 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-18T10:30:44.452 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-18T10:30:44.452 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-18T10:30:44.453 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-18T10:30:44.453 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 57569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/wmi.ko 2023-10-18T10:30:44.453 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/ 2023-10-18T10:30:44.453 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/ 2023-10-18T10:30:44.453 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 30345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/88pm860x_battery.ko 2023-10-18T10:30:44.454 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 33985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/88pm860x_charger.ko 2023-10-18T10:30:44.454 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 55057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/bq2415x_charger.ko 2023-10-18T10:30:44.454 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 54081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/bq24190_charger.ko 2023-10-18T10:30:44.454 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/bq24735-charger.ko 2023-10-18T10:30:44.454 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/cros_peripheral_charger.ko 2023-10-18T10:30:44.455 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/da9030_battery.ko 2023-10-18T10:30:44.455 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/da9052-battery.ko 2023-10-18T10:30:44.455 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 29281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/ds2760_battery.ko 2023-10-18T10:30:44.455 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 23217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/ds2780_battery.ko 2023-10-18T10:30:44.455 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 23353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/ds2781_battery.ko 2023-10-18T10:30:44.455 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 23649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/ds2782_battery.ko 2023-10-18T10:30:44.456 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/gpio-charger.ko 2023-10-18T10:30:44.456 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/isp1704_charger.ko 2023-10-18T10:30:44.456 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/lp8727_charger.ko 2023-10-18T10:30:44.456 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/max14577_charger.ko 2023-10-18T10:30:44.456 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 25897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/max17040_battery.ko 2023-10-18T10:30:44.457 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 30289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/max17042_battery.ko 2023-10-18T10:30:44.457 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/max8903_charger.ko 2023-10-18T10:30:44.457 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 23729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/max8925_power.ko 2023-10-18T10:30:44.457 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/pcf50633-charger.ko 2023-10-18T10:30:44.457 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 38009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/sbs-battery.ko 2023-10-18T10:30:44.458 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 28385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/test_power.ko 2023-10-18T10:30:44.458 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/tps65090-charger.ko 2023-10-18T10:30:44.458 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/wm831x_backup.ko 2023-10-18T10:30:44.458 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 24521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/wm831x_power.ko 2023-10-18T10:30:44.458 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 22417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/wm8350_power.ko 2023-10-18T10:30:44.459 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/ 2023-10-18T10:30:44.459 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 57729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-18T10:30:44.459 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-18T10:30:44.459 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/ 2023-10-18T10:30:44.459 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/ 2023-10-18T10:30:44.460 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-18T10:30:44.460 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-18T10:30:44.460 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-18T10:30:44.460 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 41873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/pps_core.ko 2023-10-18T10:30:44.460 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ 2023-10-18T10:30:44.461 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 66385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ptp.ko 2023-10-18T10:30:44.461 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-18T10:30:44.461 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/ 2023-10-18T10:30:44.461 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 23497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-88pm80x.ko 2023-10-18T10:30:44.461 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 24521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-88pm860x.ko 2023-10-18T10:30:44.462 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-18T10:30:44.462 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-da9052.ko 2023-10-18T10:30:44.462 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-da9055.ko 2023-10-18T10:30:44.462 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-da9063.ko 2023-10-18T10:30:44.462 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-18T10:30:44.463 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 68577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-18T10:30:44.463 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-18T10:30:44.463 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-18T10:30:44.463 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-18T10:30:44.463 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-18T10:30:44.464 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-18T10:30:44.464 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-18T10:30:44.464 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 23705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-18T10:30:44.464 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-18T10:30:44.464 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 24329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-18T10:30:44.465 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-18T10:30:44.465 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 35969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-18T10:30:44.465 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-lp8788.ko 2023-10-18T10:30:44.465 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 38097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-18T10:30:44.465 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-18T10:30:44.466 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 25945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-18T10:30:44.466 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t86.ko 2023-10-18T10:30:44.466 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-18T10:30:44.466 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max8907.ko 2023-10-18T10:30:44.466 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max8925.ko 2023-10-18T10:30:44.467 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 25809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max8997.ko 2023-10-18T10:30:44.467 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max8998.ko 2023-10-18T10:30:44.467 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-mc13xxx.ko 2023-10-18T10:30:44.467 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-18T10:30:44.467 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-palmas.ko 2023-10-18T10:30:44.468 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 46633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-18T10:30:44.468 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf50633.ko 2023-10-18T10:30:44.468 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-18T10:30:44.468 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-18T10:30:44.468 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 29881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-18T10:30:44.469 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-18T10:30:44.469 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rc5t583.ko 2023-10-18T10:30:44.469 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-18T10:30:44.469 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 33753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-18T10:30:44.469 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 27833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-18T10:30:44.469 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 25257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-18T10:30:44.470 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-18T10:30:44.470 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-s35390a.ko 2023-10-18T10:30:44.470 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-18T10:30:44.470 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-tps6586x.ko 2023-10-18T10:30:44.470 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-tps65910.ko 2023-10-18T10:30:44.471 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-wm831x.ko 2023-10-18T10:30:44.471 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-wm8350.ko 2023-10-18T10:30:44.471 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 27873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-18T10:30:44.471 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ 2023-10-18T10:30:44.471 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 77617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-18T10:30:44.472 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 57665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-sas.ko 2023-10-18T10:30:44.472 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 55169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-18T10:30:44.472 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 135185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/BusLogic.ko 2023-10-18T10:30:44.472 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/a100u2w.ko 2023-10-18T10:30:44.472 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aacraid/ 2023-10-18T10:30:44.473 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 272305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-18T10:30:44.473 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 100617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/advansys.ko 2023-10-18T10:30:44.473 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/ 2023-10-18T10:30:44.473 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 356265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-18T10:30:44.473 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 295801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-18T10:30:44.474 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic94xx/ 2023-10-18T10:30:44.474 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 189185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic94xx/aic94xx.ko 2023-10-18T10:30:44.474 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 24073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/am53c974.ko 2023-10-18T10:30:44.474 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/arcmsr/ 2023-10-18T10:30:44.474 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 106881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-18T10:30:44.475 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 38185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/atp870u.ko 2023-10-18T10:30:44.475 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/be2iscsi/ 2023-10-18T10:30:44.475 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 246985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-18T10:30:44.475 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bfa/ 2023-10-18T10:30:44.475 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 1171945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-18T10:30:44.475 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2fc/ 2023-10-18T10:30:44.476 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 237273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-18T10:30:44.476 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2i/ 2023-10-18T10:30:44.476 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 115369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-18T10:30:44.476 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 40585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ch.ko 2023-10-18T10:30:44.476 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/csiostor/ 2023-10-18T10:30:44.477 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 308921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-18T10:30:44.477 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/ 2023-10-18T10:30:44.477 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb3i/ 2023-10-18T10:30:44.477 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 76145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-18T10:30:44.478 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb4i/ 2023-10-18T10:30:44.478 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 101449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-18T10:30:44.478 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 146689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-18T10:30:44.478 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 75553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/dc395x.ko 2023-10-18T10:30:44.479 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 38137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/dmx3191d.ko 2023-10-18T10:30:44.479 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esas2r/ 2023-10-18T10:30:44.479 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 212425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-18T10:30:44.479 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 60505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esp_scsi.ko 2023-10-18T10:30:44.479 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/ 2023-10-18T10:30:44.479 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 78209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-18T10:30:44.480 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 158505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-18T10:30:44.480 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fdomain.ko 2023-10-18T10:30:44.480 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fnic/ 2023-10-18T10:30:44.480 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 263489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-18T10:30:44.480 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 201817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hpsa.ko 2023-10-18T10:30:44.481 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 51209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hptiop.ko 2023-10-18T10:30:44.481 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 49041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-18T10:30:44.481 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 35337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/imm.ko 2023-10-18T10:30:44.481 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 44057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/initio.ko 2023-10-18T10:30:44.481 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 266537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ipr.ko 2023-10-18T10:30:44.482 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 101881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ips.ko 2023-10-18T10:30:44.482 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/isci/ 2023-10-18T10:30:44.482 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 317153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/isci/isci.ko 2023-10-18T10:30:44.482 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-18T10:30:44.483 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 52961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-18T10:30:44.483 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libfc/ 2023-10-18T10:30:44.483 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 314865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-18T10:30:44.483 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 145585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libiscsi.ko 2023-10-18T10:30:44.483 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 53521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-18T10:30:44.484 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libsas/ 2023-10-18T10:30:44.484 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 216665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-18T10:30:44.484 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/lpfc/ 2023-10-18T10:30:44.484 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 2197249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-18T10:30:44.484 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/ 2023-10-18T10:30:44.485 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 73569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-18T10:30:44.485 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 33529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-18T10:30:44.485 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 310617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-18T10:30:44.485 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 84721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid.ko 2023-10-18T10:30:44.485 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mpt3sas/ 2023-10-18T10:30:44.486 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 659777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-18T10:30:44.486 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvsas/ 2023-10-18T10:30:44.486 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 119561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-18T10:30:44.486 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 78129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvumi.ko 2023-10-18T10:30:44.486 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pcmcia/ 2023-10-18T10:30:44.487 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 81625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2023-10-18T10:30:44.487 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2023-10-18T10:30:44.487 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2023-10-18T10:30:44.487 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 23857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2023-10-18T10:30:44.487 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pm8001/ 2023-10-18T10:30:44.487 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 364105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-18T10:30:44.488 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 123617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pmcraid.ko 2023-10-18T10:30:44.488 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 34513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ppa.ko 2023-10-18T10:30:44.488 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 55361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla1280.ko 2023-10-18T10:30:44.488 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/ 2023-10-18T10:30:44.488 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 1876625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-18T10:30:44.489 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 85289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-18T10:30:44.489 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla4xxx/ 2023-10-18T10:30:44.489 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 559033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-18T10:30:44.489 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qlogicfas408.ko 2023-10-18T10:30:44.490 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/raid_class.ko 2023-10-18T10:30:44.490 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 238825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_debug.ko 2023-10-18T10:30:44.490 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 173937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-18T10:30:44.490 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 279313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-18T10:30:44.490 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 97729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-18T10:30:44.491 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 73881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-18T10:30:44.491 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 44993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-18T10:30:44.491 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ses.ko 2023-10-18T10:30:44.491 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 115889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/st.ko 2023-10-18T10:30:44.491 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 46825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/stex.ko 2023-10-18T10:30:44.492 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/sym53c8xx_2/ 2023-10-18T10:30:44.492 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 162121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-18T10:30:44.492 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 42297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-18T10:30:44.492 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 50321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-18T10:30:44.492 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 40025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/wd719x.ko 2023-10-18T10:30:44.493 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 50025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/xen-scsifront.ko 2023-10-18T10:30:44.493 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ssb/ 2023-10-18T10:30:44.493 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 129441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ssb/ssb.ko 2023-10-18T10:30:44.493 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/ 2023-10-18T10:30:44.493 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/iscsi/ 2023-10-18T10:30:44.494 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 736601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-18T10:30:44.494 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/loopback/ 2023-10-18T10:30:44.494 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 49985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-18T10:30:44.494 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 39713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_file.ko 2023-10-18T10:30:44.495 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 43025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_iblock.ko 2023-10-18T10:30:44.495 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 930321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_mod.ko 2023-10-18T10:30:44.495 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 44305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_pscsi.ko 2023-10-18T10:30:44.495 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 122521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_user.ko 2023-10-18T10:30:44.495 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/tcm_fc/ 2023-10-18T10:30:44.496 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 75593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-18T10:30:44.496 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/ 2023-10-18T10:30:44.496 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/ 2023-10-18T10:30:44.496 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/ 2023-10-18T10:30:44.496 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 23481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-18T10:30:44.497 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 29321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-18T10:30:44.497 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-18T10:30:44.497 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-18T10:30:44.497 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-18T10:30:44.498 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-18T10:30:44.498 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-18T10:30:44.498 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-18T10:30:44.498 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-18T10:30:44.498 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-18T10:30:44.499 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-18T10:30:44.499 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 37065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-18T10:30:44.499 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 30729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_powerclamp.ko 2023-10-18T10:30:44.499 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-18T10:30:44.499 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-18T10:30:44.500 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-18T10:30:44.500 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ 2023-10-18T10:30:44.500 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ipwireless/ 2023-10-18T10:30:44.500 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 78425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-18T10:30:44.500 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 57865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/moxa.ko 2023-10-18T10:30:44.501 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 50953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/mxser.ko 2023-10-18T10:30:44.501 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 108601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/n_gsm.ko 2023-10-18T10:30:44.501 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 31961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/n_hdlc.ko 2023-10-18T10:30:44.501 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 67929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/nozomi.ko 2023-10-18T10:30:44.501 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/ 2023-10-18T10:30:44.502 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250/ 2023-10-18T10:30:44.502 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 35753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250/8250_dw.ko 2023-10-18T10:30:44.502 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 59497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-18T10:30:44.502 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/altera_jtaguart.ko 2023-10-18T10:30:44.502 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 22577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/altera_uart.ko 2023-10-18T10:30:44.503 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-18T10:30:44.503 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 72425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/fsl_lpuart.ko 2023-10-18T10:30:44.503 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/jsm/ 2023-10-18T10:30:44.503 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 103073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-18T10:30:44.503 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 28785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/rp2.ko 2023-10-18T10:30:44.504 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 52817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/sc16is7xx.ko 2023-10-18T10:30:44.504 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/ 2023-10-18T10:30:44.504 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/core/ 2023-10-18T10:30:44.504 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 531265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-18T10:30:44.504 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host/ 2023-10-18T10:30:44.504 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 28481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-18T10:30:44.505 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2023-10-18T10:30:44.505 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/ 2023-10-18T10:30:44.505 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 44513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio.ko 2023-10-18T10:30:44.505 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_aec.ko 2023-10-18T10:30:44.505 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_cif.ko 2023-10-18T10:30:44.506 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_dmem_genirq.ko 2023-10-18T10:30:44.506 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_mf624.ko 2023-10-18T10:30:44.506 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_netx.ko 2023-10-18T10:30:44.506 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-18T10:30:44.506 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_pdrv_genirq.ko 2023-10-18T10:30:44.507 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_sercos3.ko 2023-10-18T10:30:44.507 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/ 2023-10-18T10:30:44.507 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/c67x00/ 2023-10-18T10:30:44.507 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 73297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/c67x00/c67x00.ko 2023-10-18T10:30:44.507 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/chipidea/ 2023-10-18T10:30:44.507 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 144377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/chipidea/ci_hdrc.ko 2023-10-18T10:30:44.508 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2023-10-18T10:30:44.508 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2023-10-18T10:30:44.508 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2023-10-18T10:30:44.508 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/ 2023-10-18T10:30:44.508 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 87089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-18T10:30:44.509 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 51409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-18T10:30:44.509 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 47369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/usblp.ko 2023-10-18T10:30:44.509 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 89857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/usbtmc.ko 2023-10-18T10:30:44.509 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/common/ 2023-10-18T10:30:44.509 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 22953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/common/ulpi.ko 2023-10-18T10:30:44.510 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc2/ 2023-10-18T10:30:44.510 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc2/dwc2_pci.ko 2023-10-18T10:30:44.510 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc3/ 2023-10-18T10:30:44.510 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc3/dwc3-haps.ko 2023-10-18T10:30:44.510 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 30041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc3/dwc3-pci.ko 2023-10-18T10:30:44.511 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 444553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc3/dwc3.ko 2023-10-18T10:30:44.511 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/ 2023-10-18T10:30:44.511 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/ 2023-10-18T10:30:44.511 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 60177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/u_ether.ko 2023-10-18T10:30:44.511 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 51881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/u_serial.ko 2023-10-18T10:30:44.512 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 28841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2023-10-18T10:30:44.512 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 42449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2023-10-18T10:30:44.512 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 27953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2023-10-18T10:30:44.512 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 36513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2023-10-18T10:30:44.513 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 92489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2023-10-18T10:30:44.513 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 51049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2023-10-18T10:30:44.513 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 122081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2023-10-18T10:30:44.513 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 57617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2023-10-18T10:30:44.513 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 22185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2023-10-18T10:30:44.513 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 31265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2023-10-18T10:30:44.514 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 57729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2023-10-18T10:30:44.514 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 85609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2023-10-18T10:30:44.514 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2023-10-18T10:30:44.514 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 58113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2023-10-18T10:30:44.514 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 67337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2023-10-18T10:30:44.515 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/ 2023-10-18T10:30:44.515 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2023-10-18T10:30:44.515 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2023-10-18T10:30:44.515 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2023-10-18T10:30:44.515 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_ether.ko 2023-10-18T10:30:44.516 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 28401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2023-10-18T10:30:44.516 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_hid.ko 2023-10-18T10:30:44.516 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2023-10-18T10:30:44.516 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2023-10-18T10:30:44.516 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 32073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2023-10-18T10:30:44.517 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_printer.ko 2023-10-18T10:30:44.517 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_serial.ko 2023-10-18T10:30:44.517 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 27201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_zero.ko 2023-10-18T10:30:44.517 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 62185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2023-10-18T10:30:44.517 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2023-10-18T10:30:44.518 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 161225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/libcomposite.ko 2023-10-18T10:30:44.518 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/ 2023-10-18T10:30:44.518 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2023-10-18T10:30:44.518 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/bdc/ 2023-10-18T10:30:44.518 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 157681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2023-10-18T10:30:44.519 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 36089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/goku_udc.ko 2023-10-18T10:30:44.519 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 52721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/gr_udc.ko 2023-10-18T10:30:44.519 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 69705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2023-10-18T10:30:44.519 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 56033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/mv_udc.ko 2023-10-18T10:30:44.519 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 76057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/net2272.ko 2023-10-18T10:30:44.520 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 75073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/net2280.ko 2023-10-18T10:30:44.520 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 73169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/pch_udc.ko 2023-10-18T10:30:44.520 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 65809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2023-10-18T10:30:44.520 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 66097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2023-10-18T10:30:44.520 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 81505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2023-10-18T10:30:44.520 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 156073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/udc-core.ko 2023-10-18T10:30:44.521 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/ 2023-10-18T10:30:44.521 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/bcma-hcd.ko 2023-10-18T10:30:44.521 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 80113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/isp116x-hcd.ko 2023-10-18T10:30:44.521 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 87353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/oxu210hp-hcd.ko 2023-10-18T10:30:44.521 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 78873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/r8a66597-hcd.ko 2023-10-18T10:30:44.522 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 51329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-18T10:30:44.522 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/sl811_cs.ko 2023-10-18T10:30:44.522 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/ssb-hcd.ko 2023-10-18T10:30:44.522 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 25945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/xhci-plat-hcd.ko 2023-10-18T10:30:44.522 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/ 2023-10-18T10:30:44.523 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 34585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/mdc800.ko 2023-10-18T10:30:44.523 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/microtek.ko 2023-10-18T10:30:44.523 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ 2023-10-18T10:30:44.523 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 41721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/adutux.ko 2023-10-18T10:30:44.523 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-18T10:30:44.524 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/cypress_cy7c63.ko 2023-10-18T10:30:44.524 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 23049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/cytherm.ko 2023-10-18T10:30:44.524 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ehset.ko 2023-10-18T10:30:44.524 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/emi26.ko 2023-10-18T10:30:44.524 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/emi62.ko 2023-10-18T10:30:44.524 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ezusb.ko 2023-10-18T10:30:44.525 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/idmouse.ko 2023-10-18T10:30:44.525 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 36617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-18T10:30:44.525 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-18T10:30:44.525 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 36529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ldusb.ko 2023-10-18T10:30:44.525 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 36265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-18T10:30:44.526 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 27793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/lvstest.ko 2023-10-18T10:30:44.526 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/sisusbvga/ 2023-10-18T10:30:44.526 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 64841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-18T10:30:44.526 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-18T10:30:44.526 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usb3503.ko 2023-10-18T10:30:44.527 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 25145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usblcd.ko 2023-10-18T10:30:44.527 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 22753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-18T10:30:44.527 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 70873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usbtest.ko 2023-10-18T10:30:44.527 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 41969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/uss720.ko 2023-10-18T10:30:44.527 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 27617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/yurex.ko 2023-10-18T10:30:44.528 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/mon/ 2023-10-18T10:30:44.528 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 72953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/mon/usbmon.ko 2023-10-18T10:30:44.528 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/musb/ 2023-10-18T10:30:44.528 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 301545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/musb/musb_hdrc.ko 2023-10-18T10:30:44.528 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/ 2023-10-18T10:30:44.529 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-18T10:30:44.529 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2023-10-18T10:30:44.529 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-isp1301.ko 2023-10-18T10:30:44.529 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-tahvo.ko 2023-10-18T10:30:44.529 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ 2023-10-18T10:30:44.529 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/aircable.ko 2023-10-18T10:30:44.530 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 33329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ark3116.ko 2023-10-18T10:30:44.530 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-18T10:30:44.530 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 38105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ch341.ko 2023-10-18T10:30:44.530 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 70257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cp210x.ko 2023-10-18T10:30:44.530 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 31057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-18T10:30:44.531 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 58217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-18T10:30:44.531 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 49817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-18T10:30:44.531 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/empeg.ko 2023-10-18T10:30:44.531 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 44337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/f81232.ko 2023-10-18T10:30:44.531 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 175137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-18T10:30:44.532 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 53113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-18T10:30:44.532 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 137081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-18T10:30:44.532 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 124833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/io_ti.ko 2023-10-18T10:30:44.532 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 67769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ipaq.ko 2023-10-18T10:30:44.532 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ipw.ko 2023-10-18T10:30:44.532 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-18T10:30:44.533 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 58233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-18T10:30:44.533 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 107521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/keyspan.ko 2023-10-18T10:30:44.533 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 35617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-18T10:30:44.533 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 25665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-18T10:30:44.533 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 33689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-18T10:30:44.534 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 32953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-18T10:30:44.534 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 22033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/metro-usb.ko 2023-10-18T10:30:44.534 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 66449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mos7720.ko 2023-10-18T10:30:44.534 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 78217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mos7840.ko 2023-10-18T10:30:44.534 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 56337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mxuport.ko 2023-10-18T10:30:44.535 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/navman.ko 2023-10-18T10:30:44.535 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/omninet.ko 2023-10-18T10:30:44.535 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 24633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/opticon.ko 2023-10-18T10:30:44.535 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 190633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/option.ko 2023-10-18T10:30:44.535 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 39073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/oti6858.ko 2023-10-18T10:30:44.536 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 54353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/pl2303.ko 2023-10-18T10:30:44.536 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/qcaux.ko 2023-10-18T10:30:44.536 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 37825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/qcserial.ko 2023-10-18T10:30:44.536 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 34321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/quatech2.ko 2023-10-18T10:30:44.536 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-18T10:30:44.536 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 51625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/sierra.ko 2023-10-18T10:30:44.537 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-18T10:30:44.537 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 29993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ssu100.ko 2023-10-18T10:30:44.537 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-18T10:30:44.537 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 61489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-18T10:30:44.538 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 23729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-18T10:30:44.538 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-18T10:30:44.538 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 33201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-18T10:30:44.538 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 125609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usbserial.ko 2023-10-18T10:30:44.538 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 28753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/visor.ko 2023-10-18T10:30:44.539 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 37409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-18T10:30:44.539 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/wishbone-serial.ko 2023-10-18T10:30:44.539 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-18T10:30:44.539 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ 2023-10-18T10:30:44.539 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 52913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/uas.ko 2023-10-18T10:30:44.539 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-18T10:30:44.540 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-18T10:30:44.540 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 22049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-18T10:30:44.540 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 33497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-18T10:30:44.540 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-18T10:30:44.540 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-18T10:30:44.541 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-18T10:30:44.541 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-18T10:30:44.541 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-18T10:30:44.541 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 28441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-18T10:30:44.541 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 28905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-18T10:30:44.541 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-18T10:30:44.542 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 35337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-18T10:30:44.542 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 179969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-18T10:30:44.542 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/ 2023-10-18T10:30:44.542 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 67481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-18T10:30:44.542 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 72177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-18T10:30:44.543 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 107001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-18T10:30:44.543 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/ 2023-10-18T10:30:44.543 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/ 2023-10-18T10:30:44.543 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 154297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-18T10:30:44.543 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 29537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-18T10:30:44.543 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 108737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/vfio.ko 2023-10-18T10:30:44.544 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 77609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-18T10:30:44.544 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/ 2023-10-18T10:30:44.544 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 113873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost.ko 2023-10-18T10:30:44.544 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-18T10:30:44.544 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 58089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_net.ko 2023-10-18T10:30:44.545 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 89249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-18T10:30:44.545 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/ 2023-10-18T10:30:44.545 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/ 2023-10-18T10:30:44.545 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/88pm860x_bl.ko 2023-10-18T10:30:44.545 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/aat2870_bl.ko 2023-10-18T10:30:44.546 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 23305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/adp5520_bl.ko 2023-10-18T10:30:44.546 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 32609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/adp8860_bl.ko 2023-10-18T10:30:44.546 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 39993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/adp8870_bl.ko 2023-10-18T10:30:44.546 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-18T10:30:44.546 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/as3711_bl.ko 2023-10-18T10:30:44.547 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/bd6107.ko 2023-10-18T10:30:44.547 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/da903x_bl.ko 2023-10-18T10:30:44.547 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/da9052_bl.ko 2023-10-18T10:30:44.547 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/gpio_backlight.ko 2023-10-18T10:30:44.547 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/kb3886_bl.ko 2023-10-18T10:30:44.547 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 22017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lcd.ko 2023-10-18T10:30:44.548 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 23657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lm3533_bl.ko 2023-10-18T10:30:44.548 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lm3639_bl.ko 2023-10-18T10:30:44.548 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lv5207lp.ko 2023-10-18T10:30:44.548 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/max8925_bl.ko 2023-10-18T10:30:44.548 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/pandora_bl.ko 2023-10-18T10:30:44.549 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/pcf50633-backlight.ko 2023-10-18T10:30:44.549 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-18T10:30:44.549 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/wm831x_bl.ko 2023-10-18T10:30:44.549 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/ 2023-10-18T10:30:44.549 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ 2023-10-18T10:30:44.549 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ds2482.ko 2023-10-18T10:30:44.550 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 38985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ds2490.ko 2023-10-18T10:30:44.550 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/matrox_w1.ko 2023-10-18T10:30:44.550 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/w1-gpio.ko 2023-10-18T10:30:44.550 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/ 2023-10-18T10:30:44.551 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-18T10:30:44.551 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 22809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-18T10:30:44.551 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-18T10:30:44.551 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-18T10:30:44.551 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-18T10:30:44.552 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-18T10:30:44.552 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-18T10:30:44.552 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-18T10:30:44.552 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-18T10:30:44.552 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 7385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-18T10:30:44.552 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 56425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-18T10:30:44.553 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 102673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/wire.ko 2023-10-18T10:30:44.553 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ 2023-10-18T10:30:44.553 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/acquirewdt.ko 2023-10-18T10:30:44.553 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/advantechwdt.ko 2023-10-18T10:30:44.553 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-18T10:30:44.554 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 22969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-18T10:30:44.554 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/cpu5wdt.ko 2023-10-18T10:30:44.554 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/da9052_wdt.ko 2023-10-18T10:30:44.554 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/da9055_wdt.ko 2023-10-18T10:30:44.554 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/da9063_wdt.ko 2023-10-18T10:30:44.555 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/dw_wdt.ko 2023-10-18T10:30:44.555 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/eurotechwdt.ko 2023-10-18T10:30:44.555 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 23865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-18T10:30:44.555 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 27401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/hpwdt.ko 2023-10-18T10:30:44.555 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/i6300esb.ko 2023-10-18T10:30:44.556 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-18T10:30:44.556 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-18T10:30:44.556 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-18T10:30:44.556 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ibmasr.ko 2023-10-18T10:30:44.556 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-18T10:30:44.556 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-18T10:30:44.557 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-18T10:30:44.557 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/kempld_wdt.ko 2023-10-18T10:30:44.557 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/machzwd.ko 2023-10-18T10:30:44.557 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/mena21_wdt.ko 2023-10-18T10:30:44.557 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/menf21bmc_wdt.ko 2023-10-18T10:30:44.558 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 24673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/nv_tco.ko 2023-10-18T10:30:44.558 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/of_xilinx_wdt.ko 2023-10-18T10:30:44.558 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 22265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pc87413_wdt.ko 2023-10-18T10:30:44.558 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 38657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-18T10:30:44.558 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 31825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-18T10:30:44.559 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/retu_wdt.ko 2023-10-18T10:30:44.559 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sbc60xxwdt.ko 2023-10-18T10:30:44.559 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sbc_epx_c3.ko 2023-10-18T10:30:44.559 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-18T10:30:44.559 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 23809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sc1200wdt.ko 2023-10-18T10:30:44.559 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 23321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-18T10:30:44.560 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/smsc37b787_wdt.ko 2023-10-18T10:30:44.560 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/softdog.ko 2023-10-18T10:30:44.560 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 22273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-18T10:30:44.560 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/twl4030_wdt.ko 2023-10-18T10:30:44.560 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/via_wdt.ko 2023-10-18T10:30:44.561 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-18T10:30:44.561 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-18T10:30:44.561 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 25409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-18T10:30:44.561 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wafer5823wdt.ko 2023-10-18T10:30:44.561 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 28089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-18T10:30:44.562 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wm831x_wdt.ko 2023-10-18T10:30:44.562 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wm8350_wdt.ko 2023-10-18T10:30:44.562 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-18T10:30:44.562 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/ 2023-10-18T10:30:44.562 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 24729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-evtchn.ko 2023-10-18T10:30:44.563 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-18T10:30:44.563 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 47865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-gntdev.ko 2023-10-18T10:30:44.563 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-pciback/ 2023-10-18T10:30:44.563 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 176441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-18T10:30:44.563 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 40121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-privcmd.ko 2023-10-18T10:30:44.563 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 71249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-scsiback.ko 2023-10-18T10:30:44.564 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xenfs/ 2023-10-18T10:30:44.564 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-18T10:30:44.564 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ 2023-10-18T10:30:44.564 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/9p/ 2023-10-18T10:30:44.564 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 166425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/9p/9p.ko 2023-10-18T10:30:44.565 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/adfs/ 2023-10-18T10:30:44.565 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 61073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/adfs/adfs.ko 2023-10-18T10:30:44.565 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/affs/ 2023-10-18T10:30:44.565 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 163593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/affs/affs.ko 2023-10-18T10:30:44.565 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/afs/ 2023-10-18T10:30:44.565 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 932817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/afs/kafs.ko 2023-10-18T10:30:44.566 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/autofs/ 2023-10-18T10:30:44.566 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 101057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/autofs/autofs4.ko 2023-10-18T10:30:44.566 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/befs/ 2023-10-18T10:30:44.566 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 64545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/befs/befs.ko 2023-10-18T10:30:44.566 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/bfs/ 2023-10-18T10:30:44.567 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 41257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/bfs/bfs.ko 2023-10-18T10:30:44.567 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 34585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/binfmt_misc.ko 2023-10-18T10:30:44.567 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/btrfs/ 2023-10-18T10:30:44.567 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 3402585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/btrfs/btrfs.ko 2023-10-18T10:30:44.567 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cachefiles/ 2023-10-18T10:30:44.568 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 306073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cachefiles/cachefiles.ko 2023-10-18T10:30:44.568 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ceph/ 2023-10-18T10:30:44.568 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 1323873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ceph/ceph.ko 2023-10-18T10:30:44.568 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/coda/ 2023-10-18T10:30:44.568 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 100801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/coda/coda.ko 2023-10-18T10:30:44.569 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/configfs/ 2023-10-18T10:30:44.569 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 108393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/configfs/configfs.ko 2023-10-18T10:30:44.569 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cramfs/ 2023-10-18T10:30:44.569 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 29625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cramfs/cramfs.ko 2023-10-18T10:30:44.569 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/dlm/ 2023-10-18T10:30:44.570 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 642577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/dlm/dlm.ko 2023-10-18T10:30:44.570 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/efs/ 2023-10-18T10:30:44.570 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 36649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/efs/efs.ko 2023-10-18T10:30:44.570 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/f2fs/ 2023-10-18T10:30:44.570 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 1505025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/f2fs/f2fs.ko 2023-10-18T10:30:44.570 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/ 2023-10-18T10:30:44.571 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 27929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/msdos.ko 2023-10-18T10:30:44.571 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/freevxfs/ 2023-10-18T10:30:44.571 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 42545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/freevxfs/freevxfs.ko 2023-10-18T10:30:44.571 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fscache/ 2023-10-18T10:30:44.571 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 226481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fscache/fscache.ko 2023-10-18T10:30:44.572 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse/ 2023-10-18T10:30:44.572 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 24489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse/cuse.ko 2023-10-18T10:30:44.572 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/gfs2/ 2023-10-18T10:30:44.572 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 1231529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/gfs2/gfs2.ko 2023-10-18T10:30:44.572 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfs/ 2023-10-18T10:30:44.572 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 141209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfs/hfs.ko 2023-10-18T10:30:44.573 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfsplus/ 2023-10-18T10:30:44.573 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 222817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfsplus/hfsplus.ko 2023-10-18T10:30:44.573 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hpfs/ 2023-10-18T10:30:44.573 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 192329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hpfs/hpfs.ko 2023-10-18T10:30:44.573 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/isofs/ 2023-10-18T10:30:44.574 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 74065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/isofs/isofs.ko 2023-10-18T10:30:44.574 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/jfs/ 2023-10-18T10:30:44.574 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 419497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/jfs/jfs.ko 2023-10-18T10:30:44.574 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/lockd/ 2023-10-18T10:30:44.574 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 280313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/lockd/lockd.ko 2023-10-18T10:30:44.575 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/minix/ 2023-10-18T10:30:44.575 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 89193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/minix/minix.ko 2023-10-18T10:30:44.575 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/netfs/ 2023-10-18T10:30:44.575 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 124649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/netfs/netfs.ko 2023-10-18T10:30:44.575 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/ 2023-10-18T10:30:44.576 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/blocklayout/ 2023-10-18T10:30:44.576 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 81449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-18T10:30:44.576 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/filelayout/ 2023-10-18T10:30:44.576 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 59449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-18T10:30:44.576 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/flexfilelayout/ 2023-10-18T10:30:44.577 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 117513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-18T10:30:44.577 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 1055649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfs.ko 2023-10-18T10:30:44.577 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 66441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv2.ko 2023-10-18T10:30:44.577 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 120177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv3.ko 2023-10-18T10:30:44.577 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 2041689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv4.ko 2023-10-18T10:30:44.578 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/ 2023-10-18T10:30:44.578 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/grace.ko 2023-10-18T10:30:44.578 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-18T10:30:44.578 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfsd/ 2023-10-18T10:30:44.578 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 1516809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfsd/nfsd.ko 2023-10-18T10:30:44.579 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nilfs2/ 2023-10-18T10:30:44.579 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 592833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nilfs2/nilfs2.ko 2023-10-18T10:30:44.579 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/ 2023-10-18T10:30:44.579 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-celtic.ko 2023-10-18T10:30:44.579 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-centeuro.ko 2023-10-18T10:30:44.579 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-croatian.ko 2023-10-18T10:30:44.580 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-cyrillic.ko 2023-10-18T10:30:44.580 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-gaelic.ko 2023-10-18T10:30:44.580 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-greek.ko 2023-10-18T10:30:44.580 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-iceland.ko 2023-10-18T10:30:44.580 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-inuit.ko 2023-10-18T10:30:44.581 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-roman.ko 2023-10-18T10:30:44.581 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-romanian.ko 2023-10-18T10:30:44.581 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-turkish.ko 2023-10-18T10:30:44.581 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_ascii.ko 2023-10-18T10:30:44.581 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1250.ko 2023-10-18T10:30:44.582 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1251.ko 2023-10-18T10:30:44.582 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1255.ko 2023-10-18T10:30:44.582 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp737.ko 2023-10-18T10:30:44.582 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp775.ko 2023-10-18T10:30:44.582 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp850.ko 2023-10-18T10:30:44.582 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp852.ko 2023-10-18T10:30:44.583 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp855.ko 2023-10-18T10:30:44.583 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp857.ko 2023-10-18T10:30:44.583 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp860.ko 2023-10-18T10:30:44.583 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp861.ko 2023-10-18T10:30:44.583 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp862.ko 2023-10-18T10:30:44.584 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp863.ko 2023-10-18T10:30:44.584 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp864.ko 2023-10-18T10:30:44.584 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp865.ko 2023-10-18T10:30:44.584 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp866.ko 2023-10-18T10:30:44.584 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp869.ko 2023-10-18T10:30:44.585 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp874.ko 2023-10-18T10:30:44.585 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 94137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp932.ko 2023-10-18T10:30:44.585 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 142673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp936.ko 2023-10-18T10:30:44.585 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 165745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp949.ko 2023-10-18T10:30:44.585 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 118121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp950.ko 2023-10-18T10:30:44.585 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_euc-jp.ko 2023-10-18T10:30:44.586 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-18T10:30:44.586 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-18T10:30:44.586 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-18T10:30:44.586 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-18T10:30:44.586 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-18T10:30:44.587 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-18T10:30:44.587 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-18T10:30:44.587 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-18T10:30:44.587 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-18T10:30:44.587 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-18T10:30:44.587 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-18T10:30:44.588 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-r.ko 2023-10-18T10:30:44.588 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-18T10:30:44.588 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-u.ko 2023-10-18T10:30:44.588 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 7569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_ucs2_utils.ko 2023-10-18T10:30:44.588 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_utf8.ko 2023-10-18T10:30:44.589 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ntfs/ 2023-10-18T10:30:44.589 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 219417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ntfs/ntfs.ko 2023-10-18T10:30:44.589 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ 2023-10-18T10:30:44.589 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/cluster/ 2023-10-18T10:30:44.589 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 244409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-18T10:30:44.590 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlm/ 2023-10-18T10:30:44.590 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 440737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-18T10:30:44.590 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlmfs/ 2023-10-18T10:30:44.590 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 54049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-18T10:30:44.590 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 2592433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2.ko 2023-10-18T10:30:44.591 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-18T10:30:44.591 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 31729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-18T10:30:44.591 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 35897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-18T10:30:44.591 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/omfs/ 2023-10-18T10:30:44.591 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 53433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/omfs/omfs.ko 2023-10-18T10:30:44.591 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/overlayfs/ 2023-10-18T10:30:44.592 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 352153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/overlayfs/overlay.ko 2023-10-18T10:30:44.592 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/pstore/ 2023-10-18T10:30:44.592 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 54809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/pstore/ramoops.ko 2023-10-18T10:30:44.592 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/qnx4/ 2023-10-18T10:30:44.592 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/qnx4/qnx4.ko 2023-10-18T10:30:44.593 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/qnx6/ 2023-10-18T10:30:44.593 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 46689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/qnx6/qnx6.ko 2023-10-18T10:30:44.593 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/quota/ 2023-10-18T10:30:44.593 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 30881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/quota/quota_tree.ko 2023-10-18T10:30:44.593 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/quota/quota_v1.ko 2023-10-18T10:30:44.594 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 22017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/quota/quota_v2.ko 2023-10-18T10:30:44.594 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/reiserfs/ 2023-10-18T10:30:44.594 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 532673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/reiserfs/reiserfs.ko 2023-10-18T10:30:44.594 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/romfs/ 2023-10-18T10:30:44.594 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 28641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/romfs/romfs.ko 2023-10-18T10:30:44.595 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/ 2023-10-18T10:30:44.595 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/client/ 2023-10-18T10:30:44.595 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 2668121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/client/cifs.ko 2023-10-18T10:30:44.595 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/ 2023-10-18T10:30:44.595 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 7161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-18T10:30:44.595 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/cifs_md4.ko 2023-10-18T10:30:44.596 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/squashfs/ 2023-10-18T10:30:44.596 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 134641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/squashfs/squashfs.ko 2023-10-18T10:30:44.596 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/sysv/ 2023-10-18T10:30:44.596 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 90721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/sysv/sysv.ko 2023-10-18T10:30:44.596 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/udf/ 2023-10-18T10:30:44.597 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 247737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/udf/udf.ko 2023-10-18T10:30:44.597 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ufs/ 2023-10-18T10:30:44.597 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 162497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ufs/ufs.ko 2023-10-18T10:30:44.597 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/xfs/ 2023-10-18T10:30:44.597 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 4010729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/xfs/xfs.ko 2023-10-18T10:30:44.598 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/ 2023-10-18T10:30:44.598 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/locking/ 2023-10-18T10:30:44.598 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 75489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/locking/locktorture.ko 2023-10-18T10:30:44.598 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/time/ 2023-10-18T10:30:44.598 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/time/test_udelay.ko 2023-10-18T10:30:44.598 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 60553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/torture.ko 2023-10-18T10:30:44.599 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ 2023-10-18T10:30:44.599 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 6833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc-itu-t.ko 2023-10-18T10:30:44.599 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 6529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc7.ko 2023-10-18T10:30:44.599 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 7025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc8.ko 2023-10-18T10:30:44.599 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/ 2023-10-18T10:30:44.600 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 7097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/libarc4.ko 2023-10-18T10:30:44.600 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 37081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/libdes.ko 2023-10-18T10:30:44.600 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/interval_tree_test.ko 2023-10-18T10:30:44.600 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/libcrc32c.ko 2023-10-18T10:30:44.600 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 24953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lru_cache.ko 2023-10-18T10:30:44.601 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/ 2023-10-18T10:30:44.601 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 33801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/lz4_compress.ko 2023-10-18T10:30:44.601 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 22633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-18T10:30:44.601 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/math/ 2023-10-18T10:30:44.601 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 6617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/math/cordic.ko 2023-10-18T10:30:44.601 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/memory-notifier-error-inject.ko 2023-10-18T10:30:44.602 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/notifier-error-inject.ko 2023-10-18T10:30:44.602 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/percpu_test.ko 2023-10-18T10:30:44.602 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/pm-notifier-error-inject.ko 2023-10-18T10:30:44.602 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/raid6/ 2023-10-18T10:30:44.602 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 143433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/raid6/raid6_pq.ko 2023-10-18T10:30:44.603 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/rbtree_test.ko 2023-10-18T10:30:44.603 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/reed_solomon/ 2023-10-18T10:30:44.603 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-18T10:30:44.603 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 38841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test-kstrtox.ko 2023-10-18T10:30:44.603 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test-string_helpers.ko 2023-10-18T10:30:44.604 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 4760425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test_bpf.ko 2023-10-18T10:30:44.604 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 49089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test_firmware.ko 2023-10-18T10:30:44.604 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 6369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test_module.ko 2023-10-18T10:30:44.604 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test_user_copy.ko 2023-10-18T10:30:44.604 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_bm.ko 2023-10-18T10:30:44.605 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_fsm.ko 2023-10-18T10:30:44.605 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_kmp.ko 2023-10-18T10:30:44.605 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/xz/ 2023-10-18T10:30:44.605 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/xz/xz_dec_test.ko 2023-10-18T10:30:44.605 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/mm/ 2023-10-18T10:30:44.606 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/mm/hwpoison-inject.ko 2023-10-18T10:30:44.606 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ 2023-10-18T10:30:44.606 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/ 2023-10-18T10:30:44.606 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 55737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/6lowpan.ko 2023-10-18T10:30:44.606 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 7049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_dest.ko 2023-10-18T10:30:44.606 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 7065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-18T10:30:44.607 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 7049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_hop.ko 2023-10-18T10:30:44.607 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 6985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-18T10:30:44.607 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 7065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-18T10:30:44.607 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 7065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_routing.ko 2023-10-18T10:30:44.607 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_udp.ko 2023-10-18T10:30:44.608 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/ 2023-10-18T10:30:44.608 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 30385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/garp.ko 2023-10-18T10:30:44.608 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 34809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/mrp.ko 2023-10-18T10:30:44.608 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/p8022.ko 2023-10-18T10:30:44.608 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/psnap.ko 2023-10-18T10:30:44.608 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/stp.ko 2023-10-18T10:30:44.609 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/8021q/ 2023-10-18T10:30:44.609 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 81737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/8021q/8021q.ko 2023-10-18T10:30:44.609 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/ 2023-10-18T10:30:44.609 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 198337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet.ko 2023-10-18T10:30:44.609 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 35977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_fd.ko 2023-10-18T10:30:44.610 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_rdma.ko 2023-10-18T10:30:44.610 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 32337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_virtio.ko 2023-10-18T10:30:44.610 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/batman-adv/ 2023-10-18T10:30:44.610 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 649297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/batman-adv/batman-adv.ko 2023-10-18T10:30:44.610 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/ 2023-10-18T10:30:44.611 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 63809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/br_netfilter.ko 2023-10-18T10:30:44.611 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 777689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/bridge.ko 2023-10-18T10:30:44.611 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ 2023-10-18T10:30:44.611 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 7657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-18T10:30:44.611 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-18T10:30:44.612 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-18T10:30:44.612 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-18T10:30:44.612 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-18T10:30:44.612 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-18T10:30:44.612 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-18T10:30:44.613 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-18T10:30:44.613 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-18T10:30:44.613 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 7585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-18T10:30:44.613 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 7593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-18T10:30:44.613 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-18T10:30:44.614 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 7537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-18T10:30:44.614 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-18T10:30:44.614 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-18T10:30:44.614 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-18T10:30:44.614 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-18T10:30:44.615 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-18T10:30:44.615 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-18T10:30:44.615 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-18T10:30:44.615 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 55929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-18T10:30:44.615 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-18T10:30:44.615 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-18T10:30:44.616 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ceph/ 2023-10-18T10:30:44.616 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 1047017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ceph/libceph.ko 2023-10-18T10:30:44.616 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core/ 2023-10-18T10:30:44.616 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 125697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core/pktgen.ko 2023-10-18T10:30:44.616 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/ 2023-10-18T10:30:44.617 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 194209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp.ko 2023-10-18T10:30:44.617 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_diag.ko 2023-10-18T10:30:44.617 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 57881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_ipv4.ko 2023-10-18T10:30:44.617 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 60337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_ipv6.ko 2023-10-18T10:30:44.617 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/ 2023-10-18T10:30:44.618 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 390473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/dsa_core.ko 2023-10-18T10:30:44.618 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_brcm.ko 2023-10-18T10:30:44.618 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_dsa.ko 2023-10-18T10:30:44.618 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 7721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_none.ko 2023-10-18T10:30:44.618 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_trailer.ko 2023-10-18T10:30:44.619 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/hsr/ 2023-10-18T10:30:44.619 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 99729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/hsr/hsr.ko 2023-10-18T10:30:44.619 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ 2023-10-18T10:30:44.619 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/6lowpan/ 2023-10-18T10:30:44.619 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 45233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-18T10:30:44.619 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 254625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ieee802154.ko 2023-10-18T10:30:44.620 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 47289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-18T10:30:44.620 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ 2023-10-18T10:30:44.620 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 24729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ah4.ko 2023-10-18T10:30:44.620 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 38905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/esp4.ko 2023-10-18T10:30:44.620 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 48401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/fou.ko 2023-10-18T10:30:44.621 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/gre.ko 2023-10-18T10:30:44.621 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 48657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/inet_diag.ko 2023-10-18T10:30:44.621 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 60633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_gre.ko 2023-10-18T10:30:44.621 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 61225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_tunnel.ko 2023-10-18T10:30:44.621 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 34505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_vti.ko 2023-10-18T10:30:44.621 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ipcomp.ko 2023-10-18T10:30:44.622 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 28841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ipip.ko 2023-10-18T10:30:44.622 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ 2023-10-18T10:30:44.622 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 37641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-18T10:30:44.622 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-18T10:30:44.622 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-18T10:30:44.623 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 41673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ip_tables.ko 2023-10-18T10:30:44.623 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-18T10:30:44.623 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2023-10-18T10:30:44.623 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-18T10:30:44.623 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-18T10:30:44.624 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-18T10:30:44.624 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_filter.ko 2023-10-18T10:30:44.624 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-18T10:30:44.624 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-18T10:30:44.624 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-18T10:30:44.625 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-18T10:30:44.625 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-18T10:30:44.625 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-18T10:30:44.625 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 27625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-18T10:30:44.625 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-18T10:30:44.625 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-18T10:30:44.626 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2023-10-18T10:30:44.626 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-18T10:30:44.626 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-18T10:30:44.626 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-18T10:30:44.626 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-18T10:30:44.627 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-18T10:30:44.627 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_bic.ko 2023-10-18T10:30:44.627 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-18T10:30:44.627 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_diag.ko 2023-10-18T10:30:44.627 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-18T10:30:44.627 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_htcp.ko 2023-10-18T10:30:44.628 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_hybla.ko 2023-10-18T10:30:44.628 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_illinois.ko 2023-10-18T10:30:44.628 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_lp.ko 2023-10-18T10:30:44.628 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_scalable.ko 2023-10-18T10:30:44.628 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_vegas.ko 2023-10-18T10:30:44.629 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_veno.ko 2023-10-18T10:30:44.629 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_westwood.ko 2023-10-18T10:30:44.629 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_yeah.ko 2023-10-18T10:30:44.629 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 22601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tunnel4.ko 2023-10-18T10:30:44.629 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/udp_diag.ko 2023-10-18T10:30:44.629 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 41665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/udp_tunnel.ko 2023-10-18T10:30:44.630 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-18T10:30:44.630 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ 2023-10-18T10:30:44.630 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ah6.ko 2023-10-18T10:30:44.630 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 39537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/esp6.ko 2023-10-18T10:30:44.630 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/fou6.ko 2023-10-18T10:30:44.631 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 80897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_gre.ko 2023-10-18T10:30:44.631 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 93577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-18T10:30:44.631 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-18T10:30:44.631 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 53273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_vti.ko 2023-10-18T10:30:44.631 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ipcomp6.ko 2023-10-18T10:30:44.632 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/mip6.ko 2023-10-18T10:30:44.632 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ 2023-10-18T10:30:44.632 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 42169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-18T10:30:44.632 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2023-10-18T10:30:44.632 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-18T10:30:44.632 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-18T10:30:44.633 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-18T10:30:44.633 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 7801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-18T10:30:44.633 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-18T10:30:44.633 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-18T10:30:44.633 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-18T10:30:44.634 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-18T10:30:44.634 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-18T10:30:44.634 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-18T10:30:44.634 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-18T10:30:44.634 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-18T10:30:44.635 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-18T10:30:44.635 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-18T10:30:44.635 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-18T10:30:44.635 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 39113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-18T10:30:44.635 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-18T10:30:44.635 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 25945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-18T10:30:44.636 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-18T10:30:44.636 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-18T10:30:44.636 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-18T10:30:44.636 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-18T10:30:44.636 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-18T10:30:44.637 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 74073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/sit.ko 2023-10-18T10:30:44.637 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 24193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/tunnel6.ko 2023-10-18T10:30:44.637 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 23297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-18T10:30:44.637 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/key/ 2023-10-18T10:30:44.637 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 82833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/key/af_key.ko 2023-10-18T10:30:44.637 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/ 2023-10-18T10:30:44.638 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 125281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_core.ko 2023-10-18T10:30:44.638 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-18T10:30:44.638 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 25953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_eth.ko 2023-10-18T10:30:44.638 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 35569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ip.ko 2023-10-18T10:30:44.638 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 36713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-18T10:30:44.639 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 33001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-18T10:30:44.639 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 53241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-18T10:30:44.639 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/lapb/ 2023-10-18T10:30:44.639 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 47977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/lapb/lapb.ko 2023-10-18T10:30:44.639 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc/ 2023-10-18T10:30:44.640 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 23193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc/llc.ko 2023-10-18T10:30:44.641 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 267777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc/llc2.ko 2023-10-18T10:30:44.642 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mac802154/ 2023-10-18T10:30:44.642 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 290417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mac802154/mac802154.ko 2023-10-18T10:30:44.642 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mpls/ 2023-10-18T10:30:44.642 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mpls/mpls_gso.ko 2023-10-18T10:30:44.642 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ 2023-10-18T10:30:44.643 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ 2023-10-18T10:30:44.643 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 119585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-18T10:30:44.643 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 28353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-18T10:30:44.643 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-18T10:30:44.643 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-18T10:30:44.644 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 100297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-18T10:30:44.644 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 101441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-18T10:30:44.644 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 102817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-18T10:30:44.644 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 103481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-18T10:30:44.644 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 112849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-18T10:30:44.645 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 60329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-18T10:30:44.645 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 109593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-18T10:30:44.645 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 113129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-18T10:30:44.645 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 112585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-18T10:30:44.645 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 111585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-18T10:30:44.646 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 114465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-18T10:30:44.646 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 30025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-18T10:30:44.646 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ 2023-10-18T10:30:44.646 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 411729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-18T10:30:44.646 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-18T10:30:44.647 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 7625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-18T10:30:44.647 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-18T10:30:44.647 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-18T10:30:44.647 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-18T10:30:44.647 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 7689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-18T10:30:44.648 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 7689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-18T10:30:44.648 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-18T10:30:44.648 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-18T10:30:44.648 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 7697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-18T10:30:44.648 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-18T10:30:44.648 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 7697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-18T10:30:44.649 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-18T10:30:44.649 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 49257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conncount.ko 2023-10-18T10:30:44.649 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 408257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack.ko 2023-10-18T10:30:44.649 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-18T10:30:44.649 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-18T10:30:44.650 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 33761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-18T10:30:44.650 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 118145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-18T10:30:44.650 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 22185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-18T10:30:44.650 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-18T10:30:44.650 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 107297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-18T10:30:44.651 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 33633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-18T10:30:44.651 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-18T10:30:44.651 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 66641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-18T10:30:44.651 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-18T10:30:44.651 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-18T10:30:44.652 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-18T10:30:44.652 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 89473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_flow_table.ko 2023-10-18T10:30:44.652 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-18T10:30:44.652 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 37129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-18T10:30:44.652 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 95417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat.ko 2023-10-18T10:30:44.653 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-18T10:30:44.653 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-18T10:30:44.653 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-18T10:30:44.653 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 28929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-18T10:30:44.653 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-18T10:30:44.653 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 44065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-18T10:30:44.654 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 615313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_tables.ko 2023-10-18T10:30:44.654 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 42561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink.ko 2023-10-18T10:30:44.654 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 30233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-18T10:30:44.654 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 30569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_cttimeout.ko 2023-10-18T10:30:44.654 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 39785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-18T10:30:44.655 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 22921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-18T10:30:44.655 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 49377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-18T10:30:44.655 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-18T10:30:44.655 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 33209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_compat.ko 2023-10-18T10:30:44.655 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 40721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_ct.ko 2023-10-18T10:30:44.656 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-18T10:30:44.656 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_fib.ko 2023-10-18T10:30:44.656 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-18T10:30:44.656 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_hash.ko 2023-10-18T10:30:44.656 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_limit.ko 2023-10-18T10:30:44.657 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_log.ko 2023-10-18T10:30:44.657 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_masq.ko 2023-10-18T10:30:44.657 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_nat.ko 2023-10-18T10:30:44.657 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_numgen.ko 2023-10-18T10:30:44.657 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_osf.ko 2023-10-18T10:30:44.658 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_queue.ko 2023-10-18T10:30:44.658 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_redir.ko 2023-10-18T10:30:44.658 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_reject.ko 2023-10-18T10:30:44.658 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-18T10:30:44.658 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_socket.ko 2023-10-18T10:30:44.659 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_synproxy.ko 2023-10-18T10:30:44.659 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_tproxy.ko 2023-10-18T10:30:44.659 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 89313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/x_tables.ko 2023-10-18T10:30:44.659 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-18T10:30:44.659 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-18T10:30:44.660 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 7729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-18T10:30:44.660 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-18T10:30:44.660 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 18169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CT.ko 2023-10-18T10:30:44.660 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_DSCP.ko 2023-10-18T10:30:44.660 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_HL.ko 2023-10-18T10:30:44.661 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_HMARK.ko 2023-10-18T10:30:44.661 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 38873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-18T10:30:44.661 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_LED.ko 2023-10-18T10:30:44.661 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_LOG.ko 2023-10-18T10:30:44.661 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-18T10:30:44.662 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-18T10:30:44.662 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-18T10:30:44.662 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-18T10:30:44.662 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-18T10:30:44.662 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-18T10:30:44.663 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 14337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-18T10:30:44.663 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-18T10:30:44.663 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-18T10:30:44.663 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TEE.ko 2023-10-18T10:30:44.663 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-18T10:30:44.663 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TRACE.ko 2023-10-18T10:30:44.664 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_addrtype.ko 2023-10-18T10:30:44.664 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_bpf.ko 2023-10-18T10:30:44.664 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cgroup.ko 2023-10-18T10:30:44.664 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cluster.ko 2023-10-18T10:30:44.664 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 7505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_comment.ko 2023-10-18T10:30:44.665 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connbytes.ko 2023-10-18T10:30:44.665 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connlabel.ko 2023-10-18T10:30:44.665 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connlimit.ko 2023-10-18T10:30:44.665 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connmark.ko 2023-10-18T10:30:44.665 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_conntrack.ko 2023-10-18T10:30:44.666 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 7817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cpu.ko 2023-10-18T10:30:44.666 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_dccp.ko 2023-10-18T10:30:44.666 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 7865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_devgroup.ko 2023-10-18T10:30:44.666 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_dscp.ko 2023-10-18T10:30:44.666 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ecn.ko 2023-10-18T10:30:44.667 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_esp.ko 2023-10-18T10:30:44.667 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 43289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-18T10:30:44.667 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_helper.ko 2023-10-18T10:30:44.667 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 7865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_hl.ko 2023-10-18T10:30:44.667 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-18T10:30:44.667 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_iprange.ko 2023-10-18T10:30:44.668 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ipvs.ko 2023-10-18T10:30:44.668 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_l2tp.ko 2023-10-18T10:30:44.668 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_length.ko 2023-10-18T10:30:44.668 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_limit.ko 2023-10-18T10:30:44.668 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 7545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_mac.ko 2023-10-18T10:30:44.669 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_mark.ko 2023-10-18T10:30:44.669 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_multiport.ko 2023-10-18T10:30:44.669 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_nat.ko 2023-10-18T10:30:44.669 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_nfacct.ko 2023-10-18T10:30:44.669 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_osf.ko 2023-10-18T10:30:44.670 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_owner.ko 2023-10-18T10:30:44.670 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_physdev.ko 2023-10-18T10:30:44.670 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 7633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_pkttype.ko 2023-10-18T10:30:44.670 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_policy.ko 2023-10-18T10:30:44.670 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_quota.ko 2023-10-18T10:30:44.671 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_rateest.ko 2023-10-18T10:30:44.671 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_realm.ko 2023-10-18T10:30:44.671 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 36497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_recent.ko 2023-10-18T10:30:44.671 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_sctp.ko 2023-10-18T10:30:44.671 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 30089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_set.ko 2023-10-18T10:30:44.671 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 16089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_socket.ko 2023-10-18T10:30:44.672 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_state.ko 2023-10-18T10:30:44.672 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_statistic.ko 2023-10-18T10:30:44.672 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_string.ko 2023-10-18T10:30:44.672 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 8553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-18T10:30:44.672 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_tcpudp.ko 2023-10-18T10:30:44.673 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_time.ko 2023-10-18T10:30:44.673 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_u32.ko 2023-10-18T10:30:44.673 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netlink/ 2023-10-18T10:30:44.673 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netlink/netlink_diag.ko 2023-10-18T10:30:44.673 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/nsh/ 2023-10-18T10:30:44.674 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 12425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/nsh/nsh.ko 2023-10-18T10:30:44.674 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/ 2023-10-18T10:30:44.674 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 380033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/openvswitch.ko 2023-10-18T10:30:44.674 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-geneve.ko 2023-10-18T10:30:44.674 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-gre.ko 2023-10-18T10:30:44.675 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 11793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-18T10:30:44.675 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/packet/ 2023-10-18T10:30:44.675 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/packet/af_packet_diag.ko 2023-10-18T10:30:44.675 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/phonet/ 2023-10-18T10:30:44.675 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 110201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/phonet/phonet.ko 2023-10-18T10:30:44.675 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 54697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/phonet/pn_pep.ko 2023-10-18T10:30:44.676 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/ 2023-10-18T10:30:44.676 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 251481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds.ko 2023-10-18T10:30:44.676 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 164937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds_rdma.ko 2023-10-18T10:30:44.676 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 67377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds_tcp.ko 2023-10-18T10:30:44.676 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rxrpc/ 2023-10-18T10:30:44.677 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 819657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rxrpc/rxrpc.ko 2023-10-18T10:30:44.677 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/ 2023-10-18T10:30:44.677 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 32241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_csum.ko 2023-10-18T10:30:44.677 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 23577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_gact.ko 2023-10-18T10:30:44.677 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_ipt.ko 2023-10-18T10:30:44.677 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 34097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_mirred.ko 2023-10-18T10:30:44.678 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 23401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_nat.ko 2023-10-18T10:30:44.678 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 43449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_pedit.ko 2023-10-18T10:30:44.678 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 31961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_police.ko 2023-10-18T10:30:44.678 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_simple.ko 2023-10-18T10:30:44.678 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 35297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_skbedit.ko 2023-10-18T10:30:44.679 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 36665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_vlan.ko 2023-10-18T10:30:44.679 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_basic.ko 2023-10-18T10:30:44.679 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 33961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_bpf.ko 2023-10-18T10:30:44.679 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_cgroup.ko 2023-10-18T10:30:44.679 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 31401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_flow.ko 2023-10-18T10:30:44.680 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 31769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_fw.ko 2023-10-18T10:30:44.680 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 38489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_route.ko 2023-10-18T10:30:44.680 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 61409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_u32.ko 2023-10-18T10:30:44.680 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 7441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_cmp.ko 2023-10-18T10:30:44.680 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 13409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_ipset.ko 2023-10-18T10:30:44.680 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 33289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_meta.ko 2023-10-18T10:30:44.681 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 7897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_nbyte.ko 2023-10-18T10:30:44.681 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_text.ko 2023-10-18T10:30:44.681 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 7249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_u32.ko 2023-10-18T10:30:44.681 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 21649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_choke.ko 2023-10-18T10:30:44.681 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_codel.ko 2023-10-18T10:30:44.682 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 34633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_drr.ko 2023-10-18T10:30:44.682 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 33609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_fq.ko 2023-10-18T10:30:44.682 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 31617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_fq_codel.ko 2023-10-18T10:30:44.682 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 35433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_gred.ko 2023-10-18T10:30:44.682 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 49601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_hfsc.ko 2023-10-18T10:30:44.683 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 22089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_hhf.ko 2023-10-18T10:30:44.683 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 66393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_htb.ko 2023-10-18T10:30:44.683 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_ingress.ko 2023-10-18T10:30:44.683 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 30273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_mqprio.ko 2023-10-18T10:30:44.683 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-18T10:30:44.683 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_multiq.ko 2023-10-18T10:30:44.684 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 36137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_netem.ko 2023-10-18T10:30:44.684 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 27497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_pie.ko 2023-10-18T10:30:44.684 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_plug.ko 2023-10-18T10:30:44.684 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 24729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_prio.ko 2023-10-18T10:30:44.684 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 48793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_qfq.ko 2023-10-18T10:30:44.685 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 32905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_red.ko 2023-10-18T10:30:44.685 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 26881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_sfb.ko 2023-10-18T10:30:44.685 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 31353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_sfq.ko 2023-10-18T10:30:44.685 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 28897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_tbf.ko 2023-10-18T10:30:44.686 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 28105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_teql.ko 2023-10-18T10:30:44.686 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/ 2023-10-18T10:30:44.686 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 929745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/sctp.ko 2023-10-18T10:30:44.686 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 19033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/sctp_diag.ko 2023-10-18T10:30:44.686 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/ 2023-10-18T10:30:44.687 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/ 2023-10-18T10:30:44.687 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 336529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-18T10:30:44.687 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 54737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-18T10:30:44.687 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 1520161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/sunrpc.ko 2023-10-18T10:30:44.687 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/xprtrdma/ 2023-10-18T10:30:44.687 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 707721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-18T10:30:44.688 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/ 2023-10-18T10:30:44.688 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/diag.ko 2023-10-18T10:30:44.688 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 862641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/tipc.ko 2023-10-18T10:30:44.688 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/unix/ 2023-10-18T10:30:44.688 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/unix/unix_diag.ko 2023-10-18T10:30:44.689 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/ 2023-10-18T10:30:44.689 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 85337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-18T10:30:44.689 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 68049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-18T10:30:44.689 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 96193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock.ko 2023-10-18T10:30:44.689 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 10705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-18T10:30:44.690 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 17521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-18T10:30:44.690 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/x25/ 2023-10-18T10:30:44.690 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 125177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/x25/x25.ko 2023-10-18T10:30:44.690 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/ 2023-10-18T10:30:44.690 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 22657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/xfrm_algo.ko 2023-10-18T10:30:44.691 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 20649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-18T10:30:44.691 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 102257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/xfrm_user.ko 2023-10-18T10:30:44.691 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/ 2023-10-18T10:30:44.691 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/lib/ 2023-10-18T10:30:44.691 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 15537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/lib/irqbypass.ko 2023-10-18T10:30:44.691 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 639863 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.alias 2023-10-18T10:30:44.692 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 623919 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.alias.bin 2023-10-18T10:30:44.692 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 7663 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.builtin 2023-10-18T10:30:44.692 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 23428 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.alias.bin 2023-10-18T10:30:44.692 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 9821 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.bin 2023-10-18T10:30:44.692 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 70004 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.modinfo 2023-10-18T10:30:44.693 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 146313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.dep 2023-10-18T10:30:44.693 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 232548 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.dep.bin 2023-10-18T10:30:44.693 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 210 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.devname 2023-10-18T10:30:44.693 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 72606 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.order 2023-10-18T10:30:44.693 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 879 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.softdep 2023-10-18T10:30:44.694 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 254438 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.symbols 2023-10-18T10:30:44.694 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 309363 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.symbols.bin 2023-10-18T10:30:44.694 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./usr/ 2023-10-18T10:30:44.694 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./usr/share/ 2023-10-18T10:30:44.694 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./usr/share/doc/ 2023-10-18T10:30:44.694 INFO:teuthology.orchestra.run.smithi089.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./usr/share/doc/linux-image-6.6.0-rc5-g47672119149b/ 2023-10-18T10:30:44.695 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 175 2023-10-10 19:56 ./usr/share/doc/linux-image-6.6.0-rc5-g47672119149b/changelog.Debian.gz 2023-10-18T10:30:44.695 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r-- root/root 691 2023-10-10 19:56 ./usr/share/doc/linux-image-6.6.0-rc5-g47672119149b/copyright 2023-10-18T10:30:44.696 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-g47672119149b 2023-10-18T10:30:44.696 INFO:teuthology.task.kernel:searching for kernel 6.6.0-rc5-g47672119149b 2023-10-18T10:30:44.697 DEBUG:teuthology.orchestra.run.smithi089:> egrep '(submenu|menuentry.*6.6.0-rc5-g47672119149b).*{' /boot/grub/grub.cfg 2023-10-18T10:30:44.698 INFO:teuthology.orchestra.run.smithi154.stderr:Sourcing file `/etc/default/grub' 2023-10-18T10:30:44.698 INFO:teuthology.orchestra.run.smithi154.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-18T10:30:44.698 INFO:teuthology.orchestra.run.smithi154.stderr:Generating grub configuration file ... 2023-10-18T10:30:44.705 INFO:teuthology.orchestra.run.smithi089.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-10-18T10:30:44.705 INFO:teuthology.orchestra.run.smithi089.stdout: menuentry 'Ubuntu, with Linux 6.6.0-rc5-g47672119149b' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-rc5-g47672119149b-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-10-18T10:30:44.705 INFO:teuthology.orchestra.run.smithi089.stdout: menuentry 'Ubuntu, with Linux 6.6.0-rc5-g47672119149b (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-rc5-g47672119149b-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-10-18T10:30:44.706 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2023-10-18T10:30:44.706 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.6.0-rc5-g47672119149b 2023-10-18T10:30:44.707 DEBUG:teuthology.orchestra.run.smithi089:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2023-10-18T10:30:44.748 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi089.front.sepia.ceph.com to complete... 2023-10-18T10:30:44.878 INFO:teuthology.orchestra.run.smithi154.stderr:Found linux image: /boot/vmlinuz-6.6.0-rc5-g47672119149b 2023-10-18T10:30:44.897 INFO:teuthology.orchestra.run.smithi154.stderr:Found initrd image: /boot/initrd.img-6.6.0-rc5-g47672119149b 2023-10-18T10:30:45.148 INFO:teuthology.orchestra.run.smithi154.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-10-18T10:30:45.160 INFO:teuthology.orchestra.run.smithi154.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-10-18T10:30:45.203 INFO:teuthology.orchestra.run.smithi089.stderr:Sourcing file `/etc/default/grub' 2023-10-18T10:30:45.204 INFO:teuthology.orchestra.run.smithi089.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-18T10:30:45.225 INFO:teuthology.orchestra.run.smithi154.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-10-18T10:30:45.230 INFO:teuthology.orchestra.run.smithi154.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-10-18T10:30:45.232 INFO:teuthology.orchestra.run.smithi089.stderr:Generating grub configuration file ... 2023-10-18T10:30:45.396 INFO:teuthology.orchestra.run.smithi154.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-10-18T10:30:45.396 INFO:teuthology.orchestra.run.smithi154.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-10-18T10:30:45.397 INFO:teuthology.orchestra.run.smithi154.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-10-18T10:30:45.405 INFO:teuthology.orchestra.run.smithi089.stderr:Found linux image: /boot/vmlinuz-6.6.0-rc5-g47672119149b 2023-10-18T10:30:45.411 INFO:teuthology.orchestra.run.smithi089.stderr:Found initrd image: /boot/initrd.img-6.6.0-rc5-g47672119149b 2023-10-18T10:30:45.439 INFO:teuthology.orchestra.run.smithi154.stderr:done 2023-10-18T10:30:45.608 DEBUG:teuthology.orchestra.run.smithi154:> dpkg-deb -c /tmp/linux-image.deb 2023-10-18T10:30:45.616 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./ 2023-10-18T10:30:45.616 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./boot/ 2023-10-18T10:30:45.616 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 4415097 2023-10-10 19:56 ./boot/System.map-6.6.0-rc5-g47672119149b 2023-10-18T10:30:45.628 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 169298 2023-10-10 19:56 ./boot/config-6.6.0-rc5-g47672119149b 2023-10-18T10:30:45.628 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11352928 2023-10-10 19:56 ./boot/vmlinuz-6.6.0-rc5-g47672119149b 2023-10-18T10:30:45.648 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/ 2023-10-18T10:30:45.648 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/kernel/ 2023-10-18T10:30:45.648 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/kernel/postinst.d/ 2023-10-18T10:30:45.649 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/kernel/postrm.d/ 2023-10-18T10:30:45.649 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/kernel/preinst.d/ 2023-10-18T10:30:45.649 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/kernel/prerm.d/ 2023-10-18T10:30:45.649 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/ 2023-10-18T10:30:45.649 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/ 2023-10-18T10:30:45.650 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/ 2023-10-18T10:30:45.650 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/ 2023-10-18T10:30:45.650 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/ 2023-10-18T10:30:45.650 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/ 2023-10-18T10:30:45.650 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/ 2023-10-18T10:30:45.651 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 490233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/aesni-intel.ko 2023-10-18T10:30:45.653 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 25329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-18T10:30:45.653 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 29529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-18T10:30:45.654 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 30313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-18T10:30:45.654 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 93113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-18T10:30:45.654 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 106729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-18T10:30:45.654 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 152241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-18T10:30:45.654 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-18T10:30:45.655 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-18T10:30:45.655 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 83889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-18T10:30:45.655 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-18T10:30:45.655 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 47225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-18T10:30:45.655 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 47209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-18T10:30:45.656 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 51801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-18T10:30:45.656 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 38937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-18T10:30:45.656 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 40529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-18T10:30:45.656 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 58585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-18T10:30:45.656 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 50497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-18T10:30:45.657 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 32065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-18T10:30:45.657 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-18T10:30:45.657 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/ 2023-10-18T10:30:45.657 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/ 2023-10-18T10:30:45.657 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/mce/ 2023-10-18T10:30:45.658 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 40113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-18T10:30:45.658 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpuid.ko 2023-10-18T10:30:45.658 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/msr.ko 2023-10-18T10:30:45.658 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/ 2023-10-18T10:30:45.658 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 353761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-18T10:30:45.659 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 881713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-18T10:30:45.659 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 2276817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm.ko 2023-10-18T10:30:45.665 INFO:teuthology.orchestra.run.smithi089.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-10-18T10:30:45.672 INFO:teuthology.orchestra.run.smithi089.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-10-18T10:30:45.712 INFO:teuthology.orchestra.run.smithi089.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-10-18T10:30:45.718 INFO:teuthology.orchestra.run.smithi089.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-10-18T10:30:45.732 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ 2023-10-18T10:30:45.732 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 53761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/af_alg.ko 2023-10-18T10:30:45.732 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 23097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/algif_hash.ko 2023-10-18T10:30:45.732 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/algif_skcipher.ko 2023-10-18T10:30:45.732 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ansi_cprng.ko 2023-10-18T10:30:45.733 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/anubis.ko 2023-10-18T10:30:45.733 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/arc4.ko 2023-10-18T10:30:45.733 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/asymmetric_keys/ 2023-10-18T10:30:45.733 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2023-10-18T10:30:45.733 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/ 2023-10-18T10:30:45.734 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-18T10:30:45.734 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_pq.ko 2023-10-18T10:30:45.734 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-18T10:30:45.734 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_tx.ko 2023-10-18T10:30:45.735 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_xor.ko 2023-10-18T10:30:45.735 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/raid6test.ko 2023-10-18T10:30:45.735 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/authenc.ko 2023-10-18T10:30:45.735 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 22641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/authencesn.ko 2023-10-18T10:30:45.736 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blake2b_generic.ko 2023-10-18T10:30:45.736 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blowfish_common.ko 2023-10-18T10:30:45.736 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blowfish_generic.ko 2023-10-18T10:30:45.736 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 38873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/camellia_generic.ko 2023-10-18T10:30:45.736 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 27321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast5_generic.ko 2023-10-18T10:30:45.737 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast6_generic.ko 2023-10-18T10:30:45.737 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast_common.ko 2023-10-18T10:30:45.737 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 35097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ccm.ko 2023-10-18T10:30:45.737 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cmac.ko 2023-10-18T10:30:45.737 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crc32_generic.ko 2023-10-18T10:30:45.738 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 47505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cryptd.ko 2023-10-18T10:30:45.738 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 39505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_engine.ko 2023-10-18T10:30:45.738 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_simd.ko 2023-10-18T10:30:45.738 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 24737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_user.ko 2023-10-18T10:30:45.738 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cts.ko 2023-10-18T10:30:45.739 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/des_generic.ko 2023-10-18T10:30:45.739 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/dh_generic.ko 2023-10-18T10:30:45.739 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/echainiv.ko 2023-10-18T10:30:45.739 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 25321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/essiv.ko 2023-10-18T10:30:45.739 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/fcrypt.ko 2023-10-18T10:30:45.740 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/khazad.ko 2023-10-18T10:30:45.740 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lrw.ko 2023-10-18T10:30:45.740 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lz4.ko 2023-10-18T10:30:45.740 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lz4hc.ko 2023-10-18T10:30:45.741 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/md4.ko 2023-10-18T10:30:45.741 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/michael_mic.ko 2023-10-18T10:30:45.741 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/pcbc.ko 2023-10-18T10:30:45.741 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/pcrypt.ko 2023-10-18T10:30:45.741 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/rmd160.ko 2023-10-18T10:30:45.742 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 29393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/seed.ko 2023-10-18T10:30:45.742 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 27081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/serpent_generic.ko 2023-10-18T10:30:45.742 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 149193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/tcrypt.ko 2023-10-18T10:30:45.742 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/tea.ko 2023-10-18T10:30:45.742 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 28777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/twofish_common.ko 2023-10-18T10:30:45.743 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/twofish_generic.ko 2023-10-18T10:30:45.743 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/vmac.ko 2023-10-18T10:30:45.743 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 36185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/wp512.ko 2023-10-18T10:30:45.743 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xcbc.ko 2023-10-18T10:30:45.743 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xor.ko 2023-10-18T10:30:45.744 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xts.ko 2023-10-18T10:30:45.744 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xxhash_generic.ko 2023-10-18T10:30:45.744 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ 2023-10-18T10:30:45.744 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/ 2023-10-18T10:30:45.745 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_extlog.ko 2023-10-18T10:30:45.745 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-18T10:30:45.745 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 28793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_pad.ko 2023-10-18T10:30:45.745 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/apei/ 2023-10-18T10:30:45.745 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 31241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/apei/einj.ko 2023-10-18T10:30:45.746 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/ec_sys.ko 2023-10-18T10:30:45.746 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/platform_profile.ko 2023-10-18T10:30:45.746 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 24401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/sbs.ko 2023-10-18T10:30:45.746 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/sbshc.ko 2023-10-18T10:30:45.746 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 103025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/video.ko 2023-10-18T10:30:45.747 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ 2023-10-18T10:30:45.747 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/acard-ahci.ko 2023-10-18T10:30:45.747 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 88865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ahci.ko 2023-10-18T10:30:45.747 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ahci_platform.ko 2023-10-18T10:30:45.747 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 88289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/libahci.ko 2023-10-18T10:30:45.748 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 34985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/libahci_platform.ko 2023-10-18T10:30:45.748 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_acpi.ko 2023-10-18T10:30:45.748 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 30497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ali.ko 2023-10-18T10:30:45.748 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 33569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_amd.ko 2023-10-18T10:30:45.748 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_artop.ko 2023-10-18T10:30:45.749 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_atiixp.ko 2023-10-18T10:30:45.749 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 24945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_atp867x.ko 2023-10-18T10:30:45.749 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cmd640.ko 2023-10-18T10:30:45.749 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 25249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-18T10:30:45.749 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cypress.ko 2023-10-18T10:30:45.750 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_efar.ko 2023-10-18T10:30:45.750 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt366.ko 2023-10-18T10:30:45.750 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 33977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-18T10:30:45.750 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 23129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-18T10:30:45.750 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-18T10:30:45.751 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_it8213.ko 2023-10-18T10:30:45.751 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 35273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_it821x.ko 2023-10-18T10:30:45.751 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_jmicron.ko 2023-10-18T10:30:45.751 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 51481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_legacy.ko 2023-10-18T10:30:45.751 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_marvell.ko 2023-10-18T10:30:45.752 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_mpiix.ko 2023-10-18T10:30:45.752 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_netcell.ko 2023-10-18T10:30:45.752 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ninja32.ko 2023-10-18T10:30:45.752 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ns87410.ko 2023-10-18T10:30:45.752 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ns87415.ko 2023-10-18T10:30:45.753 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-18T10:30:45.753 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_opti.ko 2023-10-18T10:30:45.753 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_optidma.ko 2023-10-18T10:30:45.753 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 32585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-18T10:30:45.753 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 37793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-18T10:30:45.754 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-18T10:30:45.754 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_piccolo.ko 2023-10-18T10:30:45.754 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_radisys.ko 2023-10-18T10:30:45.754 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_rdc.ko 2023-10-18T10:30:45.755 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_rz1000.ko 2023-10-18T10:30:45.755 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sch.ko 2023-10-18T10:30:45.755 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 23097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_serverworks.ko 2023-10-18T10:30:45.755 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 23457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sil680.ko 2023-10-18T10:30:45.755 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-18T10:30:45.756 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_triflex.ko 2023-10-18T10:30:45.756 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 30497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_via.ko 2023-10-18T10:30:45.756 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 22793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pdc_adma.ko 2023-10-18T10:30:45.756 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 29121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_inic162x.ko 2023-10-18T10:30:45.756 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 83345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_mv.ko 2023-10-18T10:30:45.757 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 70393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_nv.ko 2023-10-18T10:30:45.757 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 37377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_promise.ko 2023-10-18T10:30:45.757 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 23041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_qstor.ko 2023-10-18T10:30:45.757 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 29273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sil.ko 2023-10-18T10:30:45.758 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 38337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sil24.ko 2023-10-18T10:30:45.758 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sis.ko 2023-10-18T10:30:45.758 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_svw.ko 2023-10-18T10:30:45.758 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 44793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sx4.ko 2023-10-18T10:30:45.759 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_uli.ko 2023-10-18T10:30:45.759 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 35065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_via.ko 2023-10-18T10:30:45.759 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_vsc.ko 2023-10-18T10:30:45.759 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/auxdisplay/ 2023-10-18T10:30:45.759 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-18T10:30:45.760 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/bcma/ 2023-10-18T10:30:45.760 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 124473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/bcma/bcma.ko 2023-10-18T10:30:45.760 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/ 2023-10-18T10:30:45.760 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/aoe/ 2023-10-18T10:30:45.760 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 113665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/aoe/aoe.ko 2023-10-18T10:30:45.760 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/drbd/ 2023-10-18T10:30:45.761 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 965585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/drbd/drbd.ko 2023-10-18T10:30:45.761 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 173225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/floppy.ko 2023-10-18T10:30:45.761 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/mtip32xx/ 2023-10-18T10:30:45.761 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 94297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-18T10:30:45.761 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 115793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/nbd.ko 2023-10-18T10:30:45.762 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/null_blk/ 2023-10-18T10:30:45.762 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 110449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-18T10:30:45.762 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 98193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/pktcdvd.ko 2023-10-18T10:30:45.762 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 263769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/rbd.ko 2023-10-18T10:30:45.762 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkback/ 2023-10-18T10:30:45.763 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 98145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-18T10:30:45.763 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/zram/ 2023-10-18T10:30:45.763 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 60217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/zram/zram.ko 2023-10-18T10:30:45.763 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ 2023-10-18T10:30:45.764 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 30705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/applicom.ko 2023-10-18T10:30:45.764 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hangcheck-timer.ko 2023-10-18T10:30:45.764 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/ 2023-10-18T10:30:45.764 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-18T10:30:45.764 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-18T10:30:45.765 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-18T10:30:45.765 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-18T10:30:45.765 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-18T10:30:45.765 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ 2023-10-18T10:30:45.766 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 30713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-18T10:30:45.766 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 148601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-18T10:30:45.766 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-18T10:30:45.766 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 139313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-18T10:30:45.767 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 70921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-18T10:30:45.767 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 53513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-18T10:30:45.767 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 36321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/lp.ko 2023-10-18T10:30:45.767 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/mwave/ 2023-10-18T10:30:45.767 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 84097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/mwave/mwave.ko 2023-10-18T10:30:45.767 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 25873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/nvram.ko 2023-10-18T10:30:45.768 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 32169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ppdev.ko 2023-10-18T10:30:45.768 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 51113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tlclk.ko 2023-10-18T10:30:45.768 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/ 2023-10-18T10:30:45.768 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-18T10:30:45.768 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2023-10-18T10:30:45.769 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 23473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2023-10-18T10:30:45.769 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 32985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2023-10-18T10:30:45.769 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 30121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-18T10:30:45.769 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 22697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-18T10:30:45.769 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 23561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/xen-tpmfront.ko 2023-10-18T10:30:45.770 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/xillybus/ 2023-10-18T10:30:45.770 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/xillybus/xillybus_class.ko 2023-10-18T10:30:45.770 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 50945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/xillybus/xillybus_core.ko 2023-10-18T10:30:45.770 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/xillybus/xillybus_pcie.ko 2023-10-18T10:30:45.770 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/clk/ 2023-10-18T10:30:45.771 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/clk/clk-palmas.ko 2023-10-18T10:30:45.771 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 47849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/clk/clk-si5351.ko 2023-10-18T10:30:45.771 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/clk/clk-twl6040.ko 2023-10-18T10:30:45.771 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/clk/clk-wm831x.ko 2023-10-18T10:30:45.771 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/ 2023-10-18T10:30:45.772 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-18T10:30:45.772 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-18T10:30:45.772 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 25297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-18T10:30:45.772 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ 2023-10-18T10:30:45.772 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ 2023-10-18T10:30:45.773 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 70801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-18T10:30:45.773 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 243025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-18T10:30:45.773 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/ 2023-10-18T10:30:45.773 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/ 2023-10-18T10:30:45.773 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_common/ 2023-10-18T10:30:45.774 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 438761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-18T10:30:45.774 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2023-10-18T10:30:45.774 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 24081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-18T10:30:45.774 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/padlock-aes.ko 2023-10-18T10:30:45.775 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/padlock-sha.ko 2023-10-18T10:30:45.775 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/virtio/ 2023-10-18T10:30:45.775 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 68857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-18T10:30:45.775 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dca/ 2023-10-18T10:30:45.775 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 29785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dca/dca.ko 2023-10-18T10:30:45.775 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ 2023-10-18T10:30:45.776 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/dw/ 2023-10-18T10:30:45.776 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-18T10:30:45.776 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ioat/ 2023-10-18T10:30:45.776 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 122569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-18T10:30:45.777 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/ 2023-10-18T10:30:45.777 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 79953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/amd64_edac.ko 2023-10-18T10:30:45.777 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 35769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/e752x_edac.ko 2023-10-18T10:30:45.777 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 63617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-18T10:30:45.777 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i3000_edac.ko 2023-10-18T10:30:45.777 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i3200_edac.ko 2023-10-18T10:30:45.778 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 22537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i5100_edac.ko 2023-10-18T10:30:45.778 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 25657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i5400_edac.ko 2023-10-18T10:30:45.778 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 24913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i7300_edac.ko 2023-10-18T10:30:45.778 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 52041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i7core_edac.ko 2023-10-18T10:30:45.778 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i82975x_edac.ko 2023-10-18T10:30:45.779 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/ie31200_edac.ko 2023-10-18T10:30:45.779 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 56825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/sb_edac.ko 2023-10-18T10:30:45.779 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/x38_edac.ko 2023-10-18T10:30:45.779 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/ 2023-10-18T10:30:45.779 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-gpio.ko 2023-10-18T10:30:45.780 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 27185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-max14577.ko 2023-10-18T10:30:45.780 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 29569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-max77693.ko 2023-10-18T10:30:45.780 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 24745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-max8997.ko 2023-10-18T10:30:45.780 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-palmas.ko 2023-10-18T10:30:45.780 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 23713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-rt8973a.ko 2023-10-18T10:30:45.781 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 27785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-sm5502.ko 2023-10-18T10:30:45.781 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/ 2023-10-18T10:30:45.781 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 34977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/dmi-sysfs.ko 2023-10-18T10:30:45.781 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/efi/ 2023-10-18T10:30:45.782 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/efi/efi-pstore.ko 2023-10-18T10:30:45.782 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-18T10:30:45.782 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/ 2023-10-18T10:30:45.782 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-adp5520.ko 2023-10-18T10:30:45.783 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-amd8111.ko 2023-10-18T10:30:45.783 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-arizona.ko 2023-10-18T10:30:45.783 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-da9052.ko 2023-10-18T10:30:45.783 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-da9055.ko 2023-10-18T10:30:45.783 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 22065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-dln2.ko 2023-10-18T10:30:45.784 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 77849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-f7188x.ko 2023-10-18T10:30:45.784 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 28937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-generic.ko 2023-10-18T10:30:45.784 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-ich.ko 2023-10-18T10:30:45.784 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-janz-ttl.ko 2023-10-18T10:30:45.784 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-kempld.ko 2023-10-18T10:30:45.785 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-lp3943.ko 2023-10-18T10:30:45.785 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-max7300.ko 2023-10-18T10:30:45.785 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-max730x.ko 2023-10-18T10:30:45.785 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 22073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-max732x.ko 2023-10-18T10:30:45.785 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 25145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-ml-ioh.ko 2023-10-18T10:30:45.786 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 47073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-pca953x.ko 2023-10-18T10:30:45.786 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-pcf857x.ko 2023-10-18T10:30:45.786 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-rdc321x.ko 2023-10-18T10:30:45.786 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-sch.ko 2023-10-18T10:30:45.786 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-sch311x.ko 2023-10-18T10:30:45.787 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-tps65912.ko 2023-10-18T10:30:45.787 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-twl4030.ko 2023-10-18T10:30:45.787 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-twl6040.ko 2023-10-18T10:30:45.787 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-18T10:30:45.788 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-vx855.ko 2023-10-18T10:30:45.788 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-wm831x.ko 2023-10-18T10:30:45.788 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-wm8350.ko 2023-10-18T10:30:45.788 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-wm8994.ko 2023-10-18T10:30:45.788 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/ 2023-10-18T10:30:45.789 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-a4tech.ko 2023-10-18T10:30:45.789 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 39169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-apple.ko 2023-10-18T10:30:45.789 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-appleir.ko 2023-10-18T10:30:45.789 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-aureal.ko 2023-10-18T10:30:45.790 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-axff.ko 2023-10-18T10:30:45.790 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-belkin.ko 2023-10-18T10:30:45.790 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-cherry.ko 2023-10-18T10:30:45.790 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-chicony.ko 2023-10-18T10:30:45.790 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 60649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-cp2112.ko 2023-10-18T10:30:45.791 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-cypress.ko 2023-10-18T10:30:45.791 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-dr.ko 2023-10-18T10:30:45.791 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-elecom.ko 2023-10-18T10:30:45.791 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-elo.ko 2023-10-18T10:30:45.791 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-emsff.ko 2023-10-18T10:30:45.792 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-ezkey.ko 2023-10-18T10:30:45.792 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gaff.ko 2023-10-18T10:30:45.792 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-generic.ko 2023-10-18T10:30:45.792 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gt683r.ko 2023-10-18T10:30:45.792 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gyration.ko 2023-10-18T10:30:45.792 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-18T10:30:45.793 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-18T10:30:45.793 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtekff.ko 2023-10-18T10:30:45.793 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-hyperv.ko 2023-10-18T10:30:45.793 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-icade.ko 2023-10-18T10:30:45.793 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-kensington.ko 2023-10-18T10:30:45.794 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-keytouch.ko 2023-10-18T10:30:45.794 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-kye.ko 2023-10-18T10:30:45.794 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lcpower.ko 2023-10-18T10:30:45.794 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-led.ko 2023-10-18T10:30:45.794 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 42361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lenovo.ko 2023-10-18T10:30:45.795 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 30761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lg-g15.ko 2023-10-18T10:30:45.795 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 58185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-18T10:30:45.795 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 101369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-18T10:30:45.795 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 80993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech.ko 2023-10-18T10:30:45.795 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 30801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-magicmouse.ko 2023-10-18T10:30:45.796 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-microsoft.ko 2023-10-18T10:30:45.796 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-monterey.ko 2023-10-18T10:30:45.796 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 50161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-multitouch.ko 2023-10-18T10:30:45.796 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 32593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-ntrig.ko 2023-10-18T10:30:45.796 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-ortek.ko 2023-10-18T10:30:45.797 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-penmount.ko 2023-10-18T10:30:45.797 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-petalynx.ko 2023-10-18T10:30:45.797 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 72937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-picolcd.ko 2023-10-18T10:30:45.797 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-pl.ko 2023-10-18T10:30:45.797 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-plantronics.ko 2023-10-18T10:30:45.798 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-primax.ko 2023-10-18T10:30:45.798 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 35457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-rmi.ko 2023-10-18T10:30:45.798 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-18T10:30:45.798 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-18T10:30:45.798 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 31425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-18T10:30:45.799 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 29729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-18T10:30:45.799 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 31129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-18T10:30:45.799 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-18T10:30:45.799 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 29489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-18T10:30:45.799 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-18T10:30:45.799 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 29313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-18T10:30:45.800 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 27937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-18T10:30:45.800 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-18T10:30:45.800 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 23337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat.ko 2023-10-18T10:30:45.800 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-saitek.ko 2023-10-18T10:30:45.800 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-samsung.ko 2023-10-18T10:30:45.801 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 36817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-18T10:30:45.801 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sjoy.ko 2023-10-18T10:30:45.801 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 51249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sony.ko 2023-10-18T10:30:45.801 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-speedlink.ko 2023-10-18T10:30:45.801 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 25297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-steelseries.ko 2023-10-18T10:30:45.802 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sunplus.ko 2023-10-18T10:30:45.802 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-18T10:30:45.802 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-tivo.ko 2023-10-18T10:30:45.802 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-tmff.ko 2023-10-18T10:30:45.802 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-topseed.ko 2023-10-18T10:30:45.803 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-twinhan.ko 2023-10-18T10:30:45.803 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 71073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-uclogic.ko 2023-10-18T10:30:45.803 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-waltop.ko 2023-10-18T10:30:45.803 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 143473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-wiimote.ko 2023-10-18T10:30:45.803 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-xinmo.ko 2023-10-18T10:30:45.804 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-zpff.ko 2023-10-18T10:30:45.804 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-zydacron.ko 2023-10-18T10:30:45.804 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 267825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid.ko 2023-10-18T10:30:45.804 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 32321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/uhid.ko 2023-10-18T10:30:45.804 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/usbhid/ 2023-10-18T10:30:45.805 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 125721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/usbhid/usbhid.ko 2023-10-18T10:30:45.805 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/usbhid/usbkbd.ko 2023-10-18T10:30:45.805 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/usbhid/usbmouse.ko 2023-10-18T10:30:45.805 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 217873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/wacom.ko 2023-10-18T10:30:45.805 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/ 2023-10-18T10:30:45.806 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 64201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_balloon.ko 2023-10-18T10:30:45.806 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 89625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_utils.ko 2023-10-18T10:30:45.806 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 305465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_vmbus.ko 2023-10-18T10:30:45.806 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ 2023-10-18T10:30:45.806 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 66521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/abituguru.ko 2023-10-18T10:30:45.807 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 61393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/abituguru3.ko 2023-10-18T10:30:45.807 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 34745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-18T10:30:45.807 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ad7414.ko 2023-10-18T10:30:45.807 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ad7418.ko 2023-10-18T10:30:45.808 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 25257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adc128d818.ko 2023-10-18T10:30:45.808 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 31729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1025.ko 2023-10-18T10:30:45.808 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 97977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1026.ko 2023-10-18T10:30:45.808 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1029.ko 2023-10-18T10:30:45.809 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 45553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1031.ko 2023-10-18T10:30:45.809 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm9240.ko 2023-10-18T10:30:45.809 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ads7828.ko 2023-10-18T10:30:45.809 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7410.ko 2023-10-18T10:30:45.809 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 22049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7411.ko 2023-10-18T10:30:45.810 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 74433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7462.ko 2023-10-18T10:30:45.810 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 39585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7470.ko 2023-10-18T10:30:45.810 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 67929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7475.ko 2023-10-18T10:30:45.810 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7x10.ko 2023-10-18T10:30:45.811 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 39073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/amc6821.ko 2023-10-18T10:30:45.811 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 55265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/applesmc.ko 2023-10-18T10:30:45.811 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 49913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asb100.ko 2023-10-18T10:30:45.811 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 58617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asc7621.ko 2023-10-18T10:30:45.812 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 40889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-18T10:30:45.812 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/atxp1.ko 2023-10-18T10:30:45.812 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 28281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/coretemp.ko 2023-10-18T10:30:45.812 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 24657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/da9052-hwmon.ko 2023-10-18T10:30:45.812 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/da9055-hwmon.ko 2023-10-18T10:30:45.813 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 44465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-18T10:30:45.813 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 99601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/dme1737.ko 2023-10-18T10:30:45.813 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ds1621.ko 2023-10-18T10:30:45.813 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ds620.ko 2023-10-18T10:30:45.813 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 29777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc1403.ko 2023-10-18T10:30:45.814 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 33185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc2103.ko 2023-10-18T10:30:45.814 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 30321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc6w201.ko 2023-10-18T10:30:45.814 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 66985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f71805f.ko 2023-10-18T10:30:45.814 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 94457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f71882fg.ko 2023-10-18T10:30:45.815 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 41401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f75375s.ko 2023-10-18T10:30:45.815 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-18T10:30:45.815 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 55449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/fschmd.ko 2023-10-18T10:30:45.815 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/g760a.ko 2023-10-18T10:30:45.815 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 24017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/g762.ko 2023-10-18T10:30:45.816 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 46801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/gl518sm.ko 2023-10-18T10:30:45.816 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 45529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/gl520sm.ko 2023-10-18T10:30:45.816 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/hih6130.ko 2023-10-18T10:30:45.816 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-18T10:30:45.816 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-18T10:30:45.817 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 22601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-18T10:30:45.817 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 34529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ibmaem.ko 2023-10-18T10:30:45.817 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ibmpex.ko 2023-10-18T10:30:45.817 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 24873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ina209.ko 2023-10-18T10:30:45.818 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ina2xx.ko 2023-10-18T10:30:45.818 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 130449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/it87.ko 2023-10-18T10:30:45.818 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/jc42.ko 2023-10-18T10:30:45.818 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 24329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/k10temp.ko 2023-10-18T10:30:45.819 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/k8temp.ko 2023-10-18T10:30:45.819 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-18T10:30:45.819 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 48601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm63.ko 2023-10-18T10:30:45.819 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm73.ko 2023-10-18T10:30:45.819 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 36777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm75.ko 2023-10-18T10:30:45.819 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm77.ko 2023-10-18T10:30:45.820 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 34657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm78.ko 2023-10-18T10:30:45.820 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 31745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm80.ko 2023-10-18T10:30:45.820 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm83.ko 2023-10-18T10:30:45.820 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 72833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm85.ko 2023-10-18T10:30:45.820 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 46153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm87.ko 2023-10-18T10:30:45.821 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 37913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm90.ko 2023-10-18T10:30:45.821 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm92.ko 2023-10-18T10:30:45.821 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 108841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm93.ko 2023-10-18T10:30:45.821 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 33393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95234.ko 2023-10-18T10:30:45.821 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95241.ko 2023-10-18T10:30:45.822 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95245.ko 2023-10-18T10:30:45.822 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 24017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc2945.ko 2023-10-18T10:30:45.822 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4151.ko 2023-10-18T10:30:45.822 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4215.ko 2023-10-18T10:30:45.822 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4222.ko 2023-10-18T10:30:45.823 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4245.ko 2023-10-18T10:30:45.823 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4260.ko 2023-10-18T10:30:45.823 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4261.ko 2023-10-18T10:30:45.823 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 33489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max16065.ko 2023-10-18T10:30:45.823 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max1619.ko 2023-10-18T10:30:45.824 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 24353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max1668.ko 2023-10-18T10:30:45.824 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max197.ko 2023-10-18T10:30:45.824 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 27129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6639.ko 2023-10-18T10:30:45.824 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 23321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6650.ko 2023-10-18T10:30:45.824 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 35777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6697.ko 2023-10-18T10:30:45.825 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/mc13783-adc.ko 2023-10-18T10:30:45.825 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/mcp3021.ko 2023-10-18T10:30:45.825 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2023-10-18T10:30:45.825 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 44129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6683.ko 2023-10-18T10:30:45.825 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 119817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-18T10:30:45.825 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 60705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6775.ko 2023-10-18T10:30:45.826 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 59737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct7802.ko 2023-10-18T10:30:45.826 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 82337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pc87360.ko 2023-10-18T10:30:45.826 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 54433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pc87427.ko 2023-10-18T10:30:45.826 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pcf8591.ko 2023-10-18T10:30:45.826 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ 2023-10-18T10:30:45.827 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-18T10:30:45.827 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-18T10:30:45.827 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 28329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-18T10:30:45.827 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-18T10:30:45.827 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-18T10:30:45.828 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-18T10:30:45.828 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-18T10:30:45.828 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 82577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-18T10:30:45.828 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-18T10:30:45.828 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 30729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-18T10:30:45.829 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-18T10:30:45.829 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-18T10:30:45.829 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/powr1220.ko 2023-10-18T10:30:45.829 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch5627.ko 2023-10-18T10:30:45.829 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 28817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch5636.ko 2023-10-18T10:30:45.830 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 27145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-18T10:30:45.830 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 32137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sht15.ko 2023-10-18T10:30:45.830 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sht21.ko 2023-10-18T10:30:45.830 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/shtc1.ko 2023-10-18T10:30:45.830 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 40105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sis5595.ko 2023-10-18T10:30:45.831 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-18T10:30:45.831 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 39193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-18T10:30:45.831 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 37385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-18T10:30:45.831 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/thmc50.ko 2023-10-18T10:30:45.831 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp102.ko 2023-10-18T10:30:45.832 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp103.ko 2023-10-18T10:30:45.832 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp401.ko 2023-10-18T10:30:45.832 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp421.ko 2023-10-18T10:30:45.832 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-18T10:30:45.832 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 41945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/via686a.ko 2023-10-18T10:30:45.833 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 52457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/vt1211.ko 2023-10-18T10:30:45.833 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 51545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/vt8231.ko 2023-10-18T10:30:45.833 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 57513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-18T10:30:45.833 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 78873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83627hf.ko 2023-10-18T10:30:45.833 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 71345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83781d.ko 2023-10-18T10:30:45.834 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 63289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83791d.ko 2023-10-18T10:30:45.834 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 71321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83792d.ko 2023-10-18T10:30:45.834 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 101473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83793.ko 2023-10-18T10:30:45.834 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 119505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83795.ko 2023-10-18T10:30:45.835 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-18T10:30:45.835 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 35625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-18T10:30:45.835 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/wm831x-hwmon.ko 2023-10-18T10:30:45.835 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/wm8350-hwmon.ko 2023-10-18T10:30:45.835 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/ 2023-10-18T10:30:45.836 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/ 2023-10-18T10:30:45.836 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-18T10:30:45.836 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-18T10:30:45.836 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/ 2023-10-18T10:30:45.836 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 27969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ali1535.ko 2023-10-18T10:30:45.837 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 28801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ali1563.ko 2023-10-18T10:30:45.837 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 27777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2023-10-18T10:30:45.837 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-18T10:30:45.837 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-18T10:30:45.837 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-18T10:30:45.838 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2023-10-18T10:30:45.838 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 6929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-18T10:30:45.838 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2023-10-18T10:30:45.838 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 50201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-18T10:30:45.838 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 23905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-18T10:30:45.839 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-18T10:30:45.839 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-18T10:30:45.839 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-dln2.ko 2023-10-18T10:30:45.839 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-gpio.ko 2023-10-18T10:30:45.839 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 62817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-18T10:30:45.840 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 22433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-18T10:30:45.840 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 47889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-18T10:30:45.840 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-kempld.ko 2023-10-18T10:30:45.840 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-18T10:30:45.840 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 22337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-18T10:30:45.841 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ocores.ko 2023-10-18T10:30:45.841 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 25145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-18T10:30:45.841 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-18T10:30:45.841 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 47241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-18T10:30:45.841 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2023-10-18T10:30:45.841 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-18T10:30:45.842 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-18T10:30:45.842 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 22313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-sis5595.ko 2023-10-18T10:30:45.842 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 29361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-sis630.ko 2023-10-18T10:30:45.842 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-18T10:30:45.843 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2023-10-18T10:30:45.843 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-18T10:30:45.843 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-18T10:30:45.843 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-18T10:30:45.843 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-18T10:30:45.844 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 47657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-xiic.ko 2023-10-18T10:30:45.844 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 22449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-mux.ko 2023-10-18T10:30:45.844 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-18T10:30:45.844 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 27073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-stub.ko 2023-10-18T10:30:45.844 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/muxes/ 2023-10-18T10:30:45.845 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2023-10-18T10:30:45.845 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2023-10-18T10:30:45.845 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 29801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2023-10-18T10:30:45.845 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ 2023-10-18T10:30:45.845 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ 2023-10-18T10:30:45.846 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 273609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-18T10:30:45.846 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 879969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-18T10:30:45.846 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 69337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-18T10:30:45.846 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 324217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-18T10:30:45.846 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 97985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-18T10:30:45.847 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 282273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-18T10:30:45.847 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 66201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-18T10:30:45.847 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/ 2023-10-18T10:30:45.847 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/cxgb4/ 2023-10-18T10:30:45.848 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 493073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-18T10:30:45.848 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx4/ 2023-10-18T10:30:45.848 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 419737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-18T10:30:45.848 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx5/ 2023-10-18T10:30:45.848 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 739025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-18T10:30:45.849 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mthca/ 2023-10-18T10:30:45.849 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 296625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-18T10:30:45.849 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/ocrdma/ 2023-10-18T10:30:45.849 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 153697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2023-10-18T10:30:45.850 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/usnic/ 2023-10-18T10:30:45.850 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 168969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2023-10-18T10:30:45.850 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ 2023-10-18T10:30:45.850 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ipoib/ 2023-10-18T10:30:45.851 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 263081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-18T10:30:45.851 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/iser/ 2023-10-18T10:30:45.851 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 100241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-18T10:30:45.851 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/isert/ 2023-10-18T10:30:45.851 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 111425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-18T10:30:45.852 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srp/ 2023-10-18T10:30:45.852 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 128753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-18T10:30:45.852 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srpt/ 2023-10-18T10:30:45.852 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 129529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-18T10:30:45.852 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/ 2023-10-18T10:30:45.853 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/evbug.ko 2023-10-18T10:30:45.853 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 24129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/ff-memless.ko 2023-10-18T10:30:45.853 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/ 2023-10-18T10:30:45.853 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-18T10:30:45.854 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-18T10:30:45.854 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 41241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/gameport.ko 2023-10-18T10:30:45.854 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 41801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/joydev.ko 2023-10-18T10:30:45.854 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/ 2023-10-18T10:30:45.855 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/adp5520-keys.ko 2023-10-18T10:30:45.855 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/adp5588-keys.ko 2023-10-18T10:30:45.855 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/adp5589-keys.ko 2023-10-18T10:30:45.855 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 24761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2023-10-18T10:30:45.855 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 36665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-18T10:30:45.856 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2023-10-18T10:30:45.856 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/lkkbd.ko 2023-10-18T10:30:45.856 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/lm8323.ko 2023-10-18T10:30:45.856 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/lm8333.ko 2023-10-18T10:30:45.857 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 24265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/matrix_keypad.ko 2023-10-18T10:30:45.857 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/max7359_keypad.ko 2023-10-18T10:30:45.857 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/mcs_touchkey.ko 2023-10-18T10:30:45.857 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2023-10-18T10:30:45.857 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/newtonkbd.ko 2023-10-18T10:30:45.858 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/opencores-kbd.ko 2023-10-18T10:30:45.858 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/qt1070.ko 2023-10-18T10:30:45.858 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/qt2160.ko 2023-10-18T10:30:45.858 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 23817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/samsung-keypad.ko 2023-10-18T10:30:45.858 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/stowaway.ko 2023-10-18T10:30:45.859 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/sunkbd.ko 2023-10-18T10:30:45.859 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/tca6416-keypad.ko 2023-10-18T10:30:45.859 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/tca8418_keypad.ko 2023-10-18T10:30:45.859 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/twl4030_keypad.ko 2023-10-18T10:30:45.859 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/xtkbd.ko 2023-10-18T10:30:45.860 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/matrix-keymap.ko 2023-10-18T10:30:45.860 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/ 2023-10-18T10:30:45.860 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 34329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/appletouch.ko 2023-10-18T10:30:45.860 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 34177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-18T10:30:45.860 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 123681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-18T10:30:45.861 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 89449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-18T10:30:45.861 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/gpio_mouse.ko 2023-10-18T10:30:45.861 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 335977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/psmouse.ko 2023-10-18T10:30:45.861 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/sermouse.ko 2023-10-18T10:30:45.861 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 23881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-18T10:30:45.862 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 22569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-18T10:30:45.862 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-18T10:30:45.862 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/rmi4/ 2023-10-18T10:30:45.862 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 109873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-18T10:30:45.862 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/ 2023-10-18T10:30:45.863 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-18T10:30:45.863 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-18T10:30:45.863 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/ct82c710.ko 2023-10-18T10:30:45.863 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-18T10:30:45.864 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/parkbd.ko 2023-10-18T10:30:45.865 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/pcips2.ko 2023-10-18T10:30:45.865 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/ps2mult.ko 2023-10-18T10:30:45.865 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 24153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/serio_raw.ko 2023-10-18T10:30:45.866 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/serport.ko 2023-10-18T10:30:45.866 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/sparse-keymap.ko 2023-10-18T10:30:45.866 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/ 2023-10-18T10:30:45.866 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/amd/ 2023-10-18T10:30:45.866 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 39289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-18T10:30:45.867 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/ 2023-10-18T10:30:45.867 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-88pm860x.ko 2023-10-18T10:30:45.867 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-adp5520.ko 2023-10-18T10:30:45.867 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 59753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-bd2802.ko 2023-10-18T10:30:45.867 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 24785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-blinkm.ko 2023-10-18T10:30:45.868 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-da903x.ko 2023-10-18T10:30:45.868 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-da9052.ko 2023-10-18T10:30:45.868 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-gpio.ko 2023-10-18T10:30:45.868 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm3530.ko 2023-10-18T10:30:45.868 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 30505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm3533.ko 2023-10-18T10:30:45.869 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm355x.ko 2023-10-18T10:30:45.869 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm3642.ko 2023-10-18T10:30:45.869 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lp3944.ko 2023-10-18T10:30:45.869 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lp8788.ko 2023-10-18T10:30:45.869 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-max8997.ko 2023-10-18T10:30:45.869 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-mc13783.ko 2023-10-18T10:30:45.870 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-menf21bmc.ko 2023-10-18T10:30:45.870 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-pca9532.ko 2023-10-18T10:30:45.870 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-pca955x.ko 2023-10-18T10:30:45.870 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-pca963x.ko 2023-10-18T10:30:45.870 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 34433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-ss4200.ko 2023-10-18T10:30:45.871 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 24865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-tca6507.ko 2023-10-18T10:30:45.871 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-wm831x-status.ko 2023-10-18T10:30:45.871 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-wm8350.ko 2023-10-18T10:30:45.871 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ 2023-10-18T10:30:45.871 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-18T10:30:45.872 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-18T10:30:45.872 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-18T10:30:45.872 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 7617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-18T10:30:45.872 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-18T10:30:45.873 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-18T10:30:45.873 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-18T10:30:45.873 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-18T10:30:45.873 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/ 2023-10-18T10:30:45.873 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/bcache/ 2023-10-18T10:30:45.874 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 654969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/bcache/bcache.ko 2023-10-18T10:30:45.874 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 40577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-bio-prison.ko 2023-10-18T10:30:45.874 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 89601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-bufio.ko 2023-10-18T10:30:45.874 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 47529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-cache-smq.ko 2023-10-18T10:30:45.874 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 156009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-cache.ko 2023-10-18T10:30:45.875 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 106897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-crypt.ko 2023-10-18T10:30:45.875 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-delay.ko 2023-10-18T10:30:45.875 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 53857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-era.ko 2023-10-18T10:30:45.875 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 28465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-flakey.ko 2023-10-18T10:30:45.875 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 41593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-log-userspace.ko 2023-10-18T10:30:45.876 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 32737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-log.ko 2023-10-18T10:30:45.876 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 49545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-mirror.ko 2023-10-18T10:30:45.876 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 78033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-multipath.ko 2023-10-18T10:30:45.876 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-queue-length.ko 2023-10-18T10:30:45.876 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 70825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-raid.ko 2023-10-18T10:30:45.877 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 36297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-region-hash.ko 2023-10-18T10:30:45.877 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-round-robin.ko 2023-10-18T10:30:45.877 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-service-time.ko 2023-10-18T10:30:45.877 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 109593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-snapshot.ko 2023-10-18T10:30:45.877 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-switch.ko 2023-10-18T10:30:45.878 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 181865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-thin-pool.ko 2023-10-18T10:30:45.878 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 51737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-verity.ko 2023-10-18T10:30:45.878 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-zero.ko 2023-10-18T10:30:45.878 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/faulty.ko 2023-10-18T10:30:45.878 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/linear.ko 2023-10-18T10:30:45.879 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 32417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/multipath.ko 2023-10-18T10:30:45.879 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/persistent-data/ 2023-10-18T10:30:45.879 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 199937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-18T10:30:45.879 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 37337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid0.ko 2023-10-18T10:30:45.879 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 113073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid1.ko 2023-10-18T10:30:45.880 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 158681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid10.ko 2023-10-18T10:30:45.880 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 395721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid456.ko 2023-10-18T10:30:45.880 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/ 2023-10-18T10:30:45.880 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/ 2023-10-18T10:30:45.880 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 178137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptbase.ko 2023-10-18T10:30:45.881 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 63385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptctl.ko 2023-10-18T10:30:45.881 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 44665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptfc.ko 2023-10-18T10:30:45.881 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 37225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptlan.ko 2023-10-18T10:30:45.881 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 123321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptsas.ko 2023-10-18T10:30:45.881 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 79041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-18T10:30:45.882 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 53081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptspi.ko 2023-10-18T10:30:45.882 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/ 2023-10-18T10:30:45.882 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/88pm800.ko 2023-10-18T10:30:45.882 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/88pm805.ko 2023-10-18T10:30:45.882 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/88pm80x.ko 2023-10-18T10:30:45.883 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/arizona-i2c.ko 2023-10-18T10:30:45.883 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 123721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/arizona.ko 2023-10-18T10:30:45.883 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/bcm590xx.ko 2023-10-18T10:30:45.883 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/cros_ec_dev.ko 2023-10-18T10:30:45.883 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 34961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/dln2.ko 2023-10-18T10:30:45.883 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/janz-cmodio.ko 2023-10-18T10:30:45.884 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 53585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/kempld-core.ko 2023-10-18T10:30:45.884 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 27529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lm3533-core.ko 2023-10-18T10:30:45.884 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lm3533-ctrlbank.ko 2023-10-18T10:30:45.884 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lp3943.ko 2023-10-18T10:30:45.885 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 54569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lpc_ich.ko 2023-10-18T10:30:45.885 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lpc_sch.ko 2023-10-18T10:30:45.885 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/max8907.ko 2023-10-18T10:30:45.885 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 29857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/mc13xxx-core.ko 2023-10-18T10:30:45.885 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/mc13xxx-i2c.ko 2023-10-18T10:30:45.885 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/menf21bmc.ko 2023-10-18T10:30:45.886 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/pcf50633-adc.ko 2023-10-18T10:30:45.886 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/pcf50633-gpio.ko 2023-10-18T10:30:45.886 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 34873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/pcf50633.ko 2023-10-18T10:30:45.886 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/rdc321x-southbridge.ko 2023-10-18T10:30:45.886 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/retu-mfd.ko 2023-10-18T10:30:45.887 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 80425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/si476x-core.ko 2023-10-18T10:30:45.887 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 48457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/sm501.ko 2023-10-18T10:30:45.887 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/ti_am335x_tscadc.ko 2023-10-18T10:30:45.887 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 51137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/tps65010.ko 2023-10-18T10:30:45.887 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/tps6507x.ko 2023-10-18T10:30:45.888 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/viperboard.ko 2023-10-18T10:30:45.888 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/vx855.ko 2023-10-18T10:30:45.888 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/wl1273-core.ko 2023-10-18T10:30:45.888 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 44345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/wm8994.ko 2023-10-18T10:30:45.888 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ 2023-10-18T10:30:45.889 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ad525x_dpot-i2c.ko 2023-10-18T10:30:45.889 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 48489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ad525x_dpot.ko 2023-10-18T10:30:45.889 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/altera-stapl/ 2023-10-18T10:30:45.889 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 46585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-18T10:30:45.889 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/apds9802als.ko 2023-10-18T10:30:45.890 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 36729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/apds990x.ko 2023-10-18T10:30:45.890 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 45593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/bh1770glc.ko 2023-10-18T10:30:45.890 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/c2port/ 2023-10-18T10:30:45.890 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2023-10-18T10:30:45.891 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 35281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/c2port/core.ko 2023-10-18T10:30:45.891 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/cb710/ 2023-10-18T10:30:45.891 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 25161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/cb710/cb710.ko 2023-10-18T10:30:45.891 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ds1682.ko 2023-10-18T10:30:45.891 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/dummy-irq.ko 2023-10-18T10:30:45.892 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/echo/ 2023-10-18T10:30:45.892 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/echo/echo.ko 2023-10-18T10:30:45.892 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/ 2023-10-18T10:30:45.892 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 37849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/at24.ko 2023-10-18T10:30:45.892 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-18T10:30:45.893 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-18T10:30:45.893 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-18T10:30:45.893 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 33009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/enclosure.ko 2023-10-18T10:30:45.893 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/genwqe/ 2023-10-18T10:30:45.893 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 152201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/genwqe/genwqe_card.ko 2023-10-18T10:30:45.894 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/hmc6352.ko 2023-10-18T10:30:45.894 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 31321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/hpilo.ko 2023-10-18T10:30:45.894 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ibmasm/ 2023-10-18T10:30:45.894 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 72129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ibmasm/ibmasm.ko 2023-10-18T10:30:45.894 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ics932s401.ko 2023-10-18T10:30:45.895 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/isl29003.ko 2023-10-18T10:30:45.895 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/isl29020.ko 2023-10-18T10:30:45.895 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/ 2023-10-18T10:30:45.895 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 42513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-18T10:30:45.895 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-18T10:30:45.896 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/ 2023-10-18T10:30:45.896 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 125209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei-me.ko 2023-10-18T10:30:45.896 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 56521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-18T10:30:45.896 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 310113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei.ko 2023-10-18T10:30:45.896 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 31329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/phantom.ko 2023-10-18T10:30:45.897 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ti-st/ 2023-10-18T10:30:45.897 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 74265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ti-st/st_drv.ko 2023-10-18T10:30:45.897 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 24305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-18T10:30:45.897 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 28849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tifm_core.ko 2023-10-18T10:30:45.897 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tsl2550.ko 2023-10-18T10:30:45.898 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 48697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_balloon.ko 2023-10-18T10:30:45.898 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_vmci/ 2023-10-18T10:30:45.898 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 183609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-18T10:30:45.898 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ 2023-10-18T10:30:45.899 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/bonding/ 2023-10-18T10:30:45.899 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 490129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/bonding/bonding.ko 2023-10-18T10:30:45.899 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/ 2023-10-18T10:30:45.899 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/b53/ 2023-10-18T10:30:45.899 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 106033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-18T10:30:45.900 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 40297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-18T10:30:45.900 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-18T10:30:45.900 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6xxx/ 2023-10-18T10:30:45.900 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 453737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-18T10:30:45.900 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dummy.ko 2023-10-18T10:30:45.901 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/eql.ko 2023-10-18T10:30:45.901 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ 2023-10-18T10:30:45.901 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/ 2023-10-18T10:30:45.901 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 45785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-18T10:30:45.901 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 42105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-18T10:30:45.902 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 130553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-18T10:30:45.902 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 68849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-18T10:30:45.902 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/ 2023-10-18T10:30:45.902 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 37649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-18T10:30:45.902 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 59689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-18T10:30:45.903 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 27201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-18T10:30:45.903 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 99113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-18T10:30:45.903 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/adaptec/ 2023-10-18T10:30:45.903 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 54521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-18T10:30:45.903 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/agere/ 2023-10-18T10:30:45.903 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 69201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-18T10:30:45.904 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/alteon/ 2023-10-18T10:30:45.904 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 59585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-18T10:30:45.904 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/altera/ 2023-10-18T10:30:45.904 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 41409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-18T10:30:45.904 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/ 2023-10-18T10:30:45.905 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 54609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-18T10:30:45.905 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 49289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-18T10:30:45.905 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 87297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-18T10:30:45.905 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/ 2023-10-18T10:30:45.905 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/alx/ 2023-10-18T10:30:45.906 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 97337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-18T10:30:45.906 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1c/ 2023-10-18T10:30:45.906 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 104673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-18T10:30:45.906 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1e/ 2023-10-18T10:30:45.906 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 90257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-18T10:30:45.907 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/ 2023-10-18T10:30:45.907 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 81569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-18T10:30:45.907 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 69729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-18T10:30:45.907 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/ 2023-10-18T10:30:45.907 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 78401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-18T10:30:45.907 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 176761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-18T10:30:45.908 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2023-10-18T10:30:45.908 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 1421497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-18T10:30:45.909 INFO:teuthology.orchestra.run.smithi089.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-10-18T10:30:45.909 INFO:teuthology.orchestra.run.smithi089.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-10-18T10:30:45.910 INFO:teuthology.orchestra.run.smithi089.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-10-18T10:30:45.910 INFO:teuthology.orchestra.run.smithi089.stderr:done 2023-10-18T10:30:46.030 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 128569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-18T10:30:46.030 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/genet/ 2023-10-18T10:30:46.031 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 132089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-18T10:30:46.031 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 312081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-18T10:30:46.031 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/ 2023-10-18T10:30:46.031 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/bna/ 2023-10-18T10:30:46.032 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 323513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-18T10:30:46.032 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/ 2023-10-18T10:30:46.032 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb/ 2023-10-18T10:30:46.032 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 193497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-18T10:30:46.032 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2023-10-18T10:30:46.033 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 347417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-18T10:30:46.033 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2023-10-18T10:30:46.033 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 1044433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-18T10:30:46.033 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2023-10-18T10:30:46.033 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 166193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-18T10:30:46.034 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2023-10-18T10:30:46.034 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 34881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-18T10:30:46.034 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/ 2023-10-18T10:30:46.034 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/enic/ 2023-10-18T10:30:46.034 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 204345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-18T10:30:46.035 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/ 2023-10-18T10:30:46.035 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/ 2023-10-18T10:30:46.035 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 69449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-18T10:30:46.035 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 61489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-18T10:30:46.035 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 141225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-18T10:30:46.036 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 51497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-18T10:30:46.036 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 62817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-18T10:30:46.036 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 31369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-18T10:30:46.036 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/ 2023-10-18T10:30:46.037 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 60873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-18T10:30:46.037 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 62833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-18T10:30:46.037 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 45721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dnet.ko 2023-10-18T10:30:46.037 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 25185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ec_bhf.ko 2023-10-18T10:30:46.037 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/ 2023-10-18T10:30:46.038 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/benet/ 2023-10-18T10:30:46.038 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 281137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-18T10:30:46.038 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 49457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-18T10:30:46.038 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 51737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-18T10:30:46.038 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/fujitsu/ 2023-10-18T10:30:46.039 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 45345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2023-10-18T10:30:46.039 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ 2023-10-18T10:30:46.039 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 90465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-18T10:30:46.039 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000/ 2023-10-18T10:30:46.039 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 295577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-18T10:30:46.039 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000e/ 2023-10-18T10:30:46.040 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 584497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-18T10:30:46.040 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/fm10k/ 2023-10-18T10:30:46.040 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 246209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-18T10:30:46.040 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/i40e/ 2023-10-18T10:30:46.040 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 925697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-18T10:30:46.041 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/iavf/ 2023-10-18T10:30:46.041 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 302849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-18T10:30:46.041 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igb/ 2023-10-18T10:30:46.041 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 514681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-18T10:30:46.041 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igbvf/ 2023-10-18T10:30:46.042 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 93593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-18T10:30:46.042 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbe/ 2023-10-18T10:30:46.042 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 734305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-18T10:30:46.042 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbevf/ 2023-10-18T10:30:46.042 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 151769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-18T10:30:46.043 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 86921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/jme.ko 2023-10-18T10:30:46.043 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/ 2023-10-18T10:30:46.043 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-18T10:30:46.043 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 102289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-18T10:30:46.043 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 122649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-18T10:30:46.044 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/ 2023-10-18T10:30:46.044 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/ 2023-10-18T10:30:46.044 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 727601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-18T10:30:46.044 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 278937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-18T10:30:46.044 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/ 2023-10-18T10:30:46.045 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2023-10-18T10:30:46.045 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 1113993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-18T10:30:46.045 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ 2023-10-18T10:30:46.045 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 53105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ks8842.ko 2023-10-18T10:30:46.045 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 40993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2023-10-18T10:30:46.046 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2023-10-18T10:30:46.046 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 95665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-18T10:30:46.046 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/ 2023-10-18T10:30:46.046 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/myri10ge/ 2023-10-18T10:30:46.046 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 108649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-18T10:30:46.047 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/ 2023-10-18T10:30:46.047 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 73441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-18T10:30:46.047 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 47777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-18T10:30:46.047 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/neterion/ 2023-10-18T10:30:46.047 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 150905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-18T10:30:46.048 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/nvidia/ 2023-10-18T10:30:46.048 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 136721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-18T10:30:46.048 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/ 2023-10-18T10:30:46.048 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 50481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-18T10:30:46.049 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 43145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-18T10:30:46.049 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/ 2023-10-18T10:30:46.049 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/netxen/ 2023-10-18T10:30:46.049 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 227897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-18T10:30:46.049 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 87081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-18T10:30:46.050 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2023-10-18T10:30:46.050 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 672049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-18T10:30:46.050 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rdc/ 2023-10-18T10:30:46.050 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 43017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-18T10:30:46.050 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/ 2023-10-18T10:30:46.051 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 67385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-18T10:30:46.051 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 83985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-18T10:30:46.051 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 29321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-18T10:30:46.051 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 189801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-18T10:30:46.051 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/samsung/ 2023-10-18T10:30:46.052 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/samsung/sxgbe/ 2023-10-18T10:30:46.052 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 104241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2023-10-18T10:30:46.052 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sfc/ 2023-10-18T10:30:46.052 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 1053921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-18T10:30:46.052 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/silan/ 2023-10-18T10:30:46.052 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 47097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-18T10:30:46.053 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/ 2023-10-18T10:30:46.053 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 54817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-18T10:30:46.053 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 63337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-18T10:30:46.053 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/ 2023-10-18T10:30:46.053 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 61601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-18T10:30:46.054 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 67193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-18T10:30:46.054 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 73409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-18T10:30:46.054 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 68177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-18T10:30:46.054 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/ 2023-10-18T10:30:46.054 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/ 2023-10-18T10:30:46.055 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2023-10-18T10:30:46.055 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 39905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-18T10:30:46.055 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2023-10-18T10:30:46.055 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 486961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-18T10:30:46.055 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/ 2023-10-18T10:30:46.056 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 103337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-18T10:30:46.056 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 166161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-18T10:30:46.056 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 66329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-18T10:30:46.056 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 78801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-18T10:30:46.056 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/tehuti/ 2023-10-18T10:30:46.057 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 50057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-18T10:30:46.057 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ti/ 2023-10-18T10:30:46.057 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 86529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-18T10:30:46.057 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/ 2023-10-18T10:30:46.057 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 80273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-18T10:30:46.058 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 83633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-18T10:30:46.058 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/ 2023-10-18T10:30:46.058 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 46241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-18T10:30:46.058 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 30849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-18T10:30:46.058 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/xircom/ 2023-10-18T10:30:46.059 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 60057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-18T10:30:46.059 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/fddi/ 2023-10-18T10:30:46.059 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 41065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/fddi/defxx.ko 2023-10-18T10:30:46.059 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/fddi/skfp/ 2023-10-18T10:30:46.059 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 216985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/fddi/skfp/skfp.ko 2023-10-18T10:30:46.060 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 77345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/geneve.ko 2023-10-18T10:30:46.060 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/hyperv/ 2023-10-18T10:30:46.060 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 235521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-18T10:30:46.060 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ieee802154/ 2023-10-18T10:30:46.060 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-18T10:30:46.061 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 25065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ifb.ko 2023-10-18T10:30:46.061 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ipvlan/ 2023-10-18T10:30:46.061 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 83769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-18T10:30:46.061 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 62209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/macvlan.ko 2023-10-18T10:30:46.061 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/macvtap.ko 2023-10-18T10:30:46.062 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/ 2023-10-18T10:30:46.062 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-18T10:30:46.062 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-18T10:30:46.062 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-gpio.ko 2023-10-18T10:30:46.062 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-18T10:30:46.063 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio.ko 2023-10-18T10:30:46.063 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mii.ko 2023-10-18T10:30:46.063 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 43953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/netconsole.ko 2023-10-18T10:30:46.063 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/nlmon.ko 2023-10-18T10:30:46.063 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/ 2023-10-18T10:30:46.064 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-18T10:30:46.064 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 38977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-18T10:30:46.064 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/ 2023-10-18T10:30:46.064 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/amd.ko 2023-10-18T10:30:46.064 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 50097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-18T10:30:46.065 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 44505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-18T10:30:46.065 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-18T10:30:46.065 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 42241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/broadcom.ko 2023-10-18T10:30:46.065 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/cicada.ko 2023-10-18T10:30:46.066 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/davicom.ko 2023-10-18T10:30:46.066 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/et1011c.ko 2023-10-18T10:30:46.066 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 24969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/icplus.ko 2023-10-18T10:30:46.066 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/lxt.ko 2023-10-18T10:30:46.066 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 82049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/marvell.ko 2023-10-18T10:30:46.067 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 76177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/micrel.ko 2023-10-18T10:30:46.067 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/national.ko 2023-10-18T10:30:46.067 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 97857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/phylink.ko 2023-10-18T10:30:46.067 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/qsemi.ko 2023-10-18T10:30:46.067 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 50065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/realtek.ko 2023-10-18T10:30:46.068 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 29985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/smsc.ko 2023-10-18T10:30:46.068 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-18T10:30:46.068 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 24289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/vitesse.ko 2023-10-18T10:30:46.068 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/plip/ 2023-10-18T10:30:46.068 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 43569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/plip/plip.ko 2023-10-18T10:30:46.069 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ 2023-10-18T10:30:46.069 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-18T10:30:46.069 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 29713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-18T10:30:46.069 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-18T10:30:46.069 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-18T10:30:46.070 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 28729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-18T10:30:46.070 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 42545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pppoe.ko 2023-10-18T10:30:46.070 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pppox.ko 2023-10-18T10:30:46.070 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 32689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pptp.ko 2023-10-18T10:30:46.070 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 28289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/sb1000.ko 2023-10-18T10:30:46.071 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip/ 2023-10-18T10:30:46.071 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 42465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip/slip.ko 2023-10-18T10:30:46.071 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/sungem_phy.ko 2023-10-18T10:30:46.071 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 64385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/tap.ko 2023-10-18T10:30:46.071 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/ 2023-10-18T10:30:46.072 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 106169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team.ko 2023-10-18T10:30:46.072 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-18T10:30:46.072 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-18T10:30:46.072 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 28881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-18T10:30:46.072 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_random.ko 2023-10-18T10:30:46.073 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-18T10:30:46.073 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 87017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/veth.ko 2023-10-18T10:30:46.073 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vmxnet3/ 2023-10-18T10:30:46.073 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 144297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-18T10:30:46.073 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vxlan/ 2023-10-18T10:30:46.074 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 297329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-18T10:30:46.074 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netback/ 2023-10-18T10:30:46.074 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 144649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-18T10:30:46.074 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ntb/ 2023-10-18T10:30:46.074 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ntb/ntb.ko 2023-10-18T10:30:46.075 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/ 2023-10-18T10:30:46.075 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/ 2023-10-18T10:30:46.075 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 264937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-18T10:30:46.075 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 102209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/nvme.ko 2023-10-18T10:30:46.075 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/ 2023-10-18T10:30:46.076 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 133841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport.ko 2023-10-18T10:30:46.076 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_cs.ko 2023-10-18T10:30:46.076 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 84465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_pc.ko 2023-10-18T10:30:46.076 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_serial.ko 2023-10-18T10:30:46.076 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/ 2023-10-18T10:30:46.077 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/ 2023-10-18T10:30:46.077 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 25009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-18T10:30:46.077 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2023-10-18T10:30:46.077 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2023-10-18T10:30:46.078 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/pci-stub.ko 2023-10-18T10:30:46.078 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 46529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/xen-pcifront.ko 2023-10-18T10:30:46.078 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/ 2023-10-18T10:30:46.078 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/i82092.ko 2023-10-18T10:30:46.078 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 145809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pcmcia.ko 2023-10-18T10:30:46.079 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 62137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pcmcia_core.ko 2023-10-18T10:30:46.079 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 34345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2023-10-18T10:30:46.079 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 34265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pd6729.ko 2023-10-18T10:30:46.079 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 112889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-18T10:30:46.079 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/phy/ 2023-10-18T10:30:46.080 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/phy/broadcom/ 2023-10-18T10:30:46.080 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2023-10-18T10:30:46.080 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/phy/samsung/ 2023-10-18T10:30:46.080 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2023-10-18T10:30:46.080 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/ 2023-10-18T10:30:46.081 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/intel/ 2023-10-18T10:30:46.081 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 70153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-18T10:30:46.081 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/ 2023-10-18T10:30:46.081 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/ 2023-10-18T10:30:46.081 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 50889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2023-10-18T10:30:46.082 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_ec.ko 2023-10-18T10:30:46.082 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2023-10-18T10:30:46.082 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 25417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2023-10-18T10:30:46.082 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2023-10-18T10:30:46.082 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2023-10-18T10:30:46.083 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2023-10-18T10:30:46.083 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/ 2023-10-18T10:30:46.083 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 67097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-18T10:30:46.083 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 39145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-18T10:30:46.083 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 36913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-18T10:30:46.084 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 59665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-18T10:30:46.084 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 31097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-18T10:30:46.084 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 127569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-18T10:30:46.084 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 36913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-18T10:30:46.084 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 49121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-18T10:30:46.084 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-18T10:30:46.085 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 40265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-18T10:30:46.085 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 22521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-18T10:30:46.085 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 36657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/hdaps.ko 2023-10-18T10:30:46.085 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/ibm_rtl.ko 2023-10-18T10:30:46.086 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel/ 2023-10-18T10:30:46.086 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-18T10:30:46.086 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2023-10-18T10:30:46.086 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 43497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-18T10:30:46.086 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-18T10:30:46.087 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-18T10:30:46.087 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 34801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-18T10:30:46.087 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 42329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-18T10:30:46.087 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-18T10:30:46.087 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-18T10:30:46.088 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-18T10:30:46.088 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-18T10:30:46.088 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-18T10:30:46.088 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 57569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/wmi.ko 2023-10-18T10:30:46.088 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/ 2023-10-18T10:30:46.089 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/ 2023-10-18T10:30:46.089 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 30345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/88pm860x_battery.ko 2023-10-18T10:30:46.089 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 33985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/88pm860x_charger.ko 2023-10-18T10:30:46.089 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 55057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/bq2415x_charger.ko 2023-10-18T10:30:46.089 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 54081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/bq24190_charger.ko 2023-10-18T10:30:46.090 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/bq24735-charger.ko 2023-10-18T10:30:46.090 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/cros_peripheral_charger.ko 2023-10-18T10:30:46.090 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/da9030_battery.ko 2023-10-18T10:30:46.090 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/da9052-battery.ko 2023-10-18T10:30:46.090 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 29281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/ds2760_battery.ko 2023-10-18T10:30:46.091 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 23217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/ds2780_battery.ko 2023-10-18T10:30:46.091 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 23353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/ds2781_battery.ko 2023-10-18T10:30:46.091 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 23649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/ds2782_battery.ko 2023-10-18T10:30:46.091 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/gpio-charger.ko 2023-10-18T10:30:46.091 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/isp1704_charger.ko 2023-10-18T10:30:46.092 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/lp8727_charger.ko 2023-10-18T10:30:46.092 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/max14577_charger.ko 2023-10-18T10:30:46.092 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 25897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/max17040_battery.ko 2023-10-18T10:30:46.092 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 30289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/max17042_battery.ko 2023-10-18T10:30:46.092 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/max8903_charger.ko 2023-10-18T10:30:46.093 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 23729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/max8925_power.ko 2023-10-18T10:30:46.093 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/pcf50633-charger.ko 2023-10-18T10:30:46.093 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 38009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/sbs-battery.ko 2023-10-18T10:30:46.093 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 28385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/test_power.ko 2023-10-18T10:30:46.093 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/tps65090-charger.ko 2023-10-18T10:30:46.094 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/wm831x_backup.ko 2023-10-18T10:30:46.094 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 24521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/wm831x_power.ko 2023-10-18T10:30:46.094 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 22417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/wm8350_power.ko 2023-10-18T10:30:46.094 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/ 2023-10-18T10:30:46.094 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 57729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-18T10:30:46.095 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-18T10:30:46.095 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/ 2023-10-18T10:30:46.095 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/ 2023-10-18T10:30:46.095 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-18T10:30:46.095 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-18T10:30:46.096 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-18T10:30:46.096 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 41873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/pps_core.ko 2023-10-18T10:30:46.096 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ 2023-10-18T10:30:46.096 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 66385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ptp.ko 2023-10-18T10:30:46.097 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-18T10:30:46.097 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/ 2023-10-18T10:30:46.097 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 23497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-88pm80x.ko 2023-10-18T10:30:46.097 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 24521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-88pm860x.ko 2023-10-18T10:30:46.097 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-18T10:30:46.098 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-da9052.ko 2023-10-18T10:30:46.098 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-da9055.ko 2023-10-18T10:30:46.098 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-da9063.ko 2023-10-18T10:30:46.098 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-18T10:30:46.099 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 68577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-18T10:30:46.099 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-18T10:30:46.099 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-18T10:30:46.099 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-18T10:30:46.099 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-18T10:30:46.100 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-18T10:30:46.100 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-18T10:30:46.100 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 23705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-18T10:30:46.100 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-18T10:30:46.100 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 24329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-18T10:30:46.101 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-18T10:30:46.101 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 35969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-18T10:30:46.101 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-lp8788.ko 2023-10-18T10:30:46.101 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 38097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-18T10:30:46.101 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-18T10:30:46.101 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 25945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-18T10:30:46.102 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t86.ko 2023-10-18T10:30:46.102 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-18T10:30:46.102 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max8907.ko 2023-10-18T10:30:46.102 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max8925.ko 2023-10-18T10:30:46.103 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 25809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max8997.ko 2023-10-18T10:30:46.103 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max8998.ko 2023-10-18T10:30:46.103 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-mc13xxx.ko 2023-10-18T10:30:46.103 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-18T10:30:46.103 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-palmas.ko 2023-10-18T10:30:46.104 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 46633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-18T10:30:46.104 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf50633.ko 2023-10-18T10:30:46.104 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-18T10:30:46.104 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-18T10:30:46.104 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 29881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-18T10:30:46.105 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-18T10:30:46.105 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rc5t583.ko 2023-10-18T10:30:46.105 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-18T10:30:46.105 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 33753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-18T10:30:46.105 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 27833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-18T10:30:46.105 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 25257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-18T10:30:46.106 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-18T10:30:46.106 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-s35390a.ko 2023-10-18T10:30:46.106 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-18T10:30:46.106 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-tps6586x.ko 2023-10-18T10:30:46.106 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-tps65910.ko 2023-10-18T10:30:46.107 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-wm831x.ko 2023-10-18T10:30:46.107 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-wm8350.ko 2023-10-18T10:30:46.107 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 27873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-18T10:30:46.107 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ 2023-10-18T10:30:46.107 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 77617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-18T10:30:46.108 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 57665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-sas.ko 2023-10-18T10:30:46.108 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 55169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-18T10:30:46.108 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 135185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/BusLogic.ko 2023-10-18T10:30:46.108 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/a100u2w.ko 2023-10-18T10:30:46.109 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aacraid/ 2023-10-18T10:30:46.109 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 272305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-18T10:30:46.109 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 100617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/advansys.ko 2023-10-18T10:30:46.109 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/ 2023-10-18T10:30:46.109 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 356265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-18T10:30:46.109 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 295801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-18T10:30:46.110 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic94xx/ 2023-10-18T10:30:46.110 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 189185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic94xx/aic94xx.ko 2023-10-18T10:30:46.110 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 24073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/am53c974.ko 2023-10-18T10:30:46.110 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/arcmsr/ 2023-10-18T10:30:46.110 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 106881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-18T10:30:46.111 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 38185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/atp870u.ko 2023-10-18T10:30:46.111 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/be2iscsi/ 2023-10-18T10:30:46.111 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 246985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-18T10:30:46.111 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bfa/ 2023-10-18T10:30:46.111 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 1171945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-18T10:30:46.112 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2fc/ 2023-10-18T10:30:46.112 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 237273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-18T10:30:46.112 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2i/ 2023-10-18T10:30:46.112 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 115369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-18T10:30:46.112 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 40585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ch.ko 2023-10-18T10:30:46.113 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/csiostor/ 2023-10-18T10:30:46.113 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 308921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-18T10:30:46.113 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/ 2023-10-18T10:30:46.113 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb3i/ 2023-10-18T10:30:46.113 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 76145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-18T10:30:46.114 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb4i/ 2023-10-18T10:30:46.114 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 101449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-18T10:30:46.114 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 146689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-18T10:30:46.114 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 75553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/dc395x.ko 2023-10-18T10:30:46.114 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 38137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/dmx3191d.ko 2023-10-18T10:30:46.115 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esas2r/ 2023-10-18T10:30:46.115 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 212425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-18T10:30:46.115 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 60505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esp_scsi.ko 2023-10-18T10:30:46.115 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/ 2023-10-18T10:30:46.116 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 78209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-18T10:30:46.116 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 158505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-18T10:30:46.116 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fdomain.ko 2023-10-18T10:30:46.116 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fnic/ 2023-10-18T10:30:46.116 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 263489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-18T10:30:46.117 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 201817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hpsa.ko 2023-10-18T10:30:46.117 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 51209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hptiop.ko 2023-10-18T10:30:46.117 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 49041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-18T10:30:46.117 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 35337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/imm.ko 2023-10-18T10:30:46.117 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 44057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/initio.ko 2023-10-18T10:30:46.118 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 266537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ipr.ko 2023-10-18T10:30:46.118 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 101881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ips.ko 2023-10-18T10:30:46.118 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/isci/ 2023-10-18T10:30:46.118 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 317153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/isci/isci.ko 2023-10-18T10:30:46.118 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-18T10:30:46.119 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 52961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-18T10:30:46.119 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libfc/ 2023-10-18T10:30:46.119 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 314865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-18T10:30:46.119 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 145585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libiscsi.ko 2023-10-18T10:30:46.119 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 53521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-18T10:30:46.119 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libsas/ 2023-10-18T10:30:46.120 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 216665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-18T10:30:46.120 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/lpfc/ 2023-10-18T10:30:46.120 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 2197249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-18T10:30:46.120 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/ 2023-10-18T10:30:46.120 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 73569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-18T10:30:46.121 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 33529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-18T10:30:46.121 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 310617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-18T10:30:46.121 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 84721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid.ko 2023-10-18T10:30:46.121 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mpt3sas/ 2023-10-18T10:30:46.121 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 659777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-18T10:30:46.122 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvsas/ 2023-10-18T10:30:46.122 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 119561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-18T10:30:46.122 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 78129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvumi.ko 2023-10-18T10:30:46.122 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pcmcia/ 2023-10-18T10:30:46.122 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 81625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2023-10-18T10:30:46.122 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2023-10-18T10:30:46.123 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2023-10-18T10:30:46.123 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 23857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2023-10-18T10:30:46.123 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pm8001/ 2023-10-18T10:30:46.123 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 364105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-18T10:30:46.123 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 123617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pmcraid.ko 2023-10-18T10:30:46.124 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 34513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ppa.ko 2023-10-18T10:30:46.124 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 55361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla1280.ko 2023-10-18T10:30:46.124 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/ 2023-10-18T10:30:46.124 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 1876625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-18T10:30:46.124 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 85289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-18T10:30:46.125 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla4xxx/ 2023-10-18T10:30:46.125 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 559033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-18T10:30:46.125 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qlogicfas408.ko 2023-10-18T10:30:46.125 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/raid_class.ko 2023-10-18T10:30:46.125 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 238825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_debug.ko 2023-10-18T10:30:46.126 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 173937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-18T10:30:46.126 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 279313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-18T10:30:46.126 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 97729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-18T10:30:46.126 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 73881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-18T10:30:46.126 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 44993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-18T10:30:46.127 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ses.ko 2023-10-18T10:30:46.127 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 115889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/st.ko 2023-10-18T10:30:46.127 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 46825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/stex.ko 2023-10-18T10:30:46.127 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/sym53c8xx_2/ 2023-10-18T10:30:46.127 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 162121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-18T10:30:46.128 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 42297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-18T10:30:46.128 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 50321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-18T10:30:46.128 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 40025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/wd719x.ko 2023-10-18T10:30:46.128 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 50025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/xen-scsifront.ko 2023-10-18T10:30:46.128 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ssb/ 2023-10-18T10:30:46.128 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 129441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ssb/ssb.ko 2023-10-18T10:30:46.129 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/ 2023-10-18T10:30:46.129 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/iscsi/ 2023-10-18T10:30:46.129 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 736601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-18T10:30:46.129 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/loopback/ 2023-10-18T10:30:46.130 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 49985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-18T10:30:46.130 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 39713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_file.ko 2023-10-18T10:30:46.130 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 43025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_iblock.ko 2023-10-18T10:30:46.130 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 930321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_mod.ko 2023-10-18T10:30:46.130 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 44305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_pscsi.ko 2023-10-18T10:30:46.131 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 122521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_user.ko 2023-10-18T10:30:46.131 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/tcm_fc/ 2023-10-18T10:30:46.131 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 75593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-18T10:30:46.131 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/ 2023-10-18T10:30:46.131 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/ 2023-10-18T10:30:46.131 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/ 2023-10-18T10:30:46.132 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 23481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-18T10:30:46.132 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 29321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-18T10:30:46.132 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-18T10:30:46.132 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-18T10:30:46.132 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-18T10:30:46.133 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-18T10:30:46.133 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-18T10:30:46.133 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-18T10:30:46.133 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-18T10:30:46.133 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-18T10:30:46.134 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-18T10:30:46.134 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 37065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-18T10:30:46.134 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 30729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_powerclamp.ko 2023-10-18T10:30:46.134 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-18T10:30:46.135 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-18T10:30:46.135 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-18T10:30:46.135 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ 2023-10-18T10:30:46.135 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ipwireless/ 2023-10-18T10:30:46.135 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 78425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-18T10:30:46.136 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 57865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/moxa.ko 2023-10-18T10:30:46.136 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 50953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/mxser.ko 2023-10-18T10:30:46.136 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 108601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/n_gsm.ko 2023-10-18T10:30:46.136 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 31961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/n_hdlc.ko 2023-10-18T10:30:46.136 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 67929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/nozomi.ko 2023-10-18T10:30:46.137 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/ 2023-10-18T10:30:46.137 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250/ 2023-10-18T10:30:46.137 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 35753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250/8250_dw.ko 2023-10-18T10:30:46.137 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 59497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-18T10:30:46.137 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/altera_jtaguart.ko 2023-10-18T10:30:46.137 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 22577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/altera_uart.ko 2023-10-18T10:30:46.138 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-18T10:30:46.138 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 72425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/fsl_lpuart.ko 2023-10-18T10:30:46.138 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/jsm/ 2023-10-18T10:30:46.138 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 103073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-18T10:30:46.138 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 28785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/rp2.ko 2023-10-18T10:30:46.139 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 52817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/sc16is7xx.ko 2023-10-18T10:30:46.139 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/ 2023-10-18T10:30:46.139 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/core/ 2023-10-18T10:30:46.139 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 531265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-18T10:30:46.139 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host/ 2023-10-18T10:30:46.140 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 28481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-18T10:30:46.140 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2023-10-18T10:30:46.140 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/ 2023-10-18T10:30:46.140 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 44513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio.ko 2023-10-18T10:30:46.140 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_aec.ko 2023-10-18T10:30:46.141 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_cif.ko 2023-10-18T10:30:46.141 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_dmem_genirq.ko 2023-10-18T10:30:46.141 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_mf624.ko 2023-10-18T10:30:46.141 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_netx.ko 2023-10-18T10:30:46.141 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-18T10:30:46.142 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_pdrv_genirq.ko 2023-10-18T10:30:46.142 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_sercos3.ko 2023-10-18T10:30:46.142 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/ 2023-10-18T10:30:46.142 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/c67x00/ 2023-10-18T10:30:46.142 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 73297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/c67x00/c67x00.ko 2023-10-18T10:30:46.143 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/chipidea/ 2023-10-18T10:30:46.143 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 144377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/chipidea/ci_hdrc.ko 2023-10-18T10:30:46.143 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2023-10-18T10:30:46.143 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2023-10-18T10:30:46.143 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2023-10-18T10:30:46.144 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/ 2023-10-18T10:30:46.144 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 87089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-18T10:30:46.144 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 51409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-18T10:30:46.144 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 47369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/usblp.ko 2023-10-18T10:30:46.144 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 89857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/usbtmc.ko 2023-10-18T10:30:46.145 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/common/ 2023-10-18T10:30:46.145 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 22953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/common/ulpi.ko 2023-10-18T10:30:46.145 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc2/ 2023-10-18T10:30:46.145 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc2/dwc2_pci.ko 2023-10-18T10:30:46.145 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc3/ 2023-10-18T10:30:46.146 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc3/dwc3-haps.ko 2023-10-18T10:30:46.146 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 30041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc3/dwc3-pci.ko 2023-10-18T10:30:46.146 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 444553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc3/dwc3.ko 2023-10-18T10:30:46.146 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/ 2023-10-18T10:30:46.146 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/ 2023-10-18T10:30:46.147 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 60177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/u_ether.ko 2023-10-18T10:30:46.147 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 51881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/u_serial.ko 2023-10-18T10:30:46.147 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 28841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2023-10-18T10:30:46.147 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 42449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2023-10-18T10:30:46.147 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 27953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2023-10-18T10:30:46.148 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 36513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2023-10-18T10:30:46.148 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 92489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2023-10-18T10:30:46.148 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 51049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2023-10-18T10:30:46.148 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 122081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2023-10-18T10:30:46.148 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 57617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2023-10-18T10:30:46.148 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 22185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2023-10-18T10:30:46.149 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 31265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2023-10-18T10:30:46.149 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 57729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2023-10-18T10:30:46.149 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 85609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2023-10-18T10:30:46.149 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2023-10-18T10:30:46.149 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 58113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2023-10-18T10:30:46.150 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 67337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2023-10-18T10:30:46.150 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/ 2023-10-18T10:30:46.150 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2023-10-18T10:30:46.150 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2023-10-18T10:30:46.150 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2023-10-18T10:30:46.151 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_ether.ko 2023-10-18T10:30:46.151 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 28401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2023-10-18T10:30:46.151 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_hid.ko 2023-10-18T10:30:46.151 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2023-10-18T10:30:46.151 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2023-10-18T10:30:46.152 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 32073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2023-10-18T10:30:46.152 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_printer.ko 2023-10-18T10:30:46.152 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_serial.ko 2023-10-18T10:30:46.152 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 27201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_zero.ko 2023-10-18T10:30:46.152 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 62185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2023-10-18T10:30:46.153 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2023-10-18T10:30:46.153 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 161225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/libcomposite.ko 2023-10-18T10:30:46.153 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/ 2023-10-18T10:30:46.153 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2023-10-18T10:30:46.153 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/bdc/ 2023-10-18T10:30:46.154 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 157681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2023-10-18T10:30:46.154 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 36089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/goku_udc.ko 2023-10-18T10:30:46.154 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 52721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/gr_udc.ko 2023-10-18T10:30:46.154 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 69705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2023-10-18T10:30:46.154 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 56033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/mv_udc.ko 2023-10-18T10:30:46.154 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 76057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/net2272.ko 2023-10-18T10:30:46.155 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 75073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/net2280.ko 2023-10-18T10:30:46.155 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 73169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/pch_udc.ko 2023-10-18T10:30:46.155 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 65809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2023-10-18T10:30:46.155 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 66097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2023-10-18T10:30:46.155 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 81505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2023-10-18T10:30:46.156 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 156073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/udc-core.ko 2023-10-18T10:30:46.156 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/ 2023-10-18T10:30:46.156 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/bcma-hcd.ko 2023-10-18T10:30:46.156 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 80113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/isp116x-hcd.ko 2023-10-18T10:30:46.156 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 87353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/oxu210hp-hcd.ko 2023-10-18T10:30:46.157 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 78873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/r8a66597-hcd.ko 2023-10-18T10:30:46.157 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 51329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-18T10:30:46.157 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/sl811_cs.ko 2023-10-18T10:30:46.157 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/ssb-hcd.ko 2023-10-18T10:30:46.157 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 25945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/xhci-plat-hcd.ko 2023-10-18T10:30:46.158 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/ 2023-10-18T10:30:46.158 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 34585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/mdc800.ko 2023-10-18T10:30:46.158 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/microtek.ko 2023-10-18T10:30:46.158 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ 2023-10-18T10:30:46.158 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 41721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/adutux.ko 2023-10-18T10:30:46.158 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-18T10:30:46.159 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/cypress_cy7c63.ko 2023-10-18T10:30:46.159 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 23049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/cytherm.ko 2023-10-18T10:30:46.159 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ehset.ko 2023-10-18T10:30:46.159 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/emi26.ko 2023-10-18T10:30:46.159 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/emi62.ko 2023-10-18T10:30:46.160 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ezusb.ko 2023-10-18T10:30:46.160 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/idmouse.ko 2023-10-18T10:30:46.160 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 36617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-18T10:30:46.160 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-18T10:30:46.160 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 36529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ldusb.ko 2023-10-18T10:30:46.161 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 36265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-18T10:30:46.161 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 27793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/lvstest.ko 2023-10-18T10:30:46.161 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/sisusbvga/ 2023-10-18T10:30:46.161 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 64841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-18T10:30:46.161 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-18T10:30:46.162 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usb3503.ko 2023-10-18T10:30:46.162 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 25145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usblcd.ko 2023-10-18T10:30:46.162 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 22753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-18T10:30:46.162 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 70873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usbtest.ko 2023-10-18T10:30:46.162 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 41969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/uss720.ko 2023-10-18T10:30:46.163 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 27617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/yurex.ko 2023-10-18T10:30:46.163 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/mon/ 2023-10-18T10:30:46.163 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 72953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/mon/usbmon.ko 2023-10-18T10:30:46.163 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/musb/ 2023-10-18T10:30:46.163 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 301545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/musb/musb_hdrc.ko 2023-10-18T10:30:46.163 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/ 2023-10-18T10:30:46.164 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-18T10:30:46.164 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2023-10-18T10:30:46.164 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-isp1301.ko 2023-10-18T10:30:46.164 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-tahvo.ko 2023-10-18T10:30:46.164 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ 2023-10-18T10:30:46.165 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/aircable.ko 2023-10-18T10:30:46.165 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 33329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ark3116.ko 2023-10-18T10:30:46.165 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-18T10:30:46.165 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 38105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ch341.ko 2023-10-18T10:30:46.165 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 70257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cp210x.ko 2023-10-18T10:30:46.166 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 31057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-18T10:30:46.166 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 58217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-18T10:30:46.166 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 49817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-18T10:30:46.166 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/empeg.ko 2023-10-18T10:30:46.166 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 44337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/f81232.ko 2023-10-18T10:30:46.167 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 175137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-18T10:30:46.167 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 53113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-18T10:30:46.167 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 137081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-18T10:30:46.167 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 124833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/io_ti.ko 2023-10-18T10:30:46.167 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 67769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ipaq.ko 2023-10-18T10:30:46.167 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ipw.ko 2023-10-18T10:30:46.168 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-18T10:30:46.168 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 58233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-18T10:30:46.168 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 107521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/keyspan.ko 2023-10-18T10:30:46.168 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 35617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-18T10:30:46.168 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 25665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-18T10:30:46.169 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 33689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-18T10:30:46.169 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 32953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-18T10:30:46.169 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 22033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/metro-usb.ko 2023-10-18T10:30:46.169 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 66449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mos7720.ko 2023-10-18T10:30:46.169 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 78217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mos7840.ko 2023-10-18T10:30:46.170 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 56337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mxuport.ko 2023-10-18T10:30:46.170 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/navman.ko 2023-10-18T10:30:46.170 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/omninet.ko 2023-10-18T10:30:46.170 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 24633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/opticon.ko 2023-10-18T10:30:46.170 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 190633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/option.ko 2023-10-18T10:30:46.171 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 39073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/oti6858.ko 2023-10-18T10:30:46.171 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 54353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/pl2303.ko 2023-10-18T10:30:46.171 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/qcaux.ko 2023-10-18T10:30:46.171 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 37825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/qcserial.ko 2023-10-18T10:30:46.171 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 34321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/quatech2.ko 2023-10-18T10:30:46.172 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-18T10:30:46.172 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 51625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/sierra.ko 2023-10-18T10:30:46.172 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-18T10:30:46.172 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 29993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ssu100.ko 2023-10-18T10:30:46.172 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-18T10:30:46.173 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 61489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-18T10:30:46.173 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 23729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-18T10:30:46.173 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-18T10:30:46.173 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 33201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-18T10:30:46.173 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 125609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usbserial.ko 2023-10-18T10:30:46.173 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 28753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/visor.ko 2023-10-18T10:30:46.174 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 37409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-18T10:30:46.174 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/wishbone-serial.ko 2023-10-18T10:30:46.174 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-18T10:30:46.174 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ 2023-10-18T10:30:46.174 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 52913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/uas.ko 2023-10-18T10:30:46.175 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-18T10:30:46.175 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-18T10:30:46.175 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 22049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-18T10:30:46.175 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 33497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-18T10:30:46.175 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-18T10:30:46.176 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-18T10:30:46.176 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-18T10:30:46.176 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-18T10:30:46.176 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-18T10:30:46.176 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 28441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-18T10:30:46.177 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 28905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-18T10:30:46.177 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-18T10:30:46.177 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 35337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-18T10:30:46.177 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 179969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-18T10:30:46.177 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/ 2023-10-18T10:30:46.177 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 67481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-18T10:30:46.178 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 72177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-18T10:30:46.178 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 107001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-18T10:30:46.178 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/ 2023-10-18T10:30:46.178 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/ 2023-10-18T10:30:46.179 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 154297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-18T10:30:46.179 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 29537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-18T10:30:46.179 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 108737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/vfio.ko 2023-10-18T10:30:46.179 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 77609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-18T10:30:46.179 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/ 2023-10-18T10:30:46.179 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 113873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost.ko 2023-10-18T10:30:46.180 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-18T10:30:46.180 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 58089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_net.ko 2023-10-18T10:30:46.180 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 89249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-18T10:30:46.180 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/ 2023-10-18T10:30:46.180 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/ 2023-10-18T10:30:46.181 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/88pm860x_bl.ko 2023-10-18T10:30:46.181 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/aat2870_bl.ko 2023-10-18T10:30:46.181 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 23305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/adp5520_bl.ko 2023-10-18T10:30:46.181 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 32609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/adp8860_bl.ko 2023-10-18T10:30:46.181 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 39993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/adp8870_bl.ko 2023-10-18T10:30:46.182 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-18T10:30:46.182 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/as3711_bl.ko 2023-10-18T10:30:46.182 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/bd6107.ko 2023-10-18T10:30:46.182 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/da903x_bl.ko 2023-10-18T10:30:46.182 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/da9052_bl.ko 2023-10-18T10:30:46.183 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/gpio_backlight.ko 2023-10-18T10:30:46.183 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/kb3886_bl.ko 2023-10-18T10:30:46.183 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 22017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lcd.ko 2023-10-18T10:30:46.183 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 23657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lm3533_bl.ko 2023-10-18T10:30:46.183 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lm3639_bl.ko 2023-10-18T10:30:46.184 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lv5207lp.ko 2023-10-18T10:30:46.184 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/max8925_bl.ko 2023-10-18T10:30:46.184 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/pandora_bl.ko 2023-10-18T10:30:46.184 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/pcf50633-backlight.ko 2023-10-18T10:30:46.184 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-18T10:30:46.184 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/wm831x_bl.ko 2023-10-18T10:30:46.185 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/ 2023-10-18T10:30:46.185 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ 2023-10-18T10:30:46.185 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ds2482.ko 2023-10-18T10:30:46.185 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 38985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ds2490.ko 2023-10-18T10:30:46.185 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/matrox_w1.ko 2023-10-18T10:30:46.186 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/w1-gpio.ko 2023-10-18T10:30:46.186 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/ 2023-10-18T10:30:46.186 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-18T10:30:46.186 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 22809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-18T10:30:46.186 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-18T10:30:46.187 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-18T10:30:46.187 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-18T10:30:46.187 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-18T10:30:46.187 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-18T10:30:46.187 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-18T10:30:46.188 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-18T10:30:46.188 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 7385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-18T10:30:46.188 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 56425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-18T10:30:46.188 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 102673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/wire.ko 2023-10-18T10:30:46.188 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ 2023-10-18T10:30:46.189 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/acquirewdt.ko 2023-10-18T10:30:46.189 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/advantechwdt.ko 2023-10-18T10:30:46.189 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-18T10:30:46.189 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 22969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-18T10:30:46.189 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/cpu5wdt.ko 2023-10-18T10:30:46.190 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/da9052_wdt.ko 2023-10-18T10:30:46.190 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/da9055_wdt.ko 2023-10-18T10:30:46.190 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/da9063_wdt.ko 2023-10-18T10:30:46.190 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/dw_wdt.ko 2023-10-18T10:30:46.190 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/eurotechwdt.ko 2023-10-18T10:30:46.191 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 23865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-18T10:30:46.191 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 27401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/hpwdt.ko 2023-10-18T10:30:46.191 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/i6300esb.ko 2023-10-18T10:30:46.191 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-18T10:30:46.192 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-18T10:30:46.192 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-18T10:30:46.192 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ibmasr.ko 2023-10-18T10:30:46.192 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-18T10:30:46.192 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-18T10:30:46.193 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-18T10:30:46.193 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/kempld_wdt.ko 2023-10-18T10:30:46.193 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/machzwd.ko 2023-10-18T10:30:46.193 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/mena21_wdt.ko 2023-10-18T10:30:46.193 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/menf21bmc_wdt.ko 2023-10-18T10:30:46.194 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 24673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/nv_tco.ko 2023-10-18T10:30:46.194 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/of_xilinx_wdt.ko 2023-10-18T10:30:46.194 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 22265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pc87413_wdt.ko 2023-10-18T10:30:46.194 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 38657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-18T10:30:46.194 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 31825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-18T10:30:46.194 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/retu_wdt.ko 2023-10-18T10:30:46.195 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sbc60xxwdt.ko 2023-10-18T10:30:46.195 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sbc_epx_c3.ko 2023-10-18T10:30:46.195 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-18T10:30:46.195 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 23809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sc1200wdt.ko 2023-10-18T10:30:46.195 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 23321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-18T10:30:46.196 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/smsc37b787_wdt.ko 2023-10-18T10:30:46.196 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/softdog.ko 2023-10-18T10:30:46.196 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 22273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-18T10:30:46.196 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/twl4030_wdt.ko 2023-10-18T10:30:46.196 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/via_wdt.ko 2023-10-18T10:30:46.196 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-18T10:30:46.197 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-18T10:30:46.197 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 25409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-18T10:30:46.197 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wafer5823wdt.ko 2023-10-18T10:30:46.197 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 28089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-18T10:30:46.197 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wm831x_wdt.ko 2023-10-18T10:30:46.198 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wm8350_wdt.ko 2023-10-18T10:30:46.198 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-18T10:30:46.198 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/ 2023-10-18T10:30:46.198 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 24729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-evtchn.ko 2023-10-18T10:30:46.199 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-18T10:30:46.199 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 47865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-gntdev.ko 2023-10-18T10:30:46.199 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-pciback/ 2023-10-18T10:30:46.199 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 176441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-18T10:30:46.199 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 40121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-privcmd.ko 2023-10-18T10:30:46.199 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 71249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-scsiback.ko 2023-10-18T10:30:46.200 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xenfs/ 2023-10-18T10:30:46.200 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-18T10:30:46.200 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ 2023-10-18T10:30:46.200 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/9p/ 2023-10-18T10:30:46.200 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 166425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/9p/9p.ko 2023-10-18T10:30:46.201 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/adfs/ 2023-10-18T10:30:46.201 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 61073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/adfs/adfs.ko 2023-10-18T10:30:46.201 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/affs/ 2023-10-18T10:30:46.201 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 163593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/affs/affs.ko 2023-10-18T10:30:46.201 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/afs/ 2023-10-18T10:30:46.202 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 932817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/afs/kafs.ko 2023-10-18T10:30:46.202 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/autofs/ 2023-10-18T10:30:46.202 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 101057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/autofs/autofs4.ko 2023-10-18T10:30:46.202 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/befs/ 2023-10-18T10:30:46.202 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 64545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/befs/befs.ko 2023-10-18T10:30:46.203 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/bfs/ 2023-10-18T10:30:46.203 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 41257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/bfs/bfs.ko 2023-10-18T10:30:46.203 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 34585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/binfmt_misc.ko 2023-10-18T10:30:46.203 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/btrfs/ 2023-10-18T10:30:46.203 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 3402585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/btrfs/btrfs.ko 2023-10-18T10:30:46.204 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cachefiles/ 2023-10-18T10:30:46.204 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 306073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cachefiles/cachefiles.ko 2023-10-18T10:30:46.204 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ceph/ 2023-10-18T10:30:46.204 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 1323873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ceph/ceph.ko 2023-10-18T10:30:46.204 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/coda/ 2023-10-18T10:30:46.205 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 100801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/coda/coda.ko 2023-10-18T10:30:46.205 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/configfs/ 2023-10-18T10:30:46.205 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 108393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/configfs/configfs.ko 2023-10-18T10:30:46.205 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cramfs/ 2023-10-18T10:30:46.205 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 29625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cramfs/cramfs.ko 2023-10-18T10:30:46.206 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/dlm/ 2023-10-18T10:30:46.206 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 642577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/dlm/dlm.ko 2023-10-18T10:30:46.206 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/efs/ 2023-10-18T10:30:46.206 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 36649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/efs/efs.ko 2023-10-18T10:30:46.206 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/f2fs/ 2023-10-18T10:30:46.207 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 1505025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/f2fs/f2fs.ko 2023-10-18T10:30:46.207 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/ 2023-10-18T10:30:46.207 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 27929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/msdos.ko 2023-10-18T10:30:46.207 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/freevxfs/ 2023-10-18T10:30:46.207 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 42545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/freevxfs/freevxfs.ko 2023-10-18T10:30:46.208 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fscache/ 2023-10-18T10:30:46.208 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 226481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fscache/fscache.ko 2023-10-18T10:30:46.208 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse/ 2023-10-18T10:30:46.208 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 24489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse/cuse.ko 2023-10-18T10:30:46.208 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/gfs2/ 2023-10-18T10:30:46.208 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 1231529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/gfs2/gfs2.ko 2023-10-18T10:30:46.209 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfs/ 2023-10-18T10:30:46.209 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 141209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfs/hfs.ko 2023-10-18T10:30:46.209 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfsplus/ 2023-10-18T10:30:46.209 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 222817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfsplus/hfsplus.ko 2023-10-18T10:30:46.209 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hpfs/ 2023-10-18T10:30:46.210 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 192329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hpfs/hpfs.ko 2023-10-18T10:30:46.210 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/isofs/ 2023-10-18T10:30:46.210 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 74065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/isofs/isofs.ko 2023-10-18T10:30:46.210 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/jfs/ 2023-10-18T10:30:46.210 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 419497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/jfs/jfs.ko 2023-10-18T10:30:46.211 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/lockd/ 2023-10-18T10:30:46.211 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 280313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/lockd/lockd.ko 2023-10-18T10:30:46.211 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/minix/ 2023-10-18T10:30:46.211 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 89193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/minix/minix.ko 2023-10-18T10:30:46.211 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/netfs/ 2023-10-18T10:30:46.212 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 124649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/netfs/netfs.ko 2023-10-18T10:30:46.212 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/ 2023-10-18T10:30:46.212 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/blocklayout/ 2023-10-18T10:30:46.212 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 81449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-18T10:30:46.212 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/filelayout/ 2023-10-18T10:30:46.213 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 59449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-18T10:30:46.213 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/flexfilelayout/ 2023-10-18T10:30:46.213 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 117513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-18T10:30:46.213 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 1055649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfs.ko 2023-10-18T10:30:46.213 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 66441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv2.ko 2023-10-18T10:30:46.213 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 120177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv3.ko 2023-10-18T10:30:46.214 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 2041689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv4.ko 2023-10-18T10:30:46.214 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/ 2023-10-18T10:30:46.214 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/grace.ko 2023-10-18T10:30:46.214 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-18T10:30:46.214 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfsd/ 2023-10-18T10:30:46.215 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 1516809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfsd/nfsd.ko 2023-10-18T10:30:46.215 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nilfs2/ 2023-10-18T10:30:46.215 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 592833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nilfs2/nilfs2.ko 2023-10-18T10:30:46.215 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/ 2023-10-18T10:30:46.215 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-celtic.ko 2023-10-18T10:30:46.216 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-centeuro.ko 2023-10-18T10:30:46.216 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-croatian.ko 2023-10-18T10:30:46.216 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-cyrillic.ko 2023-10-18T10:30:46.216 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-gaelic.ko 2023-10-18T10:30:46.216 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-greek.ko 2023-10-18T10:30:46.217 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-iceland.ko 2023-10-18T10:30:46.217 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-inuit.ko 2023-10-18T10:30:46.217 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-roman.ko 2023-10-18T10:30:46.217 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-romanian.ko 2023-10-18T10:30:46.217 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-turkish.ko 2023-10-18T10:30:46.218 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_ascii.ko 2023-10-18T10:30:46.218 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1250.ko 2023-10-18T10:30:46.218 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1251.ko 2023-10-18T10:30:46.218 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1255.ko 2023-10-18T10:30:46.218 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp737.ko 2023-10-18T10:30:46.218 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp775.ko 2023-10-18T10:30:46.219 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp850.ko 2023-10-18T10:30:46.219 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp852.ko 2023-10-18T10:30:46.219 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp855.ko 2023-10-18T10:30:46.219 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp857.ko 2023-10-18T10:30:46.220 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp860.ko 2023-10-18T10:30:46.220 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp861.ko 2023-10-18T10:30:46.220 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp862.ko 2023-10-18T10:30:46.220 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp863.ko 2023-10-18T10:30:46.220 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp864.ko 2023-10-18T10:30:46.221 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp865.ko 2023-10-18T10:30:46.221 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp866.ko 2023-10-18T10:30:46.221 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp869.ko 2023-10-18T10:30:46.221 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp874.ko 2023-10-18T10:30:46.221 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 94137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp932.ko 2023-10-18T10:30:46.221 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 142673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp936.ko 2023-10-18T10:30:46.222 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 165745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp949.ko 2023-10-18T10:30:46.222 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 118121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp950.ko 2023-10-18T10:30:46.222 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_euc-jp.ko 2023-10-18T10:30:46.222 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-18T10:30:46.222 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-18T10:30:46.223 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-18T10:30:46.223 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-18T10:30:46.223 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-18T10:30:46.223 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-18T10:30:46.223 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-18T10:30:46.224 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-18T10:30:46.224 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-18T10:30:46.224 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-18T10:30:46.224 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-18T10:30:46.224 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-r.ko 2023-10-18T10:30:46.225 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-18T10:30:46.225 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-u.ko 2023-10-18T10:30:46.225 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 7569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_ucs2_utils.ko 2023-10-18T10:30:46.225 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_utf8.ko 2023-10-18T10:30:46.225 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ntfs/ 2023-10-18T10:30:46.226 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 219417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ntfs/ntfs.ko 2023-10-18T10:30:46.226 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ 2023-10-18T10:30:46.226 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/cluster/ 2023-10-18T10:30:46.226 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 244409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-18T10:30:46.226 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlm/ 2023-10-18T10:30:46.227 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 440737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-18T10:30:46.227 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlmfs/ 2023-10-18T10:30:46.227 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 54049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-18T10:30:46.227 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 2592433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2.ko 2023-10-18T10:30:46.227 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-18T10:30:46.228 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 31729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-18T10:30:46.228 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 35897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-18T10:30:46.228 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/omfs/ 2023-10-18T10:30:46.228 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 53433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/omfs/omfs.ko 2023-10-18T10:30:46.228 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/overlayfs/ 2023-10-18T10:30:46.229 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 352153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/overlayfs/overlay.ko 2023-10-18T10:30:46.229 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/pstore/ 2023-10-18T10:30:46.229 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 54809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/pstore/ramoops.ko 2023-10-18T10:30:46.229 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/qnx4/ 2023-10-18T10:30:46.229 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/qnx4/qnx4.ko 2023-10-18T10:30:46.230 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/qnx6/ 2023-10-18T10:30:46.230 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 46689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/qnx6/qnx6.ko 2023-10-18T10:30:46.230 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/quota/ 2023-10-18T10:30:46.230 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 30881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/quota/quota_tree.ko 2023-10-18T10:30:46.230 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/quota/quota_v1.ko 2023-10-18T10:30:46.231 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 22017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/quota/quota_v2.ko 2023-10-18T10:30:46.231 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/reiserfs/ 2023-10-18T10:30:46.231 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 532673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/reiserfs/reiserfs.ko 2023-10-18T10:30:46.231 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/romfs/ 2023-10-18T10:30:46.232 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 28641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/romfs/romfs.ko 2023-10-18T10:30:46.232 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/ 2023-10-18T10:30:46.232 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/client/ 2023-10-18T10:30:46.232 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 2668121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/client/cifs.ko 2023-10-18T10:30:46.232 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/ 2023-10-18T10:30:46.232 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 7161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-18T10:30:46.233 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/cifs_md4.ko 2023-10-18T10:30:46.233 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/squashfs/ 2023-10-18T10:30:46.233 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 134641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/squashfs/squashfs.ko 2023-10-18T10:30:46.233 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/sysv/ 2023-10-18T10:30:46.233 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 90721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/sysv/sysv.ko 2023-10-18T10:30:46.234 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/udf/ 2023-10-18T10:30:46.234 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 247737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/udf/udf.ko 2023-10-18T10:30:46.234 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ufs/ 2023-10-18T10:30:46.234 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 162497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ufs/ufs.ko 2023-10-18T10:30:46.234 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/xfs/ 2023-10-18T10:30:46.235 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 4010729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/xfs/xfs.ko 2023-10-18T10:30:46.235 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/ 2023-10-18T10:30:46.235 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/locking/ 2023-10-18T10:30:46.235 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 75489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/locking/locktorture.ko 2023-10-18T10:30:46.235 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/time/ 2023-10-18T10:30:46.236 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/time/test_udelay.ko 2023-10-18T10:30:46.236 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 60553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/torture.ko 2023-10-18T10:30:46.236 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ 2023-10-18T10:30:46.236 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 6833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc-itu-t.ko 2023-10-18T10:30:46.236 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 6529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc7.ko 2023-10-18T10:30:46.237 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 7025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc8.ko 2023-10-18T10:30:46.237 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/ 2023-10-18T10:30:46.237 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 7097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/libarc4.ko 2023-10-18T10:30:46.237 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 37081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/libdes.ko 2023-10-18T10:30:46.237 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/interval_tree_test.ko 2023-10-18T10:30:46.238 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/libcrc32c.ko 2023-10-18T10:30:46.238 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 24953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lru_cache.ko 2023-10-18T10:30:46.238 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/ 2023-10-18T10:30:46.238 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 33801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/lz4_compress.ko 2023-10-18T10:30:46.238 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 22633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-18T10:30:46.239 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/math/ 2023-10-18T10:30:46.239 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 6617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/math/cordic.ko 2023-10-18T10:30:46.239 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/memory-notifier-error-inject.ko 2023-10-18T10:30:46.239 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/notifier-error-inject.ko 2023-10-18T10:30:46.239 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/percpu_test.ko 2023-10-18T10:30:46.239 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/pm-notifier-error-inject.ko 2023-10-18T10:30:46.240 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/raid6/ 2023-10-18T10:30:46.240 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 143433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/raid6/raid6_pq.ko 2023-10-18T10:30:46.240 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/rbtree_test.ko 2023-10-18T10:30:46.240 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/reed_solomon/ 2023-10-18T10:30:46.240 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-18T10:30:46.241 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 38841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test-kstrtox.ko 2023-10-18T10:30:46.241 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test-string_helpers.ko 2023-10-18T10:30:46.241 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 4760425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test_bpf.ko 2023-10-18T10:30:46.241 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 49089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test_firmware.ko 2023-10-18T10:30:46.241 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 6369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test_module.ko 2023-10-18T10:30:46.242 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test_user_copy.ko 2023-10-18T10:30:46.242 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_bm.ko 2023-10-18T10:30:46.242 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_fsm.ko 2023-10-18T10:30:46.242 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_kmp.ko 2023-10-18T10:30:46.242 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/xz/ 2023-10-18T10:30:46.243 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/xz/xz_dec_test.ko 2023-10-18T10:30:46.243 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/mm/ 2023-10-18T10:30:46.243 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/mm/hwpoison-inject.ko 2023-10-18T10:30:46.243 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ 2023-10-18T10:30:46.243 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/ 2023-10-18T10:30:46.244 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 55737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/6lowpan.ko 2023-10-18T10:30:46.244 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 7049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_dest.ko 2023-10-18T10:30:46.244 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 7065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-18T10:30:46.244 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 7049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_hop.ko 2023-10-18T10:30:46.244 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 6985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-18T10:30:46.245 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 7065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-18T10:30:46.245 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 7065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_routing.ko 2023-10-18T10:30:46.245 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_udp.ko 2023-10-18T10:30:46.245 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/ 2023-10-18T10:30:46.245 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 30385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/garp.ko 2023-10-18T10:30:46.246 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 34809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/mrp.ko 2023-10-18T10:30:46.246 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/p8022.ko 2023-10-18T10:30:46.246 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/psnap.ko 2023-10-18T10:30:46.246 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/stp.ko 2023-10-18T10:30:46.246 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/8021q/ 2023-10-18T10:30:46.247 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 81737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/8021q/8021q.ko 2023-10-18T10:30:46.247 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/ 2023-10-18T10:30:46.247 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 198337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet.ko 2023-10-18T10:30:46.247 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 35977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_fd.ko 2023-10-18T10:30:46.247 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_rdma.ko 2023-10-18T10:30:46.248 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 32337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_virtio.ko 2023-10-18T10:30:46.248 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/batman-adv/ 2023-10-18T10:30:46.248 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 649297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/batman-adv/batman-adv.ko 2023-10-18T10:30:46.248 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/ 2023-10-18T10:30:46.248 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 63809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/br_netfilter.ko 2023-10-18T10:30:46.248 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 777689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/bridge.ko 2023-10-18T10:30:46.249 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ 2023-10-18T10:30:46.249 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 7657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-18T10:30:46.249 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-18T10:30:46.249 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-18T10:30:46.249 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-18T10:30:46.250 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-18T10:30:46.250 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-18T10:30:46.250 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-18T10:30:46.250 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-18T10:30:46.250 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-18T10:30:46.251 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 7585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-18T10:30:46.251 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 7593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-18T10:30:46.251 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-18T10:30:46.251 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 7537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-18T10:30:46.251 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-18T10:30:46.252 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-18T10:30:46.252 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-18T10:30:46.252 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-18T10:30:46.252 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-18T10:30:46.252 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-18T10:30:46.252 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-18T10:30:46.253 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 55929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-18T10:30:46.253 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-18T10:30:46.253 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-18T10:30:46.253 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ceph/ 2023-10-18T10:30:46.253 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 1047017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ceph/libceph.ko 2023-10-18T10:30:46.254 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core/ 2023-10-18T10:30:46.254 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 125697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core/pktgen.ko 2023-10-18T10:30:46.254 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/ 2023-10-18T10:30:46.254 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 194209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp.ko 2023-10-18T10:30:46.255 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_diag.ko 2023-10-18T10:30:46.255 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 57881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_ipv4.ko 2023-10-18T10:30:46.255 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 60337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_ipv6.ko 2023-10-18T10:30:46.255 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/ 2023-10-18T10:30:46.255 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 390473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/dsa_core.ko 2023-10-18T10:30:46.255 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_brcm.ko 2023-10-18T10:30:46.256 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_dsa.ko 2023-10-18T10:30:46.256 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 7721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_none.ko 2023-10-18T10:30:46.256 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_trailer.ko 2023-10-18T10:30:46.256 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/hsr/ 2023-10-18T10:30:46.256 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 99729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/hsr/hsr.ko 2023-10-18T10:30:46.256 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ 2023-10-18T10:30:46.257 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/6lowpan/ 2023-10-18T10:30:46.257 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 45233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-18T10:30:46.257 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 254625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ieee802154.ko 2023-10-18T10:30:46.257 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 47289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-18T10:30:46.257 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ 2023-10-18T10:30:46.258 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 24729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ah4.ko 2023-10-18T10:30:46.258 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 38905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/esp4.ko 2023-10-18T10:30:46.258 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 48401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/fou.ko 2023-10-18T10:30:46.258 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/gre.ko 2023-10-18T10:30:46.258 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 48657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/inet_diag.ko 2023-10-18T10:30:46.259 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 60633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_gre.ko 2023-10-18T10:30:46.259 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 61225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_tunnel.ko 2023-10-18T10:30:46.259 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 34505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_vti.ko 2023-10-18T10:30:46.259 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ipcomp.ko 2023-10-18T10:30:46.259 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 28841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ipip.ko 2023-10-18T10:30:46.260 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ 2023-10-18T10:30:46.260 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 37641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-18T10:30:46.260 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-18T10:30:46.260 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-18T10:30:46.260 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 41673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ip_tables.ko 2023-10-18T10:30:46.260 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-18T10:30:46.261 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2023-10-18T10:30:46.261 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-18T10:30:46.261 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-18T10:30:46.261 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-18T10:30:46.261 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_filter.ko 2023-10-18T10:30:46.262 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-18T10:30:46.262 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-18T10:30:46.262 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-18T10:30:46.262 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-18T10:30:46.262 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-18T10:30:46.263 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-18T10:30:46.263 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 27625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-18T10:30:46.263 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-18T10:30:46.263 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-18T10:30:46.263 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2023-10-18T10:30:46.263 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-18T10:30:46.264 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-18T10:30:46.264 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-18T10:30:46.264 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-18T10:30:46.264 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-18T10:30:46.264 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_bic.ko 2023-10-18T10:30:46.265 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-18T10:30:46.265 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_diag.ko 2023-10-18T10:30:46.265 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-18T10:30:46.265 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_htcp.ko 2023-10-18T10:30:46.265 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_hybla.ko 2023-10-18T10:30:46.266 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_illinois.ko 2023-10-18T10:30:46.266 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_lp.ko 2023-10-18T10:30:46.266 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_scalable.ko 2023-10-18T10:30:46.266 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_vegas.ko 2023-10-18T10:30:46.267 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_veno.ko 2023-10-18T10:30:46.267 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_westwood.ko 2023-10-18T10:30:46.267 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_yeah.ko 2023-10-18T10:30:46.267 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 22601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tunnel4.ko 2023-10-18T10:30:46.267 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/udp_diag.ko 2023-10-18T10:30:46.268 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 41665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/udp_tunnel.ko 2023-10-18T10:30:46.268 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-18T10:30:46.268 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ 2023-10-18T10:30:46.268 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ah6.ko 2023-10-18T10:30:46.268 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 39537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/esp6.ko 2023-10-18T10:30:46.268 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/fou6.ko 2023-10-18T10:30:46.269 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 80897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_gre.ko 2023-10-18T10:30:46.269 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 93577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-18T10:30:46.269 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-18T10:30:46.269 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 53273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_vti.ko 2023-10-18T10:30:46.269 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ipcomp6.ko 2023-10-18T10:30:46.270 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/mip6.ko 2023-10-18T10:30:46.270 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ 2023-10-18T10:30:46.270 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 42169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-18T10:30:46.270 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2023-10-18T10:30:46.270 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-18T10:30:46.271 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-18T10:30:46.273 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-18T10:30:46.273 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 7801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-18T10:30:46.273 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-18T10:30:46.274 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-18T10:30:46.274 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-18T10:30:46.274 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-18T10:30:46.274 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-18T10:30:46.275 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-18T10:30:46.275 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-18T10:30:46.275 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-18T10:30:46.275 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-18T10:30:46.275 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-18T10:30:46.275 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-18T10:30:46.276 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 39113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-18T10:30:46.276 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-18T10:30:46.276 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 25945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-18T10:30:46.276 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-18T10:30:46.276 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-18T10:30:46.277 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-18T10:30:46.277 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-18T10:30:46.277 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-18T10:30:46.277 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 74073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/sit.ko 2023-10-18T10:30:46.277 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 24193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/tunnel6.ko 2023-10-18T10:30:46.277 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 23297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-18T10:30:46.278 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/key/ 2023-10-18T10:30:46.278 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 82833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/key/af_key.ko 2023-10-18T10:30:46.278 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/ 2023-10-18T10:30:46.278 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 125281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_core.ko 2023-10-18T10:30:46.279 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-18T10:30:46.279 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 25953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_eth.ko 2023-10-18T10:30:46.279 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 35569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ip.ko 2023-10-18T10:30:46.279 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 36713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-18T10:30:46.279 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 33001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-18T10:30:46.280 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 53241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-18T10:30:46.280 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/lapb/ 2023-10-18T10:30:46.280 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 47977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/lapb/lapb.ko 2023-10-18T10:30:46.280 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc/ 2023-10-18T10:30:46.280 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 23193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc/llc.ko 2023-10-18T10:30:46.281 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 267777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc/llc2.ko 2023-10-18T10:30:46.281 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mac802154/ 2023-10-18T10:30:46.281 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 290417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mac802154/mac802154.ko 2023-10-18T10:30:46.281 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mpls/ 2023-10-18T10:30:46.281 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mpls/mpls_gso.ko 2023-10-18T10:30:46.282 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ 2023-10-18T10:30:46.282 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ 2023-10-18T10:30:46.282 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 119585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-18T10:30:46.282 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 28353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-18T10:30:46.282 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-18T10:30:46.283 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-18T10:30:46.283 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 100297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-18T10:30:46.283 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 101441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-18T10:30:46.283 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 102817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-18T10:30:46.283 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 103481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-18T10:30:46.284 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 112849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-18T10:30:46.284 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 60329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-18T10:30:46.284 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 109593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-18T10:30:46.284 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 113129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-18T10:30:46.284 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 112585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-18T10:30:46.285 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 111585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-18T10:30:46.285 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 114465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-18T10:30:46.285 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 30025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-18T10:30:46.285 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ 2023-10-18T10:30:46.285 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 411729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-18T10:30:46.285 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-18T10:30:46.286 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 7625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-18T10:30:46.286 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-18T10:30:46.286 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-18T10:30:46.286 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-18T10:30:46.286 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 7689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-18T10:30:46.287 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 7689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-18T10:30:46.287 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-18T10:30:46.287 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-18T10:30:46.287 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 7697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-18T10:30:46.287 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-18T10:30:46.288 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 7697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-18T10:30:46.288 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-18T10:30:46.288 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 49257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conncount.ko 2023-10-18T10:30:46.288 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 408257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack.ko 2023-10-18T10:30:46.288 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-18T10:30:46.289 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-18T10:30:46.289 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 33761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-18T10:30:46.289 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 118145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-18T10:30:46.289 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 22185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-18T10:30:46.289 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-18T10:30:46.290 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 107297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-18T10:30:46.290 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 33633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-18T10:30:46.290 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-18T10:30:46.290 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 66641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-18T10:30:46.290 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-18T10:30:46.291 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-18T10:30:46.291 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-18T10:30:46.291 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 89473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_flow_table.ko 2023-10-18T10:30:46.291 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-18T10:30:46.291 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 37129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-18T10:30:46.292 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 95417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat.ko 2023-10-18T10:30:46.292 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-18T10:30:46.292 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-18T10:30:46.292 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-18T10:30:46.292 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 28929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-18T10:30:46.293 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-18T10:30:46.293 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 44065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-18T10:30:46.293 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 615313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_tables.ko 2023-10-18T10:30:46.293 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 42561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink.ko 2023-10-18T10:30:46.293 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 30233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-18T10:30:46.294 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 30569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_cttimeout.ko 2023-10-18T10:30:46.294 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 39785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-18T10:30:46.294 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 22921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-18T10:30:46.294 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 49377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-18T10:30:46.294 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-18T10:30:46.295 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 33209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_compat.ko 2023-10-18T10:30:46.295 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 40721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_ct.ko 2023-10-18T10:30:46.295 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-18T10:30:46.295 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_fib.ko 2023-10-18T10:30:46.295 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-18T10:30:46.296 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_hash.ko 2023-10-18T10:30:46.296 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_limit.ko 2023-10-18T10:30:46.296 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_log.ko 2023-10-18T10:30:46.296 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_masq.ko 2023-10-18T10:30:46.296 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_nat.ko 2023-10-18T10:30:46.297 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_numgen.ko 2023-10-18T10:30:46.297 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_osf.ko 2023-10-18T10:30:46.297 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_queue.ko 2023-10-18T10:30:46.297 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_redir.ko 2023-10-18T10:30:46.297 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_reject.ko 2023-10-18T10:30:46.298 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-18T10:30:46.298 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_socket.ko 2023-10-18T10:30:46.298 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_synproxy.ko 2023-10-18T10:30:46.298 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_tproxy.ko 2023-10-18T10:30:46.298 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 89313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/x_tables.ko 2023-10-18T10:30:46.299 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-18T10:30:46.299 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-18T10:30:46.299 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 7729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-18T10:30:46.299 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-18T10:30:46.299 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 18169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CT.ko 2023-10-18T10:30:46.300 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_DSCP.ko 2023-10-18T10:30:46.300 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_HL.ko 2023-10-18T10:30:46.300 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_HMARK.ko 2023-10-18T10:30:46.300 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 38873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-18T10:30:46.300 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_LED.ko 2023-10-18T10:30:46.301 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_LOG.ko 2023-10-18T10:30:46.301 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-18T10:30:46.301 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-18T10:30:46.301 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-18T10:30:46.301 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-18T10:30:46.302 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-18T10:30:46.302 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-18T10:30:46.302 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 14337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-18T10:30:46.302 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-18T10:30:46.302 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-18T10:30:46.303 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TEE.ko 2023-10-18T10:30:46.303 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-18T10:30:46.303 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TRACE.ko 2023-10-18T10:30:46.303 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_addrtype.ko 2023-10-18T10:30:46.304 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_bpf.ko 2023-10-18T10:30:46.304 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cgroup.ko 2023-10-18T10:30:46.304 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cluster.ko 2023-10-18T10:30:46.304 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 7505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_comment.ko 2023-10-18T10:30:46.304 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connbytes.ko 2023-10-18T10:30:46.304 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connlabel.ko 2023-10-18T10:30:46.305 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connlimit.ko 2023-10-18T10:30:46.305 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connmark.ko 2023-10-18T10:30:46.305 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_conntrack.ko 2023-10-18T10:30:46.305 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 7817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cpu.ko 2023-10-18T10:30:46.306 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_dccp.ko 2023-10-18T10:30:46.306 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 7865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_devgroup.ko 2023-10-18T10:30:46.306 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_dscp.ko 2023-10-18T10:30:46.306 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ecn.ko 2023-10-18T10:30:46.306 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_esp.ko 2023-10-18T10:30:46.307 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 43289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-18T10:30:46.307 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_helper.ko 2023-10-18T10:30:46.307 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 7865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_hl.ko 2023-10-18T10:30:46.307 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-18T10:30:46.307 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_iprange.ko 2023-10-18T10:30:46.307 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ipvs.ko 2023-10-18T10:30:46.308 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_l2tp.ko 2023-10-18T10:30:46.308 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_length.ko 2023-10-18T10:30:46.308 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_limit.ko 2023-10-18T10:30:46.308 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 7545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_mac.ko 2023-10-18T10:30:46.308 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_mark.ko 2023-10-18T10:30:46.309 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_multiport.ko 2023-10-18T10:30:46.309 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_nat.ko 2023-10-18T10:30:46.309 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_nfacct.ko 2023-10-18T10:30:46.309 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_osf.ko 2023-10-18T10:30:46.309 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_owner.ko 2023-10-18T10:30:46.310 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_physdev.ko 2023-10-18T10:30:46.310 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 7633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_pkttype.ko 2023-10-18T10:30:46.310 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_policy.ko 2023-10-18T10:30:46.310 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_quota.ko 2023-10-18T10:30:46.311 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_rateest.ko 2023-10-18T10:30:46.311 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_realm.ko 2023-10-18T10:30:46.311 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 36497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_recent.ko 2023-10-18T10:30:46.311 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_sctp.ko 2023-10-18T10:30:46.311 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 30089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_set.ko 2023-10-18T10:30:46.312 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 16089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_socket.ko 2023-10-18T10:30:46.312 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_state.ko 2023-10-18T10:30:46.312 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_statistic.ko 2023-10-18T10:30:46.312 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_string.ko 2023-10-18T10:30:46.312 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 8553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-18T10:30:46.312 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_tcpudp.ko 2023-10-18T10:30:46.313 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_time.ko 2023-10-18T10:30:46.313 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_u32.ko 2023-10-18T10:30:46.313 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netlink/ 2023-10-18T10:30:46.313 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netlink/netlink_diag.ko 2023-10-18T10:30:46.313 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/nsh/ 2023-10-18T10:30:46.314 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 12425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/nsh/nsh.ko 2023-10-18T10:30:46.314 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/ 2023-10-18T10:30:46.314 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 380033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/openvswitch.ko 2023-10-18T10:30:46.314 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-geneve.ko 2023-10-18T10:30:46.314 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-gre.ko 2023-10-18T10:30:46.315 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 11793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-18T10:30:46.315 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/packet/ 2023-10-18T10:30:46.315 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/packet/af_packet_diag.ko 2023-10-18T10:30:46.315 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/phonet/ 2023-10-18T10:30:46.315 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 110201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/phonet/phonet.ko 2023-10-18T10:30:46.316 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 54697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/phonet/pn_pep.ko 2023-10-18T10:30:46.316 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/ 2023-10-18T10:30:46.316 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 251481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds.ko 2023-10-18T10:30:46.316 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 164937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds_rdma.ko 2023-10-18T10:30:46.316 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 67377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds_tcp.ko 2023-10-18T10:30:46.317 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rxrpc/ 2023-10-18T10:30:46.317 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 819657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rxrpc/rxrpc.ko 2023-10-18T10:30:46.317 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/ 2023-10-18T10:30:46.317 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 32241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_csum.ko 2023-10-18T10:30:46.317 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 23577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_gact.ko 2023-10-18T10:30:46.318 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_ipt.ko 2023-10-18T10:30:46.318 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 34097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_mirred.ko 2023-10-18T10:30:46.318 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 23401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_nat.ko 2023-10-18T10:30:46.318 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 43449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_pedit.ko 2023-10-18T10:30:46.318 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 31961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_police.ko 2023-10-18T10:30:46.318 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_simple.ko 2023-10-18T10:30:46.319 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 35297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_skbedit.ko 2023-10-18T10:30:46.319 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 36665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_vlan.ko 2023-10-18T10:30:46.319 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_basic.ko 2023-10-18T10:30:46.319 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 33961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_bpf.ko 2023-10-18T10:30:46.319 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_cgroup.ko 2023-10-18T10:30:46.320 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 31401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_flow.ko 2023-10-18T10:30:46.320 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 31769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_fw.ko 2023-10-18T10:30:46.320 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 38489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_route.ko 2023-10-18T10:30:46.320 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 61409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_u32.ko 2023-10-18T10:30:46.320 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 7441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_cmp.ko 2023-10-18T10:30:46.321 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 13409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_ipset.ko 2023-10-18T10:30:46.321 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 33289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_meta.ko 2023-10-18T10:30:46.321 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 7897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_nbyte.ko 2023-10-18T10:30:46.321 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_text.ko 2023-10-18T10:30:46.322 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 7249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_u32.ko 2023-10-18T10:30:46.322 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 21649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_choke.ko 2023-10-18T10:30:46.322 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_codel.ko 2023-10-18T10:30:46.322 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 34633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_drr.ko 2023-10-18T10:30:46.322 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 33609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_fq.ko 2023-10-18T10:30:46.323 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 31617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_fq_codel.ko 2023-10-18T10:30:46.323 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 35433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_gred.ko 2023-10-18T10:30:46.323 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 49601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_hfsc.ko 2023-10-18T10:30:46.323 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 22089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_hhf.ko 2023-10-18T10:30:46.323 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 66393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_htb.ko 2023-10-18T10:30:46.324 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_ingress.ko 2023-10-18T10:30:46.324 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 30273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_mqprio.ko 2023-10-18T10:30:46.324 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-18T10:30:46.324 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_multiq.ko 2023-10-18T10:30:46.324 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 36137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_netem.ko 2023-10-18T10:30:46.325 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 27497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_pie.ko 2023-10-18T10:30:46.325 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_plug.ko 2023-10-18T10:30:46.325 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 24729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_prio.ko 2023-10-18T10:30:46.325 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 48793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_qfq.ko 2023-10-18T10:30:46.325 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 32905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_red.ko 2023-10-18T10:30:46.326 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 26881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_sfb.ko 2023-10-18T10:30:46.326 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 31353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_sfq.ko 2023-10-18T10:30:46.326 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 28897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_tbf.ko 2023-10-18T10:30:46.326 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 28105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_teql.ko 2023-10-18T10:30:46.326 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/ 2023-10-18T10:30:46.327 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 929745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/sctp.ko 2023-10-18T10:30:46.327 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 19033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/sctp_diag.ko 2023-10-18T10:30:46.327 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/ 2023-10-18T10:30:46.327 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/ 2023-10-18T10:30:46.327 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 336529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-18T10:30:46.328 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 54737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-18T10:30:46.328 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 1520161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/sunrpc.ko 2023-10-18T10:30:46.328 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/xprtrdma/ 2023-10-18T10:30:46.328 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 707721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-18T10:30:46.328 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/ 2023-10-18T10:30:46.329 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/diag.ko 2023-10-18T10:30:46.329 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 862641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/tipc.ko 2023-10-18T10:30:46.329 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/unix/ 2023-10-18T10:30:46.329 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/unix/unix_diag.ko 2023-10-18T10:30:46.329 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/ 2023-10-18T10:30:46.330 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 85337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-18T10:30:46.330 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 68049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-18T10:30:46.330 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 96193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock.ko 2023-10-18T10:30:46.330 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 10705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-18T10:30:46.330 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 17521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-18T10:30:46.331 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/x25/ 2023-10-18T10:30:46.331 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 125177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/x25/x25.ko 2023-10-18T10:30:46.331 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/ 2023-10-18T10:30:46.331 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 22657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/xfrm_algo.ko 2023-10-18T10:30:46.331 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 20649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-18T10:30:46.331 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 102257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/xfrm_user.ko 2023-10-18T10:30:46.332 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/ 2023-10-18T10:30:46.332 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/lib/ 2023-10-18T10:30:46.332 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 15537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/lib/irqbypass.ko 2023-10-18T10:30:46.332 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 639863 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.alias 2023-10-18T10:30:46.333 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 623919 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.alias.bin 2023-10-18T10:30:46.333 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 7663 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.builtin 2023-10-18T10:30:46.333 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 23428 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.alias.bin 2023-10-18T10:30:46.333 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 9821 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.bin 2023-10-18T10:30:46.333 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 70004 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.modinfo 2023-10-18T10:30:46.334 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 146313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.dep 2023-10-18T10:30:46.334 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 232548 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.dep.bin 2023-10-18T10:30:46.334 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 210 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.devname 2023-10-18T10:30:46.334 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 72606 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.order 2023-10-18T10:30:46.334 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 879 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.softdep 2023-10-18T10:30:46.334 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 254438 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.symbols 2023-10-18T10:30:46.335 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 309363 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.symbols.bin 2023-10-18T10:30:46.335 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./usr/ 2023-10-18T10:30:46.335 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./usr/share/ 2023-10-18T10:30:46.335 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./usr/share/doc/ 2023-10-18T10:30:46.335 INFO:teuthology.orchestra.run.smithi154.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./usr/share/doc/linux-image-6.6.0-rc5-g47672119149b/ 2023-10-18T10:30:46.336 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 175 2023-10-10 19:56 ./usr/share/doc/linux-image-6.6.0-rc5-g47672119149b/changelog.Debian.gz 2023-10-18T10:30:46.336 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r-- root/root 691 2023-10-10 19:56 ./usr/share/doc/linux-image-6.6.0-rc5-g47672119149b/copyright 2023-10-18T10:30:46.337 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-g47672119149b 2023-10-18T10:30:46.337 INFO:teuthology.task.kernel:searching for kernel 6.6.0-rc5-g47672119149b 2023-10-18T10:30:46.337 DEBUG:teuthology.orchestra.run.smithi154:> egrep '(submenu|menuentry.*6.6.0-rc5-g47672119149b).*{' /boot/grub/grub.cfg 2023-10-18T10:30:46.344 INFO:teuthology.orchestra.run.smithi154.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-10-18T10:30:46.345 INFO:teuthology.orchestra.run.smithi154.stdout: menuentry 'Ubuntu, with Linux 6.6.0-rc5-g47672119149b' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-rc5-g47672119149b-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-10-18T10:30:46.345 INFO:teuthology.orchestra.run.smithi154.stdout: menuentry 'Ubuntu, with Linux 6.6.0-rc5-g47672119149b (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-rc5-g47672119149b-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-10-18T10:30:46.345 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2023-10-18T10:30:46.346 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.6.0-rc5-g47672119149b 2023-10-18T10:30:46.346 DEBUG:teuthology.orchestra.run.smithi154:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2023-10-18T10:30:46.389 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi154.front.sepia.ceph.com to complete... 2023-10-18T10:30:46.765 INFO:teuthology.orchestra.run.smithi154.stderr:Sourcing file `/etc/default/grub' 2023-10-18T10:30:46.766 INFO:teuthology.orchestra.run.smithi154.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-18T10:30:46.785 INFO:teuthology.orchestra.run.smithi154.stderr:Generating grub configuration file ... 2023-10-18T10:30:46.992 INFO:teuthology.orchestra.run.smithi154.stderr:Found linux image: /boot/vmlinuz-6.6.0-rc5-g47672119149b 2023-10-18T10:30:46.998 INFO:teuthology.orchestra.run.smithi154.stderr:Found initrd image: /boot/initrd.img-6.6.0-rc5-g47672119149b 2023-10-18T10:30:47.209 INFO:teuthology.orchestra.run.smithi154.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-10-18T10:30:47.214 INFO:teuthology.orchestra.run.smithi154.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-10-18T10:30:47.255 INFO:teuthology.orchestra.run.smithi154.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-10-18T10:30:47.260 INFO:teuthology.orchestra.run.smithi154.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-10-18T10:30:47.384 INFO:teuthology.orchestra.run.smithi154.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-10-18T10:30:47.384 INFO:teuthology.orchestra.run.smithi154.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-10-18T10:30:47.384 INFO:teuthology.orchestra.run.smithi154.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-10-18T10:30:47.401 INFO:teuthology.orchestra.run.smithi154.stderr:done 2023-10-18T10:31:17.046 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-10-18T10:31:17.047 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2023-10-18T10:31:17.048 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T10:31:18.504 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-10-18T10:31:18.504 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2023-10-18T10:31:18.505 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T10:31:35.550 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.89 2023-10-18T10:31:36.826 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.154 2023-10-18T10:31:44.559 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2023-10-18T10:31:44.560 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T10:31:45.829 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2023-10-18T10:31:45.829 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T10:31:47.646 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.89 2023-10-18T10:31:48.890 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.154 2023-10-18T10:31:59.654 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2023-10-18T10:31:59.655 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T10:32:00.893 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2023-10-18T10:32:00.894 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T10:32:03.962 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.154 2023-10-18T10:32:18.042 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.89 2023-10-18T10:32:18.964 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2023-10-18T10:32:18.966 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T10:32:22.046 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.154 2023-10-18T10:32:33.054 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2023-10-18T10:32:33.055 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T10:32:33.419 DEBUG:teuthology.orchestra.run.smithi089:> true 2023-10-18T10:32:34.321 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi089.front.sepia.ceph.com' 2023-10-18T10:32:34.321 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.6.0-rc5-g47672119149b"... 2023-10-18T10:32:34.322 DEBUG:teuthology.orchestra.run.smithi089:> uname -r 2023-10-18T10:32:34.372 INFO:teuthology.orchestra.run.smithi089.stdout:6.6.0-rc5-g47672119149b 2023-10-18T10:32:34.372 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-g47672119149b vs 6.6.0-rc5-g47672119149b 2023-10-18T10:32:34.372 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-18T10:32:34.373 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-10-18T10:32:35.374 DEBUG:teuthology.parallel:result is None 2023-10-18T10:32:40.047 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2023-10-18T10:32:40.048 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T10:32:58.490 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.154 2023-10-18T10:33:19.491 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2023-10-18T10:33:19.492 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T10:33:19.721 DEBUG:teuthology.orchestra.run.smithi154:> true 2023-10-18T10:33:20.350 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi154.front.sepia.ceph.com' 2023-10-18T10:33:20.351 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.6.0-rc5-g47672119149b"... 2023-10-18T10:33:20.351 DEBUG:teuthology.orchestra.run.smithi154:> uname -r 2023-10-18T10:33:20.403 INFO:teuthology.orchestra.run.smithi154.stdout:6.6.0-rc5-g47672119149b 2023-10-18T10:33:20.403 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-g47672119149b vs 6.6.0-rc5-g47672119149b 2023-10-18T10:33:20.403 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-18T10:33:20.403 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-10-18T10:33:21.405 DEBUG:teuthology.parallel:result is None 2023-10-18T10:33:21.405 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-18T10:33:21.427 INFO:teuthology.task.internal:Creating test directory... 2023-10-18T10:33:21.427 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-18T10:33:21.431 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-18T10:33:21.440 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-18T10:33:21.448 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-18T10:33:21.458 INFO:teuthology.task.internal:Creating archive directory... 2023-10-18T10:33:21.458 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-18T10:33:21.483 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-18T10:33:21.515 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-18T10:33:21.521 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-18T10:33:21.521 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-18T10:33:21.539 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-18T10:33:21.642 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-18T10:33:21.655 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-18T10:33:21.663 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-18T10:33:21.677 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-18T10:33:21.679 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-18T10:33:21.687 INFO:teuthology.task.internal:Configuring sudo... 2023-10-18T10:33:21.687 DEBUG:teuthology.orchestra.run.smithi089:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-18T10:33:21.691 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-18T10:33:21.713 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-18T10:33:21.723 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-18T10:33:21.723 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-18T10:33:21.754 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-18T10:33:21.764 DEBUG:teuthology.orchestra.run.smithi089:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-18T10:33:21.812 DEBUG:teuthology.orchestra.run.smithi089:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-18T10:33:21.866 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:33:21.867 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-18T10:33:21.951 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-18T10:33:21.961 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-18T10:33:22.012 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T10:33:22.012 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-18T10:33:22.090 DEBUG:teuthology.orchestra.run.smithi089:> sudo service rsyslog restart 2023-10-18T10:33:22.095 DEBUG:teuthology.orchestra.run.smithi154:> sudo service rsyslog restart 2023-10-18T10:33:22.197 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-18T10:33:22.205 INFO:teuthology.task.internal:Starting timer... 2023-10-18T10:33:22.205 INFO:teuthology.run_tasks:Running task pcp... 2023-10-18T10:33:22.270 INFO:teuthology.run_tasks:Running task selinux... 2023-10-18T10:33:22.334 DEBUG:teuthology.task.selinux:Excluding smithi089: OS 'ubuntu' does not support SELinux 2023-10-18T10:33:22.335 DEBUG:teuthology.task.selinux:Excluding smithi154: OS 'ubuntu' does not support SELinux 2023-10-18T10:33:22.335 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-18T10:33:22.335 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-10-18T10:33:22.335 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-18T10:33:22.336 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-18T10:33:22.389 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-10-18T10:33:22.390 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-18T10:33:22.407 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-18T10:33:22.409 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi089.front.sepia.ceph.com,smithi154.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-18T10:42:00.385 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi089.front.sepia.ceph.com'), Remote(name='ubuntu@smithi154.front.sepia.ceph.com')] 2023-10-18T10:42:00.387 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2023-10-18T10:42:00.388 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T10:42:00.466 DEBUG:teuthology.orchestra.run.smithi089:> true 2023-10-18T10:42:00.605 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi089.front.sepia.ceph.com' 2023-10-18T10:42:00.606 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2023-10-18T10:42:00.606 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T10:42:00.684 DEBUG:teuthology.orchestra.run.smithi154:> true 2023-10-18T10:42:00.809 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi154.front.sepia.ceph.com' 2023-10-18T10:42:00.810 INFO:teuthology.run_tasks:Running task clock... 2023-10-18T10:42:00.820 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-18T10:42:00.821 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-18T10:42:00.821 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-18T10:42:00.825 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-18T10:42:00.825 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-18T10:42:00.861 INFO:teuthology.orchestra.run.smithi089.stdout:18 Oct 10:42:00 ntpd[28016]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-10-18T10:42:00.862 INFO:teuthology.orchestra.run.smithi089.stdout:18 Oct 10:42:00 ntpd[28016]: Command line: ntpd -gq 2023-10-18T10:42:00.862 INFO:teuthology.orchestra.run.smithi089.stdout:18 Oct 10:42:00 ntpd[28016]: ---------------------------------------------------- 2023-10-18T10:42:00.862 INFO:teuthology.orchestra.run.smithi089.stdout:18 Oct 10:42:00 ntpd[28016]: ntp-4 is maintained by Network Time Foundation, 2023-10-18T10:42:00.863 INFO:teuthology.orchestra.run.smithi089.stdout:18 Oct 10:42:00 ntpd[28016]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-10-18T10:42:00.863 INFO:teuthology.orchestra.run.smithi089.stdout:18 Oct 10:42:00 ntpd[28016]: corporation. Support and training for ntp-4 are 2023-10-18T10:42:00.863 INFO:teuthology.orchestra.run.smithi089.stdout:18 Oct 10:42:00 ntpd[28016]: available at https://www.nwtime.org/support 2023-10-18T10:42:00.863 INFO:teuthology.orchestra.run.smithi089.stdout:18 Oct 10:42:00 ntpd[28016]: ---------------------------------------------------- 2023-10-18T10:42:00.864 INFO:teuthology.orchestra.run.smithi089.stdout:18 Oct 10:42:00 ntpd[28016]: proto: precision = 0.043 usec (-24) 2023-10-18T10:42:00.864 INFO:teuthology.orchestra.run.smithi089.stdout:18 Oct 10:42:00 ntpd[28016]: basedate set to 2022-02-04 2023-10-18T10:42:00.864 INFO:teuthology.orchestra.run.smithi089.stdout:18 Oct 10:42:00 ntpd[28016]: gps base set to 2022-02-06 (week 2196) 2023-10-18T10:42:00.865 INFO:teuthology.orchestra.run.smithi089.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-18T10:42:00.865 INFO:teuthology.orchestra.run.smithi089.stdout:18 Oct 10:42:00 ntpd[28016]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-18T10:42:00.866 INFO:teuthology.orchestra.run.smithi089.stdout:18 Oct 10:42:00 ntpd[28016]: restrict ::: KOD does nothing without LIMITED. 2023-10-18T10:42:00.866 INFO:teuthology.orchestra.run.smithi089.stderr:restrict ::: KOD does nothing without LIMITED. 2023-10-18T10:42:00.867 INFO:teuthology.orchestra.run.smithi089.stdout:18 Oct 10:42:00 ntpd[28016]: Listen and drop on 0 v6wildcard [::]:123 2023-10-18T10:42:00.867 INFO:teuthology.orchestra.run.smithi089.stdout:18 Oct 10:42:00 ntpd[28016]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-10-18T10:42:00.867 INFO:teuthology.orchestra.run.smithi089.stdout:18 Oct 10:42:00 ntpd[28016]: Listen normally on 2 lo 127.0.0.1:123 2023-10-18T10:42:00.868 INFO:teuthology.orchestra.run.smithi089.stdout:18 Oct 10:42:00 ntpd[28016]: Listen normally on 3 enp3s0f1 172.21.15.89:123 2023-10-18T10:42:00.868 INFO:teuthology.orchestra.run.smithi089.stdout:18 Oct 10:42:00 ntpd[28016]: Listen normally on 4 lo [::1]:123 2023-10-18T10:42:00.868 INFO:teuthology.orchestra.run.smithi089.stdout:18 Oct 10:42:00 ntpd[28016]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d501%5]:123 2023-10-18T10:42:00.868 INFO:teuthology.orchestra.run.smithi089.stdout:18 Oct 10:42:00 ntpd[28016]: Listening on routing socket on fd #22 for interface updates 2023-10-18T10:42:00.896 INFO:teuthology.orchestra.run.smithi154.stdout:18 Oct 10:42:00 ntpd[27979]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-10-18T10:42:00.896 INFO:teuthology.orchestra.run.smithi154.stdout:18 Oct 10:42:00 ntpd[27979]: Command line: ntpd -gq 2023-10-18T10:42:00.897 INFO:teuthology.orchestra.run.smithi154.stdout:18 Oct 10:42:00 ntpd[27979]: ---------------------------------------------------- 2023-10-18T10:42:00.897 INFO:teuthology.orchestra.run.smithi154.stdout:18 Oct 10:42:00 ntpd[27979]: ntp-4 is maintained by Network Time Foundation, 2023-10-18T10:42:00.897 INFO:teuthology.orchestra.run.smithi154.stdout:18 Oct 10:42:00 ntpd[27979]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-10-18T10:42:00.897 INFO:teuthology.orchestra.run.smithi154.stdout:18 Oct 10:42:00 ntpd[27979]: corporation. Support and training for ntp-4 are 2023-10-18T10:42:00.897 INFO:teuthology.orchestra.run.smithi154.stdout:18 Oct 10:42:00 ntpd[27979]: available at https://www.nwtime.org/support 2023-10-18T10:42:00.898 INFO:teuthology.orchestra.run.smithi154.stdout:18 Oct 10:42:00 ntpd[27979]: ---------------------------------------------------- 2023-10-18T10:42:00.898 INFO:teuthology.orchestra.run.smithi154.stdout:18 Oct 10:42:00 ntpd[27979]: proto: precision = 0.044 usec (-24) 2023-10-18T10:42:00.899 INFO:teuthology.orchestra.run.smithi154.stdout:18 Oct 10:42:00 ntpd[27979]: basedate set to 2022-02-04 2023-10-18T10:42:00.899 INFO:teuthology.orchestra.run.smithi154.stdout:18 Oct 10:42:00 ntpd[27979]: gps base set to 2022-02-06 (week 2196) 2023-10-18T10:42:00.899 INFO:teuthology.orchestra.run.smithi154.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-18T10:42:00.900 INFO:teuthology.orchestra.run.smithi154.stdout:18 Oct 10:42:00 ntpd[27979]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-18T10:42:00.900 INFO:teuthology.orchestra.run.smithi154.stdout:18 Oct 10:42:00 ntpd[27979]: restrict ::: KOD does nothing without LIMITED. 2023-10-18T10:42:00.900 INFO:teuthology.orchestra.run.smithi154.stderr:restrict ::: KOD does nothing without LIMITED. 2023-10-18T10:42:00.901 INFO:teuthology.orchestra.run.smithi154.stdout:18 Oct 10:42:00 ntpd[27979]: Listen and drop on 0 v6wildcard [::]:123 2023-10-18T10:42:00.901 INFO:teuthology.orchestra.run.smithi154.stdout:18 Oct 10:42:00 ntpd[27979]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-10-18T10:42:00.902 INFO:teuthology.orchestra.run.smithi154.stdout:18 Oct 10:42:00 ntpd[27979]: Listen normally on 2 lo 127.0.0.1:123 2023-10-18T10:42:00.902 INFO:teuthology.orchestra.run.smithi154.stdout:18 Oct 10:42:00 ntpd[27979]: Listen normally on 3 enp3s0f1 172.21.15.154:123 2023-10-18T10:42:00.902 INFO:teuthology.orchestra.run.smithi154.stdout:18 Oct 10:42:00 ntpd[27979]: Listen normally on 4 lo [::1]:123 2023-10-18T10:42:00.902 INFO:teuthology.orchestra.run.smithi154.stdout:18 Oct 10:42:00 ntpd[27979]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7761%5]:123 2023-10-18T10:42:00.902 INFO:teuthology.orchestra.run.smithi154.stdout:18 Oct 10:42:00 ntpd[27979]: Listening on routing socket on fd #22 for interface updates 2023-10-18T10:42:01.863 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:01 ntpd[28016]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:01.864 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:01 ntpd[28016]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:01.864 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:01 ntpd[28016]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:01.864 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:01 ntpd[28016]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:01.864 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:01 ntpd[28016]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:01.865 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:01 ntpd[28016]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:01.865 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:01 ntpd[28016]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:01.865 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:01 ntpd[28016]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:01.865 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:01 ntpd[28016]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T10:42:01.865 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:01 ntpd[28016]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T10:42:01.866 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:01 ntpd[28016]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:01.866 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:01 ntpd[28016]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:01.866 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:01 ntpd[28016]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T10:42:01.866 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:01 ntpd[28016]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T10:42:01.866 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:01 ntpd[28016]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:01.867 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:01 ntpd[28016]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:01.867 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:01 ntpd[28016]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T10:42:01.867 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:01 ntpd[28016]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T10:42:02.862 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:02 ntpd[28016]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:02.862 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:02 ntpd[28016]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:03.863 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:03 ntpd[28016]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:03.863 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:03 ntpd[28016]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:03.864 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:03 ntpd[28016]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:03.864 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:03 ntpd[28016]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:03.864 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:03 ntpd[28016]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:03.865 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:03 ntpd[28016]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:03.865 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:03 ntpd[28016]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:03.865 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:03 ntpd[28016]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:03.865 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:03 ntpd[28016]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T10:42:03.865 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:03 ntpd[28016]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T10:42:03.865 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:03 ntpd[28016]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:03.866 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:03 ntpd[28016]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:03.866 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:03 ntpd[28016]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T10:42:03.866 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:03 ntpd[28016]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T10:42:03.866 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:03 ntpd[28016]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:03.866 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:03 ntpd[28016]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:03.867 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:03 ntpd[28016]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T10:42:03.867 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:03 ntpd[28016]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T10:42:03.897 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:03 ntpd[27979]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:03.897 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:03 ntpd[27979]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:03.897 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:03 ntpd[27979]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:03.897 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:03 ntpd[27979]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:03.898 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:03 ntpd[27979]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:03.898 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:03 ntpd[27979]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:03.898 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:03 ntpd[27979]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:03.898 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:03 ntpd[27979]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:03.898 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:03 ntpd[27979]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T10:42:03.899 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:03 ntpd[27979]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T10:42:03.899 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:03 ntpd[27979]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:03.899 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:03 ntpd[27979]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:03.899 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:03 ntpd[27979]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T10:42:03.899 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:03 ntpd[27979]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T10:42:05.864 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:05 ntpd[28016]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:05.864 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:05 ntpd[28016]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:05.864 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:05 ntpd[28016]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:05.864 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:05 ntpd[28016]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:05.864 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:05 ntpd[28016]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:05.865 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:05 ntpd[28016]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:05.865 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:05 ntpd[28016]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:05.865 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:05 ntpd[28016]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:05.865 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:05 ntpd[28016]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T10:42:05.866 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:05 ntpd[28016]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T10:42:05.866 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:05 ntpd[28016]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:05.866 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:05 ntpd[28016]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:05.866 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:05 ntpd[28016]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T10:42:05.866 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:05 ntpd[28016]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T10:42:05.867 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:05 ntpd[28016]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:05.867 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:05 ntpd[28016]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:05.867 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:05 ntpd[28016]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T10:42:05.867 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:05 ntpd[28016]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T10:42:05.897 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:05 ntpd[27979]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:05.898 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:05 ntpd[27979]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:05.898 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:05 ntpd[27979]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:05.898 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:05 ntpd[27979]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:05.898 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:05 ntpd[27979]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:05.899 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:05 ntpd[27979]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:05.899 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:05 ntpd[27979]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:05.899 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:05 ntpd[27979]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:05.899 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:05 ntpd[27979]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T10:42:05.899 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:05 ntpd[27979]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T10:42:05.900 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:05 ntpd[27979]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:05.900 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:05 ntpd[27979]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:05.900 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:05 ntpd[27979]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T10:42:05.900 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:05 ntpd[27979]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T10:42:05.900 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:05 ntpd[27979]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:05.901 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:05 ntpd[27979]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:05.901 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:05 ntpd[27979]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T10:42:05.901 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:05 ntpd[27979]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T10:42:07.863 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:07 ntpd[28016]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:07.863 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:07 ntpd[28016]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:07.863 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:07 ntpd[28016]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:07.864 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:07 ntpd[28016]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:07.864 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:07 ntpd[28016]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:07.864 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:07 ntpd[28016]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:07.864 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:07 ntpd[28016]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:07.864 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:07 ntpd[28016]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:07.865 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:07 ntpd[28016]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T10:42:07.865 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:07 ntpd[28016]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T10:42:07.866 INFO:teuthology.orchestra.run.smithi089.stdout:18 Oct 10:42:07 ntpd[28016]: ntpd: time slew +0.000190 s 2023-10-18T10:42:07.866 INFO:teuthology.orchestra.run.smithi089.stdout:ntpd: time slew +0.000190s 2023-10-18T10:42:07.866 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:07 ntpd[28016]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-10-18T10:42:07.867 INFO:teuthology.orchestra.run.smithi089.stderr:18 Oct 10:42:07 ntpd[28016]: can't open /var/log/ntpstats/loopstats.20231018: Permission denied 2023-10-18T10:42:07.897 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:07 ntpd[27979]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:07.898 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:07 ntpd[27979]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:07.898 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:07 ntpd[27979]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:07.898 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:07 ntpd[27979]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:07.898 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:07 ntpd[27979]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:07.899 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:07 ntpd[27979]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:07.899 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:07 ntpd[27979]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:07.899 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:07 ntpd[27979]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:07.899 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:07 ntpd[27979]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T10:42:07.900 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:07 ntpd[27979]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T10:42:07.900 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:07 ntpd[27979]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:07.900 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:07 ntpd[27979]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:07.900 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:07 ntpd[27979]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T10:42:07.900 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:07 ntpd[27979]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T10:42:07.901 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:07 ntpd[27979]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:07.901 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:07 ntpd[27979]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:07.901 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:07 ntpd[27979]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T10:42:07.901 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:07 ntpd[27979]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T10:42:07.975 INFO:teuthology.orchestra.run.smithi089.stdout: remote refid st t when poll reach delay offset jitter 2023-10-18T10:42:07.975 INFO:teuthology.orchestra.run.smithi089.stdout:============================================================================== 2023-10-18T10:42:07.976 INFO:teuthology.orchestra.run.smithi089.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-18T10:42:07.976 INFO:teuthology.orchestra.run.smithi089.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-18T10:42:07.977 INFO:teuthology.orchestra.run.smithi089.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-18T10:42:07.977 INFO:teuthology.orchestra.run.smithi089.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-18T10:42:09.898 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:09 ntpd[27979]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:09.898 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:09 ntpd[27979]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:09.898 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:09 ntpd[27979]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:09.898 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:09 ntpd[27979]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:09.899 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:09 ntpd[27979]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:09.899 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:09 ntpd[27979]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:09.899 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:09 ntpd[27979]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:09.899 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:09 ntpd[27979]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:09.899 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:09 ntpd[27979]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T10:42:09.900 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:09 ntpd[27979]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T10:42:09.900 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:09 ntpd[27979]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T10:42:09.900 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:09 ntpd[27979]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T10:42:09.900 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:09 ntpd[27979]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T10:42:09.900 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:09 ntpd[27979]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T10:42:09.901 INFO:teuthology.orchestra.run.smithi154.stdout:18 Oct 10:42:09 ntpd[27979]: ntpd: time slew -0.001159 s 2023-10-18T10:42:09.901 INFO:teuthology.orchestra.run.smithi154.stdout:ntpd: time slew -0.001159s 2023-10-18T10:42:09.902 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:09 ntpd[27979]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-10-18T10:42:09.902 INFO:teuthology.orchestra.run.smithi154.stderr:18 Oct 10:42:09 ntpd[27979]: can't open /var/log/ntpstats/loopstats.20231018: Permission denied 2023-10-18T10:42:10.000 INFO:teuthology.orchestra.run.smithi154.stdout: remote refid st t when poll reach delay offset jitter 2023-10-18T10:42:10.000 INFO:teuthology.orchestra.run.smithi154.stdout:============================================================================== 2023-10-18T10:42:10.000 INFO:teuthology.orchestra.run.smithi154.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-18T10:42:10.001 INFO:teuthology.orchestra.run.smithi154.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-18T10:42:10.001 INFO:teuthology.orchestra.run.smithi154.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-18T10:42:10.001 INFO:teuthology.orchestra.run.smithi154.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-18T10:42:10.002 INFO:teuthology.run_tasks:Running task install... 2023-10-18T10:42:10.011 DEBUG:teuthology.task.install:project ceph 2023-10-18T10:42:10.012 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '6f3e6b48891b9b50415c0822b6b870797fcda16d'}} 2023-10-18T10:42:10.012 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '6f3e6b48891b9b50415c0822b6b870797fcda16d'} 2023-10-18T10:42:10.012 INFO:teuthology.task.install:Using flavor: default 2023-10-18T10:42:10.019 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-18T10:42:10.019 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-18T10:42:10.020 DEBUG:teuthology.orchestra.run.smithi089:> sudo apt-key list | grep Ceph 2023-10-18T10:42:10.020 DEBUG:teuthology.orchestra.run.smithi154:> sudo apt-key list | grep Ceph 2023-10-18T10:42:10.129 INFO:teuthology.orchestra.run.smithi089.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-10-18T10:42:10.153 INFO:teuthology.orchestra.run.smithi154.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-10-18T10:42:10.182 INFO:teuthology.orchestra.run.smithi089.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-10-18T10:42:10.183 INFO:teuthology.orchestra.run.smithi089.stdout:uid [ unknown] Ceph.com (release key) 2023-10-18T10:42:10.184 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-10-18T10:42:10.184 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-10-18T10:42:10.185 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=6f3e6b48891b9b50415c0822b6b870797fcda16d 2023-10-18T10:42:10.206 INFO:teuthology.orchestra.run.smithi154.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-10-18T10:42:10.206 INFO:teuthology.orchestra.run.smithi154.stdout:uid [ unknown] Ceph.com (release key) 2023-10-18T10:42:10.207 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-10-18T10:42:10.207 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-10-18T10:42:10.208 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=6f3e6b48891b9b50415c0822b6b870797fcda16d 2023-10-18T10:42:10.412 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default/ 2023-10-18T10:42:10.413 INFO:teuthology.task.install.deb:Package version is 18.0.0-5610-g6f3e6b48-1jammy 2023-10-18T10:42:10.429 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default/ 2023-10-18T10:42:10.429 INFO:teuthology.task.install.deb:Package version is 18.0.0-5610-g6f3e6b48-1jammy 2023-10-18T10:42:10.523 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:42:10.523 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-10-18T10:42:10.536 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T10:42:10.537 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-10-18T10:42:10.544 DEBUG:teuthology.orchestra.run.smithi089:> sudo apt-get update 2023-10-18T10:42:10.557 DEBUG:teuthology.orchestra.run.smithi154:> sudo apt-get update 2023-10-18T10:42:10.739 INFO:teuthology.orchestra.run.smithi089.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-10-18T10:42:10.746 INFO:teuthology.orchestra.run.smithi089.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-10-18T10:42:10.749 INFO:teuthology.orchestra.run.smithi089.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2023-10-18T10:42:10.749 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-10-18T10:42:10.749 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-10-18T10:42:10.756 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-10-18T10:42:10.772 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-10-18T10:42:10.838 INFO:teuthology.orchestra.run.smithi089.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2023-10-18T10:42:10.849 INFO:teuthology.orchestra.run.smithi089.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy InRelease 2023-10-18T10:42:10.858 INFO:teuthology.orchestra.run.smithi154.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy InRelease 2023-10-18T10:42:10.900 INFO:teuthology.orchestra.run.smithi154.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2023-10-18T10:42:10.901 INFO:teuthology.orchestra.run.smithi089.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2023-10-18T10:42:10.938 INFO:teuthology.orchestra.run.smithi154.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy Release.gpg 2023-10-18T10:42:10.939 INFO:teuthology.orchestra.run.smithi089.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy Release.gpg 2023-10-18T10:42:10.988 INFO:teuthology.orchestra.run.smithi154.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,426 B] 2023-10-18T10:42:11.002 INFO:teuthology.orchestra.run.smithi089.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.5 kB] 2023-10-18T10:42:11.036 INFO:teuthology.orchestra.run.smithi154.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.5 kB] 2023-10-18T10:42:11.086 INFO:teuthology.orchestra.run.smithi089.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,426 B] 2023-10-18T10:42:11.352 INFO:teuthology.orchestra.run.smithi089.stdout:Fetched 256 kB in 1s (440 kB/s) 2023-10-18T10:42:11.356 INFO:teuthology.orchestra.run.smithi154.stdout:Fetched 28.6 kB in 1s (49.8 kB/s) 2023-10-18T10:42:12.248 INFO:teuthology.orchestra.run.smithi089.stdout:Reading package lists... 2023-10-18T10:42:12.255 INFO:teuthology.orchestra.run.smithi154.stdout:Reading package lists... 2023-10-18T10:42:12.265 DEBUG:teuthology.orchestra.run.smithi089:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-5610-g6f3e6b48-1jammy cephadm=18.0.0-5610-g6f3e6b48-1jammy ceph-mds=18.0.0-5610-g6f3e6b48-1jammy ceph-mgr=18.0.0-5610-g6f3e6b48-1jammy ceph-common=18.0.0-5610-g6f3e6b48-1jammy ceph-fuse=18.0.0-5610-g6f3e6b48-1jammy ceph-test=18.0.0-5610-g6f3e6b48-1jammy ceph-volume=18.0.0-5610-g6f3e6b48-1jammy radosgw=18.0.0-5610-g6f3e6b48-1jammy python3-rados=18.0.0-5610-g6f3e6b48-1jammy python3-rgw=18.0.0-5610-g6f3e6b48-1jammy python3-cephfs=18.0.0-5610-g6f3e6b48-1jammy python3-rbd=18.0.0-5610-g6f3e6b48-1jammy libcephfs2=18.0.0-5610-g6f3e6b48-1jammy libcephfs-dev=18.0.0-5610-g6f3e6b48-1jammy librados2=18.0.0-5610-g6f3e6b48-1jammy librbd1=18.0.0-5610-g6f3e6b48-1jammy rbd-fuse=18.0.0-5610-g6f3e6b48-1jammy python3-cephfs=18.0.0-5610-g6f3e6b48-1jammy cephfs-shell=18.0.0-5610-g6f3e6b48-1jammy cephfs-top=18.0.0-5610-g6f3e6b48-1jammy cephfs-mirror=18.0.0-5610-g6f3e6b48-1jammy 2023-10-18T10:42:12.273 DEBUG:teuthology.orchestra.run.smithi154:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-5610-g6f3e6b48-1jammy cephadm=18.0.0-5610-g6f3e6b48-1jammy ceph-mds=18.0.0-5610-g6f3e6b48-1jammy ceph-mgr=18.0.0-5610-g6f3e6b48-1jammy ceph-common=18.0.0-5610-g6f3e6b48-1jammy ceph-fuse=18.0.0-5610-g6f3e6b48-1jammy ceph-test=18.0.0-5610-g6f3e6b48-1jammy ceph-volume=18.0.0-5610-g6f3e6b48-1jammy radosgw=18.0.0-5610-g6f3e6b48-1jammy python3-rados=18.0.0-5610-g6f3e6b48-1jammy python3-rgw=18.0.0-5610-g6f3e6b48-1jammy python3-cephfs=18.0.0-5610-g6f3e6b48-1jammy python3-rbd=18.0.0-5610-g6f3e6b48-1jammy libcephfs2=18.0.0-5610-g6f3e6b48-1jammy libcephfs-dev=18.0.0-5610-g6f3e6b48-1jammy librados2=18.0.0-5610-g6f3e6b48-1jammy librbd1=18.0.0-5610-g6f3e6b48-1jammy rbd-fuse=18.0.0-5610-g6f3e6b48-1jammy python3-cephfs=18.0.0-5610-g6f3e6b48-1jammy cephfs-shell=18.0.0-5610-g6f3e6b48-1jammy cephfs-top=18.0.0-5610-g6f3e6b48-1jammy cephfs-mirror=18.0.0-5610-g6f3e6b48-1jammy 2023-10-18T10:42:12.340 INFO:teuthology.orchestra.run.smithi089.stdout:Reading package lists... 2023-10-18T10:42:12.347 INFO:teuthology.orchestra.run.smithi154.stdout:Reading package lists... 2023-10-18T10:42:12.503 INFO:teuthology.orchestra.run.smithi089.stdout:Building dependency tree... 2023-10-18T10:42:12.504 INFO:teuthology.orchestra.run.smithi089.stdout:Reading state information... 2023-10-18T10:42:12.513 INFO:teuthology.orchestra.run.smithi154.stdout:Building dependency tree... 2023-10-18T10:42:12.513 INFO:teuthology.orchestra.run.smithi154.stdout:Reading state information... 2023-10-18T10:42:12.654 INFO:teuthology.orchestra.run.smithi089.stdout:The following packages were automatically installed and are no longer required: 2023-10-18T10:42:12.654 INFO:teuthology.orchestra.run.smithi089.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-18T10:42:12.655 INFO:teuthology.orchestra.run.smithi089.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-18T10:42:12.655 INFO:teuthology.orchestra.run.smithi089.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-18T10:42:12.655 INFO:teuthology.orchestra.run.smithi089.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-10-18T10:42:12.657 INFO:teuthology.orchestra.run.smithi089.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-10-18T10:42:12.657 INFO:teuthology.orchestra.run.smithi089.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-18T10:42:12.657 INFO:teuthology.orchestra.run.smithi089.stdout:The following additional packages will be installed: 2023-10-18T10:42:12.657 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-10-18T10:42:12.658 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-10-18T10:42:12.658 INFO:teuthology.orchestra.run.smithi089.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-10-18T10:42:12.658 INFO:teuthology.orchestra.run.smithi089.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2023-10-18T10:42:12.658 INFO:teuthology.orchestra.run.smithi089.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2023-10-18T10:42:12.658 INFO:teuthology.orchestra.run.smithi089.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2023-10-18T10:42:12.659 INFO:teuthology.orchestra.run.smithi089.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2023-10-18T10:42:12.659 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2023-10-18T10:42:12.659 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2023-10-18T10:42:12.659 INFO:teuthology.orchestra.run.smithi089.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2023-10-18T10:42:12.659 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-10-18T10:42:12.660 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-10-18T10:42:12.660 INFO:teuthology.orchestra.run.smithi089.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-10-18T10:42:12.660 INFO:teuthology.orchestra.run.smithi089.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-10-18T10:42:12.660 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-10-18T10:42:12.661 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-10-18T10:42:12.661 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2023-10-18T10:42:12.661 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2023-10-18T10:42:12.661 INFO:teuthology.orchestra.run.smithi089.stdout: python3-simplejson python3-singledispatch python3-sklearn 2023-10-18T10:42:12.661 INFO:teuthology.orchestra.run.smithi089.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2023-10-18T10:42:12.662 INFO:teuthology.orchestra.run.smithi089.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2023-10-18T10:42:12.662 INFO:teuthology.orchestra.run.smithi089.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2023-10-18T10:42:12.662 INFO:teuthology.orchestra.run.smithi089.stdout: qttranslations5-l10n socat xmlstarlet 2023-10-18T10:42:12.663 INFO:teuthology.orchestra.run.smithi089.stdout:Suggested packages: 2023-10-18T10:42:12.663 INFO:teuthology.orchestra.run.smithi089.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2023-10-18T10:42:12.663 INFO:teuthology.orchestra.run.smithi089.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2023-10-18T10:42:12.663 INFO:teuthology.orchestra.run.smithi089.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2023-10-18T10:42:12.664 INFO:teuthology.orchestra.run.smithi089.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2023-10-18T10:42:12.664 INFO:teuthology.orchestra.run.smithi089.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2023-10-18T10:42:12.664 INFO:teuthology.orchestra.run.smithi089.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2023-10-18T10:42:12.664 INFO:teuthology.orchestra.run.smithi089.stdout: python3-watchdog 2023-10-18T10:42:12.664 INFO:teuthology.orchestra.run.smithi089.stdout:Recommended packages: 2023-10-18T10:42:12.665 INFO:teuthology.orchestra.run.smithi089.stdout: btrfs-tools 2023-10-18T10:42:12.666 INFO:teuthology.orchestra.run.smithi154.stdout:The following packages were automatically installed and are no longer required: 2023-10-18T10:42:12.666 INFO:teuthology.orchestra.run.smithi154.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-18T10:42:12.666 INFO:teuthology.orchestra.run.smithi154.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-18T10:42:12.667 INFO:teuthology.orchestra.run.smithi154.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-18T10:42:12.668 INFO:teuthology.orchestra.run.smithi154.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-10-18T10:42:12.668 INFO:teuthology.orchestra.run.smithi154.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-10-18T10:42:12.668 INFO:teuthology.orchestra.run.smithi154.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-18T10:42:12.668 INFO:teuthology.orchestra.run.smithi154.stdout:The following additional packages will be installed: 2023-10-18T10:42:12.669 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-10-18T10:42:12.669 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-10-18T10:42:12.669 INFO:teuthology.orchestra.run.smithi154.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-10-18T10:42:12.669 INFO:teuthology.orchestra.run.smithi154.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2023-10-18T10:42:12.671 INFO:teuthology.orchestra.run.smithi154.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2023-10-18T10:42:12.671 INFO:teuthology.orchestra.run.smithi154.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2023-10-18T10:42:12.671 INFO:teuthology.orchestra.run.smithi154.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2023-10-18T10:42:12.671 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2023-10-18T10:42:12.671 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2023-10-18T10:42:12.672 INFO:teuthology.orchestra.run.smithi154.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2023-10-18T10:42:12.672 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-10-18T10:42:12.672 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-10-18T10:42:12.672 INFO:teuthology.orchestra.run.smithi154.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-10-18T10:42:12.673 INFO:teuthology.orchestra.run.smithi154.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-10-18T10:42:12.673 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-10-18T10:42:12.673 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-10-18T10:42:12.673 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2023-10-18T10:42:12.673 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2023-10-18T10:42:12.673 INFO:teuthology.orchestra.run.smithi154.stdout: python3-simplejson python3-singledispatch python3-sklearn 2023-10-18T10:42:12.674 INFO:teuthology.orchestra.run.smithi154.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2023-10-18T10:42:12.674 INFO:teuthology.orchestra.run.smithi154.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2023-10-18T10:42:12.674 INFO:teuthology.orchestra.run.smithi154.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2023-10-18T10:42:12.674 INFO:teuthology.orchestra.run.smithi154.stdout: qttranslations5-l10n socat xmlstarlet 2023-10-18T10:42:12.674 INFO:teuthology.orchestra.run.smithi154.stdout:Suggested packages: 2023-10-18T10:42:12.675 INFO:teuthology.orchestra.run.smithi154.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2023-10-18T10:42:12.675 INFO:teuthology.orchestra.run.smithi154.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2023-10-18T10:42:12.675 INFO:teuthology.orchestra.run.smithi154.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2023-10-18T10:42:12.675 INFO:teuthology.orchestra.run.smithi154.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2023-10-18T10:42:12.675 INFO:teuthology.orchestra.run.smithi154.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2023-10-18T10:42:12.675 INFO:teuthology.orchestra.run.smithi154.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2023-10-18T10:42:12.676 INFO:teuthology.orchestra.run.smithi154.stdout: python3-watchdog 2023-10-18T10:42:12.676 INFO:teuthology.orchestra.run.smithi154.stdout:Recommended packages: 2023-10-18T10:42:12.676 INFO:teuthology.orchestra.run.smithi154.stdout: btrfs-tools 2023-10-18T10:42:12.799 INFO:teuthology.orchestra.run.smithi089.stdout:The following NEW packages will be installed: 2023-10-18T10:42:12.800 INFO:teuthology.orchestra.run.smithi089.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-10-18T10:42:12.800 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-10-18T10:42:12.800 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-10-18T10:42:12.800 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2023-10-18T10:42:12.801 INFO:teuthology.orchestra.run.smithi089.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-10-18T10:42:12.801 INFO:teuthology.orchestra.run.smithi089.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2023-10-18T10:42:12.801 INFO:teuthology.orchestra.run.smithi089.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2023-10-18T10:42:12.802 INFO:teuthology.orchestra.run.smithi089.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2023-10-18T10:42:12.802 INFO:teuthology.orchestra.run.smithi089.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2023-10-18T10:42:12.803 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-10-18T10:42:12.803 INFO:teuthology.orchestra.run.smithi089.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2023-10-18T10:42:12.803 INFO:teuthology.orchestra.run.smithi089.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2023-10-18T10:42:12.803 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2023-10-18T10:42:12.804 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2023-10-18T10:42:12.804 INFO:teuthology.orchestra.run.smithi089.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2023-10-18T10:42:12.804 INFO:teuthology.orchestra.run.smithi089.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2023-10-18T10:42:12.804 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2023-10-18T10:42:12.804 INFO:teuthology.orchestra.run.smithi089.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-10-18T10:42:12.805 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2023-10-18T10:42:12.805 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2023-10-18T10:42:12.805 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-10-18T10:42:12.805 INFO:teuthology.orchestra.run.smithi089.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-10-18T10:42:12.805 INFO:teuthology.orchestra.run.smithi089.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2023-10-18T10:42:12.806 INFO:teuthology.orchestra.run.smithi089.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2023-10-18T10:42:12.806 INFO:teuthology.orchestra.run.smithi089.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2023-10-18T10:42:12.806 INFO:teuthology.orchestra.run.smithi089.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2023-10-18T10:42:12.806 INFO:teuthology.orchestra.run.smithi089.stdout: socat xmlstarlet 2023-10-18T10:42:12.813 INFO:teuthology.orchestra.run.smithi154.stdout:The following NEW packages will be installed: 2023-10-18T10:42:12.813 INFO:teuthology.orchestra.run.smithi154.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-10-18T10:42:12.813 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-10-18T10:42:12.813 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-10-18T10:42:12.815 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2023-10-18T10:42:12.815 INFO:teuthology.orchestra.run.smithi154.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-10-18T10:42:12.816 INFO:teuthology.orchestra.run.smithi154.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2023-10-18T10:42:12.816 INFO:teuthology.orchestra.run.smithi154.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2023-10-18T10:42:12.816 INFO:teuthology.orchestra.run.smithi154.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2023-10-18T10:42:12.816 INFO:teuthology.orchestra.run.smithi154.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2023-10-18T10:42:12.816 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-10-18T10:42:12.817 INFO:teuthology.orchestra.run.smithi154.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2023-10-18T10:42:12.817 INFO:teuthology.orchestra.run.smithi154.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2023-10-18T10:42:12.817 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2023-10-18T10:42:12.817 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2023-10-18T10:42:12.817 INFO:teuthology.orchestra.run.smithi154.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2023-10-18T10:42:12.818 INFO:teuthology.orchestra.run.smithi154.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2023-10-18T10:42:12.818 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2023-10-18T10:42:12.818 INFO:teuthology.orchestra.run.smithi154.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-10-18T10:42:12.818 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2023-10-18T10:42:12.818 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2023-10-18T10:42:12.819 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-10-18T10:42:12.819 INFO:teuthology.orchestra.run.smithi154.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-10-18T10:42:12.819 INFO:teuthology.orchestra.run.smithi154.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2023-10-18T10:42:12.819 INFO:teuthology.orchestra.run.smithi154.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2023-10-18T10:42:12.819 INFO:teuthology.orchestra.run.smithi154.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2023-10-18T10:42:12.820 INFO:teuthology.orchestra.run.smithi154.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2023-10-18T10:42:12.820 INFO:teuthology.orchestra.run.smithi154.stdout: socat xmlstarlet 2023-10-18T10:42:12.869 INFO:teuthology.orchestra.run.smithi089.stdout:0 upgraded, 115 newly installed, 0 to remove and 260 not upgraded. 2023-10-18T10:42:12.869 INFO:teuthology.orchestra.run.smithi089.stdout:Need to get 183 MB of archives. 2023-10-18T10:42:12.869 INFO:teuthology.orchestra.run.smithi089.stdout:After this operation, 823 MB of additional disk space will be used. 2023-10-18T10:42:12.870 INFO:teuthology.orchestra.run.smithi089.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2023-10-18T10:42:12.889 INFO:teuthology.orchestra.run.smithi089.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-10-18T10:42:12.898 INFO:teuthology.orchestra.run.smithi089.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2023-10-18T10:42:12.945 INFO:teuthology.orchestra.run.smithi089.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2023-10-18T10:42:12.953 INFO:teuthology.orchestra.run.smithi089.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-10-18T10:42:12.972 INFO:teuthology.orchestra.run.smithi089.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-10-18T10:42:12.974 INFO:teuthology.orchestra.run.smithi089.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-10-18T10:42:12.983 INFO:teuthology.orchestra.run.smithi089.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-10-18T10:42:13.010 INFO:teuthology.orchestra.run.smithi154.stdout:0 upgraded, 115 newly installed, 0 to remove and 260 not upgraded. 2023-10-18T10:42:13.010 INFO:teuthology.orchestra.run.smithi154.stdout:Need to get 183 MB of archives. 2023-10-18T10:42:13.011 INFO:teuthology.orchestra.run.smithi154.stdout:After this operation, 823 MB of additional disk space will be used. 2023-10-18T10:42:13.011 INFO:teuthology.orchestra.run.smithi154.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2023-10-18T10:42:13.022 INFO:teuthology.orchestra.run.smithi089.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-10-18T10:42:13.024 INFO:teuthology.orchestra.run.smithi089.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-10-18T10:42:13.032 INFO:teuthology.orchestra.run.smithi089.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-10-18T10:42:13.035 INFO:teuthology.orchestra.run.smithi089.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-10-18T10:42:13.035 INFO:teuthology.orchestra.run.smithi089.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-10-18T10:42:13.041 INFO:teuthology.orchestra.run.smithi089.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-10-18T10:42:13.042 INFO:teuthology.orchestra.run.smithi089.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-10-18T10:42:13.042 INFO:teuthology.orchestra.run.smithi089.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-10-18T10:42:13.043 INFO:teuthology.orchestra.run.smithi089.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-10-18T10:42:13.044 INFO:teuthology.orchestra.run.smithi089.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-10-18T10:42:13.057 INFO:teuthology.orchestra.run.smithi089.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-10-18T10:42:13.058 INFO:teuthology.orchestra.run.smithi089.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-10-18T10:42:13.058 INFO:teuthology.orchestra.run.smithi089.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-10-18T10:42:13.059 INFO:teuthology.orchestra.run.smithi089.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-10-18T10:42:13.074 INFO:teuthology.orchestra.run.smithi089.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-10-18T10:42:13.076 INFO:teuthology.orchestra.run.smithi089.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-10-18T10:42:13.077 INFO:teuthology.orchestra.run.smithi089.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-10-18T10:42:13.078 INFO:teuthology.orchestra.run.smithi089.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-10-18T10:42:13.078 INFO:teuthology.orchestra.run.smithi089.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-10-18T10:42:13.079 INFO:teuthology.orchestra.run.smithi089.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-10-18T10:42:13.091 INFO:teuthology.orchestra.run.smithi089.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-10-18T10:42:13.092 INFO:teuthology.orchestra.run.smithi089.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-10-18T10:42:13.092 INFO:teuthology.orchestra.run.smithi089.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-10-18T10:42:13.093 INFO:teuthology.orchestra.run.smithi154.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-10-18T10:42:13.093 INFO:teuthology.orchestra.run.smithi089.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-10-18T10:42:13.108 INFO:teuthology.orchestra.run.smithi089.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-10-18T10:42:13.113 INFO:teuthology.orchestra.run.smithi089.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-10-18T10:42:13.113 INFO:teuthology.orchestra.run.smithi089.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-10-18T10:42:13.113 INFO:teuthology.orchestra.run.smithi089.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-10-18T10:42:13.114 INFO:teuthology.orchestra.run.smithi089.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-10-18T10:42:13.115 INFO:teuthology.orchestra.run.smithi089.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-10-18T10:42:13.125 INFO:teuthology.orchestra.run.smithi089.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-10-18T10:42:13.125 INFO:teuthology.orchestra.run.smithi089.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-urllib3 all 1.26.5-1~exp1 [96.4 kB] 2023-10-18T10:42:13.127 INFO:teuthology.orchestra.run.smithi089.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-10-18T10:42:13.127 INFO:teuthology.orchestra.run.smithi089.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-10-18T10:42:13.138 INFO:teuthology.orchestra.run.smithi154.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2023-10-18T10:42:13.142 INFO:teuthology.orchestra.run.smithi089.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-10-18T10:42:13.142 INFO:teuthology.orchestra.run.smithi089.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-10-18T10:42:13.145 INFO:teuthology.orchestra.run.smithi089.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-10-18T10:42:13.154 INFO:teuthology.orchestra.run.smithi089.stdout:Get:46 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.0.0-5610-g6f3e6b48-1jammy [3,412 kB] 2023-10-18T10:42:13.193 INFO:teuthology.orchestra.run.smithi089.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-10-18T10:42:13.194 INFO:teuthology.orchestra.run.smithi089.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-10-18T10:42:13.195 INFO:teuthology.orchestra.run.smithi089.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-10-18T10:42:13.196 INFO:teuthology.orchestra.run.smithi089.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-10-18T10:42:13.197 INFO:teuthology.orchestra.run.smithi089.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-10-18T10:42:13.198 INFO:teuthology.orchestra.run.smithi089.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-10-18T10:42:13.218 INFO:teuthology.orchestra.run.smithi089.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-10-18T10:42:13.220 INFO:teuthology.orchestra.run.smithi089.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-10-18T10:42:13.220 INFO:teuthology.orchestra.run.smithi089.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-10-18T10:42:13.238 INFO:teuthology.orchestra.run.smithi089.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-10-18T10:42:13.238 INFO:teuthology.orchestra.run.smithi089.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-10-18T10:42:13.239 INFO:teuthology.orchestra.run.smithi089.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-10-18T10:42:13.239 INFO:teuthology.orchestra.run.smithi089.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-10-18T10:42:13.240 INFO:teuthology.orchestra.run.smithi089.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-10-18T10:42:13.241 INFO:teuthology.orchestra.run.smithi089.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-10-18T10:42:13.249 INFO:teuthology.orchestra.run.smithi089.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-10-18T10:42:13.250 INFO:teuthology.orchestra.run.smithi089.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-10-18T10:42:13.254 INFO:teuthology.orchestra.run.smithi089.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-10-18T10:42:13.266 INFO:teuthology.orchestra.run.smithi089.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-10-18T10:42:13.268 INFO:teuthology.orchestra.run.smithi089.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-10-18T10:42:13.268 INFO:teuthology.orchestra.run.smithi089.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-10-18T10:42:13.272 INFO:teuthology.orchestra.run.smithi089.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-10-18T10:42:13.274 INFO:teuthology.orchestra.run.smithi089.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2023-10-18T10:42:13.275 INFO:teuthology.orchestra.run.smithi089.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-10-18T10:42:13.279 INFO:teuthology.orchestra.run.smithi089.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-10-18T10:42:13.283 INFO:teuthology.orchestra.run.smithi089.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-10-18T10:42:13.283 INFO:teuthology.orchestra.run.smithi089.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-10-18T10:42:13.284 INFO:teuthology.orchestra.run.smithi089.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-10-18T10:42:13.300 INFO:teuthology.orchestra.run.smithi089.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-10-18T10:42:13.301 INFO:teuthology.orchestra.run.smithi089.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-10-18T10:42:13.302 INFO:teuthology.orchestra.run.smithi089.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-10-18T10:42:13.309 INFO:teuthology.orchestra.run.smithi089.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-10-18T10:42:13.309 INFO:teuthology.orchestra.run.smithi089.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-10-18T10:42:13.310 INFO:teuthology.orchestra.run.smithi089.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-10-18T10:42:13.312 INFO:teuthology.orchestra.run.smithi089.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-10-18T10:42:13.317 INFO:teuthology.orchestra.run.smithi089.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-10-18T10:42:13.337 INFO:teuthology.orchestra.run.smithi154.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2023-10-18T10:42:13.378 INFO:teuthology.orchestra.run.smithi154.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-10-18T10:42:13.454 INFO:teuthology.orchestra.run.smithi154.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-10-18T10:42:13.467 INFO:teuthology.orchestra.run.smithi154.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-10-18T10:42:13.513 INFO:teuthology.orchestra.run.smithi154.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-10-18T10:42:13.579 INFO:teuthology.orchestra.run.smithi154.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.0.0-5610-g6f3e6b48-1jammy [3,412 kB] 2023-10-18T10:42:13.679 INFO:teuthology.orchestra.run.smithi154.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-10-18T10:42:13.688 INFO:teuthology.orchestra.run.smithi154.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-10-18T10:42:13.717 INFO:teuthology.orchestra.run.smithi154.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-10-18T10:42:13.743 INFO:teuthology.orchestra.run.smithi089.stdout:Get:83 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.0.0-5610-g6f3e6b48-1jammy [3,118 kB] 2023-10-18T10:42:13.890 INFO:teuthology.orchestra.run.smithi154.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-10-18T10:42:13.922 INFO:teuthology.orchestra.run.smithi089.stdout:Get:84 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.0.0-5610-g6f3e6b48-1jammy [669 kB] 2023-10-18T10:42:13.942 INFO:teuthology.orchestra.run.smithi089.stdout:Get:85 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.0.0-5610-g6f3e6b48-1jammy [345 kB] 2023-10-18T10:42:13.959 INFO:teuthology.orchestra.run.smithi089.stdout:Get:86 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.0.0-5610-g6f3e6b48-1jammy [23.0 kB] 2023-10-18T10:42:13.960 INFO:teuthology.orchestra.run.smithi089.stdout:Get:87 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.0.0-5610-g6f3e6b48-1jammy [170 kB] 2023-10-18T10:42:13.965 INFO:teuthology.orchestra.run.smithi089.stdout:Get:88 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.0.0-5610-g6f3e6b48-1jammy [52.4 kB] 2023-10-18T10:42:13.967 INFO:teuthology.orchestra.run.smithi089.stdout:Get:89 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.0.0-5610-g6f3e6b48-1jammy [318 kB] 2023-10-18T10:42:13.981 INFO:teuthology.orchestra.run.smithi089.stdout:Get:90 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.0.0-5610-g6f3e6b48-1jammy [6,098 kB] 2023-10-18T10:42:13.989 INFO:teuthology.orchestra.run.smithi154.stdout:Get:14 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.0.0-5610-g6f3e6b48-1jammy [3,118 kB] 2023-10-18T10:42:14.083 INFO:teuthology.orchestra.run.smithi154.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-10-18T10:42:14.218 INFO:teuthology.orchestra.run.smithi154.stdout:Get:16 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.0.0-5610-g6f3e6b48-1jammy [669 kB] 2023-10-18T10:42:14.240 INFO:teuthology.orchestra.run.smithi154.stdout:Get:17 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.0.0-5610-g6f3e6b48-1jammy [345 kB] 2023-10-18T10:42:14.289 INFO:teuthology.orchestra.run.smithi154.stdout:Get:18 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.0.0-5610-g6f3e6b48-1jammy [23.0 kB] 2023-10-18T10:42:14.289 INFO:teuthology.orchestra.run.smithi154.stdout:Get:19 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.0.0-5610-g6f3e6b48-1jammy [170 kB] 2023-10-18T10:42:14.296 INFO:teuthology.orchestra.run.smithi154.stdout:Get:20 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.0.0-5610-g6f3e6b48-1jammy [52.4 kB] 2023-10-18T10:42:14.298 INFO:teuthology.orchestra.run.smithi154.stdout:Get:21 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.0.0-5610-g6f3e6b48-1jammy [318 kB] 2023-10-18T10:42:14.309 INFO:teuthology.orchestra.run.smithi154.stdout:Get:22 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.0.0-5610-g6f3e6b48-1jammy [6,098 kB] 2023-10-18T10:42:14.334 INFO:teuthology.orchestra.run.smithi089.stdout:Get:91 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.0.0-5610-g6f3e6b48-1jammy [102 kB] 2023-10-18T10:42:14.335 INFO:teuthology.orchestra.run.smithi089.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.0.0-5610-g6f3e6b48-1jammy [428 kB] 2023-10-18T10:42:14.349 INFO:teuthology.orchestra.run.smithi089.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.0.0-5610-g6f3e6b48-1jammy [22.3 MB] 2023-10-18T10:42:14.510 INFO:teuthology.orchestra.run.smithi154.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-10-18T10:42:14.516 INFO:teuthology.orchestra.run.smithi154.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-10-18T10:42:14.517 INFO:teuthology.orchestra.run.smithi154.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-10-18T10:42:14.528 INFO:teuthology.orchestra.run.smithi154.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-10-18T10:42:14.529 INFO:teuthology.orchestra.run.smithi154.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-10-18T10:42:14.531 INFO:teuthology.orchestra.run.smithi154.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-10-18T10:42:14.533 INFO:teuthology.orchestra.run.smithi154.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-10-18T10:42:14.534 INFO:teuthology.orchestra.run.smithi154.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-10-18T10:42:14.536 INFO:teuthology.orchestra.run.smithi154.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-10-18T10:42:14.700 INFO:teuthology.orchestra.run.smithi154.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-10-18T10:42:14.786 INFO:teuthology.orchestra.run.smithi154.stdout:Get:33 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.0.0-5610-g6f3e6b48-1jammy [102 kB] 2023-10-18T10:42:14.787 INFO:teuthology.orchestra.run.smithi154.stdout:Get:34 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.0.0-5610-g6f3e6b48-1jammy [428 kB] 2023-10-18T10:42:14.803 INFO:teuthology.orchestra.run.smithi154.stdout:Get:35 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.0.0-5610-g6f3e6b48-1jammy [22.3 MB] 2023-10-18T10:42:15.152 INFO:teuthology.orchestra.run.smithi154.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-10-18T10:42:15.163 INFO:teuthology.orchestra.run.smithi154.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-10-18T10:42:15.170 INFO:teuthology.orchestra.run.smithi154.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-10-18T10:42:15.174 INFO:teuthology.orchestra.run.smithi154.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-10-18T10:42:15.193 INFO:teuthology.orchestra.run.smithi154.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-10-18T10:42:15.196 INFO:teuthology.orchestra.run.smithi154.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-10-18T10:42:15.198 INFO:teuthology.orchestra.run.smithi154.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-10-18T10:42:15.237 INFO:teuthology.orchestra.run.smithi154.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-10-18T10:42:15.252 INFO:teuthology.orchestra.run.smithi154.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-10-18T10:42:15.257 INFO:teuthology.orchestra.run.smithi154.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-10-18T10:42:15.510 INFO:teuthology.orchestra.run.smithi154.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-10-18T10:42:15.594 INFO:teuthology.orchestra.run.smithi154.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-10-18T10:42:15.656 INFO:teuthology.orchestra.run.smithi089.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.0.0-5610-g6f3e6b48-1jammy [6,820 kB] 2023-10-18T10:42:15.686 INFO:teuthology.orchestra.run.smithi154.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-10-18T10:42:15.763 INFO:teuthology.orchestra.run.smithi154.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-10-18T10:42:15.870 INFO:teuthology.orchestra.run.smithi154.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-10-18T10:42:15.960 INFO:teuthology.orchestra.run.smithi154.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-10-18T10:42:15.979 INFO:teuthology.orchestra.run.smithi154.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-urllib3 all 1.26.5-1~exp1 [96.4 kB] 2023-10-18T10:42:16.009 INFO:teuthology.orchestra.run.smithi154.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-10-18T10:42:16.022 INFO:teuthology.orchestra.run.smithi154.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-10-18T10:42:16.033 INFO:teuthology.orchestra.run.smithi089.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.0.0-5610-g6f3e6b48-1jammy [227 kB] 2023-10-18T10:42:16.038 INFO:teuthology.orchestra.run.smithi089.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.0.0-5610-g6f3e6b48-1jammy [117 kB] 2023-10-18T10:42:16.041 INFO:teuthology.orchestra.run.smithi089.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.0.0-5610-g6f3e6b48-1jammy [1,481 kB] 2023-10-18T10:42:16.060 INFO:teuthology.orchestra.run.smithi154.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-10-18T10:42:16.069 INFO:teuthology.orchestra.run.smithi154.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-10-18T10:42:16.124 INFO:teuthology.orchestra.run.smithi089.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.0.0-5610-g6f3e6b48-1jammy [9,852 kB] 2023-10-18T10:42:16.260 INFO:teuthology.orchestra.run.smithi154.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-10-18T10:42:16.565 INFO:teuthology.orchestra.run.smithi154.stdout:Get:58 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.0.0-5610-g6f3e6b48-1jammy [6,820 kB] 2023-10-18T10:42:16.693 INFO:teuthology.orchestra.run.smithi089.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.0.0-5610-g6f3e6b48-1jammy [31.5 MB] 2023-10-18T10:42:17.026 INFO:teuthology.orchestra.run.smithi154.stdout:Get:59 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.0.0-5610-g6f3e6b48-1jammy [227 kB] 2023-10-18T10:42:17.094 INFO:teuthology.orchestra.run.smithi154.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-10-18T10:42:17.095 INFO:teuthology.orchestra.run.smithi154.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-10-18T10:42:17.096 INFO:teuthology.orchestra.run.smithi154.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-10-18T10:42:17.097 INFO:teuthology.orchestra.run.smithi154.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-10-18T10:42:17.097 INFO:teuthology.orchestra.run.smithi154.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-10-18T10:42:17.098 INFO:teuthology.orchestra.run.smithi154.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-10-18T10:42:17.122 INFO:teuthology.orchestra.run.smithi154.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-10-18T10:42:17.124 INFO:teuthology.orchestra.run.smithi154.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-10-18T10:42:17.125 INFO:teuthology.orchestra.run.smithi154.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-10-18T10:42:17.192 INFO:teuthology.orchestra.run.smithi154.stdout:Get:69 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.0.0-5610-g6f3e6b48-1jammy [117 kB] 2023-10-18T10:42:17.196 INFO:teuthology.orchestra.run.smithi154.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-10-18T10:42:17.197 INFO:teuthology.orchestra.run.smithi154.stdout:Get:71 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.0.0-5610-g6f3e6b48-1jammy [1,481 kB] 2023-10-18T10:42:17.294 INFO:teuthology.orchestra.run.smithi154.stdout:Get:72 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.0.0-5610-g6f3e6b48-1jammy [9,852 kB] 2023-10-18T10:42:17.361 INFO:teuthology.orchestra.run.smithi154.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-10-18T10:42:17.620 INFO:teuthology.orchestra.run.smithi154.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-10-18T10:42:17.653 INFO:teuthology.orchestra.run.smithi154.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-10-18T10:42:17.722 INFO:teuthology.orchestra.run.smithi154.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-10-18T10:42:17.757 INFO:teuthology.orchestra.run.smithi154.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-10-18T10:42:17.775 INFO:teuthology.orchestra.run.smithi154.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-10-18T10:42:17.801 INFO:teuthology.orchestra.run.smithi154.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-10-18T10:42:17.940 INFO:teuthology.orchestra.run.smithi154.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-10-18T10:42:17.975 INFO:teuthology.orchestra.run.smithi154.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-10-18T10:42:17.996 INFO:teuthology.orchestra.run.smithi154.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-10-18T10:42:18.003 INFO:teuthology.orchestra.run.smithi154.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-10-18T10:42:18.074 INFO:teuthology.orchestra.run.smithi154.stdout:Get:84 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.0.0-5610-g6f3e6b48-1jammy [31.5 MB] 2023-10-18T10:42:18.205 INFO:teuthology.orchestra.run.smithi154.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-10-18T10:42:18.528 INFO:teuthology.orchestra.run.smithi089.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.0.0-5610-g6f3e6b48-1jammy [3,596 B] 2023-10-18T10:42:18.529 INFO:teuthology.orchestra.run.smithi089.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.0.0-5610-g6f3e6b48-1jammy [821 kB] 2023-10-18T10:42:18.583 INFO:teuthology.orchestra.run.smithi089.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.0.0-5610-g6f3e6b48-1jammy [2,157 kB] 2023-10-18T10:42:18.685 INFO:teuthology.orchestra.run.smithi154.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2023-10-18T10:42:18.691 INFO:teuthology.orchestra.run.smithi154.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-10-18T10:42:18.694 INFO:teuthology.orchestra.run.smithi089.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.0.0-5610-g6f3e6b48-1jammy [228 kB] 2023-10-18T10:42:18.715 INFO:teuthology.orchestra.run.smithi089.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.0.0-5610-g6f3e6b48-1jammy [127 kB] 2023-10-18T10:42:18.737 INFO:teuthology.orchestra.run.smithi089.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.0.0-5610-g6f3e6b48-1jammy [1,883 kB] 2023-10-18T10:42:18.817 INFO:teuthology.orchestra.run.smithi089.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.0.0-5610-g6f3e6b48-1jammy [8,615 kB] 2023-10-18T10:42:18.820 INFO:teuthology.orchestra.run.smithi154.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-10-18T10:42:18.867 INFO:teuthology.orchestra.run.smithi154.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-10-18T10:42:18.867 INFO:teuthology.orchestra.run.smithi154.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-10-18T10:42:18.872 INFO:teuthology.orchestra.run.smithi154.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-10-18T10:42:18.874 INFO:teuthology.orchestra.run.smithi154.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-10-18T10:42:18.890 INFO:teuthology.orchestra.run.smithi154.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-10-18T10:42:18.902 INFO:teuthology.orchestra.run.smithi154.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-10-18T10:42:18.959 INFO:teuthology.orchestra.run.smithi154.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-10-18T10:42:19.124 INFO:teuthology.orchestra.run.smithi154.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-10-18T10:42:19.264 INFO:teuthology.orchestra.run.smithi089.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.0.0-5610-g6f3e6b48-1jammy [3,726 B] 2023-10-18T10:42:19.264 INFO:teuthology.orchestra.run.smithi089.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.0.0-5610-g6f3e6b48-1jammy [47.6 MB] 2023-10-18T10:42:19.290 INFO:teuthology.orchestra.run.smithi154.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-10-18T10:42:19.586 INFO:teuthology.orchestra.run.smithi154.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-10-18T10:42:19.611 INFO:teuthology.orchestra.run.smithi154.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-10-18T10:42:20.882 INFO:teuthology.orchestra.run.smithi154.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.0.0-5610-g6f3e6b48-1jammy [3,596 B] 2023-10-18T10:42:20.883 INFO:teuthology.orchestra.run.smithi154.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.0.0-5610-g6f3e6b48-1jammy [821 kB] 2023-10-18T10:42:20.955 INFO:teuthology.orchestra.run.smithi154.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.0.0-5610-g6f3e6b48-1jammy [2,157 kB] 2023-10-18T10:42:21.171 INFO:teuthology.orchestra.run.smithi154.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.0.0-5610-g6f3e6b48-1jammy [228 kB] 2023-10-18T10:42:21.201 INFO:teuthology.orchestra.run.smithi154.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.0.0-5610-g6f3e6b48-1jammy [127 kB] 2023-10-18T10:42:21.204 INFO:teuthology.orchestra.run.smithi154.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.0.0-5610-g6f3e6b48-1jammy [1,883 kB] 2023-10-18T10:42:21.381 INFO:teuthology.orchestra.run.smithi154.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.0.0-5610-g6f3e6b48-1jammy [8,615 kB] 2023-10-18T10:42:21.894 INFO:teuthology.orchestra.run.smithi089.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.0.0-5610-g6f3e6b48-1jammy [234 kB] 2023-10-18T10:42:21.936 INFO:teuthology.orchestra.run.smithi089.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.0.0-5610-g6f3e6b48-1jammy [215 kB] 2023-10-18T10:42:21.961 INFO:teuthology.orchestra.run.smithi089.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.0.0-5610-g6f3e6b48-1jammy [23.8 kB] 2023-10-18T10:42:21.962 INFO:teuthology.orchestra.run.smithi089.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.0.0-5610-g6f3e6b48-1jammy [13.2 kB] 2023-10-18T10:42:21.969 INFO:teuthology.orchestra.run.smithi089.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.0.0-5610-g6f3e6b48-1jammy [28.4 kB] 2023-10-18T10:42:21.978 INFO:teuthology.orchestra.run.smithi089.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.0.0-5610-g6f3e6b48-1jammy [12.1 MB] 2023-10-18T10:42:22.500 INFO:teuthology.orchestra.run.smithi154.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.0.0-5610-g6f3e6b48-1jammy [3,726 B] 2023-10-18T10:42:22.500 INFO:teuthology.orchestra.run.smithi154.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.0.0-5610-g6f3e6b48-1jammy [47.6 MB] 2023-10-18T10:42:22.537 INFO:teuthology.orchestra.run.smithi089.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.0.0-5610-g6f3e6b48-1jammy [80.7 kB] 2023-10-18T10:42:23.114 INFO:teuthology.orchestra.run.smithi089.stdout:Fetched 183 MB in 10s (18.8 MB/s) 2023-10-18T10:42:23.255 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-pyperclip. 2023-10-18T10:42:23.327 INFO:teuthology.orchestra.run.smithi089.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132932 files and directories currently installed.) 2023-10-18T10:42:23.327 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2023-10-18T10:42:23.371 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2023-10-18T10:42:23.624 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-wcwidth. 2023-10-18T10:42:23.637 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-10-18T10:42:23.674 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-10-18T10:42:23.976 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-cmd2. 2023-10-18T10:42:23.990 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2023-10-18T10:42:24.035 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2023-10-18T10:42:24.360 INFO:teuthology.orchestra.run.smithi154.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.0.0-5610-g6f3e6b48-1jammy [234 kB] 2023-10-18T10:42:24.362 INFO:teuthology.orchestra.run.smithi154.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.0.0-5610-g6f3e6b48-1jammy [215 kB] 2023-10-18T10:42:24.364 INFO:teuthology.orchestra.run.smithi154.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.0.0-5610-g6f3e6b48-1jammy [23.8 kB] 2023-10-18T10:42:24.364 INFO:teuthology.orchestra.run.smithi154.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.0.0-5610-g6f3e6b48-1jammy [13.2 kB] 2023-10-18T10:42:24.365 INFO:teuthology.orchestra.run.smithi154.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.0.0-5610-g6f3e6b48-1jammy [28.4 kB] 2023-10-18T10:42:24.365 INFO:teuthology.orchestra.run.smithi154.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.0.0-5610-g6f3e6b48-1jammy [12.1 MB] 2023-10-18T10:42:24.379 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libfmt8:amd64. 2023-10-18T10:42:24.393 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../003-libfmt8_8.1.1+ds1-2_amd64.deb ... 2023-10-18T10:42:24.480 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2023-10-18T10:42:24.816 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-10-18T10:42:24.831 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../004-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-10-18T10:42:24.874 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-10-18T10:42:24.936 INFO:teuthology.orchestra.run.smithi154.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-63105/6f3e6b48891b9b50415c0822b6b870797fcda16d/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.0.0-5610-g6f3e6b48-1jammy [80.7 kB] 2023-10-18T10:42:25.211 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-10-18T10:42:25.226 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../005-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-10-18T10:42:25.269 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-10-18T10:42:25.508 INFO:teuthology.orchestra.run.smithi154.stdout:Fetched 183 MB in 12s (15.1 MB/s) 2023-10-18T10:42:25.631 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-10-18T10:42:25.645 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../006-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-10-18T10:42:25.655 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pyperclip. 2023-10-18T10:42:25.697 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-10-18T10:42:25.723 INFO:teuthology.orchestra.run.smithi154.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132932 files and directories currently installed.) 2023-10-18T10:42:25.726 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2023-10-18T10:42:25.770 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2023-10-18T10:42:26.056 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-wcwidth. 2023-10-18T10:42:26.070 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-10-18T10:42:26.114 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-10-18T10:42:26.142 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-10-18T10:42:26.157 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../007-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-18T10:42:26.209 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-18T10:42:26.434 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-cmd2. 2023-10-18T10:42:26.447 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2023-10-18T10:42:26.492 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2023-10-18T10:42:26.646 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-10-18T10:42:26.660 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../008-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-18T10:42:26.704 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-18T10:42:26.820 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libfmt8:amd64. 2023-10-18T10:42:26.835 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../003-libfmt8_8.1.1+ds1-2_amd64.deb ... 2023-10-18T10:42:26.920 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2023-10-18T10:42:27.049 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-10-18T10:42:27.063 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../009-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-18T10:42:27.107 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-18T10:42:27.290 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-10-18T10:42:27.304 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../004-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-10-18T10:42:27.348 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-10-18T10:42:27.410 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-10-18T10:42:27.425 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../010-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-10-18T10:42:27.469 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-10-18T10:42:27.626 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-10-18T10:42:27.641 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../005-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-10-18T10:42:27.684 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-10-18T10:42:27.822 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package librados2. 2023-10-18T10:42:27.836 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../011-librados2_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:28.004 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-10-18T10:42:28.019 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../006-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-10-18T10:42:28.062 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-10-18T10:42:28.105 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking librados2 (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:28.524 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-10-18T10:42:28.525 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package librbd1. 2023-10-18T10:42:28.539 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../007-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-18T10:42:28.539 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../012-librbd1_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:28.583 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking librbd1 (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:28.590 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-18T10:42:28.985 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libcephfs2. 2023-10-18T10:42:29.000 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../013-libcephfs2_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:29.002 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-10-18T10:42:29.017 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../008-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-18T10:42:29.035 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libcephfs2 (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:29.060 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-18T10:42:29.305 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-rados. 2023-10-18T10:42:29.319 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../014-python3-rados_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:29.363 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-rados (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:29.424 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-10-18T10:42:29.439 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../009-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-18T10:42:29.480 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-18T10:42:29.692 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-10-18T10:42:29.705 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../015-python3-ceph-argparse_18.0.0-5610-g6f3e6b48-1jammy_all.deb ... 2023-10-18T10:42:29.749 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-ceph-argparse (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:29.825 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-10-18T10:42:29.840 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../010-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-10-18T10:42:29.883 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-10-18T10:42:30.010 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-cephfs. 2023-10-18T10:42:30.024 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../016-python3-cephfs_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:30.060 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-cephfs (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:30.177 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package librados2. 2023-10-18T10:42:30.192 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../011-librados2_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:30.232 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking librados2 (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:30.329 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-ceph-common. 2023-10-18T10:42:30.344 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../017-python3-ceph-common_18.0.0-5610-g6f3e6b48-1jammy_all.deb ... 2023-10-18T10:42:30.387 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-ceph-common (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:30.699 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-prettytable. 2023-10-18T10:42:30.714 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2023-10-18T10:42:30.757 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-10-18T10:42:30.814 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package librbd1. 2023-10-18T10:42:30.828 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../012-librbd1_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:30.872 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking librbd1 (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:30.993 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-rbd. 2023-10-18T10:42:31.008 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../019-python3-rbd_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:31.052 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-rbd (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:31.283 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libcephfs2. 2023-10-18T10:42:31.297 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../013-libcephfs2_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:31.341 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libcephfs2 (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:31.404 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-10-18T10:42:31.419 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-10-18T10:42:31.462 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-10-18T10:42:31.610 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-rados. 2023-10-18T10:42:31.625 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../014-python3-rados_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:31.660 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-rados (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:31.757 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package librgw2. 2023-10-18T10:42:31.771 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../021-librgw2_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:31.815 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking librgw2 (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:31.963 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-10-18T10:42:31.977 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../015-python3-ceph-argparse_18.0.0-5610-g6f3e6b48-1jammy_all.deb ... 2023-10-18T10:42:32.021 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-ceph-argparse (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:32.256 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-cephfs. 2023-10-18T10:42:32.271 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../016-python3-cephfs_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:32.315 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-cephfs (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:32.318 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-rgw. 2023-10-18T10:42:32.332 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../022-python3-rgw_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:32.376 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-rgw (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:32.626 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-ceph-common. 2023-10-18T10:42:32.640 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../017-python3-ceph-common_18.0.0-5610-g6f3e6b48-1jammy_all.deb ... 2023-10-18T10:42:32.684 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-ceph-common (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:32.696 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package liboath0:amd64. 2023-10-18T10:42:32.710 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../023-liboath0_2.6.7-3build1_amd64.deb ... 2023-10-18T10:42:32.745 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-10-18T10:42:33.006 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-prettytable. 2023-10-18T10:42:33.021 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2023-10-18T10:42:33.023 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libradosstriper1. 2023-10-18T10:42:33.038 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../024-libradosstriper1_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:33.064 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-10-18T10:42:33.082 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libradosstriper1 (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:33.317 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-rbd. 2023-10-18T10:42:33.331 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../019-python3-rbd_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:33.375 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-rbd (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:33.644 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-common. 2023-10-18T10:42:33.658 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../025-ceph-common_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:33.702 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-common (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:33.719 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-10-18T10:42:33.734 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-10-18T10:42:33.778 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-10-18T10:42:34.122 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package librgw2. 2023-10-18T10:42:34.137 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../021-librgw2_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:34.172 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking librgw2 (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:34.617 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-rgw. 2023-10-18T10:42:34.631 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../022-python3-rgw_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:34.675 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-rgw (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:34.774 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-base. 2023-10-18T10:42:34.789 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../026-ceph-base_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:34.834 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-base (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:35.011 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package liboath0:amd64. 2023-10-18T10:42:35.025 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../023-liboath0_2.6.7-3build1_amd64.deb ... 2023-10-18T10:42:35.069 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-10-18T10:42:35.405 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libradosstriper1. 2023-10-18T10:42:35.420 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../024-libradosstriper1_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:35.472 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libradosstriper1 (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:35.503 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-10-18T10:42:35.517 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../027-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-10-18T10:42:35.561 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-10-18T10:42:35.867 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-common. 2023-10-18T10:42:35.881 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../025-ceph-common_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:35.889 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-cheroot. 2023-10-18T10:42:35.903 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../028-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-10-18T10:42:35.925 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-common (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:35.948 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-10-18T10:42:36.268 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-10-18T10:42:36.282 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../029-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-10-18T10:42:36.326 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-10-18T10:42:36.628 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-jaraco.text. 2023-10-18T10:42:36.643 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../030-python3-jaraco.text_3.6.0-2_all.deb ... 2023-10-18T10:42:36.687 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-10-18T10:42:36.990 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-10-18T10:42:37.004 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../031-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-10-18T10:42:37.048 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-10-18T10:42:37.148 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-base. 2023-10-18T10:42:37.163 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../026-ceph-base_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:37.223 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-base (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:37.351 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-tempora. 2023-10-18T10:42:37.366 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../032-python3-tempora_4.1.2-1_all.deb ... 2023-10-18T10:42:37.409 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-10-18T10:42:37.695 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-portend. 2023-10-18T10:42:37.710 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../033-python3-portend_3.0.0-1_all.deb ... 2023-10-18T10:42:37.753 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-10-18T10:42:37.935 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-10-18T10:42:37.949 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../027-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-10-18T10:42:37.993 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-10-18T10:42:38.006 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-10-18T10:42:38.021 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../034-python3-zc.lockfile_2.0-1_all.deb ... 2023-10-18T10:42:38.065 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-10-18T10:42:38.363 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-cheroot. 2023-10-18T10:42:38.378 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../028-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-10-18T10:42:38.402 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-cherrypy3. 2023-10-18T10:42:38.417 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../035-python3-cherrypy3_18.6.1-4_all.deb ... 2023-10-18T10:42:38.430 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-10-18T10:42:38.460 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-10-18T10:42:38.788 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-natsort. 2023-10-18T10:42:38.802 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../036-python3-natsort_8.0.2-1_all.deb ... 2023-10-18T10:42:38.833 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-10-18T10:42:38.837 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-10-18T10:42:38.847 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../029-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-10-18T10:42:38.899 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-10-18T10:42:39.124 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-logutils. 2023-10-18T10:42:39.139 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../037-python3-logutils_0.3.3-8_all.deb ... 2023-10-18T10:42:39.182 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-10-18T10:42:39.244 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-jaraco.text. 2023-10-18T10:42:39.259 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../030-python3-jaraco.text_3.6.0-2_all.deb ... 2023-10-18T10:42:39.302 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-10-18T10:42:39.518 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-markupsafe. 2023-10-18T10:42:39.533 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../038-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-10-18T10:42:39.577 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-10-18T10:42:39.605 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-10-18T10:42:39.620 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../031-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-10-18T10:42:39.672 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-10-18T10:42:39.863 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-mako. 2023-10-18T10:42:39.878 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../039-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-10-18T10:42:39.921 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-10-18T10:42:39.999 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-tempora. 2023-10-18T10:42:40.014 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../032-python3-tempora_4.1.2-1_all.deb ... 2023-10-18T10:42:40.058 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-10-18T10:42:40.250 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-simplegeneric. 2023-10-18T10:42:40.264 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../040-python3-simplegeneric_0.8.1-3_all.deb ... 2023-10-18T10:42:40.308 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-10-18T10:42:40.327 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-portend. 2023-10-18T10:42:40.342 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../033-python3-portend_3.0.0-1_all.deb ... 2023-10-18T10:42:40.377 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-10-18T10:42:40.594 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-singledispatch. 2023-10-18T10:42:40.609 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../041-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-10-18T10:42:40.652 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-10-18T10:42:40.856 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-10-18T10:42:40.871 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../034-python3-zc.lockfile_2.0-1_all.deb ... 2023-10-18T10:42:40.922 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-10-18T10:42:40.955 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-webob. 2023-10-18T10:42:40.970 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../042-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-10-18T10:42:41.013 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-10-18T10:42:41.325 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-cherrypy3. 2023-10-18T10:42:41.340 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../035-python3-cherrypy3_18.6.1-4_all.deb ... 2023-10-18T10:42:41.358 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-waitress. 2023-10-18T10:42:41.373 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../043-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-10-18T10:42:41.392 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-10-18T10:42:41.419 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-10-18T10:42:41.728 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-natsort. 2023-10-18T10:42:41.743 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../036-python3-natsort_8.0.2-1_all.deb ... 2023-10-18T10:42:41.752 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-tempita. 2023-10-18T10:42:41.767 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-10-18T10:42:41.778 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-10-18T10:42:41.811 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-10-18T10:42:42.082 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-logutils. 2023-10-18T10:42:42.096 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../037-python3-logutils_0.3.3-8_all.deb ... 2023-10-18T10:42:42.105 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-paste. 2023-10-18T10:42:42.121 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../045-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-10-18T10:42:42.140 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-10-18T10:42:42.155 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-10-18T10:42:42.443 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-markupsafe. 2023-10-18T10:42:42.458 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../038-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-10-18T10:42:42.475 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-10-18T10:42:42.491 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-10-18T10:42:42.501 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-10-18T10:42:42.533 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-10-18T10:42:42.787 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-mako. 2023-10-18T10:42:42.801 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../039-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-10-18T10:42:42.845 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-10-18T10:42:42.853 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-pastedeploy. 2023-10-18T10:42:42.868 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.1-1_all.deb ... 2023-10-18T10:42:42.903 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-10-18T10:42:43.156 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-webtest. 2023-10-18T10:42:43.157 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-simplegeneric. 2023-10-18T10:42:43.171 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../048-python3-webtest_2.0.35-1_all.deb ... 2023-10-18T10:42:43.172 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../040-python3-simplegeneric_0.8.1-3_all.deb ... 2023-10-18T10:42:43.207 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-10-18T10:42:43.214 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-10-18T10:42:43.510 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-singledispatch. 2023-10-18T10:42:43.525 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../041-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-10-18T10:42:43.568 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-10-18T10:42:43.683 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-pecan. 2023-10-18T10:42:43.698 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-10-18T10:42:43.852 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-10-18T10:42:43.871 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-webob. 2023-10-18T10:42:43.886 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../042-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-10-18T10:42:43.929 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-10-18T10:42:44.274 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-waitress. 2023-10-18T10:42:44.289 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../043-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-10-18T10:42:44.335 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-10-18T10:42:44.669 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-tempita. 2023-10-18T10:42:44.684 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-10-18T10:42:44.719 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-10-18T10:42:44.738 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-certifi. 2023-10-18T10:42:44.754 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../050-python3-certifi_2020.6.20-1_all.deb ... 2023-10-18T10:42:44.788 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-10-18T10:42:45.021 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-paste. 2023-10-18T10:42:45.036 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../045-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-10-18T10:42:45.080 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-10-18T10:42:45.100 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-idna. 2023-10-18T10:42:45.115 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../051-python3-idna_3.3-1_all.deb ... 2023-10-18T10:42:45.158 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-idna (3.3-1) ... 2023-10-18T10:42:45.391 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-10-18T10:42:45.405 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-10-18T10:42:45.458 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-10-18T10:42:45.490 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-urllib3. 2023-10-18T10:42:45.505 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../052-python3-urllib3_1.26.5-1~exp1_all.deb ... 2023-10-18T10:42:45.552 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1) ... 2023-10-18T10:42:45.752 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pastedeploy. 2023-10-18T10:42:45.767 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.1-1_all.deb ... 2023-10-18T10:42:45.810 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-10-18T10:42:45.872 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-requests. 2023-10-18T10:42:45.887 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../053-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-10-18T10:42:45.930 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-10-18T10:42:46.242 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-werkzeug. 2023-10-18T10:42:46.257 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-10-18T10:42:46.300 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-10-18T10:42:46.595 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-10-18T10:42:46.610 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_18.0.0-5610-g6f3e6b48-1jammy_all.deb ... 2023-10-18T10:42:46.678 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-mgr-modules-core (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:46.755 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-webtest. 2023-10-18T10:42:46.770 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../048-python3-webtest_2.0.35-1_all.deb ... 2023-10-18T10:42:46.814 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-10-18T10:42:47.041 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-10-18T10:42:47.056 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:47.091 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:47.158 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pecan. 2023-10-18T10:42:47.173 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-10-18T10:42:47.216 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-10-18T10:42:47.377 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-openssl. 2023-10-18T10:42:47.392 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../057-python3-openssl_21.0.0-1_all.deb ... 2023-10-18T10:42:47.435 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-10-18T10:42:47.563 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-certifi. 2023-10-18T10:42:47.578 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../050-python3-certifi_2020.6.20-1_all.deb ... 2023-10-18T10:42:47.621 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-10-18T10:42:47.805 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-mgr. 2023-10-18T10:42:47.820 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../058-ceph-mgr_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:47.863 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-mgr (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:47.941 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-idna. 2023-10-18T10:42:47.956 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../051-python3-idna_3.3-1_all.deb ... 2023-10-18T10:42:47.991 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-idna (3.3-1) ... 2023-10-18T10:42:48.242 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-mon. 2023-10-18T10:42:48.257 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../059-ceph-mon_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:48.300 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-mon (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:48.302 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-urllib3. 2023-10-18T10:42:48.316 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../052-python3-urllib3_1.26.5-1~exp1_all.deb ... 2023-10-18T10:42:48.360 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1) ... 2023-10-18T10:42:48.672 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-requests. 2023-10-18T10:42:48.686 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../053-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-10-18T10:42:48.730 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-10-18T10:42:49.050 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-werkzeug. 2023-10-18T10:42:49.064 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-10-18T10:42:49.108 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-10-18T10:42:49.112 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-osd. 2023-10-18T10:42:49.127 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../060-ceph-osd_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:49.170 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-osd (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:49.419 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-10-18T10:42:49.433 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_18.0.0-5610-g6f3e6b48-1jammy_all.deb ... 2023-10-18T10:42:49.477 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr-modules-core (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:49.841 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-10-18T10:42:49.854 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:49.899 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:50.160 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-openssl. 2023-10-18T10:42:50.173 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../057-python3-openssl_21.0.0-1_all.deb ... 2023-10-18T10:42:50.218 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-10-18T10:42:50.374 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph. 2023-10-18T10:42:50.389 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../061-ceph_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:50.423 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:50.580 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr. 2023-10-18T10:42:50.593 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../058-ceph-mgr_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:50.638 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:50.728 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-fuse. 2023-10-18T10:42:50.743 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../062-ceph-fuse_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:50.786 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-fuse (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:51.106 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-mds. 2023-10-18T10:42:51.121 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../063-ceph-mds_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:51.156 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-mds (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:51.200 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mon. 2023-10-18T10:42:51.213 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../059-ceph-mon_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:51.258 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mon (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:51.534 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package cephadm. 2023-10-18T10:42:51.549 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../064-cephadm_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:51.601 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking cephadm (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:51.887 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-asyncssh. 2023-10-18T10:42:51.902 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../065-python3-asyncssh_2.5.0-1_all.deb ... 2023-10-18T10:42:51.945 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-10-18T10:42:52.045 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-osd. 2023-10-18T10:42:52.060 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../060-ceph-osd_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:52.103 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-osd (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:52.256 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python-babel-localedata. 2023-10-18T10:42:52.272 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../066-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-10-18T10:42:52.314 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-10-18T10:42:53.303 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-babel. 2023-10-18T10:42:53.317 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../067-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-10-18T10:42:53.361 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-10-18T10:42:53.382 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph. 2023-10-18T10:42:53.397 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../061-ceph_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:53.440 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:53.743 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-fuse. 2023-10-18T10:42:53.758 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../062-ceph-fuse_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:53.801 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-fuse (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:53.805 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-jinja2. 2023-10-18T10:42:53.819 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../068-python3-jinja2_3.0.3-1_all.deb ... 2023-10-18T10:42:53.972 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-10-18T10:42:54.171 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mds. 2023-10-18T10:42:54.186 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../063-ceph-mds_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:54.229 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mds (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:54.326 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-10-18T10:42:54.340 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../069-ceph-mgr-cephadm_18.0.0-5610-g6f3e6b48-1jammy_all.deb ... 2023-10-18T10:42:54.393 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-mgr-cephadm (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:54.608 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package cephadm. 2023-10-18T10:42:54.622 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../064-cephadm_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:42:54.657 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking cephadm (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:54.729 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-bcrypt. 2023-10-18T10:42:54.744 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../070-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-10-18T10:42:54.813 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-10-18T10:42:54.960 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-asyncssh. 2023-10-18T10:42:54.976 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../065-python3-asyncssh_2.5.0-1_all.deb ... 2023-10-18T10:42:55.018 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-10-18T10:42:55.099 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-repoze.lru. 2023-10-18T10:42:55.114 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../071-python3-repoze.lru_0.7-2_all.deb ... 2023-10-18T10:42:55.157 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-10-18T10:42:55.297 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python-babel-localedata. 2023-10-18T10:42:55.313 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../066-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-10-18T10:42:55.346 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-10-18T10:42:55.461 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-routes. 2023-10-18T10:42:55.475 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../072-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-10-18T10:42:55.518 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-10-18T10:42:55.855 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-10-18T10:42:55.870 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../073-ceph-mgr-dashboard_18.0.0-5610-g6f3e6b48-1jammy_all.deb ... 2023-10-18T10:42:55.913 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-mgr-dashboard (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:56.302 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-babel. 2023-10-18T10:42:56.316 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../067-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-10-18T10:42:56.333 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-10-18T10:42:56.347 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../074-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-10-18T10:42:56.368 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-10-18T10:42:56.391 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-10-18T10:42:56.705 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-jinja2. 2023-10-18T10:42:56.718 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../068-python3-jinja2_3.0.3-1_all.deb ... 2023-10-18T10:42:56.763 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-10-18T10:42:56.895 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-joblib. 2023-10-18T10:42:56.910 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../075-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-10-18T10:42:56.954 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-10-18T10:42:57.092 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-10-18T10:42:57.105 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../069-ceph-mgr-cephadm_18.0.0-5610-g6f3e6b48-1jammy_all.deb ... 2023-10-18T10:42:57.149 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr-cephadm (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:57.349 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-10-18T10:42:57.364 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../076-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-10-18T10:42:57.407 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-10-18T10:42:57.486 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-bcrypt. 2023-10-18T10:42:57.500 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../070-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-10-18T10:42:57.544 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-10-18T10:42:57.685 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-sklearn. 2023-10-18T10:42:57.700 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../077-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-10-18T10:42:57.743 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-10-18T10:42:57.889 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-repoze.lru. 2023-10-18T10:42:57.903 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../071-python3-repoze.lru_0.7-2_all.deb ... 2023-10-18T10:42:57.947 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-10-18T10:42:58.225 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-routes. 2023-10-18T10:42:58.240 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../072-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-10-18T10:42:58.284 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-10-18T10:42:58.460 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-10-18T10:42:58.474 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../078-ceph-mgr-diskprediction-local_18.0.0-5610-g6f3e6b48-1jammy_all.deb ... 2023-10-18T10:42:58.519 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:58.562 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-10-18T10:42:58.576 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../073-ceph-mgr-dashboard_18.0.0-5610-g6f3e6b48-1jammy_all.deb ... 2023-10-18T10:42:58.620 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr-dashboard (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:42:59.057 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-10-18T10:42:59.071 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../074-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-10-18T10:42:59.115 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-10-18T10:42:59.505 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-cachetools. 2023-10-18T10:42:59.520 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../079-python3-cachetools_5.0.0-1_all.deb ... 2023-10-18T10:42:59.563 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-10-18T10:42:59.577 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-joblib. 2023-10-18T10:42:59.591 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../075-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-10-18T10:42:59.627 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-10-18T10:42:59.866 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-pyasn1. 2023-10-18T10:42:59.882 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../080-python3-pyasn1_0.4.8-1_all.deb ... 2023-10-18T10:42:59.924 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-10-18T10:42:59.981 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-10-18T10:42:59.994 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../076-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-10-18T10:43:00.072 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-10-18T10:43:00.261 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-10-18T10:43:00.275 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../081-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-10-18T10:43:00.319 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-10-18T10:43:00.392 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-sklearn. 2023-10-18T10:43:00.406 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../077-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-10-18T10:43:00.451 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-10-18T10:43:00.664 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-rsa. 2023-10-18T10:43:00.678 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../082-python3-rsa_4.8-1_all.deb ... 2023-10-18T10:43:00.722 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-rsa (4.8-1) ... 2023-10-18T10:43:01.069 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-google-auth. 2023-10-18T10:43:01.084 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../083-python3-google-auth_1.5.1-3_all.deb ... 2023-10-18T10:43:01.127 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-10-18T10:43:01.343 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-10-18T10:43:01.357 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../078-ceph-mgr-diskprediction-local_18.0.0-5610-g6f3e6b48-1jammy_all.deb ... 2023-10-18T10:43:01.401 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:01.430 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-10-18T10:43:01.445 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../084-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-10-18T10:43:01.480 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-10-18T10:43:01.800 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-websocket. 2023-10-18T10:43:01.815 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../085-python3-websocket_1.2.3-1_all.deb ... 2023-10-18T10:43:01.858 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-10-18T10:43:02.179 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-kubernetes. 2023-10-18T10:43:02.194 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../086-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-10-18T10:43:02.236 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-10-18T10:43:02.371 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-cachetools. 2023-10-18T10:43:02.386 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../079-python3-cachetools_5.0.0-1_all.deb ... 2023-10-18T10:43:02.429 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-10-18T10:43:02.732 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pyasn1. 2023-10-18T10:43:02.748 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../080-python3-pyasn1_0.4.8-1_all.deb ... 2023-10-18T10:43:02.890 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-10-18T10:43:02.975 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-10-18T10:43:02.990 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../087-ceph-mgr-k8sevents_18.0.0-5610-g6f3e6b48-1jammy_all.deb ... 2023-10-18T10:43:03.033 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:03.193 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-10-18T10:43:03.209 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../081-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-10-18T10:43:03.243 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-10-18T10:43:03.353 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libonig5:amd64. 2023-10-18T10:43:03.367 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../088-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-10-18T10:43:03.411 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-10-18T10:43:03.538 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-rsa. 2023-10-18T10:43:03.553 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../082-python3-rsa_4.8-1_all.deb ... 2023-10-18T10:43:03.587 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-rsa (4.8-1) ... 2023-10-18T10:43:03.706 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libjq1:amd64. 2023-10-18T10:43:03.721 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../089-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-10-18T10:43:03.756 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-10-18T10:43:04.001 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-google-auth. 2023-10-18T10:43:04.017 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../083-python3-google-auth_1.5.1-3_all.deb ... 2023-10-18T10:43:04.059 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-10-18T10:43:04.242 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package jq. 2023-10-18T10:43:04.257 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../090-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-10-18T10:43:04.300 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-10-18T10:43:04.379 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-10-18T10:43:04.395 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../084-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-10-18T10:43:04.437 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-10-18T10:43:04.547 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package socat. 2023-10-18T10:43:04.560 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../091-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-10-18T10:43:04.603 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-10-18T10:43:04.766 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-websocket. 2023-10-18T10:43:04.781 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../085-python3-websocket_1.2.3-1_all.deb ... 2023-10-18T10:43:04.816 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-10-18T10:43:04.906 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package xmlstarlet. 2023-10-18T10:43:04.922 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../092-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-10-18T10:43:04.964 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-10-18T10:43:05.127 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-kubernetes. 2023-10-18T10:43:05.143 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../086-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-10-18T10:43:05.185 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-10-18T10:43:05.443 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-test. 2023-10-18T10:43:05.458 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../093-ceph-test_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:43:05.501 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-test (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:05.858 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-10-18T10:43:05.872 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../087-ceph-mgr-k8sevents_18.0.0-5610-g6f3e6b48-1jammy_all.deb ... 2023-10-18T10:43:05.916 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:06.235 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libonig5:amd64. 2023-10-18T10:43:06.250 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../088-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-10-18T10:43:06.294 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-10-18T10:43:06.613 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libjq1:amd64. 2023-10-18T10:43:06.627 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../089-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-10-18T10:43:06.663 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-10-18T10:43:06.916 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package jq. 2023-10-18T10:43:06.931 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../090-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-10-18T10:43:06.999 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-10-18T10:43:07.235 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package socat. 2023-10-18T10:43:07.250 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../091-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-10-18T10:43:07.294 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-10-18T10:43:07.314 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-volume. 2023-10-18T10:43:07.330 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../094-ceph-volume_18.0.0-5610-g6f3e6b48-1jammy_all.deb ... 2023-10-18T10:43:07.372 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-volume (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:07.580 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package xmlstarlet. 2023-10-18T10:43:07.596 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../092-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-10-18T10:43:07.638 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-10-18T10:43:07.777 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package cephfs-mirror. 2023-10-18T10:43:07.792 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../095-cephfs-mirror_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:43:07.835 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking cephfs-mirror (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:08.001 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-test. 2023-10-18T10:43:08.016 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../093-ceph-test_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:43:08.059 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-test (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:08.147 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-colorama. 2023-10-18T10:43:08.163 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../096-python3-colorama_0.4.4-1_all.deb ... 2023-10-18T10:43:08.205 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-colorama (0.4.4-1) ... 2023-10-18T10:43:08.466 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package cephfs-shell. 2023-10-18T10:43:08.482 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../097-cephfs-shell_18.0.0-5610-g6f3e6b48-1jammy_all.deb ... 2023-10-18T10:43:08.516 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking cephfs-shell (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:08.769 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package cephfs-top. 2023-10-18T10:43:08.784 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../098-cephfs-top_18.0.0-5610-g6f3e6b48-1jammy_all.deb ... 2023-10-18T10:43:08.852 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking cephfs-top (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:09.105 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libcephfs-dev. 2023-10-18T10:43:09.120 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../099-libcephfs-dev_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:43:09.163 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libcephfs-dev (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:09.525 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package nvme-cli. 2023-10-18T10:43:09.540 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../100-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-10-18T10:43:09.583 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-10-18T10:43:09.871 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-volume. 2023-10-18T10:43:09.887 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../094-ceph-volume_18.0.0-5610-g6f3e6b48-1jammy_all.deb ... 2023-10-18T10:43:09.921 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-10-18T10:43:09.930 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-volume (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:09.936 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../101-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-10-18T10:43:10.004 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-10-18T10:43:10.335 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package cephfs-mirror. 2023-10-18T10:43:10.349 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../095-cephfs-mirror_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:43:10.385 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking cephfs-mirror (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:10.424 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-iniconfig. 2023-10-18T10:43:10.438 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../102-python3-iniconfig_1.1.1-2_all.deb ... 2023-10-18T10:43:10.474 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-10-18T10:43:10.696 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-colorama. 2023-10-18T10:43:10.711 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../096-python3-colorama_0.4.4-1_all.deb ... 2023-10-18T10:43:10.743 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-pastescript. 2023-10-18T10:43:10.757 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../103-python3-pastescript_2.0.2-4_all.deb ... 2023-10-18T10:43:10.779 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-colorama (0.4.4-1) ... 2023-10-18T10:43:10.801 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-10-18T10:43:11.043 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package cephfs-shell. 2023-10-18T10:43:11.058 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../097-cephfs-shell_18.0.0-5610-g6f3e6b48-1jammy_all.deb ... 2023-10-18T10:43:11.101 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking cephfs-shell (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:11.180 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-pluggy. 2023-10-18T10:43:11.195 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../104-python3-pluggy_0.13.0-7.1_all.deb ... 2023-10-18T10:43:11.238 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-10-18T10:43:11.304 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package cephfs-top. 2023-10-18T10:43:11.319 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../098-cephfs-top_18.0.0-5610-g6f3e6b48-1jammy_all.deb ... 2023-10-18T10:43:11.362 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking cephfs-top (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:11.533 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-psutil. 2023-10-18T10:43:11.548 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../105-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-10-18T10:43:11.590 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-10-18T10:43:11.614 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libcephfs-dev. 2023-10-18T10:43:11.630 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../099-libcephfs-dev_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:43:11.673 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libcephfs-dev (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:11.910 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-py. 2023-10-18T10:43:11.926 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../106-python3-py_1.10.0-1_all.deb ... 2023-10-18T10:43:11.960 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-py (1.10.0-1) ... 2023-10-18T10:43:12.084 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package nvme-cli. 2023-10-18T10:43:12.100 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../100-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-10-18T10:43:12.142 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-10-18T10:43:12.332 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-pygments. 2023-10-18T10:43:12.347 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../107-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-10-18T10:43:12.389 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-10-18T10:43:12.464 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-10-18T10:43:12.479 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../101-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-10-18T10:43:12.522 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-10-18T10:43:12.826 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-pyinotify. 2023-10-18T10:43:12.840 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../108-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-10-18T10:43:12.884 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-10-18T10:43:12.933 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-iniconfig. 2023-10-18T10:43:12.948 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../102-python3-iniconfig_1.1.1-2_all.deb ... 2023-10-18T10:43:12.992 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-10-18T10:43:13.170 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-toml. 2023-10-18T10:43:13.185 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../109-python3-toml_0.10.2-1_all.deb ... 2023-10-18T10:43:13.220 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-10-18T10:43:13.311 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pastescript. 2023-10-18T10:43:13.326 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../103-python3-pastescript_2.0.2-4_all.deb ... 2023-10-18T10:43:13.369 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-10-18T10:43:13.514 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-pytest. 2023-10-18T10:43:13.530 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../110-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-10-18T10:43:13.572 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-10-18T10:43:13.698 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pluggy. 2023-10-18T10:43:13.713 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../104-python3-pluggy_0.13.0-7.1_all.deb ... 2023-10-18T10:43:13.756 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-10-18T10:43:13.909 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-simplejson. 2023-10-18T10:43:13.925 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../111-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-10-18T10:43:13.967 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-10-18T10:43:14.076 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-psutil. 2023-10-18T10:43:14.091 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../105-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-10-18T10:43:14.134 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-10-18T10:43:14.246 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-10-18T10:43:14.261 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../112-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-10-18T10:43:14.295 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-10-18T10:43:14.462 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-py. 2023-10-18T10:43:14.477 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../106-python3-py_1.10.0-1_all.deb ... 2023-10-18T10:43:14.512 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-py (1.10.0-1) ... 2023-10-18T10:43:14.790 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pygments. 2023-10-18T10:43:14.805 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../107-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-10-18T10:43:14.840 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-10-18T10:43:15.024 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package radosgw. 2023-10-18T10:43:15.039 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../113-radosgw_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:43:15.074 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking radosgw (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:15.294 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pyinotify. 2023-10-18T10:43:15.309 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../108-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-10-18T10:43:15.352 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-10-18T10:43:15.652 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package rbd-fuse. 2023-10-18T10:43:15.667 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../114-rbd-fuse_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:43:15.680 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-toml. 2023-10-18T10:43:15.696 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../109-python3-toml_0.10.2-1_all.deb ... 2023-10-18T10:43:15.710 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking rbd-fuse (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:15.738 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-10-18T10:43:15.999 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pytest. 2023-10-18T10:43:16.015 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../110-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-10-18T10:43:16.021 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-10-18T10:43:16.049 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-10-18T10:43:16.290 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-10-18T10:43:16.416 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-10-18T10:43:16.666 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-10-18T10:43:16.669 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-simplejson. 2023-10-18T10:43:16.685 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../111-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-10-18T10:43:16.727 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-10-18T10:43:17.005 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-10-18T10:43:17.021 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../112-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-10-18T10:43:17.060 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-10-18T10:43:17.063 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-10-18T10:43:17.684 INFO:teuthology.orchestra.run.smithi089.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-10-18T10:43:17.748 INFO:teuthology.orchestra.run.smithi089.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2023-10-18T10:43:17.809 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package radosgw. 2023-10-18T10:43:17.824 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../113-radosgw_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:43:17.860 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up cephadm (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:17.867 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking radosgw (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:18.428 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package rbd-fuse. 2023-10-18T10:43:18.443 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../114-rbd-fuse_18.0.0-5610-g6f3e6b48-1jammy_amd64.deb ... 2023-10-18T10:43:18.487 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking rbd-fuse (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:18.755 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-10-18T10:43:18.789 INFO:teuthology.orchestra.run.smithi089.stdout:Adding system user cephadm....done 2023-10-18T10:43:18.917 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-10-18T10:43:19.025 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-10-18T10:43:19.159 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-10-18T10:43:19.193 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-10-18T10:43:19.423 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-10-18T10:43:19.444 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-10-18T10:43:19.554 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-10-18T10:43:19.808 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-10-18T10:43:20.089 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-10-18T10:43:20.355 INFO:teuthology.orchestra.run.smithi154.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-10-18T10:43:20.373 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-10-18T10:43:20.398 INFO:teuthology.orchestra.run.smithi154.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2023-10-18T10:43:20.453 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up cephadm (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:20.499 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-py (1.10.0-1) ... 2023-10-18T10:43:20.834 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-10-18T10:43:20.989 INFO:teuthology.orchestra.run.smithi154.stdout:Adding system user cephadm....done 2023-10-18T10:43:21.159 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-10-18T10:43:21.203 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-10-18T10:43:21.460 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-10-18T10:43:21.479 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-colorama (0.4.4-1) ... 2023-10-18T10:43:21.728 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-10-18T10:43:21.747 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-10-18T10:43:21.854 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-10-18T10:43:22.031 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-10-18T10:43:22.113 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-10-18T10:43:22.291 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-10-18T10:43:22.386 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-10-18T10:43:22.507 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-py (1.10.0-1) ... 2023-10-18T10:43:22.592 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-10-18T10:43:22.842 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-10-18T10:43:22.876 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-ceph-argparse (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:23.153 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-10-18T10:43:23.211 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-10-18T10:43:23.270 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2023-10-18T10:43:23.479 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-colorama (0.4.4-1) ... 2023-10-18T10:43:23.538 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-10-18T10:43:23.763 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-10-18T10:43:23.797 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-10-18T10:43:23.923 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-10-18T10:43:24.040 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-10-18T10:43:24.299 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-10-18T10:43:24.316 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-10-18T10:43:24.442 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-10-18T10:43:24.658 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-10-18T10:43:24.827 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-10-18T10:43:24.951 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-ceph-argparse (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:25.120 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-10-18T10:43:25.244 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-10-18T10:43:25.370 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2023-10-18T10:43:25.454 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-10-18T10:43:25.638 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-10-18T10:43:25.889 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-10-18T10:43:25.998 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-10-18T10:43:26.139 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-10-18T10:43:26.358 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-10-18T10:43:26.416 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-10-18T10:43:26.492 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-10-18T10:43:26.542 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-10-18T10:43:26.642 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-10-18T10:43:26.776 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-10-18T10:43:26.936 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-10-18T10:43:27.053 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-10-18T10:43:27.320 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-10-18T10:43:27.362 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-10-18T10:43:27.580 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-10-18T10:43:27.964 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-10-18T10:43:28.039 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-10-18T10:43:28.266 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2023-10-18T10:43:28.324 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-10-18T10:43:28.449 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-10-18T10:43:28.584 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-10-18T10:43:28.592 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-10-18T10:43:28.902 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-10-18T10:43:28.977 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-10-18T10:43:29.228 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-10-18T10:43:29.320 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-10-18T10:43:29.354 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-ceph-common (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:29.563 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-10-18T10:43:29.655 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-idna (3.3-1) ... 2023-10-18T10:43:29.931 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-10-18T10:43:30.006 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-10-18T10:43:30.232 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2023-10-18T10:43:30.558 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-10-18T10:43:30.927 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-10-18T10:43:31.203 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-10-18T10:43:31.237 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-18T10:43:31.328 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-ceph-common (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:31.372 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-toml (0.10.2-1) ... 2023-10-18T10:43:31.622 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-idna (3.3-1) ... 2023-10-18T10:43:31.648 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-10-18T10:43:31.774 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-10-18T10:43:31.900 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-urllib3 (1.26.5-1~exp1) ... 2023-10-18T10:43:31.973 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-10-18T10:43:32.209 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-10-18T10:43:32.477 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-10-18T10:43:32.745 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-18T10:43:32.871 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2023-10-18T10:43:32.997 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-10-18T10:43:33.195 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-18T10:43:33.313 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-toml (0.10.2-1) ... 2023-10-18T10:43:33.315 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-10-18T10:43:33.564 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-10-18T10:43:33.583 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-10-18T10:43:33.690 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-10-18T10:43:33.816 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-urllib3 (1.26.5-1~exp1) ... 2023-10-18T10:43:33.851 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-10-18T10:43:34.110 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-10-18T10:43:34.151 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-10-18T10:43:34.353 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-10-18T10:43:34.427 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-10-18T10:43:34.654 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-10-18T10:43:34.687 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-18T10:43:34.780 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-10-18T10:43:34.804 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2023-10-18T10:43:34.906 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-10-18T10:43:35.073 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-10-18T10:43:35.199 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-10-18T10:43:35.215 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-10-18T10:43:35.484 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-10-18T10:43:35.494 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-10-18T10:43:35.761 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-10-18T10:43:35.810 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-10-18T10:43:36.036 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-10-18T10:43:36.063 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-10-18T10:43:36.303 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-10-18T10:43:36.492 INFO:teuthology.orchestra.run.smithi089.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-10-18T10:43:36.571 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-10-18T10:43:36.697 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-10-18T10:43:36.774 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-10-18T10:43:36.942 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-10-18T10:43:36.990 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-10-18T10:43:37.115 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-10-18T10:43:37.327 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-10-18T10:43:37.417 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-10-18T10:43:37.594 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-portend (3.0.0-1) ... 2023-10-18T10:43:37.743 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-10-18T10:43:37.921 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-18T10:43:38.019 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-10-18T10:43:38.047 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-10-18T10:43:38.373 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-10-18T10:43:38.449 INFO:teuthology.orchestra.run.smithi154.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-10-18T10:43:38.682 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-10-18T10:43:38.697 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-10-18T10:43:38.823 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-10-18T10:43:38.967 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-10-18T10:43:39.101 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-rsa (4.8-1) ... 2023-10-18T10:43:39.216 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-10-18T10:43:39.352 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-10-18T10:43:39.484 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-portend (3.0.0-1) ... 2023-10-18T10:43:39.612 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-10-18T10:43:39.752 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-18T10:43:39.878 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-10-18T10:43:39.888 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-10-18T10:43:40.204 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-10-18T10:43:40.281 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-10-18T10:43:40.513 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-10-18T10:43:40.583 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-10-18T10:43:40.773 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-10-18T10:43:40.932 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-rsa (4.8-1) ... 2023-10-18T10:43:40.934 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-10-18T10:43:41.036 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up librados2 (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:41.171 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up librgw2 (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:41.200 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-10-18T10:43:41.399 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libsqlite3-mod-ceph (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:41.476 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-10-18T10:43:41.525 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libcephfs2 (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:41.651 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libradosstriper1 (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:41.752 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-10-18T10:43:41.777 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-10-18T10:43:42.062 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up librbd1 (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:42.145 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-10-18T10:43:42.196 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-mgr-modules-core (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:42.322 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-fuse (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:42.446 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-10-18T10:43:42.531 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-18T10:43:42.531 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-18T10:43:42.807 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-10-18T10:43:42.942 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up librados2 (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:43.076 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up librgw2 (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:43.115 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libcephfs-dev (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:43.241 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-rados (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:43.304 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libsqlite3-mod-ceph (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:43.367 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up cephfs-top (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:43.413 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libcephfs2 (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:43.485 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-10-18T10:43:43.523 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libradosstriper1 (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:43.649 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-10-18T10:43:43.951 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up librbd1 (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:44.076 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr-modules-core (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:44.202 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-fuse (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:44.437 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-18T10:43:44.437 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-18T10:43:44.806 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-rbd (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:44.932 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up rbd-fuse (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:45.033 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-rgw (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:45.087 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libcephfs-dev (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:45.134 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-cephfs (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:45.205 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-rados (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:45.235 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up cephfs-shell (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:45.331 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up cephfs-top (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:45.353 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-common (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:45.499 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-10-18T10:43:45.916 INFO:teuthology.orchestra.run.smithi089.stdout:Adding group ceph....done 2023-10-18T10:43:46.283 INFO:teuthology.orchestra.run.smithi089.stdout:Adding system user ceph....done 2023-10-18T10:43:46.469 INFO:teuthology.orchestra.run.smithi089.stdout:Setting system user ceph properties....done 2023-10-18T10:43:46.486 INFO:teuthology.orchestra.run.smithi089.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-10-18T10:43:46.622 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-10-18T10:43:46.788 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-rbd (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:46.913 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up rbd-fuse (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:46.972 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-10-18T10:43:47.022 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-rgw (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:47.132 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-cephfs (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:47.258 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up cephfs-shell (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:47.647 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up cephfs-mirror (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:47.692 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-common (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:47.855 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-18T10:43:47.856 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-18T10:43:48.262 INFO:teuthology.orchestra.run.smithi154.stdout:Adding group ceph....done 2023-10-18T10:43:48.464 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-test (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:48.590 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up radosgw (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:48.655 INFO:teuthology.orchestra.run.smithi154.stdout:Adding system user ceph....done 2023-10-18T10:43:48.808 INFO:teuthology.orchestra.run.smithi154.stdout:Setting system user ceph properties....done 2023-10-18T10:43:48.825 INFO:teuthology.orchestra.run.smithi154.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-10-18T10:43:48.839 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-18T10:43:48.839 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-18T10:43:48.959 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-10-18T10:43:49.349 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-10-18T10:43:49.726 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-base (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:50.047 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-10-18T10:43:50.111 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up cephfs-mirror (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:50.362 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-18T10:43:50.362 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-18T10:43:50.728 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-mds (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:50.977 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-18T10:43:50.977 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-18T10:43:50.988 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-test (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:51.114 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up radosgw (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:51.393 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-18T10:43:51.394 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-18T10:43:51.555 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-mgr (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:51.766 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-18T10:43:51.767 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-18T10:43:52.282 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-base (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:52.340 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-osd (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:52.641 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-18T10:43:52.642 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-18T10:43:52.659 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-10-18T10:43:53.293 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mds (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:53.333 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-mgr-k8sevents (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:53.442 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:53.532 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-18T10:43:53.532 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-18T10:43:53.584 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-mon (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:53.794 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-18T10:43:53.795 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-18T10:43:54.136 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:54.349 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-18T10:43:54.350 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-18T10:43:54.436 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-mgr-cephadm (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:54.587 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:54.713 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-mgr-dashboard (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:54.856 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-volume (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:54.974 INFO:teuthology.orchestra.run.smithi089.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-10-18T10:43:55.004 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-osd (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:55.307 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-18T10:43:55.307 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-18T10:43:55.922 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr-k8sevents (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:56.023 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:56.167 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mon (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:56.375 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-18T10:43:56.375 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-18T10:43:56.394 INFO:teuthology.orchestra.run.smithi089.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-10-18T10:43:56.877 INFO:teuthology.orchestra.run.smithi089.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-18T10:43:56.881 DEBUG:teuthology.orchestra.run.smithi089:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-10-18T10:43:56.956 INFO:teuthology.orchestra.run.smithi089.stdout:Reading package lists... 2023-10-18T10:43:57.001 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr-cephadm (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:57.125 INFO:teuthology.orchestra.run.smithi089.stdout:Building dependency tree... 2023-10-18T10:43:57.126 INFO:teuthology.orchestra.run.smithi089.stdout:Reading state information... 2023-10-18T10:43:57.143 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:57.270 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr-dashboard (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:57.278 INFO:teuthology.orchestra.run.smithi089.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2023-10-18T10:43:57.279 INFO:teuthology.orchestra.run.smithi089.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2023-10-18T10:43:57.279 INFO:teuthology.orchestra.run.smithi089.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2023-10-18T10:43:57.280 INFO:teuthology.orchestra.run.smithi089.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2023-10-18T10:43:57.280 INFO:teuthology.orchestra.run.smithi089.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2023-10-18T10:43:57.280 INFO:teuthology.orchestra.run.smithi089.stdout:The following packages were automatically installed and are no longer required: 2023-10-18T10:43:57.281 INFO:teuthology.orchestra.run.smithi089.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-18T10:43:57.281 INFO:teuthology.orchestra.run.smithi089.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-18T10:43:57.281 INFO:teuthology.orchestra.run.smithi089.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-18T10:43:57.282 INFO:teuthology.orchestra.run.smithi089.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-10-18T10:43:57.282 INFO:teuthology.orchestra.run.smithi089.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-10-18T10:43:57.282 INFO:teuthology.orchestra.run.smithi089.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-18T10:43:57.284 INFO:teuthology.orchestra.run.smithi089.stdout:The following additional packages will be installed: 2023-10-18T10:43:57.285 INFO:teuthology.orchestra.run.smithi089.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2023-10-18T10:43:57.285 INFO:teuthology.orchestra.run.smithi089.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2023-10-18T10:43:57.286 INFO:teuthology.orchestra.run.smithi089.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2023-10-18T10:43:57.286 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2023-10-18T10:43:57.286 INFO:teuthology.orchestra.run.smithi089.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-18T10:43:57.287 INFO:teuthology.orchestra.run.smithi089.stdout:Suggested packages: 2023-10-18T10:43:57.287 INFO:teuthology.orchestra.run.smithi089.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2023-10-18T10:43:57.287 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2023-10-18T10:43:57.288 INFO:teuthology.orchestra.run.smithi089.stdout: libengine-pkcs11-openssl 2023-10-18T10:43:57.330 INFO:teuthology.orchestra.run.smithi089.stdout:The following NEW packages will be installed: 2023-10-18T10:43:57.332 INFO:teuthology.orchestra.run.smithi089.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2023-10-18T10:43:57.332 INFO:teuthology.orchestra.run.smithi089.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2023-10-18T10:43:57.333 INFO:teuthology.orchestra.run.smithi089.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2023-10-18T10:43:57.333 INFO:teuthology.orchestra.run.smithi089.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2023-10-18T10:43:57.333 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2023-10-18T10:43:57.334 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2023-10-18T10:43:57.334 INFO:teuthology.orchestra.run.smithi089.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-18T10:43:57.335 INFO:teuthology.orchestra.run.smithi089.stdout:The following packages will be upgraded: 2023-10-18T10:43:57.335 INFO:teuthology.orchestra.run.smithi089.stdout: libssl3 2023-10-18T10:43:57.412 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-volume (18.0.0-5610-g6f3e6b48-1jammy) ... 2023-10-18T10:43:57.539 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-10-18T10:43:57.541 INFO:teuthology.orchestra.run.smithi089.stdout:1 upgraded, 39 newly installed, 0 to remove and 259 not upgraded. 2023-10-18T10:43:57.542 INFO:teuthology.orchestra.run.smithi089.stdout:Need to get 54.6 MB of archives. 2023-10-18T10:43:57.542 INFO:teuthology.orchestra.run.smithi089.stdout:After this operation, 202 MB of additional disk space will be used. 2023-10-18T10:43:57.542 INFO:teuthology.orchestra.run.smithi089.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2023-10-18T10:43:58.036 INFO:teuthology.orchestra.run.smithi089.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.10 [1,901 kB] 2023-10-18T10:43:58.295 INFO:teuthology.orchestra.run.smithi089.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2023-10-18T10:43:58.329 INFO:teuthology.orchestra.run.smithi089.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2023-10-18T10:43:58.331 INFO:teuthology.orchestra.run.smithi089.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2023-10-18T10:43:58.336 INFO:teuthology.orchestra.run.smithi089.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2023-10-18T10:43:58.339 INFO:teuthology.orchestra.run.smithi089.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2023-10-18T10:43:58.340 INFO:teuthology.orchestra.run.smithi089.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 libbluetooth3 amd64 5.64-0ubuntu1 [86.9 kB] 2023-10-18T10:43:58.343 INFO:teuthology.orchestra.run.smithi089.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2023-10-18T10:43:58.346 INFO:teuthology.orchestra.run.smithi089.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2023-10-18T10:43:58.511 INFO:teuthology.orchestra.run.smithi089.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2023-10-18T10:43:58.511 INFO:teuthology.orchestra.run.smithi089.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2023-10-18T10:43:58.724 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-10-18T10:43:58.762 INFO:teuthology.orchestra.run.smithi089.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2023-10-18T10:43:59.197 INFO:teuthology.orchestra.run.smithi154.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-18T10:43:59.201 DEBUG:teuthology.orchestra.run.smithi154:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-10-18T10:43:59.277 INFO:teuthology.orchestra.run.smithi154.stdout:Reading package lists... 2023-10-18T10:43:59.442 INFO:teuthology.orchestra.run.smithi154.stdout:Building dependency tree... 2023-10-18T10:43:59.443 INFO:teuthology.orchestra.run.smithi154.stdout:Reading state information... 2023-10-18T10:43:59.588 INFO:teuthology.orchestra.run.smithi154.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2023-10-18T10:43:59.589 INFO:teuthology.orchestra.run.smithi154.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2023-10-18T10:43:59.589 INFO:teuthology.orchestra.run.smithi154.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2023-10-18T10:43:59.589 INFO:teuthology.orchestra.run.smithi154.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2023-10-18T10:43:59.589 INFO:teuthology.orchestra.run.smithi154.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2023-10-18T10:43:59.590 INFO:teuthology.orchestra.run.smithi154.stdout:The following packages were automatically installed and are no longer required: 2023-10-18T10:43:59.590 INFO:teuthology.orchestra.run.smithi154.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-18T10:43:59.590 INFO:teuthology.orchestra.run.smithi154.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-18T10:43:59.590 INFO:teuthology.orchestra.run.smithi154.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-18T10:43:59.590 INFO:teuthology.orchestra.run.smithi154.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-10-18T10:43:59.592 INFO:teuthology.orchestra.run.smithi154.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-10-18T10:43:59.592 INFO:teuthology.orchestra.run.smithi154.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-18T10:43:59.592 INFO:teuthology.orchestra.run.smithi154.stdout:The following additional packages will be installed: 2023-10-18T10:43:59.592 INFO:teuthology.orchestra.run.smithi154.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2023-10-18T10:43:59.593 INFO:teuthology.orchestra.run.smithi154.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2023-10-18T10:43:59.593 INFO:teuthology.orchestra.run.smithi154.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2023-10-18T10:43:59.593 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2023-10-18T10:43:59.593 INFO:teuthology.orchestra.run.smithi154.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-18T10:43:59.593 INFO:teuthology.orchestra.run.smithi154.stdout:Suggested packages: 2023-10-18T10:43:59.594 INFO:teuthology.orchestra.run.smithi154.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2023-10-18T10:43:59.594 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2023-10-18T10:43:59.594 INFO:teuthology.orchestra.run.smithi154.stdout: libengine-pkcs11-openssl 2023-10-18T10:43:59.639 INFO:teuthology.orchestra.run.smithi154.stdout:The following NEW packages will be installed: 2023-10-18T10:43:59.640 INFO:teuthology.orchestra.run.smithi154.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2023-10-18T10:43:59.640 INFO:teuthology.orchestra.run.smithi154.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2023-10-18T10:43:59.641 INFO:teuthology.orchestra.run.smithi154.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2023-10-18T10:43:59.641 INFO:teuthology.orchestra.run.smithi154.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2023-10-18T10:43:59.641 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2023-10-18T10:43:59.642 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2023-10-18T10:43:59.642 INFO:teuthology.orchestra.run.smithi154.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-18T10:43:59.642 INFO:teuthology.orchestra.run.smithi154.stdout:The following packages will be upgraded: 2023-10-18T10:43:59.643 INFO:teuthology.orchestra.run.smithi154.stdout: libssl3 2023-10-18T10:43:59.772 INFO:teuthology.orchestra.run.smithi089.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2023-10-18T10:43:59.774 INFO:teuthology.orchestra.run.smithi089.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2023-10-18T10:43:59.774 INFO:teuthology.orchestra.run.smithi089.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2023-10-18T10:43:59.799 INFO:teuthology.orchestra.run.smithi089.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2023-10-18T10:43:59.800 INFO:teuthology.orchestra.run.smithi089.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2023-10-18T10:43:59.804 INFO:teuthology.orchestra.run.smithi089.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.9-0ubuntu0.22.04.1 [141 kB] 2023-10-18T10:43:59.806 INFO:teuthology.orchestra.run.smithi089.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2023-10-18T10:43:59.813 INFO:teuthology.orchestra.run.smithi089.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2023-10-18T10:43:59.825 INFO:teuthology.orchestra.run.smithi154.stdout:1 upgraded, 39 newly installed, 0 to remove and 259 not upgraded. 2023-10-18T10:43:59.825 INFO:teuthology.orchestra.run.smithi154.stdout:Need to get 54.6 MB of archives. 2023-10-18T10:43:59.826 INFO:teuthology.orchestra.run.smithi154.stdout:After this operation, 202 MB of additional disk space will be used. 2023-10-18T10:43:59.826 INFO:teuthology.orchestra.run.smithi154.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2023-10-18T10:43:59.978 INFO:teuthology.orchestra.run.smithi089.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.10 [2,372 kB] 2023-10-18T10:44:00.321 INFO:teuthology.orchestra.run.smithi154.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.10 [1,901 kB] 2023-10-18T10:44:00.578 INFO:teuthology.orchestra.run.smithi154.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2023-10-18T10:44:00.612 INFO:teuthology.orchestra.run.smithi154.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2023-10-18T10:44:00.614 INFO:teuthology.orchestra.run.smithi154.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2023-10-18T10:44:00.619 INFO:teuthology.orchestra.run.smithi154.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2023-10-18T10:44:00.622 INFO:teuthology.orchestra.run.smithi154.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2023-10-18T10:44:00.623 INFO:teuthology.orchestra.run.smithi154.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 libbluetooth3 amd64 5.64-0ubuntu1 [86.9 kB] 2023-10-18T10:44:00.627 INFO:teuthology.orchestra.run.smithi154.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2023-10-18T10:44:00.629 INFO:teuthology.orchestra.run.smithi154.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2023-10-18T10:44:00.718 INFO:teuthology.orchestra.run.smithi089.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2023-10-18T10:44:00.721 INFO:teuthology.orchestra.run.smithi089.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2023-10-18T10:44:00.745 INFO:teuthology.orchestra.run.smithi089.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2023-10-18T10:44:00.794 INFO:teuthology.orchestra.run.smithi154.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2023-10-18T10:44:00.794 INFO:teuthology.orchestra.run.smithi154.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2023-10-18T10:44:00.822 INFO:teuthology.orchestra.run.smithi089.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2023-10-18T10:44:00.830 INFO:teuthology.orchestra.run.smithi089.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2023-10-18T10:44:00.831 INFO:teuthology.orchestra.run.smithi089.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2023-10-18T10:44:00.832 INFO:teuthology.orchestra.run.smithi089.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2023-10-18T10:44:00.833 INFO:teuthology.orchestra.run.smithi089.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.9-0ubuntu0.22.04.1 [1,222 kB] 2023-10-18T10:44:00.861 INFO:teuthology.orchestra.run.smithi089.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2023-10-18T10:44:00.864 INFO:teuthology.orchestra.run.smithi089.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.9-0ubuntu0.22.04.1 [16.1 MB] 2023-10-18T10:44:01.040 INFO:teuthology.orchestra.run.smithi154.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2023-10-18T10:44:01.490 INFO:teuthology.orchestra.run.smithi089.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2023-10-18T10:44:01.572 INFO:teuthology.orchestra.run.smithi089.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2023-10-18T10:44:01.572 INFO:teuthology.orchestra.run.smithi089.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2023-10-18T10:44:01.572 INFO:teuthology.orchestra.run.smithi089.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2023-10-18T10:44:01.662 INFO:teuthology.orchestra.run.smithi089.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2023-10-18T10:44:01.667 INFO:teuthology.orchestra.run.smithi089.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2023-10-18T10:44:02.060 INFO:teuthology.orchestra.run.smithi154.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2023-10-18T10:44:02.062 INFO:teuthology.orchestra.run.smithi154.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2023-10-18T10:44:02.062 INFO:teuthology.orchestra.run.smithi154.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2023-10-18T10:44:02.065 INFO:teuthology.orchestra.run.smithi154.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2023-10-18T10:44:02.065 INFO:teuthology.orchestra.run.smithi154.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2023-10-18T10:44:02.070 INFO:teuthology.orchestra.run.smithi154.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.9-0ubuntu0.22.04.1 [141 kB] 2023-10-18T10:44:02.071 INFO:teuthology.orchestra.run.smithi154.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2023-10-18T10:44:02.096 INFO:teuthology.orchestra.run.smithi154.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2023-10-18T10:44:02.130 INFO:teuthology.orchestra.run.smithi089.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2023-10-18T10:44:02.132 INFO:teuthology.orchestra.run.smithi089.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2023-10-18T10:44:02.260 INFO:teuthology.orchestra.run.smithi154.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.10 [2,372 kB] 2023-10-18T10:44:02.523 INFO:teuthology.orchestra.run.smithi089.stdout:Preconfiguring packages ... 2023-10-18T10:44:02.675 INFO:teuthology.orchestra.run.smithi089.stdout:Fetched 54.6 MB in 5s (11.4 MB/s) 2023-10-18T10:44:02.793 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package flex. 2023-10-18T10:44:02.864 INFO:teuthology.orchestra.run.smithi089.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139908 files and directories currently installed.) 2023-10-18T10:44:02.868 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2023-10-18T10:44:02.942 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking flex (2.6.4-8build2) ... 2023-10-18T10:44:03.002 INFO:teuthology.orchestra.run.smithi154.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2023-10-18T10:44:03.002 INFO:teuthology.orchestra.run.smithi154.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2023-10-18T10:44:03.029 INFO:teuthology.orchestra.run.smithi154.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2023-10-18T10:44:03.106 INFO:teuthology.orchestra.run.smithi154.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2023-10-18T10:44:03.114 INFO:teuthology.orchestra.run.smithi154.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2023-10-18T10:44:03.115 INFO:teuthology.orchestra.run.smithi154.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2023-10-18T10:44:03.115 INFO:teuthology.orchestra.run.smithi154.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2023-10-18T10:44:03.116 INFO:teuthology.orchestra.run.smithi154.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.9-0ubuntu0.22.04.1 [1,222 kB] 2023-10-18T10:44:03.144 INFO:teuthology.orchestra.run.smithi154.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2023-10-18T10:44:03.148 INFO:teuthology.orchestra.run.smithi154.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.9-0ubuntu0.22.04.1 [16.1 MB] 2023-10-18T10:44:03.478 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.10_amd64.deb ... 2023-10-18T10:44:03.647 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.10) over (3.0.2-0ubuntu1.7) ... 2023-10-18T10:44:03.771 INFO:teuthology.orchestra.run.smithi154.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2023-10-18T10:44:03.853 INFO:teuthology.orchestra.run.smithi154.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2023-10-18T10:44:03.854 INFO:teuthology.orchestra.run.smithi154.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2023-10-18T10:44:03.854 INFO:teuthology.orchestra.run.smithi154.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2023-10-18T10:44:03.946 INFO:teuthology.orchestra.run.smithi154.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2023-10-18T10:44:03.950 INFO:teuthology.orchestra.run.smithi154.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2023-10-18T10:44:04.045 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.10) ... 2023-10-18T10:44:04.414 INFO:teuthology.orchestra.run.smithi154.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2023-10-18T10:44:04.415 INFO:teuthology.orchestra.run.smithi154.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2023-10-18T10:44:04.573 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package bison. 2023-10-18T10:44:04.645 INFO:teuthology.orchestra.run.smithi089.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139981 files and directories currently installed.) 2023-10-18T10:44:04.649 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2023-10-18T10:44:04.684 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2023-10-18T10:44:04.810 INFO:teuthology.orchestra.run.smithi154.stdout:Preconfiguring packages ... 2023-10-18T10:44:04.947 INFO:teuthology.orchestra.run.smithi154.stdout:Fetched 54.6 MB in 5s (11.4 MB/s) 2023-10-18T10:44:05.034 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-10-18T10:44:05.048 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2023-10-18T10:44:05.082 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package flex. 2023-10-18T10:44:05.092 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2023-10-18T10:44:05.154 INFO:teuthology.orchestra.run.smithi154.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139908 files and directories currently installed.) 2023-10-18T10:44:05.158 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2023-10-18T10:44:05.214 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking flex (2.6.4-8build2) ... 2023-10-18T10:44:05.454 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package dump. 2023-10-18T10:44:05.469 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2023-10-18T10:44:05.546 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking dump (0.4b47-3) ... 2023-10-18T10:44:05.775 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.10_amd64.deb ... 2023-10-18T10:44:05.833 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package indent. 2023-10-18T10:44:05.848 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2023-10-18T10:44:05.883 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2023-10-18T10:44:05.953 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.10) over (3.0.2-0ubuntu1.7) ... 2023-10-18T10:44:06.110 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-10-18T10:44:06.125 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2023-10-18T10:44:06.169 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2023-10-18T10:44:06.542 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.10) ... 2023-10-18T10:44:06.543 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-10-18T10:44:06.547 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1_amd64.deb ... 2023-10-18T10:44:06.590 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-10-18T10:44:06.867 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-10-18T10:44:06.882 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2023-10-18T10:44:06.917 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2023-10-18T10:44:06.928 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package bison. 2023-10-18T10:44:07.001 INFO:teuthology.orchestra.run.smithi154.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139981 files and directories currently installed.) 2023-10-18T10:44:07.004 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2023-10-18T10:44:07.065 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2023-10-18T10:44:07.203 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libfl2:amd64. 2023-10-18T10:44:07.218 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2023-10-18T10:44:07.270 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2023-10-18T10:44:07.473 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-10-18T10:44:07.488 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2023-10-18T10:44:07.522 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libfl-dev:amd64. 2023-10-18T10:44:07.532 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2023-10-18T10:44:07.538 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2023-10-18T10:44:07.581 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2023-10-18T10:44:07.826 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package dump. 2023-10-18T10:44:07.841 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2023-10-18T10:44:07.866 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libjson-perl. 2023-10-18T10:44:07.876 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking dump (0.4b47-3) ... 2023-10-18T10:44:07.882 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2023-10-18T10:44:07.925 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libjson-perl (4.04000-1) ... 2023-10-18T10:44:08.155 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package indent. 2023-10-18T10:44:08.169 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2023-10-18T10:44:08.213 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2023-10-18T10:44:08.236 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libllvm14:amd64. 2023-10-18T10:44:08.252 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2023-10-18T10:44:08.286 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-10-18T10:44:08.552 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-10-18T10:44:08.566 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2023-10-18T10:44:08.602 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2023-10-18T10:44:08.922 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-10-18T10:44:08.937 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1_amd64.deb ... 2023-10-18T10:44:08.980 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-10-18T10:44:09.241 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-10-18T10:44:09.256 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2023-10-18T10:44:09.299 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2023-10-18T10:44:09.611 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libfl2:amd64. 2023-10-18T10:44:09.627 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2023-10-18T10:44:09.673 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-10-18T10:44:09.677 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2023-10-18T10:44:09.688 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-10-18T10:44:09.722 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-10-18T10:44:09.905 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libfl-dev:amd64. 2023-10-18T10:44:09.921 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2023-10-18T10:44:09.942 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libmbim-proxy. 2023-10-18T10:44:09.957 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-10-18T10:44:09.965 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2023-10-18T10:44:09.992 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-10-18T10:44:10.233 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libjson-perl. 2023-10-18T10:44:10.248 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2023-10-18T10:44:10.291 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libjson-perl (4.04000-1) ... 2023-10-18T10:44:10.294 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-10-18T10:44:10.310 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-10-18T10:44:10.352 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-10-18T10:44:10.594 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libllvm14:amd64. 2023-10-18T10:44:10.610 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2023-10-18T10:44:10.652 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-10-18T10:44:10.722 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libndp0:amd64. 2023-10-18T10:44:10.738 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2023-10-18T10:44:10.780 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2023-10-18T10:44:11.108 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libnm0:amd64. 2023-10-18T10:44:11.124 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2023-10-18T10:44:11.166 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-10-18T10:44:11.477 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libpq5:amd64. 2023-10-18T10:44:11.493 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../16-libpq5_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-18T10:44:11.527 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-10-18T10:44:11.931 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-10-18T10:44:11.947 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-10-18T10:44:11.989 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-10-18T10:44:12.030 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-10-18T10:44:12.046 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-10-18T10:44:12.088 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-10-18T10:44:12.284 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libqmi-proxy. 2023-10-18T10:44:12.299 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-10-18T10:44:12.316 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libmbim-proxy. 2023-10-18T10:44:12.331 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-10-18T10:44:12.342 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-10-18T10:44:12.374 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-10-18T10:44:12.577 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-10-18T10:44:12.593 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.10_amd64.deb ... 2023-10-18T10:44:12.627 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.10) ... 2023-10-18T10:44:12.685 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-10-18T10:44:12.701 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-10-18T10:44:12.743 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-10-18T10:44:13.113 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libndp0:amd64. 2023-10-18T10:44:13.128 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2023-10-18T10:44:13.139 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-10-18T10:44:13.154 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2023-10-18T10:44:13.171 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2023-10-18T10:44:13.197 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2023-10-18T10:44:13.525 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libnm0:amd64. 2023-10-18T10:44:13.541 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2023-10-18T10:44:13.559 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package modemmanager. 2023-10-18T10:44:13.574 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-10-18T10:44:13.575 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-10-18T10:44:13.617 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-10-18T10:44:13.919 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libpq5:amd64. 2023-10-18T10:44:13.935 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../16-libpq5_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-18T10:44:13.977 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-10-18T10:44:14.273 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package network-manager. 2023-10-18T10:44:14.288 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2023-10-18T10:44:14.330 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-10-18T10:44:14.340 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2023-10-18T10:44:14.346 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-10-18T10:44:14.388 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-10-18T10:44:14.674 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libqmi-proxy. 2023-10-18T10:44:14.691 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-10-18T10:44:14.732 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-10-18T10:44:14.897 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ppp. 2023-10-18T10:44:14.912 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2023-10-18T10:44:14.934 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-10-18T10:44:14.950 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.10_amd64.deb ... 2023-10-18T10:44:14.993 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.10) ... 2023-10-18T10:44:15.247 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2023-10-18T10:44:15.538 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-10-18T10:44:15.553 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2023-10-18T10:44:15.568 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package pptp-linux. 2023-10-18T10:44:15.582 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2023-10-18T10:44:15.596 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2023-10-18T10:44:15.618 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2023-10-18T10:44:15.879 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package network-manager-pptp. 2023-10-18T10:44:15.893 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2023-10-18T10:44:15.932 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package modemmanager. 2023-10-18T10:44:15.937 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2023-10-18T10:44:15.948 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-10-18T10:44:15.982 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-10-18T10:44:16.240 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package postgresql-client-common. 2023-10-18T10:44:16.255 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2023-10-18T10:44:16.324 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking postgresql-client-common (238) ... 2023-10-18T10:44:16.585 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package postgresql-client-14. 2023-10-18T10:44:16.600 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../27-postgresql-client-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-18T10:44:16.613 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package network-manager. 2023-10-18T10:44:16.629 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2023-10-18T10:44:16.635 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-18T10:44:16.671 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2023-10-18T10:44:17.271 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ppp. 2023-10-18T10:44:17.286 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2023-10-18T10:44:17.350 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package postgresql-common. 2023-10-18T10:44:17.365 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2023-10-18T10:44:17.427 INFO:teuthology.orchestra.run.smithi089.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-10-18T10:44:17.482 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking postgresql-common (238) ... 2023-10-18T10:44:17.636 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2023-10-18T10:44:17.941 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package postgresql-14. 2023-10-18T10:44:17.956 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../29-postgresql-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-18T10:44:17.975 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package pptp-linux. 2023-10-18T10:44:17.989 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2023-10-18T10:44:18.002 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-18T10:44:18.033 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2023-10-18T10:44:18.277 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package network-manager-pptp. 2023-10-18T10:44:18.292 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2023-10-18T10:44:18.336 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2023-10-18T10:44:19.195 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package postgresql. 2023-10-18T10:44:19.199 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package postgresql-client-common. 2023-10-18T10:44:19.209 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2023-10-18T10:44:19.214 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2023-10-18T10:44:19.244 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking postgresql (14+238) ... 2023-10-18T10:44:19.257 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking postgresql-client-common (238) ... 2023-10-18T10:44:19.463 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package postgresql-client. 2023-10-18T10:44:19.478 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2023-10-18T10:44:19.522 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking postgresql-client (14+238) ... 2023-10-18T10:44:19.561 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package postgresql-client-14. 2023-10-18T10:44:19.576 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../27-postgresql-client-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-18T10:44:19.619 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-18T10:44:19.749 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package postgresql-contrib. 2023-10-18T10:44:19.763 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2023-10-18T10:44:19.807 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking postgresql-contrib (14+238) ... 2023-10-18T10:44:20.160 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package tcl8.6. 2023-10-18T10:44:20.175 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2023-10-18T10:44:20.207 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package postgresql-common. 2023-10-18T10:44:20.218 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2023-10-18T10:44:20.222 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2023-10-18T10:44:20.283 INFO:teuthology.orchestra.run.smithi154.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-10-18T10:44:20.322 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking postgresql-common (238) ... 2023-10-18T10:44:20.496 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package tcl. 2023-10-18T10:44:20.511 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2023-10-18T10:44:20.558 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking tcl (8.6.11+1build2) ... 2023-10-18T10:44:20.715 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package postgresql-14. 2023-10-18T10:44:20.729 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../29-postgresql-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-18T10:44:20.776 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-18T10:44:20.890 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package wpasupplicant. 2023-10-18T10:44:20.906 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2023-10-18T10:44:20.948 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2023-10-18T10:44:21.319 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-10-18T10:44:21.335 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2023-10-18T10:44:21.377 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2023-10-18T10:44:21.764 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package usb-modeswitch. 2023-10-18T10:44:21.780 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2023-10-18T10:44:21.839 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2023-10-18T10:44:21.927 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package postgresql. 2023-10-18T10:44:21.941 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2023-10-18T10:44:21.976 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking postgresql (14+238) ... 2023-10-18T10:44:22.115 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up postgresql-client-common (238) ... 2023-10-18T10:44:22.204 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package postgresql-client. 2023-10-18T10:44:22.218 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2023-10-18T10:44:22.262 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking postgresql-client (14+238) ... 2023-10-18T10:44:22.362 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up flex (2.6.4-8build2) ... 2023-10-18T10:44:22.488 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2023-10-18T10:44:22.531 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package postgresql-contrib. 2023-10-18T10:44:22.545 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2023-10-18T10:44:22.589 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking postgresql-contrib (14+238) ... 2023-10-18T10:44:22.614 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-10-18T10:44:22.740 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2023-10-18T10:44:22.858 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2023-10-18T10:44:22.891 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package tcl8.6. 2023-10-18T10:44:22.906 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2023-10-18T10:44:22.950 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2023-10-18T10:44:22.958 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-18T10:44:23.219 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package tcl. 2023-10-18T10:44:23.234 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2023-10-18T10:44:23.273 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking tcl (8.6.11+1build2) ... 2023-10-18T10:44:23.647 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package wpasupplicant. 2023-10-18T10:44:23.662 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2023-10-18T10:44:23.705 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2023-10-18T10:44:23.957 INFO:teuthology.orchestra.run.smithi089.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-10-18T10:44:24.043 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-10-18T10:44:24.048 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2023-10-18T10:44:24.058 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2023-10-18T10:44:24.109 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2023-10-18T10:44:24.403 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-18T10:44:24.403 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-18T10:44:24.438 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package usb-modeswitch. 2023-10-18T10:44:24.453 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2023-10-18T10:44:24.504 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2023-10-18T10:44:24.787 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up postgresql-client-common (238) ... 2023-10-18T10:44:24.994 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up flex (2.6.4-8build2) ... 2023-10-18T10:44:25.076 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2023-10-18T10:44:25.120 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2023-10-18T10:44:25.210 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.10) ... 2023-10-18T10:44:25.254 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-10-18T10:44:25.336 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2023-10-18T10:44:25.380 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2023-10-18T10:44:25.462 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2023-10-18T10:44:25.498 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2023-10-18T10:44:25.563 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2023-10-18T10:44:25.615 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-18T10:44:25.690 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-10-18T10:44:25.799 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2023-10-18T10:44:26.512 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-10-18T10:44:26.602 INFO:teuthology.orchestra.run.smithi154.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-10-18T10:44:26.641 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up dump (0.4b47-3) ... 2023-10-18T10:44:26.705 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2023-10-18T10:44:26.736 INFO:teuthology.orchestra.run.smithi089.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-10-18T10:44:26.834 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-10-18T10:44:27.074 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-18T10:44:27.075 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-18T10:44:27.602 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-10-18T10:44:27.716 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2023-10-18T10:44:27.719 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up tcl (8.6.11+1build2) ... 2023-10-18T10:44:27.829 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libjson-perl (4.04000-1) ... 2023-10-18T10:44:27.842 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.10) ... 2023-10-18T10:44:27.976 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2023-10-18T10:44:27.988 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2023-10-18T10:44:28.078 INFO:teuthology.orchestra.run.smithi089.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2023-10-18T10:44:28.111 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2023-10-18T10:44:28.140 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2023-10-18T10:44:28.270 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2023-10-18T10:44:28.274 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2023-10-18T10:44:28.396 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-10-18T10:44:28.451 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2023-10-18T10:44:28.522 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2023-10-18T10:44:28.576 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-10-18T10:44:28.694 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2023-10-18T10:44:28.837 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up postgresql-client (14+238) ... 2023-10-18T10:44:28.963 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2023-10-18T10:44:29.227 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-10-18T10:44:29.378 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up dump (0.4b47-3) ... 2023-10-18T10:44:29.474 INFO:teuthology.orchestra.run.smithi154.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-10-18T10:44:29.539 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-10-18T10:44:29.601 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:44:29.601 INFO:teuthology.orchestra.run.smithi089.stdout:The following network interfaces were found in /etc/network/interfaces 2023-10-18T10:44:29.601 INFO:teuthology.orchestra.run.smithi089.stdout:which means they are currently configured by ifupdown: 2023-10-18T10:44:29.601 INFO:teuthology.orchestra.run.smithi089.stdout:- enp3s0f1 2023-10-18T10:44:29.602 INFO:teuthology.orchestra.run.smithi089.stdout:If you want to manage those interfaces with NetworkManager instead 2023-10-18T10:44:29.602 INFO:teuthology.orchestra.run.smithi089.stdout:remove their configuration from /etc/network/interfaces. 2023-10-18T10:44:29.602 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:44:29.665 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-10-18T10:44:29.734 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-10-18T10:44:29.782 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up tcl (8.6.11+1build2) ... 2023-10-18T10:44:29.883 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libjson-perl (4.04000-1) ... 2023-10-18T10:44:30.000 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2023-10-18T10:44:30.099 INFO:teuthology.orchestra.run.smithi154.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2023-10-18T10:44:30.142 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-10-18T10:44:30.169 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2023-10-18T10:44:30.295 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2023-10-18T10:44:30.497 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2023-10-18T10:44:30.569 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-10-18T10:44:30.622 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-10-18T10:44:30.748 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2023-10-18T10:44:30.891 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up postgresql-client (14+238) ... 2023-10-18T10:44:31.001 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2023-10-18T10:44:31.249 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-10-18T10:44:31.392 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2023-10-18T10:44:31.551 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2023-10-18T10:44:31.610 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-18T10:44:31.610 INFO:teuthology.orchestra.run.smithi154.stdout:The following network interfaces were found in /etc/network/interfaces 2023-10-18T10:44:31.610 INFO:teuthology.orchestra.run.smithi154.stdout:which means they are currently configured by ifupdown: 2023-10-18T10:44:31.611 INFO:teuthology.orchestra.run.smithi154.stdout:- enp3s0f1 2023-10-18T10:44:31.611 INFO:teuthology.orchestra.run.smithi154.stdout:If you want to manage those interfaces with NetworkManager instead 2023-10-18T10:44:31.611 INFO:teuthology.orchestra.run.smithi154.stdout:remove their configuration from /etc/network/interfaces. 2023-10-18T10:44:31.611 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-18T10:44:31.677 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up postgresql-common (238) ... 2023-10-18T10:44:31.742 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-10-18T10:44:32.146 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-10-18T10:44:32.577 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-10-18T10:44:32.644 INFO:teuthology.orchestra.run.smithi089.stdout:Adding user postgres to group ssl-cert 2023-10-18T10:44:32.921 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:44:32.921 INFO:teuthology.orchestra.run.smithi089.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-10-18T10:44:33.181 INFO:teuthology.orchestra.run.smithi089.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-10-18T10:44:33.181 INFO:teuthology.orchestra.run.smithi089.stdout:Removing obsolete dictionary files: 2023-10-18T10:44:33.253 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-10-18T10:44:33.396 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2023-10-18T10:44:33.522 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2023-10-18T10:44:33.649 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up postgresql-common (238) ... 2023-10-18T10:44:33.906 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-10-18T10:44:34.512 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-18T10:44:34.860 INFO:teuthology.orchestra.run.smithi089.stdout:Creating new PostgreSQL cluster 14/main ... 2023-10-18T10:44:34.878 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2023-10-18T10:44:34.894 INFO:teuthology.orchestra.run.smithi154.stdout:Adding user postgres to group ssl-cert 2023-10-18T10:44:34.938 INFO:teuthology.orchestra.run.smithi089.stdout:The files belonging to this database system will be owned by user "postgres". 2023-10-18T10:44:34.938 INFO:teuthology.orchestra.run.smithi089.stdout:This user must also own the server process. 2023-10-18T10:44:34.938 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:44:34.938 INFO:teuthology.orchestra.run.smithi089.stdout:The database cluster will be initialized with locale "en_US". 2023-10-18T10:44:34.939 INFO:teuthology.orchestra.run.smithi089.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-10-18T10:44:34.939 INFO:teuthology.orchestra.run.smithi089.stdout:The default text search configuration will be set to "english". 2023-10-18T10:44:34.939 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:44:34.939 INFO:teuthology.orchestra.run.smithi089.stdout:Data page checksums are disabled. 2023-10-18T10:44:34.939 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:44:34.940 INFO:teuthology.orchestra.run.smithi089.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2023-10-18T10:44:34.940 INFO:teuthology.orchestra.run.smithi089.stdout:creating subdirectories ... ok 2023-10-18T10:44:34.940 INFO:teuthology.orchestra.run.smithi089.stdout:selecting dynamic shared memory implementation ... posix 2023-10-18T10:44:34.963 INFO:teuthology.orchestra.run.smithi089.stdout:selecting default max_connections ... 100 2023-10-18T10:44:34.998 INFO:teuthology.orchestra.run.smithi089.stdout:selecting default shared_buffers ... 128MB 2023-10-18T10:44:35.012 INFO:teuthology.orchestra.run.smithi089.stdout:selecting default time zone ... Etc/UTC 2023-10-18T10:44:35.013 INFO:teuthology.orchestra.run.smithi089.stdout:creating configuration files ... ok 2023-10-18T10:44:35.171 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-18T10:44:35.171 INFO:teuthology.orchestra.run.smithi154.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-10-18T10:44:35.181 INFO:teuthology.orchestra.run.smithi089.stdout:running bootstrap script ... ok 2023-10-18T10:44:35.421 INFO:teuthology.orchestra.run.smithi154.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-10-18T10:44:35.422 INFO:teuthology.orchestra.run.smithi154.stdout:Removing obsolete dictionary files: 2023-10-18T10:44:35.786 INFO:teuthology.orchestra.run.smithi089.stdout:performing post-bootstrap initialization ... ok 2023-10-18T10:44:36.125 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-10-18T10:44:36.363 INFO:teuthology.orchestra.run.smithi089.stdout:syncing data to disk ... ok 2023-10-18T10:44:36.700 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-18T10:44:36.888 INFO:teuthology.orchestra.run.smithi089.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-10-18T10:44:37.050 INFO:teuthology.orchestra.run.smithi154.stdout:Creating new PostgreSQL cluster 14/main ... 2023-10-18T10:44:37.067 INFO:teuthology.orchestra.run.smithi154.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2023-10-18T10:44:37.124 INFO:teuthology.orchestra.run.smithi154.stdout:The files belonging to this database system will be owned by user "postgres". 2023-10-18T10:44:37.124 INFO:teuthology.orchestra.run.smithi154.stdout:This user must also own the server process. 2023-10-18T10:44:37.124 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-18T10:44:37.124 INFO:teuthology.orchestra.run.smithi154.stdout:The database cluster will be initialized with locale "en_US". 2023-10-18T10:44:37.125 INFO:teuthology.orchestra.run.smithi154.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-10-18T10:44:37.125 INFO:teuthology.orchestra.run.smithi154.stdout:The default text search configuration will be set to "english". 2023-10-18T10:44:37.125 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-18T10:44:37.125 INFO:teuthology.orchestra.run.smithi154.stdout:Data page checksums are disabled. 2023-10-18T10:44:37.125 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-18T10:44:37.125 INFO:teuthology.orchestra.run.smithi154.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2023-10-18T10:44:37.126 INFO:teuthology.orchestra.run.smithi154.stdout:creating subdirectories ... ok 2023-10-18T10:44:37.126 INFO:teuthology.orchestra.run.smithi154.stdout:selecting dynamic shared memory implementation ... posix 2023-10-18T10:44:37.150 INFO:teuthology.orchestra.run.smithi154.stdout:selecting default max_connections ... 100 2023-10-18T10:44:37.185 INFO:teuthology.orchestra.run.smithi154.stdout:selecting default shared_buffers ... 128MB 2023-10-18T10:44:37.325 INFO:teuthology.orchestra.run.smithi154.stdout:selecting default time zone ... Etc/UTC 2023-10-18T10:44:37.325 INFO:teuthology.orchestra.run.smithi154.stdout:creating configuration files ... ok 2023-10-18T10:44:37.369 INFO:teuthology.orchestra.run.smithi154.stdout:running bootstrap script ... ok 2023-10-18T10:44:37.995 INFO:teuthology.orchestra.run.smithi154.stdout:performing post-bootstrap initialization ... ok 2023-10-18T10:44:38.600 INFO:teuthology.orchestra.run.smithi154.stdout:syncing data to disk ... ok 2023-10-18T10:44:39.105 INFO:teuthology.orchestra.run.smithi154.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-10-18T10:44:39.970 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-10-18T10:44:40.096 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up postgresql-contrib (14+238) ... 2023-10-18T10:44:40.222 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up postgresql (14+238) ... 2023-10-18T10:44:40.348 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-10-18T10:44:40.474 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-10-18T10:44:40.735 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-10-18T10:44:40.735 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-10-18T10:44:41.460 INFO:teuthology.orchestra.run.smithi089.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-10-18T10:44:41.770 INFO:teuthology.orchestra.run.smithi089.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-10-18T10:44:41.921 INFO:teuthology.orchestra.run.smithi089.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-10-18T10:44:42.149 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-10-18T10:44:42.259 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up postgresql-contrib (14+238) ... 2023-10-18T10:44:42.385 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up postgresql (14+238) ... 2023-10-18T10:44:42.510 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-10-18T10:44:42.637 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-10-18T10:44:42.898 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-10-18T10:44:42.898 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-10-18T10:44:43.375 INFO:teuthology.orchestra.run.smithi089.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2023-10-18T10:44:43.475 INFO:teuthology.orchestra.run.smithi089.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-10-18T10:44:43.605 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-10-18T10:44:43.916 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-10-18T10:44:44.051 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-10-18T10:44:44.131 INFO:teuthology.orchestra.run.smithi089.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-18T10:44:44.136 DEBUG:teuthology.parallel:result is None 2023-10-18T10:44:45.512 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2023-10-18T10:44:45.613 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-10-18T10:44:46.270 INFO:teuthology.orchestra.run.smithi154.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-18T10:44:46.274 DEBUG:teuthology.parallel:result is None 2023-10-18T10:44:46.275 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=6f3e6b48891b9b50415c0822b6b870797fcda16d 2023-10-18T10:44:46.423 DEBUG:teuthology.orchestra.run.smithi089:> dpkg-query -W -f '${Version}' ceph 2023-10-18T10:44:46.448 INFO:teuthology.orchestra.run.smithi089.stdout:18.0.0-5610-g6f3e6b48-1jammy 2023-10-18T10:44:46.448 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-5610-g6f3e6b48-1jammy 2023-10-18T10:44:46.449 INFO:teuthology.task.install:The correct ceph version 18.0.0-5610-g6f3e6b48-1jammy is installed. 2023-10-18T10:44:46.449 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=6f3e6b48891b9b50415c0822b6b870797fcda16d 2023-10-18T10:44:46.594 DEBUG:teuthology.orchestra.run.smithi154:> dpkg-query -W -f '${Version}' ceph 2023-10-18T10:44:46.616 INFO:teuthology.orchestra.run.smithi154.stdout:18.0.0-5610-g6f3e6b48-1jammy 2023-10-18T10:44:46.616 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-5610-g6f3e6b48-1jammy 2023-10-18T10:44:46.617 INFO:teuthology.task.install:The correct ceph version 18.0.0-5610-g6f3e6b48-1jammy is installed. 2023-10-18T10:44:46.617 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-18T10:44:46.618 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:44:46.618 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-18T10:44:46.641 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T10:44:46.642 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-18T10:44:46.679 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-18T10:44:46.680 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:44:46.680 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/daemon-helper 2023-10-18T10:44:46.705 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-18T10:44:46.768 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T10:44:46.769 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/daemon-helper 2023-10-18T10:44:46.789 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-18T10:44:46.854 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-18T10:44:46.854 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:44:46.855 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-18T10:44:46.878 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-18T10:44:46.944 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T10:44:46.945 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-18T10:44:46.965 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-18T10:44:47.030 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-18T10:44:47.030 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:44:47.031 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/stdin-killer 2023-10-18T10:44:47.053 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-18T10:44:47.116 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T10:44:47.117 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/stdin-killer 2023-10-18T10:44:47.137 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-18T10:44:47.202 INFO:teuthology.run_tasks:Running task ceph... 2023-10-18T10:44:47.267 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-18T10:44:47.268 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 777 /var/log/ceph 2023-10-18T10:44:47.272 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 777 /var/log/ceph 2023-10-18T10:44:47.292 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-18T10:44:47.293 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-18T10:44:47.335 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-18T10:44:47.354 INFO:tasks.ceph:Creating extra log directories... 2023-10-18T10:44:47.355 DEBUG:teuthology.orchestra.run.smithi089:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-18T10:44:47.399 DEBUG:teuthology.orchestra.run.smithi154:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-18T10:44:47.430 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-18T10:44:47.430 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-18T10:44:47.431 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-18T10:44:47.431 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:44:47.432 DEBUG:teuthology.orchestra.run.smithi089:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-18T10:44:47.469 DEBUG:teuthology.orchestra.run.smithi089:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-18T10:44:47.566 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T10:44:47.567 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-18T10:44:47.575 DEBUG:teuthology.orchestra.run.smithi154:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-18T10:44:47.674 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-18T10:44:47.675 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'but it is still running', 'slow request', 'MDS_CLIENT_LATE_RELEASE', 't responding to mclientcaps', 'file system flag refuse_client_session is set', 'Degraded data redundancy', 'MDS_CLIENTS_LAGGY', 'Reduced data availability'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-18T10:44:47.675 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/dparmar-2023-10-18_10:19:04-fs:functional-wip-63105-distro-default-smithi/7431296', 'branch': 'wip-63105', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-recovery}', 'email': None, 'first_in_suite': False, 'job_id': '7431296', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'dparmar-2023-10-18_10:19:04-fs:functional-wip-63105-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-63105'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'but it is still running', 'slow request', 'MDS_CLIENT_LATE_RELEASE', 't responding to mclientcaps', 'file system flag refuse_client_session is set', 'Degraded data redundancy', 'MDS_CLIENTS_LAGGY', 'Reduced data availability'], 'sha1': '6f3e6b48891b9b50415c0822b6b870797fcda16d'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '6f3e6b48891b9b50415c0822b6b870797fcda16d'}}, 'kclient': {'syntax': 'v1'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-63105', 'sha1': '22c76d63886ee29a8c38dcd6634331fbd621f64a'}}, 'owner': 'scheduled_dparmar@teuthology', 'priority': 55, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7825, 'sha1': '6f3e6b48891b9b50415c0822b6b870797fcda16d', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-63105', 'suite_path': '/home/teuthworker/src/github.com_dparmar18_ceph_22c76d63886ee29a8c38dcd6634331fbd621f64a/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/dparmar18/ceph', 'suite_sha1': '22c76d63886ee29a8c38dcd6634331fbd621f64a', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi089.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK1OYlDgi4O1/6ng/uXhZ8MUwlMXFIH1rfYalkqOVAKHxilpUJbwZU6muYWI8DFrjmTYl+hk901BCq77/ZnsJAE=', 'smithi154.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD206tYTbUO+0hlBk/fd4mL1UBAKK4y+NSWI2uFM6AlUPiOt/tJXuysZRQtUD6n/+FEL5LTD7erHsfj2X4n3pZ4='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '6f3e6b48891b9b50415c0822b6b870797fcda16d'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_client_recovery']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '202b180cb047e798fb131047314a862593f45403', 'timestamp': '2023-10-18_10:19:04', 'tube': 'smithi', 'user': 'dparmar', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-18T10:44:47.676 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-18T10:44:47.680 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-18T10:44:47.689 DEBUG:teuthology.orchestra.run.smithi089:> sudo install -d -m0777 -- /var/run/ceph 2023-10-18T10:44:47.731 DEBUG:teuthology.orchestra.run.smithi154:> sudo install -d -m0777 -- /var/run/ceph 2023-10-18T10:44:47.754 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:44:47.754 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/scratch_devs of=/dev/stdout 2023-10-18T10:44:47.800 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-18T10:44:47.801 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_1 2023-10-18T10:44:47.852 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-18T10:44:47.852 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T10:44:47.852 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 892 Links: 1 2023-10-18T10:44:47.853 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T10:44:47.853 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2023-10-18 10:38:27.558610832 +0000 2023-10-18T10:44:47.853 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2023-10-18 10:38:27.174614262 +0000 2023-10-18T10:44:47.853 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2023-10-18 10:38:27.174614262 +0000 2023-10-18T10:44:47.853 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2023-10-18 10:38:27.174614262 +0000 2023-10-18T10:44:47.854 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-18T10:44:47.915 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2023-10-18T10:44:47.916 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2023-10-18T10:44:47.916 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000369251 s, 1.4 MB/s 2023-10-18T10:44:47.917 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-18T10:44:47.970 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_2 2023-10-18T10:44:48.025 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-18T10:44:48.025 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T10:44:48.025 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 909 Links: 1 2023-10-18T10:44:48.025 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T10:44:48.026 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2023-10-18 10:38:28.226604865 +0000 2023-10-18T10:44:48.026 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2023-10-18 10:38:27.834608367 +0000 2023-10-18T10:44:48.026 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2023-10-18 10:38:27.834608367 +0000 2023-10-18T10:44:48.026 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2023-10-18 10:38:27.834608367 +0000 2023-10-18T10:44:48.027 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-18T10:44:48.087 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2023-10-18T10:44:48.087 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2023-10-18T10:44:48.088 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000383648 s, 1.3 MB/s 2023-10-18T10:44:48.089 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-18T10:44:48.147 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_3 2023-10-18T10:44:48.196 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-18T10:44:48.196 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T10:44:48.196 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 926 Links: 1 2023-10-18T10:44:48.196 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T10:44:48.197 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2023-10-18 10:38:28.898598863 +0000 2023-10-18T10:44:48.197 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2023-10-18 10:38:28.506602363 +0000 2023-10-18T10:44:48.197 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2023-10-18 10:38:28.506602363 +0000 2023-10-18T10:44:48.197 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2023-10-18 10:38:28.506602363 +0000 2023-10-18T10:44:48.198 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-18T10:44:48.258 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2023-10-18T10:44:48.259 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2023-10-18T10:44:48.259 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000372292 s, 1.4 MB/s 2023-10-18T10:44:48.260 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-18T10:44:48.313 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_4 2023-10-18T10:44:48.369 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-18T10:44:48.369 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T10:44:48.370 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 942 Links: 1 2023-10-18T10:44:48.370 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T10:44:48.371 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2023-10-18 10:38:29.586592718 +0000 2023-10-18T10:44:48.371 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2023-10-18 10:38:29.182596325 +0000 2023-10-18T10:44:48.371 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2023-10-18 10:38:29.182596325 +0000 2023-10-18T10:44:48.371 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2023-10-18 10:38:29.182596325 +0000 2023-10-18T10:44:48.372 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-18T10:44:48.430 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2023-10-18T10:44:48.431 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2023-10-18T10:44:48.431 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000381982 s, 1.3 MB/s 2023-10-18T10:44:48.433 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-18T10:44:48.486 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-18T10:44:48.487 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T10:44:48.487 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/scratch_devs of=/dev/stdout 2023-10-18T10:44:48.497 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-18T10:44:48.497 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_1 2023-10-18T10:44:48.548 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-18T10:44:48.548 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T10:44:48.548 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 886 Links: 1 2023-10-18T10:44:48.548 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T10:44:48.549 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2023-10-18 10:38:26.709420756 +0000 2023-10-18T10:44:48.549 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2023-10-18 10:38:26.313424293 +0000 2023-10-18T10:44:48.549 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2023-10-18 10:38:26.313424293 +0000 2023-10-18T10:44:48.549 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2023-10-18 10:38:26.313424293 +0000 2023-10-18T10:44:48.550 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-18T10:44:48.610 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2023-10-18T10:44:48.610 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2023-10-18T10:44:48.611 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000387911 s, 1.3 MB/s 2023-10-18T10:44:48.612 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-18T10:44:48.665 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_2 2023-10-18T10:44:48.718 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-18T10:44:48.718 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T10:44:48.718 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 902 Links: 1 2023-10-18T10:44:48.719 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T10:44:48.719 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2023-10-18 10:38:27.337415146 +0000 2023-10-18T10:44:48.719 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2023-10-18 10:38:26.965418469 +0000 2023-10-18T10:44:48.719 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2023-10-18 10:38:26.965418469 +0000 2023-10-18T10:44:48.719 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2023-10-18 10:38:26.965418469 +0000 2023-10-18T10:44:48.720 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-18T10:44:48.779 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2023-10-18T10:44:48.779 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2023-10-18T10:44:48.779 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000384622 s, 1.3 MB/s 2023-10-18T10:44:48.781 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-18T10:44:48.833 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_3 2023-10-18T10:44:48.883 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-18T10:44:48.884 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T10:44:48.884 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 918 Links: 1 2023-10-18T10:44:48.884 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T10:44:48.884 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2023-10-18 10:38:28.017409073 +0000 2023-10-18T10:44:48.884 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2023-10-18 10:38:27.613412681 +0000 2023-10-18T10:44:48.885 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2023-10-18 10:38:27.613412681 +0000 2023-10-18T10:44:48.885 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2023-10-18 10:38:27.613412681 +0000 2023-10-18T10:44:48.885 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-18T10:44:48.946 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2023-10-18T10:44:48.947 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2023-10-18T10:44:48.947 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000402514 s, 1.3 MB/s 2023-10-18T10:44:48.948 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-18T10:44:49.002 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_4 2023-10-18T10:44:49.055 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-18T10:44:49.056 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T10:44:49.056 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 936 Links: 1 2023-10-18T10:44:49.056 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T10:44:49.056 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2023-10-18 10:38:28.685403106 +0000 2023-10-18T10:44:49.057 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2023-10-18 10:38:28.301406536 +0000 2023-10-18T10:44:49.057 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2023-10-18 10:38:28.301406536 +0000 2023-10-18T10:44:49.057 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2023-10-18 10:38:28.301406536 +0000 2023-10-18T10:44:49.058 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-18T10:44:49.118 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2023-10-18T10:44:49.119 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2023-10-18T10:44:49.119 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000638652 s, 802 kB/s 2023-10-18T10:44:49.120 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-18T10:44:49.173 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-18T10:44:49.174 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi089.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi154.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-18T10:44:49.174 INFO:tasks.ceph:Generating config... 2023-10-18T10:44:49.175 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-18T10:44:49.176 INFO:tasks.ceph:[client] debug client = 20 2023-10-18T10:44:49.176 INFO:tasks.ceph:[client] debug ms = 1 2023-10-18T10:44:49.176 INFO:tasks.ceph:[client] pid file = 2023-10-18T10:44:49.176 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-18T10:44:49.177 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-18T10:44:49.177 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-10-18T10:44:49.177 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-18T10:44:49.177 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-10-18T10:44:49.178 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-18T10:44:49.178 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-18T10:44:49.178 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-18T10:44:49.179 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-18T10:44:49.179 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-18T10:44:49.179 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-18T10:44:49.179 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-18T10:44:49.180 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-18T10:44:49.180 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-18T10:44:49.180 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-18T10:44:49.181 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-18T10:44:49.181 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-18T10:44:49.181 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-18T10:44:49.181 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-18T10:44:49.182 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-18T10:44:49.182 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-18T10:44:49.182 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-10-18T10:44:49.182 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-10-18T10:44:49.183 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-18T10:44:49.183 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-18T10:44:49.183 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-10-18T10:44:49.184 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-18T10:44:49.184 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-18T10:44:49.184 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-18T10:44:49.184 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-18T10:44:49.185 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-18T10:44:49.185 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-18T10:44:49.185 INFO:tasks.ceph:Setting up mon.a... 2023-10-18T10:44:49.186 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-18T10:44:49.232 INFO:teuthology.orchestra.run.smithi089.stdout:creating /etc/ceph/ceph.keyring 2023-10-18T10:44:49.241 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-18T10:44:49.332 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-18T10:44:49.353 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.89'), ('mon.b', '172.21.15.154'), ('mon.c', '[v2:172.21.15.154:3301,v1:172.21.15.154:6790]')] 2023-10-18T10:44:49.354 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.89,172.21.15.154,[v2:172.21.15.154:3301,v1:172.21.15.154:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-18T10:44:49.355 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:44:49.355 DEBUG:teuthology.orchestra.run.smithi089:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-18T10:44:49.404 DEBUG:teuthology.orchestra.run.smithi089:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.89 --add b 172.21.15.154 --addv c '[v2:172.21.15.154:3301,v1:172.21.15.154:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-18T10:44:49.494 INFO:teuthology.orchestra.run.smithi089.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-18T10:44:49.495 INFO:teuthology.orchestra.run.smithi089.stdout:monmaptool: generated fsid 1352f62a-2330-4e96-afcd-b8e87cd82667 2023-10-18T10:44:49.495 INFO:teuthology.orchestra.run.smithi089.stdout:setting min_mon_release = pacific 2023-10-18T10:44:49.495 INFO:teuthology.orchestra.run.smithi089.stdout:epoch 0 2023-10-18T10:44:49.495 INFO:teuthology.orchestra.run.smithi089.stdout:fsid 1352f62a-2330-4e96-afcd-b8e87cd82667 2023-10-18T10:44:49.496 INFO:teuthology.orchestra.run.smithi089.stdout:last_changed 2023-10-18T10:44:49.498620+0000 2023-10-18T10:44:49.496 INFO:teuthology.orchestra.run.smithi089.stdout:created 2023-10-18T10:44:49.498620+0000 2023-10-18T10:44:49.496 INFO:teuthology.orchestra.run.smithi089.stdout:min_mon_release 16 (pacific) 2023-10-18T10:44:49.496 INFO:teuthology.orchestra.run.smithi089.stdout:election_strategy: 1 2023-10-18T10:44:49.496 INFO:teuthology.orchestra.run.smithi089.stdout:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2023-10-18T10:44:49.497 INFO:teuthology.orchestra.run.smithi089.stdout:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2023-10-18T10:44:49.497 INFO:teuthology.orchestra.run.smithi089.stdout:2: [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] mon.c 2023-10-18T10:44:49.497 INFO:teuthology.orchestra.run.smithi089.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-18T10:44:49.498 DEBUG:teuthology.orchestra.run.smithi089:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-18T10:44:49.507 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 1352f62a-2330-4e96-afcd-b8e87cd82667... 2023-10-18T10:44:49.508 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-18T10:44:49.555 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-18T10:44:49.595 INFO:teuthology.orchestra.run.smithi089.stdout:[global] 2023-10-18T10:44:49.596 INFO:teuthology.orchestra.run.smithi089.stdout: chdir = "" 2023-10-18T10:44:49.596 INFO:teuthology.orchestra.run.smithi089.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-18T10:44:49.596 INFO:teuthology.orchestra.run.smithi089.stdout: auth supported = cephx 2023-10-18T10:44:49.596 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:44:49.597 INFO:teuthology.orchestra.run.smithi089.stdout: filestore xattr use omap = true 2023-10-18T10:44:49.597 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:44:49.597 INFO:teuthology.orchestra.run.smithi089.stdout: mon clock drift allowed = 1.000 2023-10-18T10:44:49.597 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:44:49.597 INFO:teuthology.orchestra.run.smithi089.stdout: osd crush chooseleaf type = 0 2023-10-18T10:44:49.598 INFO:teuthology.orchestra.run.smithi089.stdout: auth debug = true 2023-10-18T10:44:49.598 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:44:49.598 INFO:teuthology.orchestra.run.smithi089.stdout: ms die on old message = true 2023-10-18T10:44:49.598 INFO:teuthology.orchestra.run.smithi089.stdout: ms die on bug = true 2023-10-18T10:44:49.598 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:44:49.599 INFO:teuthology.orchestra.run.smithi089.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-18T10:44:49.599 INFO:teuthology.orchestra.run.smithi089.stdout: mon pg warn max object skew = 0 2023-10-18T10:44:49.599 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:44:49.599 INFO:teuthology.orchestra.run.smithi089.stdout: # disable pg_autoscaler by default for new pools 2023-10-18T10:44:49.599 INFO:teuthology.orchestra.run.smithi089.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-18T10:44:49.600 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:44:49.600 INFO:teuthology.orchestra.run.smithi089.stdout: osd pool default size = 2 2023-10-18T10:44:49.600 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:44:49.600 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd allow primary affinity = true 2023-10-18T10:44:49.600 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd allow pg remap = true 2023-10-18T10:44:49.601 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on legacy crush tunables = false 2023-10-18T10:44:49.601 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on crush straw calc version zero = false 2023-10-18T10:44:49.601 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on no sortbitwise = false 2023-10-18T10:44:49.601 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on osd down out interval zero = false 2023-10-18T10:44:49.601 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on too few osds = false 2023-10-18T10:44:49.602 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-18T10:44:49.602 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-18T10:44:49.602 INFO:teuthology.orchestra.run.smithi089.stdout: mon_allow_pool_size_one = true 2023-10-18T10:44:49.602 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:44:49.602 INFO:teuthology.orchestra.run.smithi089.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-18T10:44:49.603 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:44:49.603 INFO:teuthology.orchestra.run.smithi089.stdout: osd default data pool replay window = 5 2023-10-18T10:44:49.603 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:44:49.603 INFO:teuthology.orchestra.run.smithi089.stdout: mon allow pool delete = true 2023-10-18T10:44:49.603 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:44:49.604 INFO:teuthology.orchestra.run.smithi089.stdout: mon cluster log file level = debug 2023-10-18T10:44:49.604 INFO:teuthology.orchestra.run.smithi089.stdout: debug asserts on shutdown = true 2023-10-18T10:44:49.604 INFO:teuthology.orchestra.run.smithi089.stdout: mon health detail to clog = false 2023-10-18T10:44:49.604 INFO:teuthology.orchestra.run.smithi089.stdout: mon host = "172.21.15.89,172.21.15.154,[v2:172.21.15.154:3301,v1:172.21.15.154:6790]" 2023-10-18T10:44:49.604 INFO:teuthology.orchestra.run.smithi089.stdout: ms die on skipped message = False 2023-10-18T10:44:49.605 INFO:teuthology.orchestra.run.smithi089.stdout: fsid = 1352f62a-2330-4e96-afcd-b8e87cd82667 2023-10-18T10:44:49.605 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:44:49.605 INFO:teuthology.orchestra.run.smithi089.stdout:[osd] 2023-10-18T10:44:49.605 INFO:teuthology.orchestra.run.smithi089.stdout: osd journal size = 100 2023-10-18T10:44:49.605 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:44:49.605 INFO:teuthology.orchestra.run.smithi089.stdout: osd scrub load threshold = 5.0 2023-10-18T10:44:49.606 INFO:teuthology.orchestra.run.smithi089.stdout: osd scrub max interval = 600 2023-10-18T10:44:49.606 INFO:teuthology.orchestra.run.smithi089.stdout: osd mclock profile = high_recovery_ops 2023-10-18T10:44:49.606 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:44:49.606 INFO:teuthology.orchestra.run.smithi089.stdout: osd recover clone overlap = true 2023-10-18T10:44:49.606 INFO:teuthology.orchestra.run.smithi089.stdout: osd recovery max chunk = 1048576 2023-10-18T10:44:49.607 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:44:49.607 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug shutdown = true 2023-10-18T10:44:49.607 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug op order = true 2023-10-18T10:44:49.607 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug verify stray on activate = true 2023-10-18T10:44:49.608 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:44:49.608 INFO:teuthology.orchestra.run.smithi089.stdout: osd open classes on start = true 2023-10-18T10:44:49.608 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug pg log writeout = true 2023-10-18T10:44:49.608 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:44:49.609 INFO:teuthology.orchestra.run.smithi089.stdout: osd deep scrub update digest min age = 30 2023-10-18T10:44:49.609 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:44:49.609 INFO:teuthology.orchestra.run.smithi089.stdout: osd map max advance = 10 2023-10-18T10:44:49.609 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:44:49.609 INFO:teuthology.orchestra.run.smithi089.stdout: journal zero on create = true 2023-10-18T10:44:49.610 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:44:49.610 INFO:teuthology.orchestra.run.smithi089.stdout: filestore ondisk finisher threads = 3 2023-10-18T10:44:49.610 INFO:teuthology.orchestra.run.smithi089.stdout: filestore apply finisher threads = 3 2023-10-18T10:44:49.610 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:44:49.610 INFO:teuthology.orchestra.run.smithi089.stdout: bdev debug aio = true 2023-10-18T10:44:49.611 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug misdirected ops = true 2023-10-18T10:44:49.611 INFO:teuthology.orchestra.run.smithi089.stdout: bluestore block size = 96636764160 2023-10-18T10:44:49.611 INFO:teuthology.orchestra.run.smithi089.stdout: bluestore fsck on mount = True 2023-10-18T10:44:49.611 INFO:teuthology.orchestra.run.smithi089.stdout: debug bluefs = 20 2023-10-18T10:44:49.612 INFO:teuthology.orchestra.run.smithi089.stdout: debug bluestore = 20 2023-10-18T10:44:49.612 INFO:teuthology.orchestra.run.smithi089.stdout: debug ms = 1 2023-10-18T10:44:49.612 INFO:teuthology.orchestra.run.smithi089.stdout: debug osd = 20 2023-10-18T10:44:49.612 INFO:teuthology.orchestra.run.smithi089.stdout: debug rocksdb = 10 2023-10-18T10:44:49.612 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd backfillfull_ratio = 0.85 2023-10-18T10:44:49.613 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd full ratio = 0.9 2023-10-18T10:44:49.613 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd nearfull ratio = 0.8 2023-10-18T10:44:49.613 INFO:teuthology.orchestra.run.smithi089.stdout: osd failsafe full ratio = 0.95 2023-10-18T10:44:49.613 INFO:teuthology.orchestra.run.smithi089.stdout: osd objectstore = bluestore 2023-10-18T10:44:49.613 INFO:teuthology.orchestra.run.smithi089.stdout: osd op complaint time = 180 2023-10-18T10:44:49.613 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:44:49.614 INFO:teuthology.orchestra.run.smithi089.stdout:[mgr] 2023-10-18T10:44:49.614 INFO:teuthology.orchestra.run.smithi089.stdout: debug ms = 1 2023-10-18T10:44:49.614 INFO:teuthology.orchestra.run.smithi089.stdout: debug mgr = 20 2023-10-18T10:44:49.614 INFO:teuthology.orchestra.run.smithi089.stdout: debug mon = 20 2023-10-18T10:44:49.614 INFO:teuthology.orchestra.run.smithi089.stdout: debug auth = 20 2023-10-18T10:44:49.615 INFO:teuthology.orchestra.run.smithi089.stdout: mon reweight min pgs per osd = 4 2023-10-18T10:44:49.615 INFO:teuthology.orchestra.run.smithi089.stdout: mon reweight min bytes per osd = 10 2023-10-18T10:44:49.615 INFO:teuthology.orchestra.run.smithi089.stdout: mgr/telemetry/nag = false 2023-10-18T10:44:49.615 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:44:49.615 INFO:teuthology.orchestra.run.smithi089.stdout:[mon] 2023-10-18T10:44:49.616 INFO:teuthology.orchestra.run.smithi089.stdout: debug ms = 1 2023-10-18T10:44:49.616 INFO:teuthology.orchestra.run.smithi089.stdout: debug mon = 20 2023-10-18T10:44:49.616 INFO:teuthology.orchestra.run.smithi089.stdout: debug paxos = 20 2023-10-18T10:44:49.616 INFO:teuthology.orchestra.run.smithi089.stdout: debug auth = 20 2023-10-18T10:44:49.617 INFO:teuthology.orchestra.run.smithi089.stdout: mon data avail warn = 5 2023-10-18T10:44:49.617 INFO:teuthology.orchestra.run.smithi089.stdout: mon mgr mkfs grace = 240 2023-10-18T10:44:49.617 INFO:teuthology.orchestra.run.smithi089.stdout: mon reweight min pgs per osd = 4 2023-10-18T10:44:49.617 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd reporter subtree level = osd 2023-10-18T10:44:49.617 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd prime pg temp = true 2023-10-18T10:44:49.617 INFO:teuthology.orchestra.run.smithi089.stdout: mon reweight min bytes per osd = 10 2023-10-18T10:44:49.618 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:44:49.618 INFO:teuthology.orchestra.run.smithi089.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-18T10:44:49.618 INFO:teuthology.orchestra.run.smithi089.stdout: auth mon ticket ttl = 660 # 11m 2023-10-18T10:44:49.618 INFO:teuthology.orchestra.run.smithi089.stdout: auth service ticket ttl = 240 # 4m 2023-10-18T10:44:49.618 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:44:49.619 INFO:teuthology.orchestra.run.smithi089.stdout: # don't complain about insecure global_id in the test suite 2023-10-18T10:44:49.619 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-18T10:44:49.619 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-18T10:44:49.619 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:44:49.619 INFO:teuthology.orchestra.run.smithi089.stdout: # 1m isn't quite enough 2023-10-18T10:44:49.620 INFO:teuthology.orchestra.run.smithi089.stdout: mon_down_mkfs_grace = 2m 2023-10-18T10:44:49.620 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:44:49.620 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_filestore_osds = false 2023-10-18T10:44:49.620 INFO:teuthology.orchestra.run.smithi089.stdout: mon op complaint time = 120 2023-10-18T10:44:49.620 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:44:49.620 INFO:teuthology.orchestra.run.smithi089.stdout:[client] 2023-10-18T10:44:49.621 INFO:teuthology.orchestra.run.smithi089.stdout: rgw cache enabled = true 2023-10-18T10:44:49.621 INFO:teuthology.orchestra.run.smithi089.stdout: rgw enable ops log = true 2023-10-18T10:44:49.621 INFO:teuthology.orchestra.run.smithi089.stdout: rgw enable usage log = true 2023-10-18T10:44:49.621 INFO:teuthology.orchestra.run.smithi089.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-18T10:44:49.621 INFO:teuthology.orchestra.run.smithi089.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-18T10:44:49.622 INFO:teuthology.orchestra.run.smithi089.stdout: client mount timeout = 600 2023-10-18T10:44:49.622 INFO:teuthology.orchestra.run.smithi089.stdout: debug client = 20 2023-10-18T10:44:49.622 INFO:teuthology.orchestra.run.smithi089.stdout: debug ms = 1 2023-10-18T10:44:49.622 INFO:teuthology.orchestra.run.smithi089.stdout: pid file = "" 2023-10-18T10:44:49.622 INFO:teuthology.orchestra.run.smithi089.stdout: rados mon op timeout = 900 2023-10-18T10:44:49.623 INFO:teuthology.orchestra.run.smithi089.stdout: rados osd op timeout = 900 2023-10-18T10:44:49.623 INFO:teuthology.orchestra.run.smithi089.stdout:[mon.a] 2023-10-18T10:44:49.623 INFO:teuthology.orchestra.run.smithi089.stdout:[mon.b] 2023-10-18T10:44:49.623 INFO:teuthology.orchestra.run.smithi089.stdout:[mon.c] 2023-10-18T10:44:49.623 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.a] 2023-10-18T10:44:49.623 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.c] 2023-10-18T10:44:49.624 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.b] 2023-10-18T10:44:49.624 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.d] 2023-10-18T10:44:49.624 INFO:teuthology.orchestra.run.smithi089.stdout:[mds] 2023-10-18T10:44:49.624 INFO:teuthology.orchestra.run.smithi089.stdout: debug mds = 20 2023-10-18T10:44:49.624 INFO:teuthology.orchestra.run.smithi089.stdout: debug mds balancer = 20 2023-10-18T10:44:49.624 INFO:teuthology.orchestra.run.smithi089.stdout: debug ms = 1 2023-10-18T10:44:49.625 INFO:teuthology.orchestra.run.smithi089.stdout: mds debug frag = True 2023-10-18T10:44:49.625 INFO:teuthology.orchestra.run.smithi089.stdout: mds debug scatterstat = True 2023-10-18T10:44:49.625 INFO:teuthology.orchestra.run.smithi089.stdout: mds op complaint time = 180 2023-10-18T10:44:49.625 INFO:teuthology.orchestra.run.smithi089.stdout: mds verify scatter = True 2023-10-18T10:44:49.625 INFO:teuthology.orchestra.run.smithi089.stdout: osd op complaint time = 180 2023-10-18T10:44:49.626 INFO:teuthology.orchestra.run.smithi089.stdout: rados mon op timeout = 900 2023-10-18T10:44:49.626 INFO:teuthology.orchestra.run.smithi089.stdout: rados osd op timeout = 900 2023-10-18T10:44:49.627 INFO:teuthology.orchestra.run.smithi154.stdout:[global] 2023-10-18T10:44:49.628 INFO:teuthology.orchestra.run.smithi154.stdout: chdir = "" 2023-10-18T10:44:49.628 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-18T10:44:49.628 INFO:teuthology.orchestra.run.smithi154.stdout: auth supported = cephx 2023-10-18T10:44:49.628 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-18T10:44:49.628 INFO:teuthology.orchestra.run.smithi154.stdout: filestore xattr use omap = true 2023-10-18T10:44:49.629 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-18T10:44:49.629 INFO:teuthology.orchestra.run.smithi154.stdout: mon clock drift allowed = 1.000 2023-10-18T10:44:49.629 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-18T10:44:49.629 INFO:teuthology.orchestra.run.smithi154.stdout: osd crush chooseleaf type = 0 2023-10-18T10:44:49.629 INFO:teuthology.orchestra.run.smithi154.stdout: auth debug = true 2023-10-18T10:44:49.630 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-18T10:44:49.630 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on old message = true 2023-10-18T10:44:49.630 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on bug = true 2023-10-18T10:44:49.630 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-18T10:44:49.631 INFO:teuthology.orchestra.run.smithi154.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-18T10:44:49.631 INFO:teuthology.orchestra.run.smithi154.stdout: mon pg warn max object skew = 0 2023-10-18T10:44:49.631 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-18T10:44:49.631 INFO:teuthology.orchestra.run.smithi154.stdout: # disable pg_autoscaler by default for new pools 2023-10-18T10:44:49.631 INFO:teuthology.orchestra.run.smithi154.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-18T10:44:49.631 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-18T10:44:49.632 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default size = 2 2023-10-18T10:44:49.632 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-18T10:44:49.632 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow primary affinity = true 2023-10-18T10:44:49.632 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow pg remap = true 2023-10-18T10:44:49.632 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on legacy crush tunables = false 2023-10-18T10:44:49.633 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on crush straw calc version zero = false 2023-10-18T10:44:49.633 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on no sortbitwise = false 2023-10-18T10:44:49.633 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on osd down out interval zero = false 2023-10-18T10:44:49.633 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on too few osds = false 2023-10-18T10:44:49.633 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-18T10:44:49.634 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-18T10:44:49.634 INFO:teuthology.orchestra.run.smithi154.stdout: mon_allow_pool_size_one = true 2023-10-18T10:44:49.634 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-18T10:44:49.634 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-18T10:44:49.634 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-18T10:44:49.635 INFO:teuthology.orchestra.run.smithi154.stdout: osd default data pool replay window = 5 2023-10-18T10:44:49.635 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-18T10:44:49.635 INFO:teuthology.orchestra.run.smithi154.stdout: mon allow pool delete = true 2023-10-18T10:44:49.635 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-18T10:44:49.635 INFO:teuthology.orchestra.run.smithi154.stdout: mon cluster log file level = debug 2023-10-18T10:44:49.635 INFO:teuthology.orchestra.run.smithi154.stdout: debug asserts on shutdown = true 2023-10-18T10:44:49.636 INFO:teuthology.orchestra.run.smithi154.stdout: mon health detail to clog = false 2023-10-18T10:44:49.636 INFO:teuthology.orchestra.run.smithi154.stdout: mon host = "172.21.15.89,172.21.15.154,[v2:172.21.15.154:3301,v1:172.21.15.154:6790]" 2023-10-18T10:44:49.636 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on skipped message = False 2023-10-18T10:44:49.636 INFO:teuthology.orchestra.run.smithi154.stdout: fsid = 1352f62a-2330-4e96-afcd-b8e87cd82667 2023-10-18T10:44:49.636 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-18T10:44:49.637 INFO:teuthology.orchestra.run.smithi154.stdout:[osd] 2023-10-18T10:44:49.637 INFO:teuthology.orchestra.run.smithi154.stdout: osd journal size = 100 2023-10-18T10:44:49.637 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-18T10:44:49.637 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub load threshold = 5.0 2023-10-18T10:44:49.637 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub max interval = 600 2023-10-18T10:44:49.638 INFO:teuthology.orchestra.run.smithi154.stdout: osd mclock profile = high_recovery_ops 2023-10-18T10:44:49.638 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-18T10:44:49.638 INFO:teuthology.orchestra.run.smithi154.stdout: osd recover clone overlap = true 2023-10-18T10:44:49.638 INFO:teuthology.orchestra.run.smithi154.stdout: osd recovery max chunk = 1048576 2023-10-18T10:44:49.638 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-18T10:44:49.639 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug shutdown = true 2023-10-18T10:44:49.639 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug op order = true 2023-10-18T10:44:49.639 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug verify stray on activate = true 2023-10-18T10:44:49.639 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-18T10:44:49.639 INFO:teuthology.orchestra.run.smithi154.stdout: osd open classes on start = true 2023-10-18T10:44:49.640 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug pg log writeout = true 2023-10-18T10:44:49.640 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-18T10:44:49.640 INFO:teuthology.orchestra.run.smithi154.stdout: osd deep scrub update digest min age = 30 2023-10-18T10:44:49.640 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-18T10:44:49.640 INFO:teuthology.orchestra.run.smithi154.stdout: osd map max advance = 10 2023-10-18T10:44:49.641 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-18T10:44:49.641 INFO:teuthology.orchestra.run.smithi154.stdout: journal zero on create = true 2023-10-18T10:44:49.641 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-18T10:44:49.641 INFO:teuthology.orchestra.run.smithi154.stdout: filestore ondisk finisher threads = 3 2023-10-18T10:44:49.641 INFO:teuthology.orchestra.run.smithi154.stdout: filestore apply finisher threads = 3 2023-10-18T10:44:49.642 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-18T10:44:49.642 INFO:teuthology.orchestra.run.smithi154.stdout: bdev debug aio = true 2023-10-18T10:44:49.642 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug misdirected ops = true 2023-10-18T10:44:49.642 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore block size = 96636764160 2023-10-18T10:44:49.642 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore fsck on mount = True 2023-10-18T10:44:49.642 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluefs = 20 2023-10-18T10:44:49.643 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluestore = 20 2023-10-18T10:44:49.643 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-10-18T10:44:49.643 INFO:teuthology.orchestra.run.smithi154.stdout: debug osd = 20 2023-10-18T10:44:49.643 INFO:teuthology.orchestra.run.smithi154.stdout: debug rocksdb = 10 2023-10-18T10:44:49.643 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd backfillfull_ratio = 0.85 2023-10-18T10:44:49.644 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd full ratio = 0.9 2023-10-18T10:44:49.644 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd nearfull ratio = 0.8 2023-10-18T10:44:49.644 INFO:teuthology.orchestra.run.smithi154.stdout: osd failsafe full ratio = 0.95 2023-10-18T10:44:49.644 INFO:teuthology.orchestra.run.smithi154.stdout: osd objectstore = bluestore 2023-10-18T10:44:49.644 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2023-10-18T10:44:49.645 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-18T10:44:49.645 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr] 2023-10-18T10:44:49.645 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-10-18T10:44:49.645 INFO:teuthology.orchestra.run.smithi154.stdout: debug mgr = 20 2023-10-18T10:44:49.645 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2023-10-18T10:44:49.646 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2023-10-18T10:44:49.646 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2023-10-18T10:44:49.646 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2023-10-18T10:44:49.646 INFO:teuthology.orchestra.run.smithi154.stdout: mgr/telemetry/nag = false 2023-10-18T10:44:49.646 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-18T10:44:49.647 INFO:teuthology.orchestra.run.smithi154.stdout:[mon] 2023-10-18T10:44:49.647 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-10-18T10:44:49.647 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2023-10-18T10:44:49.647 INFO:teuthology.orchestra.run.smithi154.stdout: debug paxos = 20 2023-10-18T10:44:49.647 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2023-10-18T10:44:49.648 INFO:teuthology.orchestra.run.smithi154.stdout: mon data avail warn = 5 2023-10-18T10:44:49.648 INFO:teuthology.orchestra.run.smithi154.stdout: mon mgr mkfs grace = 240 2023-10-18T10:44:49.648 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2023-10-18T10:44:49.648 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd reporter subtree level = osd 2023-10-18T10:44:49.648 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd prime pg temp = true 2023-10-18T10:44:49.649 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2023-10-18T10:44:49.649 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-18T10:44:49.649 INFO:teuthology.orchestra.run.smithi154.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-18T10:44:49.649 INFO:teuthology.orchestra.run.smithi154.stdout: auth mon ticket ttl = 660 # 11m 2023-10-18T10:44:49.649 INFO:teuthology.orchestra.run.smithi154.stdout: auth service ticket ttl = 240 # 4m 2023-10-18T10:44:49.649 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-18T10:44:49.650 INFO:teuthology.orchestra.run.smithi154.stdout: # don't complain about insecure global_id in the test suite 2023-10-18T10:44:49.650 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-18T10:44:49.650 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-18T10:44:49.650 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-18T10:44:49.650 INFO:teuthology.orchestra.run.smithi154.stdout: # 1m isn't quite enough 2023-10-18T10:44:49.651 INFO:teuthology.orchestra.run.smithi154.stdout: mon_down_mkfs_grace = 2m 2023-10-18T10:44:49.651 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-18T10:44:49.651 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_filestore_osds = false 2023-10-18T10:44:49.651 INFO:teuthology.orchestra.run.smithi154.stdout: mon op complaint time = 120 2023-10-18T10:44:49.651 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-18T10:44:49.652 INFO:teuthology.orchestra.run.smithi154.stdout:[client] 2023-10-18T10:44:49.652 INFO:teuthology.orchestra.run.smithi154.stdout: rgw cache enabled = true 2023-10-18T10:44:49.652 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable ops log = true 2023-10-18T10:44:49.652 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable usage log = true 2023-10-18T10:44:49.652 INFO:teuthology.orchestra.run.smithi154.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-18T10:44:49.652 INFO:teuthology.orchestra.run.smithi154.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-18T10:44:49.653 INFO:teuthology.orchestra.run.smithi154.stdout: client mount timeout = 600 2023-10-18T10:44:49.653 INFO:teuthology.orchestra.run.smithi154.stdout: debug client = 20 2023-10-18T10:44:49.653 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-10-18T10:44:49.653 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = "" 2023-10-18T10:44:49.653 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2023-10-18T10:44:49.653 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2023-10-18T10:44:49.653 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.a] 2023-10-18T10:44:49.654 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.b] 2023-10-18T10:44:49.654 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.c] 2023-10-18T10:44:49.654 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.a] 2023-10-18T10:44:49.654 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.c] 2023-10-18T10:44:49.654 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.b] 2023-10-18T10:44:49.655 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.d] 2023-10-18T10:44:49.655 INFO:teuthology.orchestra.run.smithi154.stdout:[mds] 2023-10-18T10:44:49.655 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds = 20 2023-10-18T10:44:49.655 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds balancer = 20 2023-10-18T10:44:49.655 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-10-18T10:44:49.655 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug frag = True 2023-10-18T10:44:49.656 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug scatterstat = True 2023-10-18T10:44:49.656 INFO:teuthology.orchestra.run.smithi154.stdout: mds op complaint time = 180 2023-10-18T10:44:49.656 INFO:teuthology.orchestra.run.smithi154.stdout: mds verify scatter = True 2023-10-18T10:44:49.656 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2023-10-18T10:44:49.656 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2023-10-18T10:44:49.657 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2023-10-18T10:44:49.657 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-18T10:44:49.657 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-18T10:44:49.728 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-18T10:44:49.728 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:44:49.729 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-18T10:44:49.738 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:44:49.738 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-18T10:44:49.788 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi089.front.sepia.ceph.com 2023-10-18T10:44:49.788 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:44:49.788 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-18T10:44:49.789 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-18T10:44:49.864 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:44:49.864 DEBUG:teuthology.orchestra.run.smithi089:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-18T10:44:49.916 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi154.front.sepia.ceph.com 2023-10-18T10:44:49.916 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T10:44:49.916 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-18T10:44:49.917 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-18T10:44:49.953 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T10:44:49.953 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-18T10:44:50.007 INFO:tasks.ceph:Setting up mon nodes... 2023-10-18T10:44:50.007 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-18T10:44:50.008 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-18T10:44:50.062 INFO:teuthology.orchestra.run.smithi089.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-18T10:44:50.067 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-18T10:44:50.128 INFO:teuthology.orchestra.run.smithi154.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-18T10:44:50.188 INFO:tasks.ceph:Setting up mds nodes... 2023-10-18T10:44:50.189 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-18T10:44:50.246 INFO:teuthology.orchestra.run.smithi089.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-18T10:44:50.250 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-18T10:44:50.271 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-18T10:44:50.366 INFO:teuthology.orchestra.run.smithi089.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-18T10:44:50.370 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-18T10:44:50.391 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-18T10:44:50.449 INFO:teuthology.orchestra.run.smithi154.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-18T10:44:50.454 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-18T10:44:50.475 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-18T10:44:50.574 INFO:teuthology.orchestra.run.smithi154.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-18T10:44:50.579 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-18T10:44:50.599 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-18T10:44:50.600 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-18T10:44:50.644 INFO:teuthology.orchestra.run.smithi089.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-18T10:44:50.661 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-18T10:44:50.745 INFO:teuthology.orchestra.run.smithi089.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-18T10:44:50.762 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-18T10:44:50.805 INFO:teuthology.orchestra.run.smithi154.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-18T10:44:50.822 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-18T10:44:50.904 INFO:teuthology.orchestra.run.smithi154.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-18T10:44:50.921 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-18T10:44:50.922 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi089.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi154.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-18T10:44:50.922 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-18T10:44:50.942 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-18T10:44:50.943 INFO:tasks.ceph:role: osd.0 2023-10-18T10:44:50.943 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi089.front.sepia.ceph.com 2023-10-18T10:44:50.943 DEBUG:teuthology.orchestra.run.smithi089:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-18T10:44:51.010 INFO:teuthology.orchestra.run.smithi089.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-18T10:44:51.011 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-18T10:44:51.011 INFO:teuthology.orchestra.run.smithi089.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-18T10:44:51.011 INFO:teuthology.orchestra.run.smithi089.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-18T10:44:51.011 INFO:teuthology.orchestra.run.smithi089.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-18T10:44:51.012 INFO:teuthology.orchestra.run.smithi089.stdout: = sunit=0 swidth=0 blks 2023-10-18T10:44:51.012 INFO:teuthology.orchestra.run.smithi089.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-18T10:44:51.012 INFO:teuthology.orchestra.run.smithi089.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-18T10:44:51.012 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-18T10:44:51.013 INFO:teuthology.orchestra.run.smithi089.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-18T10:44:51.469 INFO:teuthology.orchestra.run.smithi089.stdout:Discarding blocks...Done. 2023-10-18T10:44:51.472 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi089.front.sepia.ceph.com -o noatime 2023-10-18T10:44:51.472 DEBUG:teuthology.orchestra.run.smithi089:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-18T10:44:51.586 DEBUG:teuthology.orchestra.run.smithi089:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-18T10:44:51.603 INFO:teuthology.orchestra.run.smithi089.stderr:sudo: /sbin/restorecon: command not found 2023-10-18T10:44:51.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:44:51.604 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-18T10:44:51.667 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-18T10:44:51.667 INFO:tasks.ceph:role: osd.1 2023-10-18T10:44:51.667 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi089.front.sepia.ceph.com 2023-10-18T10:44:51.668 DEBUG:teuthology.orchestra.run.smithi089:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-18T10:44:51.734 INFO:teuthology.orchestra.run.smithi089.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-18T10:44:51.735 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-18T10:44:51.735 INFO:teuthology.orchestra.run.smithi089.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-18T10:44:51.735 INFO:teuthology.orchestra.run.smithi089.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-18T10:44:51.735 INFO:teuthology.orchestra.run.smithi089.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-18T10:44:51.736 INFO:teuthology.orchestra.run.smithi089.stdout: = sunit=0 swidth=0 blks 2023-10-18T10:44:51.736 INFO:teuthology.orchestra.run.smithi089.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-18T10:44:51.736 INFO:teuthology.orchestra.run.smithi089.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-18T10:44:51.736 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-18T10:44:51.737 INFO:teuthology.orchestra.run.smithi089.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-18T10:44:52.245 INFO:teuthology.orchestra.run.smithi089.stdout:Discarding blocks...Done. 2023-10-18T10:44:52.248 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi089.front.sepia.ceph.com -o noatime 2023-10-18T10:44:52.249 DEBUG:teuthology.orchestra.run.smithi089:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-18T10:44:52.409 DEBUG:teuthology.orchestra.run.smithi089:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-18T10:44:52.427 INFO:teuthology.orchestra.run.smithi089.stderr:sudo: /sbin/restorecon: command not found 2023-10-18T10:44:52.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:44:52.429 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-18T10:44:52.498 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-18T10:44:52.498 INFO:tasks.ceph:role: osd.2 2023-10-18T10:44:52.498 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi089.front.sepia.ceph.com 2023-10-18T10:44:52.499 DEBUG:teuthology.orchestra.run.smithi089:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-18T10:44:52.568 INFO:teuthology.orchestra.run.smithi089.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-18T10:44:52.569 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-18T10:44:52.569 INFO:teuthology.orchestra.run.smithi089.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-18T10:44:52.569 INFO:teuthology.orchestra.run.smithi089.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-18T10:44:52.569 INFO:teuthology.orchestra.run.smithi089.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-18T10:44:52.570 INFO:teuthology.orchestra.run.smithi089.stdout: = sunit=0 swidth=0 blks 2023-10-18T10:44:52.570 INFO:teuthology.orchestra.run.smithi089.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-18T10:44:52.570 INFO:teuthology.orchestra.run.smithi089.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-18T10:44:52.570 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-18T10:44:52.570 INFO:teuthology.orchestra.run.smithi089.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-18T10:44:53.050 INFO:teuthology.orchestra.run.smithi089.stdout:Discarding blocks...Done. 2023-10-18T10:44:53.054 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi089.front.sepia.ceph.com -o noatime 2023-10-18T10:44:53.054 DEBUG:teuthology.orchestra.run.smithi089:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-18T10:44:53.152 DEBUG:teuthology.orchestra.run.smithi089:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-18T10:44:53.169 INFO:teuthology.orchestra.run.smithi089.stderr:sudo: /sbin/restorecon: command not found 2023-10-18T10:44:53.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:44:53.171 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-18T10:44:53.233 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-18T10:44:53.233 INFO:tasks.ceph:role: osd.3 2023-10-18T10:44:53.233 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi089.front.sepia.ceph.com 2023-10-18T10:44:53.234 DEBUG:teuthology.orchestra.run.smithi089:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-18T10:44:53.304 INFO:teuthology.orchestra.run.smithi089.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-18T10:44:53.305 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-18T10:44:53.305 INFO:teuthology.orchestra.run.smithi089.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-18T10:44:53.305 INFO:teuthology.orchestra.run.smithi089.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-18T10:44:53.305 INFO:teuthology.orchestra.run.smithi089.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-18T10:44:53.306 INFO:teuthology.orchestra.run.smithi089.stdout: = sunit=0 swidth=0 blks 2023-10-18T10:44:53.306 INFO:teuthology.orchestra.run.smithi089.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-18T10:44:53.306 INFO:teuthology.orchestra.run.smithi089.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-18T10:44:53.306 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-18T10:44:53.306 INFO:teuthology.orchestra.run.smithi089.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-18T10:44:53.755 INFO:teuthology.orchestra.run.smithi089.stdout:Discarding blocks...Done. 2023-10-18T10:44:53.757 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi089.front.sepia.ceph.com -o noatime 2023-10-18T10:44:53.758 DEBUG:teuthology.orchestra.run.smithi089:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-18T10:44:53.871 DEBUG:teuthology.orchestra.run.smithi089:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-18T10:44:53.889 INFO:teuthology.orchestra.run.smithi089.stderr:sudo: /sbin/restorecon: command not found 2023-10-18T10:44:53.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:44:53.890 DEBUG:teuthology.orchestra.run.smithi089:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-18T10:44:54.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:44:54.017+0000 7f7f3ca22640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-18T10:44:54.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:44:54.021+0000 7f7f3ca22640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-18T10:44:54.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:44:54.021+0000 7f7f3ca22640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-18T10:44:54.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:44:54.021+0000 7f7f3ca22640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-18T10:44:54.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:44:54.021+0000 7f7f3ca22640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-18T10:44:54.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:44:54.025+0000 7f7f3ca22640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-18T10:44:57.048 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-18T10:44:57.070 DEBUG:teuthology.orchestra.run.smithi089:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-18T10:44:57.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:44:57.169+0000 7f6c95a46640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-18T10:44:57.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:44:57.169+0000 7f6c95a46640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-18T10:44:57.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:44:57.169+0000 7f6c95a46640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-18T10:44:57.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:44:57.169+0000 7f6c95a46640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-18T10:44:57.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:44:57.169+0000 7f6c95a46640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-18T10:44:57.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:44:57.173+0000 7f6c95a46640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-18T10:45:00.155 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-18T10:45:00.175 DEBUG:teuthology.orchestra.run.smithi089:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-18T10:45:00.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:00.273+0000 7f92ccfc6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-18T10:45:00.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:00.273+0000 7f92ccfc6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-18T10:45:00.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:00.273+0000 7f92ccfc6640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-18T10:45:00.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:00.273+0000 7f92ccfc6640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-18T10:45:00.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:00.273+0000 7f92ccfc6640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-18T10:45:00.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:00.273+0000 7f92ccfc6640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-18T10:45:03.294 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-18T10:45:03.316 DEBUG:teuthology.orchestra.run.smithi089:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-18T10:45:03.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:03.413+0000 7fb0ab13c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-18T10:45:03.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:03.413+0000 7fb0ab13c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-18T10:45:03.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:03.413+0000 7fb0ab13c640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-18T10:45:03.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:03.413+0000 7fb0ab13c640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-18T10:45:03.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:03.413+0000 7fb0ab13c640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-18T10:45:03.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:03.417+0000 7fb0ab13c640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-18T10:45:06.402 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-18T10:45:06.423 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-18T10:45:06.447 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-18T10:45:06.447 INFO:tasks.ceph:role: osd.4 2023-10-18T10:45:06.447 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi154.front.sepia.ceph.com 2023-10-18T10:45:06.448 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-18T10:45:06.519 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-18T10:45:06.520 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-18T10:45:06.520 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-18T10:45:06.520 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-18T10:45:06.520 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-18T10:45:06.520 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2023-10-18T10:45:06.521 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-18T10:45:06.521 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-18T10:45:06.521 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-18T10:45:06.521 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-18T10:45:06.970 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2023-10-18T10:45:06.973 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2023-10-18T10:45:06.973 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-18T10:45:07.079 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-18T10:45:07.097 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: /sbin/restorecon: command not found 2023-10-18T10:45:07.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:45:07.099 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-18T10:45:07.163 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-18T10:45:07.163 INFO:tasks.ceph:role: osd.5 2023-10-18T10:45:07.164 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi154.front.sepia.ceph.com 2023-10-18T10:45:07.164 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-18T10:45:07.234 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-18T10:45:07.235 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-18T10:45:07.235 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-18T10:45:07.235 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-18T10:45:07.235 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-18T10:45:07.235 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2023-10-18T10:45:07.236 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-18T10:45:07.236 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-18T10:45:07.236 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-18T10:45:07.236 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-18T10:45:07.721 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2023-10-18T10:45:07.724 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2023-10-18T10:45:07.725 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-18T10:45:07.882 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-18T10:45:07.900 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: /sbin/restorecon: command not found 2023-10-18T10:45:07.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:45:07.902 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-18T10:45:07.963 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-18T10:45:07.963 INFO:tasks.ceph:role: osd.6 2023-10-18T10:45:07.963 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi154.front.sepia.ceph.com 2023-10-18T10:45:07.964 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-18T10:45:08.035 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-18T10:45:08.035 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-18T10:45:08.035 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-18T10:45:08.035 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-18T10:45:08.035 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-18T10:45:08.036 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2023-10-18T10:45:08.036 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-18T10:45:08.036 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-18T10:45:08.036 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-18T10:45:08.036 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-18T10:45:08.517 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2023-10-18T10:45:08.520 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2023-10-18T10:45:08.521 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-18T10:45:08.651 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-18T10:45:08.669 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: /sbin/restorecon: command not found 2023-10-18T10:45:08.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:45:08.671 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-18T10:45:08.735 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-18T10:45:08.736 INFO:tasks.ceph:role: osd.7 2023-10-18T10:45:08.736 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi154.front.sepia.ceph.com 2023-10-18T10:45:08.736 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-18T10:45:08.808 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-18T10:45:08.808 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-18T10:45:08.808 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-18T10:45:08.808 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-18T10:45:08.809 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-18T10:45:08.809 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2023-10-18T10:45:08.809 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-18T10:45:08.809 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-18T10:45:08.809 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-18T10:45:08.810 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-18T10:45:09.259 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2023-10-18T10:45:09.263 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2023-10-18T10:45:09.263 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-18T10:45:09.365 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-18T10:45:09.383 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: /sbin/restorecon: command not found 2023-10-18T10:45:09.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:45:09.384 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-18T10:45:09.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:09.518+0000 7f2c034cc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-18T10:45:09.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:09.526+0000 7f2c034cc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-18T10:45:09.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:09.526+0000 7f2c034cc640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-18T10:45:09.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:09.526+0000 7f2c034cc640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-18T10:45:09.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:09.526+0000 7f2c034cc640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-18T10:45:09.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:09.530+0000 7f2c034cc640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-18T10:45:12.527 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-18T10:45:12.548 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-18T10:45:12.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:12.650+0000 7f5fced2d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-18T10:45:12.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:12.650+0000 7f5fced2d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-18T10:45:12.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:12.650+0000 7f5fced2d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-18T10:45:12.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:12.650+0000 7f5fced2d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-18T10:45:12.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:12.650+0000 7f5fced2d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-18T10:45:12.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:12.654+0000 7f5fced2d640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-18T10:45:15.642 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-18T10:45:15.663 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-18T10:45:15.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:15.762+0000 7f6e8e73c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-18T10:45:15.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:15.762+0000 7f6e8e73c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-18T10:45:15.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:15.762+0000 7f6e8e73c640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-18T10:45:15.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:15.762+0000 7f6e8e73c640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-18T10:45:15.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:15.762+0000 7f6e8e73c640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-18T10:45:15.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:15.766+0000 7f6e8e73c640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-18T10:45:17.678 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:45:17.682 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:45:18.752 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-18T10:45:18.774 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-18T10:45:18.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:18.870+0000 7f5bfa6b4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-18T10:45:18.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:18.870+0000 7f5bfa6b4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-18T10:45:18.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:18.870+0000 7f5bfa6b4640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-18T10:45:18.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:18.870+0000 7f5bfa6b4640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-18T10:45:18.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:18.870+0000 7f5bfa6b4640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-18T10:45:18.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:18.874+0000 7f5bfa6b4640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-18T10:45:21.849 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-18T10:45:21.870 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-18T10:45:21.871 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:45:21.871 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-18T10:45:21.894 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:45:21.895 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-18T10:45:21.957 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:45:21.957 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-18T10:45:22.020 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:45:22.021 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-18T10:45:22.084 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:45:22.085 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-18T10:45:22.153 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:45:22.153 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-18T10:45:22.216 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:45:22.217 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-18T10:45:22.280 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T10:45:22.281 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-18T10:45:22.302 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T10:45:22.302 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-18T10:45:22.366 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T10:45:22.367 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-18T10:45:22.431 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T10:45:22.431 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-18T10:45:22.494 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T10:45:22.495 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-18T10:45:22.562 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T10:45:22.562 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-18T10:45:22.630 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T10:45:22.631 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-18T10:45:22.694 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:45:22.695 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-18T10:45:22.704 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:45:22.704 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-18T10:45:22.760 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T10:45:22.761 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-18T10:45:22.770 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T10:45:22.771 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-18T10:45:22.822 INFO:tasks.ceph:Adding keys to all mons... 2023-10-18T10:45:22.823 DEBUG:teuthology.orchestra.run.smithi089:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-18T10:45:22.827 DEBUG:teuthology.orchestra.run.smithi154:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-18T10:45:22.871 INFO:teuthology.orchestra.run.smithi089.stdout:[mgr.y] 2023-10-18T10:45:22.871 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQAity9la0P9AxAAGS2Tg2NOUwGXRgWNRpU1Xg== 2023-10-18T10:45:22.871 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.a] 2023-10-18T10:45:22.872 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQAity9l3YjmDhAApyQSVPKVH9p2kMmY8L12sw== 2023-10-18T10:45:22.872 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.c] 2023-10-18T10:45:22.872 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQAity9lJQIVFhAAqPlG3bmFHpxQZlY4J0Kk3Q== 2023-10-18T10:45:22.873 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.0] 2023-10-18T10:45:22.873 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQAmty9lMm5nARAA0jHQUL2hduxEzO1ImPmP1g== 2023-10-18T10:45:22.873 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.1] 2023-10-18T10:45:22.873 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQApty9lN0ZtChAAmkzQ69dcnGUj1jCMv/tksQ== 2023-10-18T10:45:22.874 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.2] 2023-10-18T10:45:22.874 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQAsty9l07p8EBAAPJVssCSLmw5sG9oM2OzJSQ== 2023-10-18T10:45:22.874 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.3] 2023-10-18T10:45:22.874 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQAvty9lnCP3GBAAfRyC++rI5Ha+iVN0wSMcNQ== 2023-10-18T10:45:22.875 INFO:teuthology.orchestra.run.smithi089.stdout:[mgr.x] 2023-10-18T10:45:22.875 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQAity9liyzXBxAAuc/Y/Gay+OYIj9TFZ6q2Dw== 2023-10-18T10:45:22.875 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.b] 2023-10-18T10:45:22.875 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQAity9llE78GhAAZCzWHVS+Cy8PoENP2E7RbQ== 2023-10-18T10:45:22.876 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.d] 2023-10-18T10:45:22.876 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQAity9laNByIhAAa8uuDEq77DKA67YeY+7a4A== 2023-10-18T10:45:22.876 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.4] 2023-10-18T10:45:22.876 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQA1ty9lwM0aHxAAXuBTefDP1w4rxukHFgqCqQ== 2023-10-18T10:45:22.877 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.5] 2023-10-18T10:45:22.877 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQA4ty9lalrVJhAAHaF+tMmtF8KN9Z4eFZrI8A== 2023-10-18T10:45:22.877 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.6] 2023-10-18T10:45:22.877 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQA7ty9lZT6iLRAApQbbomgjYQZrjfG0Jbj7dg== 2023-10-18T10:45:22.878 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.7] 2023-10-18T10:45:22.878 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQA+ty9lZlwFNBAAEt+82Hbl4mIWuvT/L8A/OA== 2023-10-18T10:45:22.878 INFO:teuthology.orchestra.run.smithi089.stdout:[client.0] 2023-10-18T10:45:22.879 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQAity9lApuiJhAAUNpuF1S7ZLrsIuRG0mrVNw== 2023-10-18T10:45:22.879 INFO:teuthology.orchestra.run.smithi089.stdout:[client.2] 2023-10-18T10:45:22.879 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQAity9l7HqvLBAAYrtttxbjU+BBFgtOl3bvjg== 2023-10-18T10:45:22.879 INFO:teuthology.orchestra.run.smithi089.stdout:[client.1] 2023-10-18T10:45:22.880 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQAity9lyrouMBAAOsZT3MhcFH5guAKKbmJFuw== 2023-10-18T10:45:22.880 INFO:teuthology.orchestra.run.smithi089.stdout:[client.3] 2023-10-18T10:45:22.880 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQAity9l/VAfNhAA6jHkCCY8phqDIvJLuRu/Rw== 2023-10-18T10:45:22.885 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr.y] 2023-10-18T10:45:22.885 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAity9la0P9AxAAGS2Tg2NOUwGXRgWNRpU1Xg== 2023-10-18T10:45:22.886 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.a] 2023-10-18T10:45:22.886 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAity9l3YjmDhAApyQSVPKVH9p2kMmY8L12sw== 2023-10-18T10:45:22.886 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.c] 2023-10-18T10:45:22.887 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAity9lJQIVFhAAqPlG3bmFHpxQZlY4J0Kk3Q== 2023-10-18T10:45:22.887 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.0] 2023-10-18T10:45:22.887 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAmty9lMm5nARAA0jHQUL2hduxEzO1ImPmP1g== 2023-10-18T10:45:22.887 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.1] 2023-10-18T10:45:22.888 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQApty9lN0ZtChAAmkzQ69dcnGUj1jCMv/tksQ== 2023-10-18T10:45:22.888 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.2] 2023-10-18T10:45:22.888 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAsty9l07p8EBAAPJVssCSLmw5sG9oM2OzJSQ== 2023-10-18T10:45:22.888 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.3] 2023-10-18T10:45:22.889 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAvty9lnCP3GBAAfRyC++rI5Ha+iVN0wSMcNQ== 2023-10-18T10:45:22.889 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr.x] 2023-10-18T10:45:22.889 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAity9liyzXBxAAuc/Y/Gay+OYIj9TFZ6q2Dw== 2023-10-18T10:45:22.889 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.b] 2023-10-18T10:45:22.890 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAity9llE78GhAAZCzWHVS+Cy8PoENP2E7RbQ== 2023-10-18T10:45:22.890 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.d] 2023-10-18T10:45:22.890 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAity9laNByIhAAa8uuDEq77DKA67YeY+7a4A== 2023-10-18T10:45:22.890 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.4] 2023-10-18T10:45:22.891 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQA1ty9lwM0aHxAAXuBTefDP1w4rxukHFgqCqQ== 2023-10-18T10:45:22.891 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.5] 2023-10-18T10:45:22.891 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQA4ty9lalrVJhAAHaF+tMmtF8KN9Z4eFZrI8A== 2023-10-18T10:45:22.891 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.6] 2023-10-18T10:45:22.892 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQA7ty9lZT6iLRAApQbbomgjYQZrjfG0Jbj7dg== 2023-10-18T10:45:22.892 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.7] 2023-10-18T10:45:22.892 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQA+ty9lZlwFNBAAEt+82Hbl4mIWuvT/L8A/OA== 2023-10-18T10:45:22.892 INFO:teuthology.orchestra.run.smithi154.stdout:[client.0] 2023-10-18T10:45:22.893 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAity9lApuiJhAAUNpuF1S7ZLrsIuRG0mrVNw== 2023-10-18T10:45:22.893 INFO:teuthology.orchestra.run.smithi154.stdout:[client.2] 2023-10-18T10:45:22.893 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAity9l7HqvLBAAYrtttxbjU+BBFgtOl3bvjg== 2023-10-18T10:45:22.893 INFO:teuthology.orchestra.run.smithi154.stdout:[client.1] 2023-10-18T10:45:22.894 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAity9lyrouMBAAOsZT3MhcFH5guAKKbmJFuw== 2023-10-18T10:45:22.894 INFO:teuthology.orchestra.run.smithi154.stdout:[client.3] 2023-10-18T10:45:22.894 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAity9l/VAfNhAA6jHkCCY8phqDIvJLuRu/Rw== 2023-10-18T10:45:22.895 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-18T10:45:22.927 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-18T10:45:22.995 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T10:45:23.000 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T10:45:23.060 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T10:45:23.065 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T10:45:23.124 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T10:45:23.130 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T10:45:23.184 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T10:45:23.188 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T10:45:23.243 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T10:45:23.246 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T10:45:23.301 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T10:45:23.305 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T10:45:23.359 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-18T10:45:23.362 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-18T10:45:23.417 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T10:45:23.421 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T10:45:23.477 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T10:45:23.481 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T10:45:23.539 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T10:45:23.542 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T10:45:23.596 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T10:45:23.600 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T10:45:23.654 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T10:45:23.658 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T10:45:23.711 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T10:45:23.716 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T10:45:23.771 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-18T10:45:23.775 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-18T10:45:23.833 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-18T10:45:23.838 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-18T10:45:23.897 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-18T10:45:23.901 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-18T10:45:23.959 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-18T10:45:23.962 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-18T10:45:24.019 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-18T10:45:24.020 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-18T10:45:24.040 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-18T10:45:24.158 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-18T10:45:24.179 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-18T10:45:24.199 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-18T10:45:24.317 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-18T10:45:24.338 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-18T10:45:24.401 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-18T10:45:24.519 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-18T10:45:24.540 DEBUG:teuthology.orchestra.run.smithi089:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-18T10:45:24.544 DEBUG:teuthology.orchestra.run.smithi154:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-18T10:45:24.589 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-18T10:45:24.590 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-18T10:45:24.591 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-18T10:45:24.598 INFO:tasks.ceph.mon.a:Started 2023-10-18T10:45:24.598 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-18T10:45:24.599 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-18T10:45:24.637 INFO:tasks.ceph.mon.b:Started 2023-10-18T10:45:24.637 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-18T10:45:24.638 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-18T10:45:24.642 INFO:tasks.ceph.mon.c:Started 2023-10-18T10:45:24.642 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-18T10:45:24.643 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-18T10:45:24.643 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-18T10:45:24.647 INFO:tasks.ceph.mgr.y:Started 2023-10-18T10:45:24.647 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-18T10:45:24.647 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-18T10:45:24.651 INFO:tasks.ceph.mgr.x:Started 2023-10-18T10:45:24.652 INFO:tasks.ceph:Setting crush tunables to default 2023-10-18T10:45:24.652 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --cluster ceph osd crush tunables default 2023-10-18T10:45:24.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:24.793+0000 7f6e7b59e640 1 Processor -- start 2023-10-18T10:45:24.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:24.793+0000 7f6e7b59e640 1 -- start start 2023-10-18T10:45:24.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:24.793+0000 7f6e7b59e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6e7c15c5d0 0x7f6e7c0d1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:24.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:24.793+0000 7f6e7b59e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6e7c0cd860 0x7f6e7c0d1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:24.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:24.793+0000 7f6e7b59e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e7c0cee70 0x7f6e7c0cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:24.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:24.793+0000 7f6e7b59e640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e7c0d15c0 con 0x7f6e7c0cd860 2023-10-18T10:45:24.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:24.793+0000 7f6e7b59e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e7c0cf800 con 0x7f6e7c15c5d0 2023-10-18T10:45:24.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:24.793+0000 7f6e7b59e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c0d0520 con 0x7f6e7c0cee70 2023-10-18T10:45:24.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:24.793+0000 7f6e79d9b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6e7c0cd860 0x7f6e7c0d1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:24.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:24.793+0000 7f6e79d9b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6e7c0cd860 0x7f6e7c0d1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59054/0 (socket says 172.21.15.89:59054) 2023-10-18T10:45:24.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:24.793+0000 7f6e79d9b640 1 -- 172.21.15.89:0/3888767860 learned_addr learned my addr 172.21.15.89:0/3888767860 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:45:24.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:24.793+0000 7f6e7ad9d640 1 --2- 172.21.15.89:0/3888767860 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e7c0cee70 0x7f6e7c0cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:24.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:24.793+0000 7f6e79d9b640 1 -- 172.21.15.89:0/3888767860 >> v2:172.21.15.89:3300/0 conn(0x7f6e7c0cd860 msgr2=0x7f6e7c0d1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-18T10:45:24.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:24.793+0000 7f6e79d9b640 1 -- 172.21.15.89:0/3888767860 >> v2:172.21.15.89:3300/0 conn(0x7f6e7c0cd860 msgr2=0x7f6e7c0d1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-18T10:45:24.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:24.793+0000 7f6e79d9b640 1 --2- 172.21.15.89:0/3888767860 >> v2:172.21.15.89:3300/0 conn(0x7f6e7c0cd860 0x7f6e7c0d1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-18T10:45:24.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:24.793+0000 7f6e79d9b640 1 --2- 172.21.15.89:0/3888767860 >> v2:172.21.15.89:3300/0 conn(0x7f6e7c0cd860 0x7f6e7c0d1e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:45:24.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:24.793+0000 7f6e7ad9d640 1 -- 172.21.15.89:0/3888767860 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e7c0cee70 msgr2=0x7f6e7c0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-18T10:45:24.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:24.793+0000 7f6e7a59c640 1 --2- 172.21.15.89:0/3888767860 >> v2:172.21.15.154:3300/0 conn(0x7f6e7c15c5d0 0x7f6e7c0d1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:24.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:24.793+0000 7f6e7ad9d640 1 -- 172.21.15.89:0/3888767860 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e7c0cee70 msgr2=0x7f6e7c0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-18T10:45:24.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:24.793+0000 7f6e7ad9d640 1 --2- 172.21.15.89:0/3888767860 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e7c0cee70 0x7f6e7c0cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-18T10:45:24.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:24.797+0000 7f6e7ad9d640 1 --2- 172.21.15.89:0/3888767860 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e7c0cee70 0x7f6e7c0cf2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:45:24.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:24.797+0000 7f6e7a59c640 1 -- 172.21.15.89:0/3888767860 >> v2:172.21.15.154:3300/0 conn(0x7f6e7c15c5d0 msgr2=0x7f6e7c0d1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-18T10:45:24.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:24.797+0000 7f6e7a59c640 1 -- 172.21.15.89:0/3888767860 >> v2:172.21.15.154:3300/0 conn(0x7f6e7c15c5d0 msgr2=0x7f6e7c0d1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-18T10:45:24.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:24.797+0000 7f6e7a59c640 1 --2- 172.21.15.89:0/3888767860 >> v2:172.21.15.154:3300/0 conn(0x7f6e7c15c5d0 0x7f6e7c0d1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-18T10:45:24.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:24.797+0000 7f6e7a59c640 1 --2- 172.21.15.89:0/3888767860 >> v2:172.21.15.154:3300/0 conn(0x7f6e7c15c5d0 0x7f6e7c0d1930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:45:24.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:24.997+0000 7f6e79d9b640 1 --2- 172.21.15.89:0/3888767860 >> v2:172.21.15.89:3300/0 conn(0x7f6e7c0cd860 0x7f6e7c0d1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:24.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:24.997+0000 7f6e79d9b640 1 -- 172.21.15.89:0/3888767860 >> v2:172.21.15.89:3300/0 conn(0x7f6e7c0cd860 msgr2=0x7f6e7c0d1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-18T10:45:24.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:24.997+0000 7f6e79d9b640 1 -- 172.21.15.89:0/3888767860 >> v2:172.21.15.89:3300/0 conn(0x7f6e7c0cd860 msgr2=0x7f6e7c0d1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-18T10:45:24.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:24.997+0000 7f6e79d9b640 1 --2- 172.21.15.89:0/3888767860 >> v2:172.21.15.89:3300/0 conn(0x7f6e7c0cd860 0x7f6e7c0d1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-18T10:45:24.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:24.997+0000 7f6e79d9b640 1 --2- 172.21.15.89:0/3888767860 >> v2:172.21.15.89:3300/0 conn(0x7f6e7c0cd860 0x7f6e7c0d1e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T10:45:25.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:25.001+0000 7f6e7a59c640 1 --2- 172.21.15.89:0/3888767860 >> v2:172.21.15.154:3300/0 conn(0x7f6e7c15c5d0 0x7f6e7c0d1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:25.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:25.001+0000 7f6e7ad9d640 1 --2- 172.21.15.89:0/3888767860 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e7c0cee70 0x7f6e7c0cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:25.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:25.001+0000 7f6e7ad9d640 1 -- 172.21.15.89:0/3888767860 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e7c0cee70 msgr2=0x7f6e7c0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-18T10:45:25.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:25.001+0000 7f6e7ad9d640 1 -- 172.21.15.89:0/3888767860 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e7c0cee70 msgr2=0x7f6e7c0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-18T10:45:25.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:25.001+0000 7f6e7ad9d640 1 --2- 172.21.15.89:0/3888767860 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e7c0cee70 0x7f6e7c0cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-18T10:45:25.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:25.001+0000 7f6e7ad9d640 1 --2- 172.21.15.89:0/3888767860 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e7c0cee70 0x7f6e7c0cf2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T10:45:25.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:25.001+0000 7f6e7a59c640 1 -- 172.21.15.89:0/3888767860 >> v2:172.21.15.154:3300/0 conn(0x7f6e7c15c5d0 msgr2=0x7f6e7c0d1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-18T10:45:25.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:25.001+0000 7f6e7a59c640 1 -- 172.21.15.89:0/3888767860 >> v2:172.21.15.154:3300/0 conn(0x7f6e7c15c5d0 msgr2=0x7f6e7c0d1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-18T10:45:25.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:25.001+0000 7f6e7a59c640 1 --2- 172.21.15.89:0/3888767860 >> v2:172.21.15.154:3300/0 conn(0x7f6e7c15c5d0 0x7f6e7c0d1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-18T10:45:25.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:25.001+0000 7f6e7a59c640 1 --2- 172.21.15.89:0/3888767860 >> v2:172.21.15.154:3300/0 conn(0x7f6e7c15c5d0 0x7f6e7c0d1930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T10:45:25.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:25.401+0000 7f6e79d9b640 1 --2- 172.21.15.89:0/3888767860 >> v2:172.21.15.89:3300/0 conn(0x7f6e7c0cd860 0x7f6e7c0d1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:25.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:25.401+0000 7f6e79d9b640 1 -- 172.21.15.89:0/3888767860 >> v2:172.21.15.89:3300/0 conn(0x7f6e7c0cd860 msgr2=0x7f6e7c0d1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-18T10:45:25.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:25.401+0000 7f6e79d9b640 1 -- 172.21.15.89:0/3888767860 >> v2:172.21.15.89:3300/0 conn(0x7f6e7c0cd860 msgr2=0x7f6e7c0d1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-18T10:45:25.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:25.401+0000 7f6e79d9b640 1 --2- 172.21.15.89:0/3888767860 >> v2:172.21.15.89:3300/0 conn(0x7f6e7c0cd860 0x7f6e7c0d1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-18T10:45:25.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:25.401+0000 7f6e79d9b640 1 --2- 172.21.15.89:0/3888767860 >> v2:172.21.15.89:3300/0 conn(0x7f6e7c0cd860 0x7f6e7c0d1e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-18T10:45:25.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:25.405+0000 7f6e7a59c640 1 --2- 172.21.15.89:0/3888767860 >> v2:172.21.15.154:3300/0 conn(0x7f6e7c15c5d0 0x7f6e7c0d1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:25.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:25.405+0000 7f6e7ad9d640 1 --2- 172.21.15.89:0/3888767860 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e7c0cee70 0x7f6e7c0cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:25.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:25.405+0000 7f6e7a59c640 1 -- 172.21.15.89:0/3888767860 >> v2:172.21.15.154:3300/0 conn(0x7f6e7c15c5d0 msgr2=0x7f6e7c0d1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-18T10:45:25.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:25.405+0000 7f6e7a59c640 1 -- 172.21.15.89:0/3888767860 >> v2:172.21.15.154:3300/0 conn(0x7f6e7c15c5d0 msgr2=0x7f6e7c0d1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-18T10:45:25.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:25.405+0000 7f6e7a59c640 1 --2- 172.21.15.89:0/3888767860 >> v2:172.21.15.154:3300/0 conn(0x7f6e7c15c5d0 0x7f6e7c0d1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-18T10:45:25.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:25.405+0000 7f6e7ad9d640 1 -- 172.21.15.89:0/3888767860 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e7c0cee70 msgr2=0x7f6e7c0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-18T10:45:25.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:25.405+0000 7f6e7ad9d640 1 -- 172.21.15.89:0/3888767860 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e7c0cee70 msgr2=0x7f6e7c0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-18T10:45:25.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:25.405+0000 7f6e7ad9d640 1 --2- 172.21.15.89:0/3888767860 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e7c0cee70 0x7f6e7c0cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-18T10:45:25.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:25.405+0000 7f6e7a59c640 1 --2- 172.21.15.89:0/3888767860 >> v2:172.21.15.154:3300/0 conn(0x7f6e7c15c5d0 0x7f6e7c0d1930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-18T10:45:25.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:25.405+0000 7f6e7ad9d640 1 --2- 172.21.15.89:0/3888767860 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e7c0cee70 0x7f6e7c0cf2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-18T10:45:26.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:26.205+0000 7f6e79d9b640 1 --2- 172.21.15.89:0/3888767860 >> v2:172.21.15.89:3300/0 conn(0x7f6e7c0cd860 0x7f6e7c0d1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:26.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:26.205+0000 7f6e79d9b640 1 -- 172.21.15.89:0/3888767860 >> v2:172.21.15.89:3300/0 conn(0x7f6e7c0cd860 msgr2=0x7f6e7c0d1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-18T10:45:26.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:26.205+0000 7f6e79d9b640 1 -- 172.21.15.89:0/3888767860 >> v2:172.21.15.89:3300/0 conn(0x7f6e7c0cd860 msgr2=0x7f6e7c0d1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-18T10:45:26.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:26.205+0000 7f6e79d9b640 1 --2- 172.21.15.89:0/3888767860 >> v2:172.21.15.89:3300/0 conn(0x7f6e7c0cd860 0x7f6e7c0d1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-18T10:45:26.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:26.205+0000 7f6e79d9b640 1 --2- 172.21.15.89:0/3888767860 >> v2:172.21.15.89:3300/0 conn(0x7f6e7c0cd860 0x7f6e7c0d1e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-18T10:45:26.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:26.209+0000 7f6e7a59c640 1 --2- 172.21.15.89:0/3888767860 >> v2:172.21.15.154:3300/0 conn(0x7f6e7c15c5d0 0x7f6e7c0d1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:26.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:26.209+0000 7f6e7ad9d640 1 --2- 172.21.15.89:0/3888767860 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e7c0cee70 0x7f6e7c0cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:26.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:26.209+0000 7f6e7a59c640 1 -- 172.21.15.89:0/3888767860 >> v2:172.21.15.154:3300/0 conn(0x7f6e7c15c5d0 msgr2=0x7f6e7c0d1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-18T10:45:26.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:26.209+0000 7f6e7ad9d640 1 -- 172.21.15.89:0/3888767860 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e7c0cee70 msgr2=0x7f6e7c0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-18T10:45:26.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:26.209+0000 7f6e7a59c640 1 -- 172.21.15.89:0/3888767860 >> v2:172.21.15.154:3300/0 conn(0x7f6e7c15c5d0 msgr2=0x7f6e7c0d1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-18T10:45:26.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:26.209+0000 7f6e7ad9d640 1 -- 172.21.15.89:0/3888767860 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e7c0cee70 msgr2=0x7f6e7c0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-18T10:45:26.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:26.209+0000 7f6e7a59c640 1 --2- 172.21.15.89:0/3888767860 >> v2:172.21.15.154:3300/0 conn(0x7f6e7c15c5d0 0x7f6e7c0d1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-18T10:45:26.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:26.209+0000 7f6e7ad9d640 1 --2- 172.21.15.89:0/3888767860 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e7c0cee70 0x7f6e7c0cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-18T10:45:26.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:26.209+0000 7f6e7a59c640 1 --2- 172.21.15.89:0/3888767860 >> v2:172.21.15.154:3300/0 conn(0x7f6e7c15c5d0 0x7f6e7c0d1930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-18T10:45:26.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:26.209+0000 7f6e7ad9d640 1 --2- 172.21.15.89:0/3888767860 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e7c0cee70 0x7f6e7c0cf2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-18T10:45:27.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:27.793+0000 7f6e6bfff640 1 -- 172.21.15.89:0/3888767860 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e7c0cee70 msgr2=0x7f6e7c0cf2c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T10:45:27.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:27.793+0000 7f6e6bfff640 1 --2- 172.21.15.89:0/3888767860 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e7c0cee70 0x7f6e7c0cf2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:27.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:27.793+0000 7f6e6bfff640 1 -- 172.21.15.89:0/3888767860 >> v2:172.21.15.154:3300/0 conn(0x7f6e7c15c5d0 msgr2=0x7f6e7c0d1930 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T10:45:27.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:27.793+0000 7f6e6bfff640 1 --2- 172.21.15.89:0/3888767860 >> v2:172.21.15.154:3300/0 conn(0x7f6e7c15c5d0 0x7f6e7c0d1930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:27.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:27.793+0000 7f6e6bfff640 1 -- 172.21.15.89:0/3888767860 >> v2:172.21.15.89:3300/0 conn(0x7f6e7c0cd860 msgr2=0x7f6e7c0d1e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T10:45:27.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:27.793+0000 7f6e6bfff640 1 --2- 172.21.15.89:0/3888767860 >> v2:172.21.15.89:3300/0 conn(0x7f6e7c0cd860 0x7f6e7c0d1e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:27.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:27.793+0000 7f6e6bfff640 1 -- 172.21.15.89:0/3888767860 --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e7c0d0800 con 0x7f6e640073e0 2023-10-18T10:45:27.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:27.793+0000 7f6e6bfff640 1 -- 172.21.15.89:0/3888767860 --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e6400c130 con 0x7f6e640036c0 2023-10-18T10:45:29.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.965+0000 7f6e7959a640 1 -- 172.21.15.89:0/3888767860 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3559732254 0 0) 0x7f6e7c0d0800 con 0x7f6e640073e0 2023-10-18T10:45:29.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.965+0000 7f6e7959a640 1 -- 172.21.15.89:0/3888767860 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e5c003530 con 0x7f6e640073e0 2023-10-18T10:45:29.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.965+0000 7f6e7959a640 1 -- 172.21.15.89:0/3888767860 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2288082513 0 0) 0x7f6e5c003530 con 0x7f6e640073e0 2023-10-18T10:45:29.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.965+0000 7f6e7959a640 1 -- 172.21.15.89:0/3888767860 >> v1:172.21.15.154:6789/0 conn(0x7f6e640036c0 legacy=0x7f6e64005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:29.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.965+0000 7f6e7959a640 1 -- 172.21.15.89:0/3888767860 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e6400d370 con 0x7f6e640073e0 2023-10-18T10:45:29.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.965+0000 7f6e7959a640 1 -- 172.21.15.89:0/3888767860 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7f6e7000c570 con 0x7f6e640073e0 2023-10-18T10:45:29.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.965+0000 7f6e7959a640 1 -- 172.21.15.89:0/3888767860 >> v1:172.21.15.89:6789/0 conn(0x7f6e640073e0 legacy=0x7f6e64009880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:29.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.965+0000 7f6e7ad9d640 1 -- 172.21.15.89:0/3888767860 reap_dead start 2023-10-18T10:45:29.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.965+0000 7f6e7959a640 1 --2- 172.21.15.89:0/3888767860 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e5c003a70 0x7f6e5c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:29.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.965+0000 7f6e7959a640 1 --2- 172.21.15.89:0/3888767860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e5c0047d0 0x7f6e5c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:29.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.965+0000 7f6e7959a640 1 --2- 172.21.15.89:0/3888767860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e5c009980 0x7f6e5c00bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:29.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.965+0000 7f6e7959a640 1 -- 172.21.15.89:0/3888767860 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c00c390 con 0x7f6e5c009980 2023-10-18T10:45:29.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.965+0000 7f6e7959a640 1 -- 172.21.15.89:0/3888767860 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c00c500 con 0x7f6e5c0047d0 2023-10-18T10:45:29.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.965+0000 7f6e7959a640 1 -- 172.21.15.89:0/3888767860 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c00c6c0 con 0x7f6e5c003a70 2023-10-18T10:45:29.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.965+0000 7f6e7ad9d640 1 --2- 172.21.15.89:0/3888767860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e5c009980 0x7f6e5c00bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:29.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.965+0000 7f6e7a59c640 1 --2- 172.21.15.89:0/3888767860 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e5c003a70 0x7f6e5c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:29.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.965+0000 7f6e79d9b640 1 --2- 172.21.15.89:0/3888767860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e5c0047d0 0x7f6e5c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:29.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.965+0000 7f6e7ad9d640 1 -- 172.21.15.89:0/3888767860 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e5c003a70 msgr2=0x7f6e5c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:29.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.965+0000 7f6e7ad9d640 1 --2- 172.21.15.89:0/3888767860 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e5c003a70 0x7f6e5c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:29.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.965+0000 7f6e7ad9d640 1 -- 172.21.15.89:0/3888767860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e5c0047d0 msgr2=0x7f6e5c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:29.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.965+0000 7f6e7ad9d640 1 --2- 172.21.15.89:0/3888767860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e5c0047d0 0x7f6e5c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:29.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.965+0000 7f6e7ad9d640 1 -- 172.21.15.89:0/3888767860 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e5c003530 con 0x7f6e5c009980 2023-10-18T10:45:29.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.969+0000 7f6e7ad9d640 1 --2- 172.21.15.89:0/3888767860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e5c009980 0x7f6e5c00bd70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c0cfba0 tx=0x7f6e7401ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57296b1a63c8e52a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:29.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.973+0000 7f6e7959a640 1 -- 172.21.15.89:0/3888767860 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e740a5be0 con 0x7f6e5c009980 2023-10-18T10:45:29.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.973+0000 7f6e7959a640 1 -- 172.21.15.89:0/3888767860 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e740a9070 con 0x7f6e5c009980 2023-10-18T10:45:29.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.973+0000 7f6e7959a640 1 -- 172.21.15.89:0/3888767860 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e740af400 con 0x7f6e5c009980 2023-10-18T10:45:29.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.973+0000 7f6e7b59e640 1 -- 172.21.15.89:0/3888767860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e5c009980 msgr2=0x7f6e5c00bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:29.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.973+0000 7f6e7b59e640 1 --2- 172.21.15.89:0/3888767860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e5c009980 0x7f6e5c00bd70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c0cfba0 tx=0x7f6e7401ce20 comp rx=0 tx=0).stop 2023-10-18T10:45:29.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.973+0000 7f6e7b59e640 1 -- 172.21.15.89:0/3888767860 shutdown_connections 2023-10-18T10:45:29.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.973+0000 7f6e7b59e640 1 --2- 172.21.15.89:0/3888767860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e5c009980 0x7f6e5c00bd70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:29.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.973+0000 7f6e7b59e640 1 --2- 172.21.15.89:0/3888767860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e5c0047d0 0x7f6e5c004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:29.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.973+0000 7f6e7b59e640 1 --2- 172.21.15.89:0/3888767860 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e5c003a70 0x7f6e5c003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:29.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.973+0000 7f6e7b59e640 1 -- 172.21.15.89:0/3888767860 >> 172.21.15.89:0/3888767860 conn(0x7f6e7c0b3f50 msgr2=0x7f6e7c0b4360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:29.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.973+0000 7f6e7b59e640 1 -- 172.21.15.89:0/3888767860 shutdown_connections 2023-10-18T10:45:29.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.973+0000 7f6e7b59e640 1 -- 172.21.15.89:0/3888767860 wait complete. 2023-10-18T10:45:29.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.973+0000 7f6e7b59e640 1 Processor -- start 2023-10-18T10:45:29.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.973+0000 7f6e7b59e640 1 -- start start 2023-10-18T10:45:29.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.977+0000 7f6e7b59e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e5c003a70 0x7f6e7c153c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:29.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.977+0000 7f6e7b59e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e5c0047d0 0x7f6e7c156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:29.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.977+0000 7f6e7b59e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e5c009980 0x7f6e7c15e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:29.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.977+0000 7f6e7b59e640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c158230 con 0x7f6e5c003a70 2023-10-18T10:45:29.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.977+0000 7f6e7b59e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c155360 con 0x7f6e5c009980 2023-10-18T10:45:29.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.977+0000 7f6e7b59e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c1555b0 con 0x7f6e5c0047d0 2023-10-18T10:45:29.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.977+0000 7f6e7a59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e5c003a70 0x7f6e7c153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:29.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.977+0000 7f6e7a59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e5c003a70 0x7f6e7c153c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59152/0 (socket says 172.21.15.89:59152) 2023-10-18T10:45:29.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.977+0000 7f6e7a59c640 1 -- 172.21.15.89:0/1138539788 learned_addr learned my addr 172.21.15.89:0/1138539788 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:45:29.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.977+0000 7f6e79d9b640 1 --2- 172.21.15.89:0/1138539788 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e5c0047d0 0x7f6e7c156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:29.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.977+0000 7f6e7ad9d640 1 --2- 172.21.15.89:0/1138539788 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e5c009980 0x7f6e7c15e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:29.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.977+0000 7f6e7a59c640 1 -- 172.21.15.89:0/1138539788 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e5c0047d0 msgr2=0x7f6e7c156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:29.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.977+0000 7f6e7a59c640 1 --2- 172.21.15.89:0/1138539788 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e5c0047d0 0x7f6e7c156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:29.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.977+0000 7f6e7a59c640 1 -- 172.21.15.89:0/1138539788 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e5c009980 msgr2=0x7f6e7c15e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:29.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.977+0000 7f6e7a59c640 1 --2- 172.21.15.89:0/1138539788 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e5c009980 0x7f6e7c15e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:29.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.977+0000 7f6e7a59c640 1 -- 172.21.15.89:0/1138539788 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7c155890 con 0x7f6e5c003a70 2023-10-18T10:45:29.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.977+0000 7f6e7a59c640 1 --2- 172.21.15.89:0/1138539788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e5c003a70 0x7f6e7c153c20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c004630 tx=0x7f6e6c014690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:29.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.977+0000 7f6e6affd640 1 -- 172.21.15.89:0/1138539788 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c014ec0 con 0x7f6e5c003a70 2023-10-18T10:45:29.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.977+0000 7f6e6affd640 1 -- 172.21.15.89:0/1138539788 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e6c00f040 con 0x7f6e5c003a70 2023-10-18T10:45:29.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.977+0000 7f6e7b59e640 1 -- 172.21.15.89:0/1138539788 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e7c155b70 con 0x7f6e5c003a70 2023-10-18T10:45:29.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.977+0000 7f6e6affd640 1 -- 172.21.15.89:0/1138539788 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c017640 con 0x7f6e5c003a70 2023-10-18T10:45:29.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.977+0000 7f6e7b59e640 1 -- 172.21.15.89:0/1138539788 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e7c156130 con 0x7f6e5c003a70 2023-10-18T10:45:29.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.977+0000 7f6e6affd640 1 -- 172.21.15.89:0/1138539788 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6e6c017aa0 con 0x7f6e5c003a70 2023-10-18T10:45:29.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.977+0000 7f6e6affd640 1 -- 172.21.15.89:0/1138539788 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6e6c01f3c0 con 0x7f6e5c003a70 2023-10-18T10:45:29.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:29.977+0000 7f6e7b59e640 1 -- 172.21.15.89:0/1138539788 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e7c073b50 con 0x7f6e5c003a70 2023-10-18T10:45:30.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:30.001+0000 7f6e6affd640 1 -- 172.21.15.89:0/1138539788 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f6e6c025060 con 0x7f6e5c003a70 2023-10-18T10:45:30.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:30.061+0000 7f6e7b59e640 1 -- 172.21.15.89:0/1138539788 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f6e7c154160 con 0x7f6e5c003a70 2023-10-18T10:45:30.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:30.065+0000 7f6e6affd640 1 -- 172.21.15.89:0/1138539788 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f6e6c01b030 con 0x7f6e5c003a70 2023-10-18T10:45:30.068 INFO:teuthology.orchestra.run.smithi089.stderr:adjusted tunables profile to default 2023-10-18T10:45:30.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:30.069+0000 7f6e7b59e640 1 -- 172.21.15.89:0/1138539788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e5c003a70 msgr2=0x7f6e7c153c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:30.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:30.069+0000 7f6e7b59e640 1 --2- 172.21.15.89:0/1138539788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e5c003a70 0x7f6e7c153c20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c004630 tx=0x7f6e6c014690 comp rx=0 tx=0).stop 2023-10-18T10:45:30.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:30.069+0000 7f6e7b59e640 1 -- 172.21.15.89:0/1138539788 shutdown_connections 2023-10-18T10:45:30.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:30.069+0000 7f6e7b59e640 1 --2- 172.21.15.89:0/1138539788 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e5c0047d0 0x7f6e7c156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:30.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:30.069+0000 7f6e7b59e640 1 --2- 172.21.15.89:0/1138539788 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e5c009980 0x7f6e7c15e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:30.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:30.069+0000 7f6e7b59e640 1 --2- 172.21.15.89:0/1138539788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e5c003a70 0x7f6e7c153c20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:30.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:30.069+0000 7f6e7b59e640 1 -- 172.21.15.89:0/1138539788 >> 172.21.15.89:0/1138539788 conn(0x7f6e7c0b3f50 msgr2=0x7f6e7c0b5340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:30.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:30.069+0000 7f6e7b59e640 1 -- 172.21.15.89:0/1138539788 shutdown_connections 2023-10-18T10:45:30.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:30.069+0000 7f6e7b59e640 1 -- 172.21.15.89:0/1138539788 wait complete. 2023-10-18T10:45:30.088 INFO:tasks.ceph:check_enable_crimson: False 2023-10-18T10:45:30.088 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-18T10:45:30.089 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:45:30.089 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-18T10:45:30.110 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:45:30.110 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-18T10:45:30.170 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:45:30.171 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-18T10:45:30.191 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-18T10:45:30.194+0000 7f3f05055180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T10:45:30.230 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:45:30.233 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-18T10:45:30.290 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T10:45:30.291 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-18T10:45:30.311 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T10:45:30.312 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-18T10:45:30.374 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T10:45:30.374 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-18T10:45:30.440 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T10:45:30.440 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-18T10:45:30.502 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 842c9db2-b707-471b-a554-c3d6314ac38b 0 2023-10-18T10:45:30.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.682+0000 7fc06ff67640 1 Processor -- start 2023-10-18T10:45:30.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.682+0000 7fc06ff67640 1 -- start start 2023-10-18T10:45:30.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.686+0000 7fc06ff67640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc0680b1860 0x7fc0680cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:30.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.686+0000 7fc06ff67640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc0680b1cc0 0x7fc0680cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:30.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.686+0000 7fc06ff67640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0680d1e50 con 0x7fc06815c550 2023-10-18T10:45:30.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.686+0000 7fc06ff67640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0680d2130 con 0x7fc0680b1cc0 2023-10-18T10:45:30.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.686+0000 7fc06ff67640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0680d2410 con 0x7fc0680b1860 2023-10-18T10:45:30.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.686+0000 7fc06d502640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc0680b1860 0x7fc0680cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:30.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.686+0000 7fc06d502640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc0680b1860 0x7fc0680cf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60614/0 (socket says 172.21.15.154:60614) 2023-10-18T10:45:30.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.686+0000 7fc06d502640 1 -- 172.21.15.154:0/3699209133 learned_addr learned my addr 172.21.15.154:0/3699209133 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-18T10:45:30.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.686+0000 7fc06e504640 1 --2- 172.21.15.154:0/3699209133 >> v2:172.21.15.89:3300/0 conn(0x7fc0680b1cc0 0x7fc0680cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:30.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.686+0000 7fc06d502640 1 -- 172.21.15.154:0/3699209133 >> v2:172.21.15.89:3300/0 conn(0x7fc0680b1cc0 msgr2=0x7fc0680cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:30.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.686+0000 7fc06d502640 1 --2- 172.21.15.154:0/3699209133 >> v2:172.21.15.89:3300/0 conn(0x7fc0680b1cc0 0x7fc0680cfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:30.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.686+0000 7fc06d502640 1 -- 172.21.15.154:0/3699209133 >> v1:172.21.15.89:6789/0 conn(0x7fc06815c550 legacy=0x7fc0680d2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:30.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.686+0000 7fc06d502640 1 -- 172.21.15.154:0/3699209133 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0680d26f0 con 0x7fc0680b1860 2023-10-18T10:45:30.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.690+0000 7fc06d502640 1 --2- 172.21.15.154:0/3699209133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0680b1860 0x7fc0680cf610 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc0680b4e60 tx=0x7fc05801c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59078fb41aba07f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:30.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.690+0000 7fc06cd01640 1 -- 172.21.15.154:0/3699209133 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0580aa650 con 0x7fc0680b1860 2023-10-18T10:45:30.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.690+0000 7fc06cd01640 1 -- 172.21.15.154:0/3699209133 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0580aa7b0 con 0x7fc0680b1860 2023-10-18T10:45:30.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.690+0000 7fc06cd01640 1 -- 172.21.15.154:0/3699209133 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0580aab40 con 0x7fc0680b1860 2023-10-18T10:45:30.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.690+0000 7fc06ff67640 1 -- 172.21.15.154:0/3699209133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0680b1860 msgr2=0x7fc0680cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:30.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.690+0000 7fc06ff67640 1 --2- 172.21.15.154:0/3699209133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0680b1860 0x7fc0680cf610 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc0680b4e60 tx=0x7fc05801c4d0 comp rx=0 tx=0).stop 2023-10-18T10:45:30.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.690+0000 7fc06ff67640 1 -- 172.21.15.154:0/3699209133 shutdown_connections 2023-10-18T10:45:30.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.690+0000 7fc06ff67640 1 --2- 172.21.15.154:0/3699209133 >> v2:172.21.15.89:3300/0 conn(0x7fc0680b1cc0 0x7fc0680cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:30.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.690+0000 7fc06ff67640 1 --2- 172.21.15.154:0/3699209133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0680b1860 0x7fc0680cf610 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:30.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.690+0000 7fc06ff67640 1 -- 172.21.15.154:0/3699209133 >> 172.21.15.154:0/3699209133 conn(0x7fc068069d00 msgr2=0x7fc06806a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:30.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.690+0000 7fc06ff67640 1 -- 172.21.15.154:0/3699209133 shutdown_connections 2023-10-18T10:45:30.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.690+0000 7fc06ff67640 1 -- 172.21.15.154:0/3699209133 wait complete. 2023-10-18T10:45:30.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.690+0000 7fc06ff67640 1 Processor -- start 2023-10-18T10:45:30.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.690+0000 7fc06ff67640 1 -- start start 2023-10-18T10:45:30.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.694+0000 7fc06ff67640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0680b1860 0x7fc068147a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:30.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.694+0000 7fc06ff67640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0680b1cc0 0x7fc0680d0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:30.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.694+0000 7fc06ff67640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc06815c550 0x7fc0680d0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:30.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.694+0000 7fc06ff67640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0680b6910 con 0x7fc0680b1cc0 2023-10-18T10:45:30.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.694+0000 7fc06ff67640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc068147fc0 con 0x7fc0680b1860 2023-10-18T10:45:30.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.694+0000 7fc06ff67640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0681561f0 con 0x7fc06815c550 2023-10-18T10:45:30.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.694+0000 7fc06dd03640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0680b1860 0x7fc068147a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:30.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.694+0000 7fc06dd03640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0680b1860 0x7fc068147a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60622/0 (socket says 172.21.15.154:60622) 2023-10-18T10:45:30.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.694+0000 7fc06dd03640 1 -- 172.21.15.154:0/3688495091 learned_addr learned my addr 172.21.15.154:0/3688495091 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-18T10:45:30.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.694+0000 7fc06e504640 1 --2- 172.21.15.154:0/3688495091 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc06815c550 0x7fc0680d0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:30.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.694+0000 7fc06d502640 1 --2- 172.21.15.154:0/3688495091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0680b1cc0 0x7fc0680d0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:30.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.694+0000 7fc06dd03640 1 -- 172.21.15.154:0/3688495091 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc06815c550 msgr2=0x7fc0680d0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:30.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.694+0000 7fc06dd03640 1 --2- 172.21.15.154:0/3688495091 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc06815c550 0x7fc0680d0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:30.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.694+0000 7fc06dd03640 1 -- 172.21.15.154:0/3688495091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0680b1cc0 msgr2=0x7fc0680d0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:30.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.694+0000 7fc06dd03640 1 --2- 172.21.15.154:0/3688495091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0680b1cc0 0x7fc0680d0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:30.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.694+0000 7fc06dd03640 1 -- 172.21.15.154:0/3688495091 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0580094d0 con 0x7fc0680b1860 2023-10-18T10:45:30.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.694+0000 7fc06dd03640 1 --2- 172.21.15.154:0/3688495091 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0680b1860 0x7fc068147a80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc06400c3c0 tx=0x7fc06400c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:30.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.694+0000 7fc05e7fc640 1 -- 172.21.15.154:0/3688495091 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0640052d0 con 0x7fc0680b1860 2023-10-18T10:45:30.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.694+0000 7fc05e7fc640 1 -- 172.21.15.154:0/3688495091 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc064005430 con 0x7fc0680b1860 2023-10-18T10:45:30.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.694+0000 7fc06ff67640 1 -- 172.21.15.154:0/3688495091 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc068156440 con 0x7fc0680b1860 2023-10-18T10:45:30.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.694+0000 7fc05e7fc640 1 -- 172.21.15.154:0/3688495091 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc064003840 con 0x7fc0680b1860 2023-10-18T10:45:30.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.694+0000 7fc06ff67640 1 -- 172.21.15.154:0/3688495091 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc068156a00 con 0x7fc0680b1860 2023-10-18T10:45:30.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.694+0000 7fc05e7fc640 1 -- 172.21.15.154:0/3688495091 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc064003ca0 con 0x7fc0680b1860 2023-10-18T10:45:30.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.694+0000 7fc05e7fc640 1 -- 172.21.15.154:0/3688495091 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc06401c780 con 0x7fc0680b1860 2023-10-18T10:45:30.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.694+0000 7fc06ff67640 1 -- 172.21.15.154:0/3688495091 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc06806a2f0 con 0x7fc0680b1860 2023-10-18T10:45:30.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.714+0000 7fc05e7fc640 1 -- 172.21.15.154:0/3688495091 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fc064017070 con 0x7fc0680b1860 2023-10-18T10:45:30.744 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-18T10:45:30.746+0000 7f3f05055180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T10:45:30.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.794+0000 7fc06ff67640 1 -- 172.21.15.154:0/3688495091 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd new", "uuid": "842c9db2-b707-471b-a554-c3d6314ac38b", "id": 0} v 0) v1 -- 0x7fc0680cf7b0 con 0x7fc0680b1860 2023-10-18T10:45:30.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.798+0000 7fc05e7fc640 1 -- 172.21.15.154:0/3688495091 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "842c9db2-b707-471b-a554-c3d6314ac38b", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc06401c940 con 0x7fc0680b1860 2023-10-18T10:45:30.798 INFO:teuthology.orchestra.run.smithi154.stdout:0 2023-10-18T10:45:30.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.802+0000 7fc06ff67640 1 -- 172.21.15.154:0/3688495091 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0680b1860 msgr2=0x7fc068147a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:30.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.802+0000 7fc06ff67640 1 --2- 172.21.15.154:0/3688495091 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0680b1860 0x7fc068147a80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc06400c3c0 tx=0x7fc06400c880 comp rx=0 tx=0).stop 2023-10-18T10:45:30.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.802+0000 7fc06ff67640 1 -- 172.21.15.154:0/3688495091 shutdown_connections 2023-10-18T10:45:30.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.802+0000 7fc06ff67640 1 --2- 172.21.15.154:0/3688495091 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc06815c550 0x7fc0680d0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:30.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.802+0000 7fc06ff67640 1 --2- 172.21.15.154:0/3688495091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0680b1cc0 0x7fc0680d0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:30.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.802+0000 7fc06ff67640 1 --2- 172.21.15.154:0/3688495091 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0680b1860 0x7fc068147a80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:30.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.802+0000 7fc06ff67640 1 -- 172.21.15.154:0/3688495091 >> 172.21.15.154:0/3688495091 conn(0x7fc068069d00 msgr2=0x7fc0680b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:30.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.802+0000 7fc06ff67640 1 -- 172.21.15.154:0/3688495091 shutdown_connections 2023-10-18T10:45:30.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.802+0000 7fc06ff67640 1 -- 172.21.15.154:0/3688495091 wait complete. 2023-10-18T10:45:30.818 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 0835cc88-e19b-4a4e-8857-d30c7e6804ef 1 2023-10-18T10:45:30.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.958+0000 7fe5268bd640 1 Processor -- start 2023-10-18T10:45:30.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.958+0000 7fe5268bd640 1 -- start start 2023-10-18T10:45:30.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.958+0000 7fe5268bd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe52015c550 0x7fe5200d2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:30.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.958+0000 7fe5268bd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe5200b1860 0x7fe5200cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:30.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.958+0000 7fe5268bd640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5200d1e50 con 0x7fe5200b1cc0 2023-10-18T10:45:30.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.958+0000 7fe5268bd640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5200d2130 con 0x7fe5200b1860 2023-10-18T10:45:30.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.958+0000 7fe5268bd640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5200d2410 con 0x7fe52015c550 2023-10-18T10:45:30.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.958+0000 7fe51ffff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe52015c550 0x7fe5200d2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:30.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.958+0000 7fe51ffff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe52015c550 0x7fe5200d2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:54644/0 (socket says 172.21.15.154:54644) 2023-10-18T10:45:30.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.958+0000 7fe51ffff640 1 -- 172.21.15.154:0/1677704983 learned_addr learned my addr 172.21.15.154:0/1677704983 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-18T10:45:30.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.958+0000 7fe51f7fe640 1 --2- 172.21.15.154:0/1677704983 >> v2:172.21.15.89:3300/0 conn(0x7fe5200b1860 0x7fe5200cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:30.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.958+0000 7fe51ffff640 1 -- 172.21.15.154:0/1677704983 >> v2:172.21.15.89:3300/0 conn(0x7fe5200b1860 msgr2=0x7fe5200cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:30.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.958+0000 7fe51ffff640 1 --2- 172.21.15.154:0/1677704983 >> v2:172.21.15.89:3300/0 conn(0x7fe5200b1860 0x7fe5200cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:30.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.958+0000 7fe51ffff640 1 -- 172.21.15.154:0/1677704983 >> v1:172.21.15.89:6789/0 conn(0x7fe5200b1cc0 legacy=0x7fe5200b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:30.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.958+0000 7fe51ffff640 1 -- 172.21.15.154:0/1677704983 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5200d26f0 con 0x7fe52015c550 2023-10-18T10:45:30.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.958+0000 7fe51f7fe640 1 --2- 172.21.15.154:0/1677704983 >> v2:172.21.15.89:3300/0 conn(0x7fe5200b1860 0x7fe5200cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:45:30.961 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-18T10:45:30.962+0000 7f3f05055180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T10:45:30.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.962+0000 7fe51ffff640 1 --2- 172.21.15.154:0/1677704983 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe52015c550 0x7fe5200d2ef0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe510009870 tx=0x7fe51001c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dd3a8ef8ea31ca1 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:30.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.962+0000 7fe51effd640 1 -- 172.21.15.154:0/1677704983 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5100ac600 con 0x7fe52015c550 2023-10-18T10:45:30.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.962+0000 7fe51effd640 1 -- 172.21.15.154:0/1677704983 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5100a8050 con 0x7fe52015c550 2023-10-18T10:45:30.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.962+0000 7fe51effd640 1 -- 172.21.15.154:0/1677704983 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5100ac980 con 0x7fe52015c550 2023-10-18T10:45:30.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.962+0000 7fe5268bd640 1 -- 172.21.15.154:0/1677704983 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe52015c550 msgr2=0x7fe5200d2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:30.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.962+0000 7fe5268bd640 1 --2- 172.21.15.154:0/1677704983 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe52015c550 0x7fe5200d2ef0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe510009870 tx=0x7fe51001c4d0 comp rx=0 tx=0).stop 2023-10-18T10:45:30.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.962+0000 7fe5268bd640 1 -- 172.21.15.154:0/1677704983 shutdown_connections 2023-10-18T10:45:30.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.962+0000 7fe5268bd640 1 --2- 172.21.15.154:0/1677704983 >> v2:172.21.15.89:3300/0 conn(0x7fe5200b1860 0x7fe5200cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:30.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.962+0000 7fe5268bd640 1 --2- 172.21.15.154:0/1677704983 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe52015c550 0x7fe5200d2ef0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:30.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.962+0000 7fe5268bd640 1 -- 172.21.15.154:0/1677704983 >> 172.21.15.154:0/1677704983 conn(0x7fe520069d00 msgr2=0x7fe52006a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:30.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.962+0000 7fe5268bd640 1 -- 172.21.15.154:0/1677704983 shutdown_connections 2023-10-18T10:45:30.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.962+0000 7fe5268bd640 1 -- 172.21.15.154:0/1677704983 wait complete. 2023-10-18T10:45:30.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.966+0000 7fe5268bd640 1 Processor -- start 2023-10-18T10:45:30.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.966+0000 7fe5268bd640 1 -- start start 2023-10-18T10:45:30.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.966+0000 7fe5268bd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5200b1860 0x7fe52015e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:30.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.970+0000 7fe5268bd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5200b1cc0 0x7fe520155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:30.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.970+0000 7fe5268bd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5200c89e0 0x7fe5200c6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:30.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.970+0000 7fe5268bd640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5200c62f0 con 0x7fe5200c89e0 2023-10-18T10:45:30.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.970+0000 7fe5268bd640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5201560e0 con 0x7fe5200b1860 2023-10-18T10:45:30.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.970+0000 7fe5268bd640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe52015e7f0 con 0x7fe5200b1cc0 2023-10-18T10:45:30.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.970+0000 7fe51ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5200b1860 0x7fe52015e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:30.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.970+0000 7fe51ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5200b1860 0x7fe52015e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60632/0 (socket says 172.21.15.154:60632) 2023-10-18T10:45:30.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.970+0000 7fe51ffff640 1 -- 172.21.15.154:0/1967673131 learned_addr learned my addr 172.21.15.154:0/1967673131 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-18T10:45:30.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.970+0000 7fe51f7fe640 1 --2- 172.21.15.154:0/1967673131 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5200b1cc0 0x7fe520155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:30.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.970+0000 7fe524e5a640 1 --2- 172.21.15.154:0/1967673131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5200c89e0 0x7fe5200c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:30.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.970+0000 7fe51ffff640 1 -- 172.21.15.154:0/1967673131 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5200b1cc0 msgr2=0x7fe520155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:30.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.970+0000 7fe51ffff640 1 --2- 172.21.15.154:0/1967673131 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5200b1cc0 0x7fe520155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:30.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.970+0000 7fe51ffff640 1 -- 172.21.15.154:0/1967673131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5200c89e0 msgr2=0x7fe5200c6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:30.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.970+0000 7fe51ffff640 1 --2- 172.21.15.154:0/1967673131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5200c89e0 0x7fe5200c6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:30.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.970+0000 7fe51ffff640 1 -- 172.21.15.154:0/1967673131 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5100094d0 con 0x7fe5200b1860 2023-10-18T10:45:30.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.970+0000 7fe51f7fe640 1 --2- 172.21.15.154:0/1967673131 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5200b1cc0 0x7fe520155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:45:30.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.970+0000 7fe51ffff640 1 --2- 172.21.15.154:0/1967673131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5200b1860 0x7fe52015e160 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe51001ab80 tx=0x7fe51001ad10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:30.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.970+0000 7fe51cff9640 1 -- 172.21.15.154:0/1967673131 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5100b7620 con 0x7fe5200b1860 2023-10-18T10:45:30.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.970+0000 7fe51cff9640 1 -- 172.21.15.154:0/1967673131 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5100b77a0 con 0x7fe5200b1860 2023-10-18T10:45:30.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.970+0000 7fe5268bd640 1 -- 172.21.15.154:0/1967673131 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe52015ead0 con 0x7fe5200b1860 2023-10-18T10:45:30.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.970+0000 7fe51cff9640 1 -- 172.21.15.154:0/1967673131 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5100b7b50 con 0x7fe5200b1860 2023-10-18T10:45:30.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.970+0000 7fe5268bd640 1 -- 172.21.15.154:0/1967673131 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe52015f090 con 0x7fe5200b1860 2023-10-18T10:45:30.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.970+0000 7fe51cff9640 1 -- 172.21.15.154:0/1967673131 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe5100b3040 con 0x7fe5200b1860 2023-10-18T10:45:30.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.970+0000 7fe51cff9640 1 -- 172.21.15.154:0/1967673131 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fe5100b57a0 con 0x7fe5200b1860 2023-10-18T10:45:30.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.970+0000 7fe5268bd640 1 -- 172.21.15.154:0/1967673131 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5200b7a00 con 0x7fe5200b1860 2023-10-18T10:45:30.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:30.974+0000 7fe51cff9640 1 -- 172.21.15.154:0/1967673131 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fe5100b1070 con 0x7fe5200b1860 2023-10-18T10:45:30.978 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-18T10:45:30.977+0000 7fda4d0d5180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T10:45:31.039 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.041+0000 7fe5268bd640 1 -- 172.21.15.154:0/1967673131 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0835cc88-e19b-4a4e-8857-d30c7e6804ef", "id": 1} v 0) v1 -- 0x7fe5200b6dd0 con 0x7fe5200b1860 2023-10-18T10:45:31.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.049+0000 7fe51cff9640 1 -- 172.21.15.154:0/1967673131 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0835cc88-e19b-4a4e-8857-d30c7e6804ef", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe5100c3460 con 0x7fe5200b1860 2023-10-18T10:45:31.047 INFO:teuthology.orchestra.run.smithi154.stdout:1 2023-10-18T10:45:31.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.049+0000 7fe5268bd640 1 -- 172.21.15.154:0/1967673131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5200b1860 msgr2=0x7fe52015e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:31.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.049+0000 7fe5268bd640 1 --2- 172.21.15.154:0/1967673131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5200b1860 0x7fe52015e160 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe51001ab80 tx=0x7fe51001ad10 comp rx=0 tx=0).stop 2023-10-18T10:45:31.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.049+0000 7fe5268bd640 1 -- 172.21.15.154:0/1967673131 shutdown_connections 2023-10-18T10:45:31.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.049+0000 7fe5268bd640 1 --2- 172.21.15.154:0/1967673131 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5200b1cc0 0x7fe520155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:31.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.049+0000 7fe5268bd640 1 --2- 172.21.15.154:0/1967673131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5200c89e0 0x7fe5200c6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:31.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.049+0000 7fe5268bd640 1 --2- 172.21.15.154:0/1967673131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5200b1860 0x7fe52015e160 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:31.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.049+0000 7fe5268bd640 1 -- 172.21.15.154:0/1967673131 >> 172.21.15.154:0/1967673131 conn(0x7fe520069d00 msgr2=0x7fe5200b72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:31.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.049+0000 7fe5268bd640 1 -- 172.21.15.154:0/1967673131 shutdown_connections 2023-10-18T10:45:31.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.049+0000 7fe5268bd640 1 -- 172.21.15.154:0/1967673131 wait complete. 2023-10-18T10:45:31.067 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 95eb0a15-8b08-4e57-8b37-34096114f81f 2 2023-10-18T10:45:31.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.205+0000 7fec8e0ec640 1 Processor -- start 2023-10-18T10:45:31.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.205+0000 7fec8e0ec640 1 -- start start 2023-10-18T10:45:31.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.205+0000 7fec8e0ec640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fec880b1860 0x7fec880cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:31.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.205+0000 7fec8e0ec640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec880d1e50 con 0x7fec8815c550 2023-10-18T10:45:31.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.205+0000 7fec8e0ec640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec880d2130 con 0x7fec880b1cc0 2023-10-18T10:45:31.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.205+0000 7fec8e0ec640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec880d2410 con 0x7fec880b1860 2023-10-18T10:45:31.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.205+0000 7fec7ffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fec880b1860 0x7fec880cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:31.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.205+0000 7fec87fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fec880b1cc0 0x7fec880b55f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:37684/0 (socket says 172.21.15.154:37684) 2023-10-18T10:45:31.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.205+0000 7fec87fff640 1 -- 172.21.15.154:0/1611074952 learned_addr learned my addr 172.21.15.154:0/1611074952 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-10-18T10:45:31.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.205+0000 7fec7ffff640 1 -- 172.21.15.154:0/1611074952 >> v1:172.21.15.154:6789/0 conn(0x7fec880b1cc0 legacy=0x7fec880b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:31.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.205+0000 7fec7ffff640 1 -- 172.21.15.154:0/1611074952 >> v1:172.21.15.89:6789/0 conn(0x7fec8815c550 legacy=0x7fec880d2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:31.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.205+0000 7fec7ffff640 1 -- 172.21.15.154:0/1611074952 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec880d26f0 con 0x7fec880b1860 2023-10-18T10:45:31.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.209+0000 7fec7ffff640 1 --2- 172.21.15.154:0/1611074952 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec880b1860 0x7fec880cf610 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fec700097c0 tx=0x7fec7001c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=538bd6f0356a8f51 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:31.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.209+0000 7fec86ffd640 1 -- 172.21.15.154:0/1611074952 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec700ad660 con 0x7fec880b1860 2023-10-18T10:45:31.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.209+0000 7fec86ffd640 1 -- 172.21.15.154:0/1611074952 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec700ad7c0 con 0x7fec880b1860 2023-10-18T10:45:31.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.209+0000 7fec86ffd640 1 -- 172.21.15.154:0/1611074952 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec700adb50 con 0x7fec880b1860 2023-10-18T10:45:31.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.209+0000 7fec8e0ec640 1 -- 172.21.15.154:0/1611074952 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec880b1860 msgr2=0x7fec880cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:31.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.209+0000 7fec8e0ec640 1 --2- 172.21.15.154:0/1611074952 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec880b1860 0x7fec880cf610 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fec700097c0 tx=0x7fec7001c450 comp rx=0 tx=0).stop 2023-10-18T10:45:31.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.209+0000 7fec8e0ec640 1 -- 172.21.15.154:0/1611074952 shutdown_connections 2023-10-18T10:45:31.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.209+0000 7fec8e0ec640 1 --2- 172.21.15.154:0/1611074952 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec880b1860 0x7fec880cf610 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:31.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.209+0000 7fec8e0ec640 1 -- 172.21.15.154:0/1611074952 >> 172.21.15.154:0/1611074952 conn(0x7fec88069d00 msgr2=0x7fec8806a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:31.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.209+0000 7fec8e0ec640 1 -- 172.21.15.154:0/1611074952 shutdown_connections 2023-10-18T10:45:31.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.209+0000 7fec8e0ec640 1 -- 172.21.15.154:0/1611074952 wait complete. 2023-10-18T10:45:31.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.213+0000 7fec8e0ec640 1 Processor -- start 2023-10-18T10:45:31.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.213+0000 7fec8e0ec640 1 -- start start 2023-10-18T10:45:31.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.213+0000 7fec8e0ec640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec880b1860 0x7fec8814acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:31.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.213+0000 7fec8e0ec640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec880b1cc0 0x7fec8814d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:31.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.213+0000 7fec8e0ec640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec8815c550 0x7fec8815bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:31.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.213+0000 7fec8e0ec640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8814c8f0 con 0x7fec880b1860 2023-10-18T10:45:31.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.213+0000 7fec8e0ec640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8814cbd0 con 0x7fec8815c550 2023-10-18T10:45:31.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.213+0000 7fec8e0ec640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec8814ceb0 con 0x7fec880b1cc0 2023-10-18T10:45:31.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.213+0000 7fec7ffff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec880b1cc0 0x7fec8814d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:31.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.213+0000 7fec87fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec8815c550 0x7fec8815bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:31.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.213+0000 7fec7ffff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec880b1cc0 0x7fec8814d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:54666/0 (socket says 172.21.15.154:54666) 2023-10-18T10:45:31.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.213+0000 7fec7ffff640 1 -- 172.21.15.154:0/3597243531 learned_addr learned my addr 172.21.15.154:0/3597243531 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-18T10:45:31.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.213+0000 7fec877fe640 1 --2- 172.21.15.154:0/3597243531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec880b1860 0x7fec8814acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:31.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.213+0000 7fec7ffff640 1 -- 172.21.15.154:0/3597243531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec8815c550 msgr2=0x7fec8815bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:31.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.213+0000 7fec7ffff640 1 --2- 172.21.15.154:0/3597243531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec8815c550 0x7fec8815bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:31.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.213+0000 7fec7ffff640 1 -- 172.21.15.154:0/3597243531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec880b1860 msgr2=0x7fec8814acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:31.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.213+0000 7fec7ffff640 1 --2- 172.21.15.154:0/3597243531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec880b1860 0x7fec8814acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:31.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.213+0000 7fec7ffff640 1 -- 172.21.15.154:0/3597243531 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec70009420 con 0x7fec880b1cc0 2023-10-18T10:45:31.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.213+0000 7fec7ffff640 1 --2- 172.21.15.154:0/3597243531 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec880b1cc0 0x7fec8814d990 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fec700b0040 tx=0x7fec700191f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:31.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.213+0000 7fec84ff9640 1 -- 172.21.15.154:0/3597243531 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec700b6820 con 0x7fec880b1cc0 2023-10-18T10:45:31.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.213+0000 7fec84ff9640 1 -- 172.21.15.154:0/3597243531 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec700b69c0 con 0x7fec880b1cc0 2023-10-18T10:45:31.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.213+0000 7fec84ff9640 1 -- 172.21.15.154:0/3597243531 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec700b6d70 con 0x7fec880b1cc0 2023-10-18T10:45:31.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.213+0000 7fec8e0ec640 1 -- 172.21.15.154:0/3597243531 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec8814d190 con 0x7fec880b1cc0 2023-10-18T10:45:31.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.213+0000 7fec8e0ec640 1 -- 172.21.15.154:0/3597243531 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec8815de60 con 0x7fec880b1cc0 2023-10-18T10:45:31.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.217+0000 7fec84ff9640 1 -- 172.21.15.154:0/3597243531 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fec700b4040 con 0x7fec880b1cc0 2023-10-18T10:45:31.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.217+0000 7fec84ff9640 1 -- 172.21.15.154:0/3597243531 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fec700bd780 con 0x7fec880b1cc0 2023-10-18T10:45:31.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.217+0000 7fec8e0ec640 1 -- 172.21.15.154:0/3597243531 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec8814cd10 con 0x7fec880b1cc0 2023-10-18T10:45:31.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.217+0000 7fec84ff9640 1 -- 172.21.15.154:0/3597243531 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fec8814cd10 con 0x7fec880b1cc0 2023-10-18T10:45:31.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.285+0000 7fec8e0ec640 1 -- 172.21.15.154:0/3597243531 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd new", "uuid": "95eb0a15-8b08-4e57-8b37-34096114f81f", "id": 2} v 0) v1 -- 0x7fec8806a330 con 0x7fec880b1cc0 2023-10-18T10:45:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.293+0000 7fec84ff9640 1 -- 172.21.15.154:0/3597243531 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "95eb0a15-8b08-4e57-8b37-34096114f81f", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fec700c4060 con 0x7fec880b1cc0 2023-10-18T10:45:31.292 INFO:teuthology.orchestra.run.smithi154.stdout:2 2023-10-18T10:45:31.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.293+0000 7fec8e0ec640 1 -- 172.21.15.154:0/3597243531 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec880b1cc0 msgr2=0x7fec8814d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:31.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.293+0000 7fec8e0ec640 1 --2- 172.21.15.154:0/3597243531 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec880b1cc0 0x7fec8814d990 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fec700b0040 tx=0x7fec700191f0 comp rx=0 tx=0).stop 2023-10-18T10:45:31.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.293+0000 7fec8e0ec640 1 -- 172.21.15.154:0/3597243531 shutdown_connections 2023-10-18T10:45:31.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.293+0000 7fec8e0ec640 1 --2- 172.21.15.154:0/3597243531 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec880b1cc0 0x7fec8814d990 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:31.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.293+0000 7fec8e0ec640 1 --2- 172.21.15.154:0/3597243531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec8815c550 0x7fec8815bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:31.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.293+0000 7fec8e0ec640 1 --2- 172.21.15.154:0/3597243531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec880b1860 0x7fec8814acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:31.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.293+0000 7fec8e0ec640 1 -- 172.21.15.154:0/3597243531 >> 172.21.15.154:0/3597243531 conn(0x7fec88069d00 msgr2=0x7fec880b6290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:31.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.293+0000 7fec8e0ec640 1 -- 172.21.15.154:0/3597243531 shutdown_connections 2023-10-18T10:45:31.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.293+0000 7fec8e0ec640 1 -- 172.21.15.154:0/3597243531 wait complete. 2023-10-18T10:45:31.311 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 40cd481e-d2af-4210-8d4a-add39171432d 3 2023-10-18T10:45:31.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.449+0000 7f7a34061640 1 Processor -- start 2023-10-18T10:45:31.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.449+0000 7f7a34061640 1 -- start start 2023-10-18T10:45:31.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.449+0000 7f7a34061640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7a2c1607e0 0x7f7a2c164f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:31.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.449+0000 7f7a34061640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a2c069b30 0x7f7a2c069fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:31.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.449+0000 7f7a34061640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a2c0cd8f0 con 0x7f7a2c0ccd70 2023-10-18T10:45:31.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.449+0000 7f7a34061640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a2c164500 con 0x7f7a2c1607e0 2023-10-18T10:45:31.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.449+0000 7f7a34061640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2c06a4f0 con 0x7f7a2c069b30 2023-10-18T10:45:31.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.449+0000 7f7a31dfd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7a2c1607e0 0x7f7a2c164f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:31.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.449+0000 7f7a31dfd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7a2c1607e0 0x7f7a2c164f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60642/0 (socket says 172.21.15.154:60642) 2023-10-18T10:45:31.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.449+0000 7f7a31dfd640 1 -- 172.21.15.154:0/3333038159 learned_addr learned my addr 172.21.15.154:0/3333038159 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-18T10:45:31.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.449+0000 7f7a315fc640 1 --2- 172.21.15.154:0/3333038159 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a2c069b30 0x7f7a2c069fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:31.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.449+0000 7f7a31dfd640 1 -- 172.21.15.154:0/3333038159 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a2c069b30 msgr2=0x7f7a2c069fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:31.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.449+0000 7f7a31dfd640 1 --2- 172.21.15.154:0/3333038159 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a2c069b30 0x7f7a2c069fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:31.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.449+0000 7f7a31dfd640 1 -- 172.21.15.154:0/3333038159 >> v1:172.21.15.89:6789/0 conn(0x7f7a2c0ccd70 legacy=0x7f7a2c0cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:31.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.449+0000 7f7a31dfd640 1 -- 172.21.15.154:0/3333038159 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a2c161a40 con 0x7f7a2c1607e0 2023-10-18T10:45:31.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.453+0000 7f7a31dfd640 1 --2- 172.21.15.154:0/3333038159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a2c1607e0 0x7f7a2c164f10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7a28009870 tx=0x7f7a2801c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=530dd20494785e97 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:31.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.453+0000 7f7a30dfb640 1 -- 172.21.15.154:0/3333038159 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a280a7070 con 0x7f7a2c1607e0 2023-10-18T10:45:31.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.453+0000 7f7a30dfb640 1 -- 172.21.15.154:0/3333038159 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a280ab430 con 0x7f7a2c1607e0 2023-10-18T10:45:31.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.453+0000 7f7a30dfb640 1 -- 172.21.15.154:0/3333038159 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a280ab7c0 con 0x7f7a2c1607e0 2023-10-18T10:45:31.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.453+0000 7f7a34061640 1 -- 172.21.15.154:0/3333038159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a2c1607e0 msgr2=0x7f7a2c164f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:31.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.453+0000 7f7a34061640 1 --2- 172.21.15.154:0/3333038159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a2c1607e0 0x7f7a2c164f10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7a28009870 tx=0x7f7a2801c540 comp rx=0 tx=0).stop 2023-10-18T10:45:31.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.453+0000 7f7a34061640 1 -- 172.21.15.154:0/3333038159 shutdown_connections 2023-10-18T10:45:31.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.453+0000 7f7a34061640 1 --2- 172.21.15.154:0/3333038159 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a2c069b30 0x7f7a2c069fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:31.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.453+0000 7f7a34061640 1 --2- 172.21.15.154:0/3333038159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a2c1607e0 0x7f7a2c164f10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:31.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.453+0000 7f7a34061640 1 -- 172.21.15.154:0/3333038159 >> 172.21.15.154:0/3333038159 conn(0x7f7a2c0ced10 msgr2=0x7f7a2c0cf120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:31.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.453+0000 7f7a34061640 1 -- 172.21.15.154:0/3333038159 shutdown_connections 2023-10-18T10:45:31.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.453+0000 7f7a34061640 1 -- 172.21.15.154:0/3333038159 wait complete. 2023-10-18T10:45:31.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.457+0000 7f7a34061640 1 Processor -- start 2023-10-18T10:45:31.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.457+0000 7f7a34061640 1 -- start start 2023-10-18T10:45:31.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.457+0000 7f7a34061640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a2c069b30 0x7f7a2c15a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:31.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.457+0000 7f7a34061640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a2c0ccd70 0x7f7a2c0b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:31.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.457+0000 7f7a34061640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a2c1607e0 0x7f7a2c0c4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:31.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.457+0000 7f7a34061640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2c0c27e0 con 0x7f7a2c0ccd70 2023-10-18T10:45:31.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.457+0000 7f7a34061640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2c0c4c00 con 0x7f7a2c069b30 2023-10-18T10:45:31.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.457+0000 7f7a34061640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2c0c4ee0 con 0x7f7a2c1607e0 2023-10-18T10:45:31.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.457+0000 7f7a31dfd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a2c069b30 0x7f7a2c15a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:31.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.457+0000 7f7a325fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a2c1607e0 0x7f7a2c0c4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:31.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.457+0000 7f7a31dfd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a2c069b30 0x7f7a2c15a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60646/0 (socket says 172.21.15.154:60646) 2023-10-18T10:45:31.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.457+0000 7f7a31dfd640 1 -- 172.21.15.154:0/947171033 learned_addr learned my addr 172.21.15.154:0/947171033 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-18T10:45:31.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.457+0000 7f7a315fc640 1 --2- 172.21.15.154:0/947171033 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a2c0ccd70 0x7f7a2c0b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:31.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.457+0000 7f7a31dfd640 1 -- 172.21.15.154:0/947171033 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a2c1607e0 msgr2=0x7f7a2c0c4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:31.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.457+0000 7f7a31dfd640 1 --2- 172.21.15.154:0/947171033 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a2c1607e0 0x7f7a2c0c4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:31.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.457+0000 7f7a31dfd640 1 -- 172.21.15.154:0/947171033 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a2c0ccd70 msgr2=0x7f7a2c0b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:31.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.457+0000 7f7a31dfd640 1 --2- 172.21.15.154:0/947171033 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a2c0ccd70 0x7f7a2c0b2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:31.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.457+0000 7f7a31dfd640 1 -- 172.21.15.154:0/947171033 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a280094d0 con 0x7f7a2c069b30 2023-10-18T10:45:31.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.457+0000 7f7a31dfd640 1 --2- 172.21.15.154:0/947171033 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a2c069b30 0x7f7a2c15a050 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7a280ad040 tx=0x7f7a280192e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:31.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.457+0000 7f7a227fc640 1 -- 172.21.15.154:0/947171033 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a280bc070 con 0x7f7a2c069b30 2023-10-18T10:45:31.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.457+0000 7f7a227fc640 1 -- 172.21.15.154:0/947171033 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a280b4420 con 0x7f7a2c069b30 2023-10-18T10:45:31.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.457+0000 7f7a34061640 1 -- 172.21.15.154:0/947171033 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a2c0c51c0 con 0x7f7a2c069b30 2023-10-18T10:45:31.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.461+0000 7f7a227fc640 1 -- 172.21.15.154:0/947171033 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a280b47f0 con 0x7f7a2c069b30 2023-10-18T10:45:31.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.461+0000 7f7a34061640 1 -- 172.21.15.154:0/947171033 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a2c1f9260 con 0x7f7a2c069b30 2023-10-18T10:45:31.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.461+0000 7f7a227fc640 1 -- 172.21.15.154:0/947171033 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7a280b4c50 con 0x7f7a2c069b30 2023-10-18T10:45:31.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.461+0000 7f7a227fc640 1 -- 172.21.15.154:0/947171033 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f7a280c1b90 con 0x7f7a2c069b30 2023-10-18T10:45:31.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.461+0000 7f7a34061640 1 -- 172.21.15.154:0/947171033 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a2c0cd760 con 0x7f7a2c069b30 2023-10-18T10:45:31.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.461+0000 7f7a227fc640 1 -- 172.21.15.154:0/947171033 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f7a280c1d50 con 0x7f7a2c069b30 2023-10-18T10:45:31.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.529+0000 7f7a34061640 1 -- 172.21.15.154:0/947171033 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd new", "uuid": "40cd481e-d2af-4210-8d4a-add39171432d", "id": 3} v 0) v1 -- 0x7f7a2c066ca0 con 0x7f7a2c069b30 2023-10-18T10:45:31.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.537+0000 7f7a227fc640 1 -- 172.21.15.154:0/947171033 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "40cd481e-d2af-4210-8d4a-add39171432d", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7a280c1410 con 0x7f7a2c069b30 2023-10-18T10:45:31.536 INFO:teuthology.orchestra.run.smithi154.stdout:3 2023-10-18T10:45:31.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.537+0000 7f7a34061640 1 -- 172.21.15.154:0/947171033 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a2c069b30 msgr2=0x7f7a2c15a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:31.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.537+0000 7f7a34061640 1 --2- 172.21.15.154:0/947171033 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a2c069b30 0x7f7a2c15a050 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7a280ad040 tx=0x7f7a280192e0 comp rx=0 tx=0).stop 2023-10-18T10:45:31.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.537+0000 7f7a34061640 1 -- 172.21.15.154:0/947171033 shutdown_connections 2023-10-18T10:45:31.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.537+0000 7f7a34061640 1 --2- 172.21.15.154:0/947171033 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a2c1607e0 0x7f7a2c0c4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:31.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.537+0000 7f7a34061640 1 --2- 172.21.15.154:0/947171033 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a2c0ccd70 0x7f7a2c0b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:31.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.537+0000 7f7a34061640 1 --2- 172.21.15.154:0/947171033 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a2c069b30 0x7f7a2c15a050 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:31.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.537+0000 7f7a34061640 1 -- 172.21.15.154:0/947171033 >> 172.21.15.154:0/947171033 conn(0x7f7a2c0ced10 msgr2=0x7f7a2c1633c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:31.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.537+0000 7f7a34061640 1 -- 172.21.15.154:0/947171033 shutdown_connections 2023-10-18T10:45:31.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.537+0000 7f7a34061640 1 -- 172.21.15.154:0/947171033 wait complete. 2023-10-18T10:45:31.550 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-18T10:45:31.549+0000 7fda4d0d5180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T10:45:31.556 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 2fa04448-d520-4f8c-a9b0-684f526019a5 4 2023-10-18T10:45:31.636 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-18T10:45:31.637+0000 7f3f05055180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T10:45:31.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.693+0000 7fbb83b2c640 1 Processor -- start 2023-10-18T10:45:31.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.697+0000 7fbb83b2c640 1 -- start start 2023-10-18T10:45:31.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.697+0000 7fbb83b2c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbb7c0cfd20 0x7fbb7c0b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:31.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.697+0000 7fbb83b2c640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb7c0d1e50 con 0x7fbb7c15c550 2023-10-18T10:45:31.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.697+0000 7fbb83b2c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb7c0d2130 con 0x7fbb7c0b1860 2023-10-18T10:45:31.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.697+0000 7fbb83b2c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c0d2410 con 0x7fbb7c0cfd20 2023-10-18T10:45:31.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.697+0000 7fbb810c7640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fbb7c0b1860 0x7fbb7c0cf610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:37700/0 (socket says 172.21.15.154:37700) 2023-10-18T10:45:31.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.697+0000 7fbb810c7640 1 -- 172.21.15.154:0/1559048313 learned_addr learned my addr 172.21.15.154:0/1559048313 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-10-18T10:45:31.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.697+0000 7fbb820c9640 1 --2- 172.21.15.154:0/1559048313 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbb7c0cfd20 0x7fbb7c0b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:31.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.697+0000 7fbb808c6640 1 -- 172.21.15.154:0/1559048313 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1206164561 0 0) 0x7fbb7c0d2130 con 0x7fbb7c0b1860 2023-10-18T10:45:31.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.697+0000 7fbb808c6640 1 -- 172.21.15.154:0/1559048313 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb68003530 con 0x7fbb7c0b1860 2023-10-18T10:45:31.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.697+0000 7fbb820c9640 1 -- 172.21.15.154:0/1559048313 >> v1:172.21.15.154:6789/0 conn(0x7fbb7c0b1860 legacy=0x7fbb7c0cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:31.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.697+0000 7fbb808c6640 1 -- 172.21.15.154:0/1559048313 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1440978830 0 0) 0x7fbb7c0d1e50 con 0x7fbb7c15c550 2023-10-18T10:45:31.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.697+0000 7fbb820c9640 1 -- 172.21.15.154:0/1559048313 >> v1:172.21.15.89:6789/0 conn(0x7fbb7c15c550 legacy=0x7fbb7c0d2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:31.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.697+0000 7fbb820c9640 1 -- 172.21.15.154:0/1559048313 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7c0d26f0 con 0x7fbb7c0cfd20 2023-10-18T10:45:31.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.701+0000 7fbb820c9640 1 --2- 172.21.15.154:0/1559048313 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbb7c0cfd20 0x7fbb7c0b55f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbb74009dc0 tx=0x7fbb7401c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f9af610eeb73c94 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:31.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.701+0000 7fbb808c6640 1 -- 172.21.15.154:0/1559048313 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb740a5960 con 0x7fbb7c0cfd20 2023-10-18T10:45:31.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.701+0000 7fbb808c6640 1 -- 172.21.15.154:0/1559048313 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb740a5ac0 con 0x7fbb7c0cfd20 2023-10-18T10:45:31.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.701+0000 7fbb808c6640 1 -- 172.21.15.154:0/1559048313 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb740a5e50 con 0x7fbb7c0cfd20 2023-10-18T10:45:31.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.701+0000 7fbb83b2c640 1 -- 172.21.15.154:0/1559048313 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbb7c0cfd20 msgr2=0x7fbb7c0b55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:31.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.701+0000 7fbb83b2c640 1 --2- 172.21.15.154:0/1559048313 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbb7c0cfd20 0x7fbb7c0b55f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbb74009dc0 tx=0x7fbb7401c940 comp rx=0 tx=0).stop 2023-10-18T10:45:31.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.701+0000 7fbb83b2c640 1 -- 172.21.15.154:0/1559048313 shutdown_connections 2023-10-18T10:45:31.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.701+0000 7fbb83b2c640 1 --2- 172.21.15.154:0/1559048313 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbb7c0cfd20 0x7fbb7c0b55f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:31.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.701+0000 7fbb83b2c640 1 -- 172.21.15.154:0/1559048313 >> 172.21.15.154:0/1559048313 conn(0x7fbb7c069d00 msgr2=0x7fbb7c06a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:31.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.701+0000 7fbb83b2c640 1 -- 172.21.15.154:0/1559048313 shutdown_connections 2023-10-18T10:45:31.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.701+0000 7fbb83b2c640 1 -- 172.21.15.154:0/1559048313 wait complete. 2023-10-18T10:45:31.703 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-18T10:45:31.701+0000 7f3f05055180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T10:45:31.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.701+0000 7fbb83b2c640 1 Processor -- start 2023-10-18T10:45:31.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.705+0000 7fbb83b2c640 1 -- start start 2023-10-18T10:45:31.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.705+0000 7fbb83b2c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb7c0b1860 0x7fbb7c14b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:31.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.705+0000 7fbb83b2c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb7c0cfd20 0x7fbb7c14f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:31.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.705+0000 7fbb83b2c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbb7c15c550 0x7fbb7c14f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:31.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.705+0000 7fbb83b2c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c14e030 con 0x7fbb7c0b1860 2023-10-18T10:45:31.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.705+0000 7fbb83b2c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c14e560 con 0x7fbb7c0cfd20 2023-10-18T10:45:31.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.705+0000 7fbb83b2c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c150f70 con 0x7fbb7c15c550 2023-10-18T10:45:31.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.705+0000 7fbb820c9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbb7c15c550 0x7fbb7c14f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:31.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.705+0000 7fbb810c7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb7c0cfd20 0x7fbb7c14f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:31.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.705+0000 7fbb820c9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbb7c15c550 0x7fbb7c14f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:54696/0 (socket says 172.21.15.154:54696) 2023-10-18T10:45:31.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.705+0000 7fbb820c9640 1 -- 172.21.15.154:0/608840616 learned_addr learned my addr 172.21.15.154:0/608840616 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-18T10:45:31.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.705+0000 7fbb818c8640 1 --2- 172.21.15.154:0/608840616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb7c0b1860 0x7fbb7c14b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:31.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.705+0000 7fbb820c9640 1 -- 172.21.15.154:0/608840616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb7c0cfd20 msgr2=0x7fbb7c14f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:31.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.705+0000 7fbb820c9640 1 --2- 172.21.15.154:0/608840616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb7c0cfd20 0x7fbb7c14f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:31.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.705+0000 7fbb820c9640 1 -- 172.21.15.154:0/608840616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb7c0b1860 msgr2=0x7fbb7c14b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:31.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.705+0000 7fbb820c9640 1 --2- 172.21.15.154:0/608840616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb7c0b1860 0x7fbb7c14b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:31.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.705+0000 7fbb820c9640 1 -- 172.21.15.154:0/608840616 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb74009a70 con 0x7fbb7c15c550 2023-10-18T10:45:31.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.705+0000 7fbb810c7640 1 --2- 172.21.15.154:0/608840616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb7c0cfd20 0x7fbb7c14f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:45:31.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.705+0000 7fbb820c9640 1 --2- 172.21.15.154:0/608840616 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbb7c15c550 0x7fbb7c14f540 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbb74019550 tx=0x7fbb740196e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:31.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.705+0000 7fbb727fc640 1 -- 172.21.15.154:0/608840616 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb740aede0 con 0x7fbb7c15c550 2023-10-18T10:45:31.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.705+0000 7fbb727fc640 1 -- 172.21.15.154:0/608840616 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb74009450 con 0x7fbb7c15c550 2023-10-18T10:45:31.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.705+0000 7fbb727fc640 1 -- 172.21.15.154:0/608840616 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb740b7650 con 0x7fbb7c15c550 2023-10-18T10:45:31.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.705+0000 7fbb83b2c640 1 -- 172.21.15.154:0/608840616 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb7c14e800 con 0x7fbb7c15c550 2023-10-18T10:45:31.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.705+0000 7fbb83b2c640 1 -- 172.21.15.154:0/608840616 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb7c150c60 con 0x7fbb7c15c550 2023-10-18T10:45:31.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.705+0000 7fbb727fc640 1 -- 172.21.15.154:0/608840616 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbb740b4050 con 0x7fbb7c15c550 2023-10-18T10:45:31.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.705+0000 7fbb727fc640 1 -- 172.21.15.154:0/608840616 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fbb740b7d00 con 0x7fbb7c15c550 2023-10-18T10:45:31.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.705+0000 7fbb83b2c640 1 -- 172.21.15.154:0/608840616 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb7c0ce090 con 0x7fbb7c15c550 2023-10-18T10:45:31.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.709+0000 7fbb727fc640 1 -- 172.21.15.154:0/608840616 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fbb740b2070 con 0x7fbb7c15c550 2023-10-18T10:45:31.745 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-18T10:45:31.745+0000 7fda4d0d5180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T10:45:31.763 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-18T10:45:31.765+0000 7f3f05055180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T10:45:31.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.773+0000 7fbb83b2c640 1 -- 172.21.15.154:0/608840616 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2fa04448-d520-4f8c-a9b0-684f526019a5", "id": 4} v 0) v1 -- 0x7fbb7c15f290 con 0x7fbb7c15c550 2023-10-18T10:45:31.782 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.781+0000 7fbb727fc640 1 -- 172.21.15.154:0/608840616 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2fa04448-d520-4f8c-a9b0-684f526019a5", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbb740b77b0 con 0x7fbb7c15c550 2023-10-18T10:45:31.782 INFO:teuthology.orchestra.run.smithi154.stdout:4 2023-10-18T10:45:31.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.785+0000 7fbb83b2c640 1 -- 172.21.15.154:0/608840616 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbb7c15c550 msgr2=0x7fbb7c14f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:31.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.785+0000 7fbb83b2c640 1 --2- 172.21.15.154:0/608840616 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbb7c15c550 0x7fbb7c14f540 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbb74019550 tx=0x7fbb740196e0 comp rx=0 tx=0).stop 2023-10-18T10:45:31.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.785+0000 7fbb83b2c640 1 -- 172.21.15.154:0/608840616 shutdown_connections 2023-10-18T10:45:31.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.785+0000 7fbb83b2c640 1 --2- 172.21.15.154:0/608840616 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbb7c15c550 0x7fbb7c14f540 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:31.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.785+0000 7fbb83b2c640 1 --2- 172.21.15.154:0/608840616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb7c0cfd20 0x7fbb7c14f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:31.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.785+0000 7fbb83b2c640 1 --2- 172.21.15.154:0/608840616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb7c0b1860 0x7fbb7c14b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:31.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.785+0000 7fbb83b2c640 1 -- 172.21.15.154:0/608840616 >> 172.21.15.154:0/608840616 conn(0x7fbb7c069d00 msgr2=0x7fbb7c06a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:31.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.785+0000 7fbb83b2c640 1 -- 172.21.15.154:0/608840616 shutdown_connections 2023-10-18T10:45:31.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.785+0000 7fbb83b2c640 1 -- 172.21.15.154:0/608840616 wait complete. 2023-10-18T10:45:31.801 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 93b639fd-01bb-4821-a426-7460cc1abb66 5 2023-10-18T10:45:31.856 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-18T10:45:31.857+0000 7f3f05055180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T10:45:31.918 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-18T10:45:31.917+0000 7f3f05055180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T10:45:31.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.937+0000 7faafd4ca640 1 Processor -- start 2023-10-18T10:45:31.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.937+0000 7faafd4ca640 1 -- start start 2023-10-18T10:45:31.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.941+0000 7faafd4ca640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faaf815c940 0x7faaf80d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:31.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.941+0000 7faafd4ca640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7faaf80d2f40 0x7faaf80d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:31.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.941+0000 7faafd4ca640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faaf80d0ca0 0x7faaf80cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:31.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.941+0000 7faafd4ca640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaf80d1130 con 0x7faaf80d0ca0 2023-10-18T10:45:31.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.941+0000 7faafd4ca640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaf815da50 con 0x7faaf80d2f40 2023-10-18T10:45:31.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.941+0000 7faafd4ca640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaf815ce40 con 0x7faaf815c940 2023-10-18T10:45:31.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.941+0000 7faaf6ffd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faaf815c940 0x7faaf80d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:31.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.941+0000 7faaf67fc640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7faaf80d2f40 0x7faaf80d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:31.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.941+0000 7faaf6ffd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faaf815c940 0x7faaf80d2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:54698/0 (socket says 172.21.15.154:54698) 2023-10-18T10:45:31.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.941+0000 7faaf6ffd640 1 -- 172.21.15.154:0/1391110399 learned_addr learned my addr 172.21.15.154:0/1391110399 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-18T10:45:31.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.941+0000 7faaf77fe640 1 --2- 172.21.15.154:0/1391110399 >> v2:172.21.15.89:3300/0 conn(0x7faaf80d0ca0 0x7faaf80cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:31.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.941+0000 7faaf6ffd640 1 -- 172.21.15.154:0/1391110399 >> v2:172.21.15.154:3300/0 conn(0x7faaf80d2f40 msgr2=0x7faaf80d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:31.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.941+0000 7faaf6ffd640 1 --2- 172.21.15.154:0/1391110399 >> v2:172.21.15.154:3300/0 conn(0x7faaf80d2f40 0x7faaf80d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:31.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.941+0000 7faaf6ffd640 1 -- 172.21.15.154:0/1391110399 >> v2:172.21.15.89:3300/0 conn(0x7faaf80d0ca0 msgr2=0x7faaf80cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:31.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.941+0000 7faaf6ffd640 1 --2- 172.21.15.154:0/1391110399 >> v2:172.21.15.89:3300/0 conn(0x7faaf80d0ca0 0x7faaf80cf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:31.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.941+0000 7faaf6ffd640 1 -- 172.21.15.154:0/1391110399 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf815d120 con 0x7faaf815c940 2023-10-18T10:45:31.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.941+0000 7faaf67fc640 1 --2- 172.21.15.154:0/1391110399 >> v2:172.21.15.154:3300/0 conn(0x7faaf80d2f40 0x7faaf80d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:45:31.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.945+0000 7faaf6ffd640 1 --2- 172.21.15.154:0/1391110399 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faaf815c940 0x7faaf80d2a00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faaf80b5240 tx=0x7faae801c210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a701153ed431d80f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:31.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.945+0000 7faaf5ffb640 1 -- 172.21.15.154:0/1391110399 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae80ab600 con 0x7faaf815c940 2023-10-18T10:45:31.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.945+0000 7faaf5ffb640 1 -- 172.21.15.154:0/1391110399 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faae80a7050 con 0x7faaf815c940 2023-10-18T10:45:31.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.945+0000 7faaf5ffb640 1 -- 172.21.15.154:0/1391110399 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae80ab980 con 0x7faaf815c940 2023-10-18T10:45:31.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.945+0000 7faafd4ca640 1 -- 172.21.15.154:0/1391110399 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faaf815c940 msgr2=0x7faaf80d2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:31.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.945+0000 7faafd4ca640 1 --2- 172.21.15.154:0/1391110399 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faaf815c940 0x7faaf80d2a00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faaf80b5240 tx=0x7faae801c210 comp rx=0 tx=0).stop 2023-10-18T10:45:31.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.945+0000 7faafd4ca640 1 -- 172.21.15.154:0/1391110399 shutdown_connections 2023-10-18T10:45:31.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.945+0000 7faafd4ca640 1 --2- 172.21.15.154:0/1391110399 >> v2:172.21.15.89:3300/0 conn(0x7faaf80d0ca0 0x7faaf80cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:31.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.945+0000 7faafd4ca640 1 --2- 172.21.15.154:0/1391110399 >> v2:172.21.15.154:3300/0 conn(0x7faaf80d2f40 0x7faaf80d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:31.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.945+0000 7faafd4ca640 1 --2- 172.21.15.154:0/1391110399 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faaf815c940 0x7faaf80d2a00 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:31.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.945+0000 7faafd4ca640 1 -- 172.21.15.154:0/1391110399 >> 172.21.15.154:0/1391110399 conn(0x7faaf8089fd0 msgr2=0x7faaf809b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:31.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.945+0000 7faafd4ca640 1 -- 172.21.15.154:0/1391110399 shutdown_connections 2023-10-18T10:45:31.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.945+0000 7faafd4ca640 1 -- 172.21.15.154:0/1391110399 wait complete. 2023-10-18T10:45:31.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.945+0000 7faafd4ca640 1 Processor -- start 2023-10-18T10:45:31.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.945+0000 7faafd4ca640 1 -- start start 2023-10-18T10:45:31.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.949+0000 7faafd4ca640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faaf80d0ca0 0x7faaf8155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:31.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.949+0000 7faafd4ca640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faaf80b6290 0x7faaf80b6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:31.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.949+0000 7faafd4ca640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faaf80c7300 0x7faaf80c7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:31.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.949+0000 7faafd4ca640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf815bea0 con 0x7faaf80b6290 2023-10-18T10:45:31.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.949+0000 7faafd4ca640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf80b6c50 con 0x7faaf80d0ca0 2023-10-18T10:45:31.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.949+0000 7faafd4ca640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaf8156110 con 0x7faaf80c7300 2023-10-18T10:45:31.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.949+0000 7faaf6ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faaf80d0ca0 0x7faaf8155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:31.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.949+0000 7faaf6ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faaf80d0ca0 0x7faaf8155bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60678/0 (socket says 172.21.15.154:60678) 2023-10-18T10:45:31.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.949+0000 7faaf6ffd640 1 -- 172.21.15.154:0/2617915478 learned_addr learned my addr 172.21.15.154:0/2617915478 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-18T10:45:31.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.949+0000 7faaf77fe640 1 --2- 172.21.15.154:0/2617915478 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faaf80c7300 0x7faaf80c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:31.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.949+0000 7faaf67fc640 1 --2- 172.21.15.154:0/2617915478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faaf80b6290 0x7faaf80b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:31.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.949+0000 7faaf6ffd640 1 -- 172.21.15.154:0/2617915478 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faaf80c7300 msgr2=0x7faaf80c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:31.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.949+0000 7faaf6ffd640 1 --2- 172.21.15.154:0/2617915478 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faaf80c7300 0x7faaf80c7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:31.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.949+0000 7faaf6ffd640 1 -- 172.21.15.154:0/2617915478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faaf80b6290 msgr2=0x7faaf80b6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:31.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.949+0000 7faaf6ffd640 1 --2- 172.21.15.154:0/2617915478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faaf80b6290 0x7faaf80b6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:31.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.949+0000 7faaf6ffd640 1 -- 172.21.15.154:0/2617915478 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaec009420 con 0x7faaf80d0ca0 2023-10-18T10:45:31.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.949+0000 7faaf6ffd640 1 --2- 172.21.15.154:0/2617915478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faaf80d0ca0 0x7faaf8155bd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7faae80ae040 tx=0x7faae801aa50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:31.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.949+0000 7faadf7fe640 1 -- 172.21.15.154:0/2617915478 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae80b6420 con 0x7faaf80d0ca0 2023-10-18T10:45:31.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.949+0000 7faadf7fe640 1 -- 172.21.15.154:0/2617915478 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faae80b65a0 con 0x7faaf80d0ca0 2023-10-18T10:45:31.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.949+0000 7faadf7fe640 1 -- 172.21.15.154:0/2617915478 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae80b6950 con 0x7faaf80d0ca0 2023-10-18T10:45:31.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.949+0000 7faafd4ca640 1 -- 172.21.15.154:0/2617915478 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faae80094d0 con 0x7faaf80d0ca0 2023-10-18T10:45:31.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.949+0000 7faafd4ca640 1 -- 172.21.15.154:0/2617915478 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaf80c7f50 con 0x7faaf80d0ca0 2023-10-18T10:45:31.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.949+0000 7faadf7fe640 1 -- 172.21.15.154:0/2617915478 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7faae80b2040 con 0x7faaf80d0ca0 2023-10-18T10:45:31.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.949+0000 7faadf7fe640 1 -- 172.21.15.154:0/2617915478 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7faae80b6db0 con 0x7faaf80d0ca0 2023-10-18T10:45:31.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.949+0000 7faafd4ca640 1 -- 172.21.15.154:0/2617915478 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaf804dbe0 con 0x7faaf80d0ca0 2023-10-18T10:45:31.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:31.953+0000 7faadf7fe640 1 -- 172.21.15.154:0/2617915478 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7faae80b0070 con 0x7faaf80d0ca0 2023-10-18T10:45:32.013 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-18T10:45:32.013+0000 7f3f05055180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T10:45:32.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.021+0000 7faafd4ca640 1 -- 172.21.15.154:0/2617915478 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd new", "uuid": "93b639fd-01bb-4821-a426-7460cc1abb66", "id": 5} v 0) v1 -- 0x7faaf80b85d0 con 0x7faaf80d0ca0 2023-10-18T10:45:32.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.025+0000 7faadf7fe640 1 -- 172.21.15.154:0/2617915478 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "93b639fd-01bb-4821-a426-7460cc1abb66", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7faae80a4e60 con 0x7faaf80d0ca0 2023-10-18T10:45:32.026 INFO:teuthology.orchestra.run.smithi154.stdout:5 2023-10-18T10:45:32.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.029+0000 7faafd4ca640 1 -- 172.21.15.154:0/2617915478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faaf80d0ca0 msgr2=0x7faaf8155bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:32.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.029+0000 7faafd4ca640 1 --2- 172.21.15.154:0/2617915478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faaf80d0ca0 0x7faaf8155bd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7faae80ae040 tx=0x7faae801aa50 comp rx=0 tx=0).stop 2023-10-18T10:45:32.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.029+0000 7faafd4ca640 1 -- 172.21.15.154:0/2617915478 shutdown_connections 2023-10-18T10:45:32.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.029+0000 7faafd4ca640 1 --2- 172.21.15.154:0/2617915478 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faaf80c7300 0x7faaf80c7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:32.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.029+0000 7faafd4ca640 1 --2- 172.21.15.154:0/2617915478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faaf80b6290 0x7faaf80b6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:32.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.029+0000 7faafd4ca640 1 --2- 172.21.15.154:0/2617915478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faaf80d0ca0 0x7faaf8155bd0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:32.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.029+0000 7faafd4ca640 1 -- 172.21.15.154:0/2617915478 >> 172.21.15.154:0/2617915478 conn(0x7faaf8089fd0 msgr2=0x7faaf815edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:32.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.029+0000 7faafd4ca640 1 -- 172.21.15.154:0/2617915478 shutdown_connections 2023-10-18T10:45:32.029 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.029+0000 7faafd4ca640 1 -- 172.21.15.154:0/2617915478 wait complete. 2023-10-18T10:45:32.045 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 81f3fc7f-239c-47de-a40b-34bf19289823 6 2023-10-18T10:45:32.077 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-18T10:45:32.077+0000 7f3f05055180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T10:45:32.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.181+0000 7ff5d0d8f640 1 Processor -- start 2023-10-18T10:45:32.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.181+0000 7ff5d0d8f640 1 -- start start 2023-10-18T10:45:32.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.185+0000 7ff5d0d8f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5cc0d0760 0x7ff5cc0d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:32.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.185+0000 7ff5d0d8f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff5cc0cd160 0x7ff5cc0cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:32.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.185+0000 7ff5d0d8f640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5cc0d1100 con 0x7ff5cc15c940 2023-10-18T10:45:32.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.185+0000 7ff5d0d8f640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5cc15da50 con 0x7ff5cc0cd160 2023-10-18T10:45:32.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.185+0000 7ff5d0d8f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc0d3110 con 0x7ff5cc0d0760 2023-10-18T10:45:32.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.185+0000 7ff5c3fff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5cc0d0760 0x7ff5cc0d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:32.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.185+0000 7ff5ca59c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff5cc15c940 0x7ff5cc0d2a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:37710/0 (socket says 172.21.15.154:37710) 2023-10-18T10:45:32.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.185+0000 7ff5ca59c640 1 -- 172.21.15.154:0/405434207 learned_addr learned my addr 172.21.15.154:0/405434207 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-10-18T10:45:32.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.185+0000 7ff5cad9d640 1 --2- 172.21.15.154:0/405434207 >> v2:172.21.15.89:3300/0 conn(0x7ff5cc0cd160 0x7ff5cc0cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:32.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.185+0000 7ff5c9d9b640 1 -- 172.21.15.154:0/405434207 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1830162650 0 0) 0x7ff5cc0d1100 con 0x7ff5cc15c940 2023-10-18T10:45:32.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.185+0000 7ff5c9d9b640 1 -- 172.21.15.154:0/405434207 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5b8003530 con 0x7ff5cc15c940 2023-10-18T10:45:32.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.185+0000 7ff5c3fff640 1 -- 172.21.15.154:0/405434207 >> v2:172.21.15.89:3300/0 conn(0x7ff5cc0cd160 msgr2=0x7ff5cc0cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:32.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.185+0000 7ff5c3fff640 1 --2- 172.21.15.154:0/405434207 >> v2:172.21.15.89:3300/0 conn(0x7ff5cc0cd160 0x7ff5cc0cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:32.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.185+0000 7ff5c3fff640 1 -- 172.21.15.154:0/405434207 >> v1:172.21.15.154:6789/0 conn(0x7ff5cc15c940 legacy=0x7ff5cc0d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:32.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.185+0000 7ff5c3fff640 1 -- 172.21.15.154:0/405434207 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5cc15ce40 con 0x7ff5cc0d0760 2023-10-18T10:45:32.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.185+0000 7ff5cad9d640 1 --2- 172.21.15.154:0/405434207 >> v2:172.21.15.89:3300/0 conn(0x7ff5cc0cd160 0x7ff5cc0cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:45:32.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.185+0000 7ff5c3fff640 1 --2- 172.21.15.154:0/405434207 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5cc0d0760 0x7ff5cc0d0bc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40097c0 tx=0x7ff5b401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd6da1749b52fe71 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:32.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.189+0000 7ff5c9d9b640 1 -- 172.21.15.154:0/405434207 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b40ab650 con 0x7ff5cc0d0760 2023-10-18T10:45:32.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.189+0000 7ff5c9d9b640 1 -- 172.21.15.154:0/405434207 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b40ab7b0 con 0x7ff5cc0d0760 2023-10-18T10:45:32.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.189+0000 7ff5c9d9b640 1 -- 172.21.15.154:0/405434207 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b40abb40 con 0x7ff5cc0d0760 2023-10-18T10:45:32.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.189+0000 7ff5d0d8f640 1 -- 172.21.15.154:0/405434207 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5cc0d0760 msgr2=0x7ff5cc0d0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:32.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.189+0000 7ff5d0d8f640 1 --2- 172.21.15.154:0/405434207 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5cc0d0760 0x7ff5cc0d0bc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40097c0 tx=0x7ff5b401c450 comp rx=0 tx=0).stop 2023-10-18T10:45:32.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.189+0000 7ff5d0d8f640 1 -- 172.21.15.154:0/405434207 shutdown_connections 2023-10-18T10:45:32.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.189+0000 7ff5d0d8f640 1 --2- 172.21.15.154:0/405434207 >> v2:172.21.15.89:3300/0 conn(0x7ff5cc0cd160 0x7ff5cc0cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:32.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.189+0000 7ff5d0d8f640 1 --2- 172.21.15.154:0/405434207 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5cc0d0760 0x7ff5cc0d0bc0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:32.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.189+0000 7ff5d0d8f640 1 -- 172.21.15.154:0/405434207 >> 172.21.15.154:0/405434207 conn(0x7ff5cc089fd0 msgr2=0x7ff5cc09b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:32.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.189+0000 7ff5d0d8f640 1 -- 172.21.15.154:0/405434207 shutdown_connections 2023-10-18T10:45:32.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.189+0000 7ff5d0d8f640 1 -- 172.21.15.154:0/405434207 wait complete. 2023-10-18T10:45:32.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.189+0000 7ff5d0d8f640 1 Processor -- start 2023-10-18T10:45:32.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.189+0000 7ff5d0d8f640 1 -- start start 2023-10-18T10:45:32.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.189+0000 7ff5d0d8f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5cc0cd160 0x7ff5cc151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:32.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.189+0000 7ff5d0d8f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5cc0d0760 0x7ff5cc0cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:32.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.193+0000 7ff5d0d8f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5cc1f4cb0 0x7ff5cc0b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:32.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.193+0000 7ff5d0d8f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc1f5260 con 0x7ff5cc1f4cb0 2023-10-18T10:45:32.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.193+0000 7ff5d0d8f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc1f5540 con 0x7ff5cc0d0760 2023-10-18T10:45:32.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.193+0000 7ff5d0d8f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc1f5820 con 0x7ff5cc0cd160 2023-10-18T10:45:32.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.193+0000 7ff5ca59c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5cc0cd160 0x7ff5cc151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:32.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.193+0000 7ff5c3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5cc0d0760 0x7ff5cc0cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:32.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.193+0000 7ff5ca59c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5cc0cd160 0x7ff5cc151b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:54724/0 (socket says 172.21.15.154:54724) 2023-10-18T10:45:32.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.193+0000 7ff5ca59c640 1 -- 172.21.15.154:0/162115394 learned_addr learned my addr 172.21.15.154:0/162115394 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-18T10:45:32.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.193+0000 7ff5cad9d640 1 --2- 172.21.15.154:0/162115394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5cc1f4cb0 0x7ff5cc0b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:32.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.193+0000 7ff5ca59c640 1 -- 172.21.15.154:0/162115394 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5cc0d0760 msgr2=0x7ff5cc0cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:32.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.193+0000 7ff5ca59c640 1 --2- 172.21.15.154:0/162115394 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5cc0d0760 0x7ff5cc0cbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:32.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.193+0000 7ff5ca59c640 1 -- 172.21.15.154:0/162115394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5cc1f4cb0 msgr2=0x7ff5cc0b5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:32.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.193+0000 7ff5ca59c640 1 --2- 172.21.15.154:0/162115394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5cc1f4cb0 0x7ff5cc0b5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:32.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.193+0000 7ff5ca59c640 1 -- 172.21.15.154:0/162115394 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b4009420 con 0x7ff5cc0cd160 2023-10-18T10:45:32.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.193+0000 7ff5cad9d640 1 --2- 172.21.15.154:0/162115394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5cc1f4cb0 0x7ff5cc0b5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:45:32.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.193+0000 7ff5ca59c640 1 --2- 172.21.15.154:0/162115394 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5cc0cd160 0x7ff5cc151b10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00c3c0 tx=0x7ff5bc00c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:32.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.193+0000 7ff5c2ffd640 1 -- 172.21.15.154:0/162115394 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc005cf0 con 0x7ff5cc0cd160 2023-10-18T10:45:32.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.193+0000 7ff5c2ffd640 1 -- 172.21.15.154:0/162115394 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5bc005e50 con 0x7ff5cc0cd160 2023-10-18T10:45:32.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.193+0000 7ff5c2ffd640 1 -- 172.21.15.154:0/162115394 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc0072f0 con 0x7ff5cc0cd160 2023-10-18T10:45:32.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.193+0000 7ff5d0d8f640 1 -- 172.21.15.154:0/162115394 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5cc1f5b00 con 0x7ff5cc0cd160 2023-10-18T10:45:32.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.193+0000 7ff5d0d8f640 1 -- 172.21.15.154:0/162115394 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5cc152050 con 0x7ff5cc0cd160 2023-10-18T10:45:32.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.193+0000 7ff5c2ffd640 1 -- 172.21.15.154:0/162115394 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff5bc01c440 con 0x7ff5cc0cd160 2023-10-18T10:45:32.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.193+0000 7ff5c2ffd640 1 -- 172.21.15.154:0/162115394 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7ff5bc0129f0 con 0x7ff5cc0cd160 2023-10-18T10:45:32.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.193+0000 7ff5d0d8f640 1 -- 172.21.15.154:0/162115394 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5cc0b3990 con 0x7ff5cc0cd160 2023-10-18T10:45:32.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.197+0000 7ff5c2ffd640 1 -- 172.21.15.154:0/162115394 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7ff5bc017070 con 0x7ff5cc0cd160 2023-10-18T10:45:32.241 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-18T10:45:32.241+0000 7f3f05055180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T10:45:32.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.261+0000 7ff5d0d8f640 1 -- 172.21.15.154:0/162115394 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd new", "uuid": "81f3fc7f-239c-47de-a40b-34bf19289823", "id": 6} v 0) v1 -- 0x7ff5cc04dbe0 con 0x7ff5cc0cd160 2023-10-18T10:45:32.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.269+0000 7ff5c2ffd640 1 -- 172.21.15.154:0/162115394 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "81f3fc7f-239c-47de-a40b-34bf19289823", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff5bc0124d0 con 0x7ff5cc0cd160 2023-10-18T10:45:32.267 INFO:teuthology.orchestra.run.smithi154.stdout:6 2023-10-18T10:45:32.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.269+0000 7ff5d0d8f640 1 -- 172.21.15.154:0/162115394 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5cc0cd160 msgr2=0x7ff5cc151b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:32.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.269+0000 7ff5d0d8f640 1 --2- 172.21.15.154:0/162115394 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5cc0cd160 0x7ff5cc151b10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00c3c0 tx=0x7ff5bc00c880 comp rx=0 tx=0).stop 2023-10-18T10:45:32.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.269+0000 7ff5d0d8f640 1 -- 172.21.15.154:0/162115394 shutdown_connections 2023-10-18T10:45:32.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.269+0000 7ff5d0d8f640 1 --2- 172.21.15.154:0/162115394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5cc1f4cb0 0x7ff5cc0b5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:32.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.269+0000 7ff5d0d8f640 1 --2- 172.21.15.154:0/162115394 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5cc0d0760 0x7ff5cc0cbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:32.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.269+0000 7ff5d0d8f640 1 --2- 172.21.15.154:0/162115394 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5cc0cd160 0x7ff5cc151b10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:32.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.269+0000 7ff5d0d8f640 1 -- 172.21.15.154:0/162115394 >> 172.21.15.154:0/162115394 conn(0x7ff5cc089fd0 msgr2=0x7ff5cc0d0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:32.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.269+0000 7ff5d0d8f640 1 -- 172.21.15.154:0/162115394 shutdown_connections 2023-10-18T10:45:32.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.269+0000 7ff5d0d8f640 1 -- 172.21.15.154:0/162115394 wait complete. 2023-10-18T10:45:32.287 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new ceb2d886-a2ee-4974-9079-948986ac3778 7 2023-10-18T10:45:32.313 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-18T10:45:32.313+0000 7f3f05055180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T10:45:32.375 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-18T10:45:32.372+0000 7fda4d0d5180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T10:45:32.437 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-18T10:45:32.436+0000 7fda4d0d5180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T10:45:32.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.449+0000 7f3c78688640 1 Processor -- start 2023-10-18T10:45:32.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.449+0000 7f3c78688640 1 -- start start 2023-10-18T10:45:32.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.453+0000 7f3c78688640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3c700b1860 0x7f3c700cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:32.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.453+0000 7f3c78688640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c700d1e50 con 0x7f3c7015c550 2023-10-18T10:45:32.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.453+0000 7f3c78688640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c700d2130 con 0x7f3c700b1cc0 2023-10-18T10:45:32.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.453+0000 7f3c78688640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c700d2410 con 0x7f3c700b1860 2023-10-18T10:45:32.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.453+0000 7f3c75c23640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3c700b1860 0x7f3c700cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:32.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.453+0000 7f3c75c23640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3c700b1860 0x7f3c700cf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60696/0 (socket says 172.21.15.154:60696) 2023-10-18T10:45:32.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.453+0000 7f3c75c23640 1 -- 172.21.15.154:0/3274590945 learned_addr learned my addr 172.21.15.154:0/3274590945 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-18T10:45:32.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.453+0000 7f3c76c25640 1 --1- 172.21.15.154:0/3274590945 >> v1:172.21.15.154:6789/0 conn(0x7f3c700b1cc0 0x7f3c700b55f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:37720/0 (socket says 172.21.15.154:37720) 2023-10-18T10:45:32.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.453+0000 7f3c75c23640 1 -- 172.21.15.154:0/3274590945 >> v1:172.21.15.154:6789/0 conn(0x7f3c700b1cc0 legacy=0x7f3c700b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:32.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.453+0000 7f3c75c23640 1 -- 172.21.15.154:0/3274590945 >> v1:172.21.15.89:6789/0 conn(0x7f3c7015c550 legacy=0x7f3c700d2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:32.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.453+0000 7f3c75c23640 1 -- 172.21.15.154:0/3274590945 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c700d26f0 con 0x7f3c700b1860 2023-10-18T10:45:32.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.453+0000 7f3c75422640 1 -- 172.21.15.154:0/3274590945 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 737373026 0 0) 0x7f3c700d2130 con 0x7f3c700b1cc0 2023-10-18T10:45:32.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.457+0000 7f3c75c23640 1 --2- 172.21.15.154:0/3274590945 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c700b1860 0x7f3c700cf610 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3c60009870 tx=0x7f3c6001c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6bc1824b2bd0464 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:32.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.457+0000 7f3c75422640 1 -- 172.21.15.154:0/3274590945 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c600ab620 con 0x7f3c700b1860 2023-10-18T10:45:32.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.457+0000 7f3c75422640 1 -- 172.21.15.154:0/3274590945 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c600ab780 con 0x7f3c700b1860 2023-10-18T10:45:32.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.457+0000 7f3c75422640 1 -- 172.21.15.154:0/3274590945 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c600abb10 con 0x7f3c700b1860 2023-10-18T10:45:32.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.457+0000 7f3c78688640 1 -- 172.21.15.154:0/3274590945 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c700b1860 msgr2=0x7f3c700cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:32.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.457+0000 7f3c78688640 1 --2- 172.21.15.154:0/3274590945 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c700b1860 0x7f3c700cf610 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3c60009870 tx=0x7f3c6001c510 comp rx=0 tx=0).stop 2023-10-18T10:45:32.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.457+0000 7f3c78688640 1 -- 172.21.15.154:0/3274590945 shutdown_connections 2023-10-18T10:45:32.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.457+0000 7f3c78688640 1 --2- 172.21.15.154:0/3274590945 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c700b1860 0x7f3c700cf610 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:32.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.457+0000 7f3c78688640 1 -- 172.21.15.154:0/3274590945 >> 172.21.15.154:0/3274590945 conn(0x7f3c70069d00 msgr2=0x7f3c7006a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:32.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.457+0000 7f3c78688640 1 -- 172.21.15.154:0/3274590945 shutdown_connections 2023-10-18T10:45:32.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.457+0000 7f3c78688640 1 -- 172.21.15.154:0/3274590945 wait complete. 2023-10-18T10:45:32.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.457+0000 7f3c78688640 1 Processor -- start 2023-10-18T10:45:32.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.457+0000 7f3c78688640 1 -- start start 2023-10-18T10:45:32.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.461+0000 7f3c78688640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c700b1860 0x7f3c70156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:32.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.461+0000 7f3c78688640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c700b1cc0 0x7f3c700cc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:32.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.461+0000 7f3c78688640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c7015c550 0x7f3c70146200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:32.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.461+0000 7f3c78688640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c701570c0 con 0x7f3c700b1cc0 2023-10-18T10:45:32.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.461+0000 7f3c78688640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c700cb0e0 con 0x7f3c7015c550 2023-10-18T10:45:32.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.461+0000 7f3c78688640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7006bf10 con 0x7f3c700b1860 2023-10-18T10:45:32.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.461+0000 7f3c76424640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c700b1860 0x7f3c70156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:32.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.461+0000 7f3c76c25640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c7015c550 0x7f3c70146200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:32.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.461+0000 7f3c76424640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c700b1860 0x7f3c70156b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:54740/0 (socket says 172.21.15.154:54740) 2023-10-18T10:45:32.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.461+0000 7f3c76424640 1 -- 172.21.15.154:0/1689741851 learned_addr learned my addr 172.21.15.154:0/1689741851 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-18T10:45:32.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.461+0000 7f3c75c23640 1 --2- 172.21.15.154:0/1689741851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c700b1cc0 0x7f3c700cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:32.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.461+0000 7f3c76424640 1 -- 172.21.15.154:0/1689741851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c7015c550 msgr2=0x7f3c70146200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:32.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.461+0000 7f3c76424640 1 --2- 172.21.15.154:0/1689741851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c7015c550 0x7f3c70146200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:32.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.461+0000 7f3c76424640 1 -- 172.21.15.154:0/1689741851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c700b1cc0 msgr2=0x7f3c700cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:32.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.461+0000 7f3c76424640 1 --2- 172.21.15.154:0/1689741851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c700b1cc0 0x7f3c700cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:32.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.461+0000 7f3c76424640 1 -- 172.21.15.154:0/1689741851 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c600094d0 con 0x7f3c700b1860 2023-10-18T10:45:32.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.461+0000 7f3c76c25640 1 --2- 172.21.15.154:0/1689741851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c7015c550 0x7f3c70146200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:45:32.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.461+0000 7f3c76424640 1 --2- 172.21.15.154:0/1689741851 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c700b1860 0x7f3c70156b80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c00c3c0 tx=0x7f3c6c00c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:32.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.461+0000 7f3c66ffd640 1 -- 172.21.15.154:0/1689741851 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c0052d0 con 0x7f3c700b1860 2023-10-18T10:45:32.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.461+0000 7f3c66ffd640 1 -- 172.21.15.154:0/1689741851 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c6c005430 con 0x7f3c700b1860 2023-10-18T10:45:32.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.461+0000 7f3c66ffd640 1 -- 172.21.15.154:0/1689741851 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c003840 con 0x7f3c700b1860 2023-10-18T10:45:32.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.461+0000 7f3c78688640 1 -- 172.21.15.154:0/1689741851 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c7006d500 con 0x7f3c700b1860 2023-10-18T10:45:32.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.461+0000 7f3c78688640 1 -- 172.21.15.154:0/1689741851 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c700ca7b0 con 0x7f3c700b1860 2023-10-18T10:45:32.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.461+0000 7f3c66ffd640 1 -- 172.21.15.154:0/1689741851 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3c6c003ca0 con 0x7f3c700b1860 2023-10-18T10:45:32.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.461+0000 7f3c66ffd640 1 -- 172.21.15.154:0/1689741851 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f3c6c011ea0 con 0x7f3c700b1860 2023-10-18T10:45:32.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.461+0000 7f3c78688640 1 -- 172.21.15.154:0/1689741851 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c700b1640 con 0x7f3c700b1860 2023-10-18T10:45:32.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.465+0000 7f3c66ffd640 1 -- 172.21.15.154:0/1689741851 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f3c6c017070 con 0x7f3c700b1860 2023-10-18T10:45:32.497 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-18T10:45:32.496+0000 7fda4d0d5180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T10:45:32.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.533+0000 7f3c78688640 1 -- 172.21.15.154:0/1689741851 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ceb2d886-a2ee-4974-9079-948986ac3778", "id": 7} v 0) v1 -- 0x7f3c7015f270 con 0x7f3c700b1860 2023-10-18T10:45:32.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.541+0000 7f3c66ffd640 1 -- 172.21.15.154:0/1689741851 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ceb2d886-a2ee-4974-9079-948986ac3778", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3c6c011590 con 0x7f3c700b1860 2023-10-18T10:45:32.539 INFO:teuthology.orchestra.run.smithi154.stdout:7 2023-10-18T10:45:32.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.541+0000 7f3c78688640 1 -- 172.21.15.154:0/1689741851 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c700b1860 msgr2=0x7f3c70156b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:32.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.541+0000 7f3c78688640 1 --2- 172.21.15.154:0/1689741851 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c700b1860 0x7f3c70156b80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c00c3c0 tx=0x7f3c6c00c880 comp rx=0 tx=0).stop 2023-10-18T10:45:32.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.541+0000 7f3c78688640 1 -- 172.21.15.154:0/1689741851 shutdown_connections 2023-10-18T10:45:32.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.541+0000 7f3c78688640 1 --2- 172.21.15.154:0/1689741851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c7015c550 0x7f3c70146200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:32.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.541+0000 7f3c78688640 1 --2- 172.21.15.154:0/1689741851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c700b1cc0 0x7f3c700cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:32.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.541+0000 7f3c78688640 1 --2- 172.21.15.154:0/1689741851 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c700b1860 0x7f3c70156b80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:32.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.541+0000 7f3c78688640 1 -- 172.21.15.154:0/1689741851 >> 172.21.15.154:0/1689741851 conn(0x7f3c70069d00 msgr2=0x7f3c700d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:32.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.541+0000 7f3c78688640 1 -- 172.21.15.154:0/1689741851 shutdown_connections 2023-10-18T10:45:32.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-18T10:45:32.541+0000 7f3c78688640 1 -- 172.21.15.154:0/1689741851 wait complete. 2023-10-18T10:45:32.557 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-18T10:45:32.558 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-18T10:45:32.562 INFO:tasks.ceph.osd.0:Started 2023-10-18T10:45:32.562 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-18T10:45:32.562 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-18T10:45:32.566 INFO:tasks.ceph.osd.1:Started 2023-10-18T10:45:32.566 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-18T10:45:32.566 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-18T10:45:32.570 INFO:tasks.ceph.osd.2:Started 2023-10-18T10:45:32.570 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-18T10:45:32.571 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-18T10:45:32.576 INFO:tasks.ceph.osd.3:Started 2023-10-18T10:45:32.576 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-18T10:45:32.577 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-18T10:45:32.580 INFO:tasks.ceph.osd.4:Started 2023-10-18T10:45:32.581 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-18T10:45:32.581 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-18T10:45:32.585 INFO:tasks.ceph.osd.5:Started 2023-10-18T10:45:32.585 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-18T10:45:32.585 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-18T10:45:32.589 INFO:tasks.ceph.osd.6:Started 2023-10-18T10:45:32.589 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-18T10:45:32.590 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-18T10:45:32.590 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-18T10:45:32.588+0000 7fda4d0d5180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T10:45:32.593 INFO:tasks.ceph.osd.7:Started 2023-10-18T10:45:32.594 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:45:32.676 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-18T10:45:32.672+0000 7fda4d0d5180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T10:45:32.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.764+0000 7fd3c52e6640 1 Processor -- start 2023-10-18T10:45:32.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.764+0000 7fd3c52e6640 1 -- start start 2023-10-18T10:45:32.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.764+0000 7fd3c52e6640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3c015c940 0x7fd3c00d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:32.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.764+0000 7fd3c52e6640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd3c00d0e70 0x7fd3c00cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.764+0000 7fd3c52e6640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3c015da50 con 0x7fd3c00d2f40 2023-10-18T10:45:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.764+0000 7fd3c52e6640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3c015ce40 con 0x7fd3c00d0e70 2023-10-18T10:45:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.764+0000 7fd3c52e6640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c015d120 con 0x7fd3c015c940 2023-10-18T10:45:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.764+0000 7fd3bf7fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd3c00d0e70 0x7fd3c00cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:32.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.764+0000 7fd3bf7fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd3c00d0e70 0x7fd3c00cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46620/0 (socket says 172.21.15.89:46620) 2023-10-18T10:45:32.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.764+0000 7fd3bf7fe640 1 -- 172.21.15.89:0/121383545 learned_addr learned my addr 172.21.15.89:0/121383545 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:45:32.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.764+0000 7fd3be7fc640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd3c00d2f40 0x7fd3c00d0760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48182/0 (socket says 172.21.15.89:48182) 2023-10-18T10:45:32.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.764+0000 7fd3beffd640 1 --2- 172.21.15.89:0/121383545 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3c015c940 0x7fd3c00d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:32.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.764+0000 7fd3bf7fe640 1 -- 172.21.15.89:0/121383545 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3c015c940 msgr2=0x7fd3c00d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:32.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.764+0000 7fd3bf7fe640 1 --2- 172.21.15.89:0/121383545 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3c015c940 0x7fd3c00d2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:32.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.764+0000 7fd3bf7fe640 1 -- 172.21.15.89:0/121383545 >> v1:172.21.15.89:6789/0 conn(0x7fd3c00d2f40 legacy=0x7fd3c00d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:32.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.764+0000 7fd3bf7fe640 1 -- 172.21.15.89:0/121383545 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c015d400 con 0x7fd3c00d0e70 2023-10-18T10:45:32.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.768+0000 7fd3bf7fe640 1 --2- 172.21.15.89:0/121383545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3c00d0e70 0x7fd3c00cd160 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8009e80 tx=0x7fd3b801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=933dfee1495b7443 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:32.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.768+0000 7fd3bdffb640 1 -- 172.21.15.89:0/121383545 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b80a5820 con 0x7fd3c00d0e70 2023-10-18T10:45:32.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.768+0000 7fd3bdffb640 1 -- 172.21.15.89:0/121383545 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b80a5980 con 0x7fd3c00d0e70 2023-10-18T10:45:32.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.768+0000 7fd3bdffb640 1 -- 172.21.15.89:0/121383545 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b80a5d10 con 0x7fd3c00d0e70 2023-10-18T10:45:32.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.768+0000 7fd3c52e6640 1 -- 172.21.15.89:0/121383545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3c00d0e70 msgr2=0x7fd3c00cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:32.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.768+0000 7fd3c52e6640 1 --2- 172.21.15.89:0/121383545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3c00d0e70 0x7fd3c00cd160 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8009e80 tx=0x7fd3b801c9c0 comp rx=0 tx=0).stop 2023-10-18T10:45:32.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.768+0000 7fd3c52e6640 1 -- 172.21.15.89:0/121383545 shutdown_connections 2023-10-18T10:45:32.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.768+0000 7fd3c52e6640 1 --2- 172.21.15.89:0/121383545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3c00d0e70 0x7fd3c00cd160 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:32.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.768+0000 7fd3c52e6640 1 --2- 172.21.15.89:0/121383545 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3c015c940 0x7fd3c00d2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:32.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.768+0000 7fd3c52e6640 1 -- 172.21.15.89:0/121383545 >> 172.21.15.89:0/121383545 conn(0x7fd3c0089fd0 msgr2=0x7fd3c009b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:32.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.768+0000 7fd3c52e6640 1 -- 172.21.15.89:0/121383545 shutdown_connections 2023-10-18T10:45:32.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.768+0000 7fd3c52e6640 1 -- 172.21.15.89:0/121383545 wait complete. 2023-10-18T10:45:32.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.768+0000 7fd3c52e6640 1 Processor -- start 2023-10-18T10:45:32.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.772+0000 7fd3c52e6640 1 -- start start 2023-10-18T10:45:32.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.772+0000 7fd3c52e6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3c00d0e70 0x7fd3c00b7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:32.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.772+0000 7fd3c52e6640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3c01423a0 0x7fd3c0141900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:32.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.772+0000 7fd3c52e6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3c00cbfa0 0x7fd3c0159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:32.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.772+0000 7fd3c52e6640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c00b7890 con 0x7fd3c00cbfa0 2023-10-18T10:45:32.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.772+0000 7fd3c52e6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c00b7590 con 0x7fd3c00d0e70 2023-10-18T10:45:32.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.772+0000 7fd3c52e6640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c0154d20 con 0x7fd3c01423a0 2023-10-18T10:45:32.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.772+0000 7fd3bf7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3c00cbfa0 0x7fd3c0159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:32.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.772+0000 7fd3beffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3c00d0e70 0x7fd3c00b7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:32.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.772+0000 7fd3bf7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3c00cbfa0 0x7fd3c0159f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46634/0 (socket says 172.21.15.89:46634) 2023-10-18T10:45:32.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.772+0000 7fd3bf7fe640 1 -- 172.21.15.89:0/1824837238 learned_addr learned my addr 172.21.15.89:0/1824837238 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:45:32.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.772+0000 7fd3be7fc640 1 --2- 172.21.15.89:0/1824837238 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3c01423a0 0x7fd3c0141900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:32.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.772+0000 7fd3bf7fe640 1 -- 172.21.15.89:0/1824837238 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3c01423a0 msgr2=0x7fd3c0141900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:32.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.772+0000 7fd3bf7fe640 1 --2- 172.21.15.89:0/1824837238 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3c01423a0 0x7fd3c0141900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:32.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.772+0000 7fd3bf7fe640 1 -- 172.21.15.89:0/1824837238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3c00d0e70 msgr2=0x7fd3c00b7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:32.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.772+0000 7fd3bf7fe640 1 --2- 172.21.15.89:0/1824837238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3c00d0e70 0x7fd3c00b7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:32.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.772+0000 7fd3bf7fe640 1 -- 172.21.15.89:0/1824837238 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b8009b30 con 0x7fd3c00cbfa0 2023-10-18T10:45:32.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.772+0000 7fd3beffd640 1 --2- 172.21.15.89:0/1824837238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3c00d0e70 0x7fd3c00b7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:45:32.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.772+0000 7fd3bf7fe640 1 --2- 172.21.15.89:0/1824837238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3c00cbfa0 0x7fd3c0159f20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd3b80af040 tx=0x7fd3b8019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:32.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.772+0000 7fd3a77fe640 1 -- 172.21.15.89:0/1824837238 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b80b8670 con 0x7fd3c00cbfa0 2023-10-18T10:45:32.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.772+0000 7fd3a77fe640 1 -- 172.21.15.89:0/1824837238 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b80b87d0 con 0x7fd3c00cbfa0 2023-10-18T10:45:32.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.772+0000 7fd3a77fe640 1 -- 172.21.15.89:0/1824837238 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b80b8b80 con 0x7fd3c00cbfa0 2023-10-18T10:45:32.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.772+0000 7fd3c52e6640 1 -- 172.21.15.89:0/1824837238 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3c0155000 con 0x7fd3c00cbfa0 2023-10-18T10:45:32.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.772+0000 7fd3c52e6640 1 -- 172.21.15.89:0/1824837238 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c00b8260 con 0x7fd3c00cbfa0 2023-10-18T10:45:32.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.772+0000 7fd3a77fe640 1 -- 172.21.15.89:0/1824837238 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd3b80c3460 con 0x7fd3c00cbfa0 2023-10-18T10:45:32.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.772+0000 7fd3a77fe640 1 -- 172.21.15.89:0/1824837238 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fd3b80b6b10 con 0x7fd3c00cbfa0 2023-10-18T10:45:32.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.772+0000 7fd3c52e6640 1 -- 172.21.15.89:0/1824837238 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3c00b7a40 con 0x7fd3c00cbfa0 2023-10-18T10:45:32.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.776+0000 7fd3a77fe640 1 -- 172.21.15.89:0/1824837238 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fd3c00b7a40 con 0x7fd3c00cbfa0 2023-10-18T10:45:32.782 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-18T10:45:32.780+0000 7fda4d0d5180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T10:45:32.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.832+0000 7fd3c52e6640 1 -- 172.21.15.89:0/1824837238 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3c004dbe0 con 0x7fd3c00cbfa0 2023-10-18T10:45:32.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.832+0000 7fd3a77fe640 1 -- 172.21.15.89:0/1824837238 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fd3b80b6e50 con 0x7fd3c00cbfa0 2023-10-18T10:45:32.835 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:45:32.835 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":10,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:45:32.538820+0000","last_up_change":"0.000000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:45:32.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.832+0000 7fd3c52e6640 1 -- 172.21.15.89:0/1824837238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3c00cbfa0 msgr2=0x7fd3c0159f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:32.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.832+0000 7fd3c52e6640 1 --2- 172.21.15.89:0/1824837238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3c00cbfa0 0x7fd3c0159f20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd3b80af040 tx=0x7fd3b8019760 comp rx=0 tx=0).stop 2023-10-18T10:45:32.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.832+0000 7fd3c52e6640 1 -- 172.21.15.89:0/1824837238 shutdown_connections 2023-10-18T10:45:32.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.832+0000 7fd3c52e6640 1 --2- 172.21.15.89:0/1824837238 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3c01423a0 0x7fd3c0141900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:32.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.832+0000 7fd3c52e6640 1 --2- 172.21.15.89:0/1824837238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3c00cbfa0 0x7fd3c0159f20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:32.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.832+0000 7fd3c52e6640 1 --2- 172.21.15.89:0/1824837238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3c00d0e70 0x7fd3c00b7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:32.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.832+0000 7fd3c52e6640 1 -- 172.21.15.89:0/1824837238 >> 172.21.15.89:0/1824837238 conn(0x7fd3c0089fd0 msgr2=0x7fd3c0158df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:32.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.832+0000 7fd3c52e6640 1 -- 172.21.15.89:0/1824837238 shutdown_connections 2023-10-18T10:45:32.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:32.832+0000 7fd3c52e6640 1 -- 172.21.15.89:0/1824837238 wait complete. 2023-10-18T10:45:32.844 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-18T10:45:32.844+0000 7fda4d0d5180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T10:45:32.855 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-18T10:45:32.856 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-18T10:45:33.003 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-18T10:45:33.000+0000 7fda4d0d5180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T10:45:33.032 INFO:tasks.ceph.mgr.x.smithi154.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-10-18T10:45:33.032 INFO:tasks.ceph.mgr.x.smithi154.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-10-18T10:45:33.033 INFO:tasks.ceph.mgr.x.smithi154.stderr: from numpy import show_config as show_numpy_config 2023-10-18T10:45:33.072 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-18T10:45:33.068+0000 7fda4d0d5180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T10:45:33.080 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-18T10:45:33.081+0000 7f3f05055180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T10:45:33.213 INFO:tasks.ceph.osd.0.smithi089.stderr:2023-10-18T10:45:33.212+0000 7f5750cae640 -1 Falling back to public interface 2023-10-18T10:45:33.214 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-18T10:45:33.213+0000 7f3f05055180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:45:33.245 INFO:tasks.ceph.osd.2.smithi089.stderr:2023-10-18T10:45:33.244+0000 7f687dd59640 -1 Falling back to public interface 2023-10-18T10:45:33.257 INFO:tasks.ceph.osd.1.smithi089.stderr:2023-10-18T10:45:33.256+0000 7f4d18984640 -1 Falling back to public interface 2023-10-18T10:45:33.261 INFO:tasks.ceph.osd.3.smithi089.stderr:2023-10-18T10:45:33.260+0000 7fafe2719640 -1 Falling back to public interface 2023-10-18T10:45:33.291 INFO:tasks.ceph.osd.4.smithi154.stderr:2023-10-18T10:45:33.293+0000 7f9955107640 -1 Falling back to public interface 2023-10-18T10:45:33.320 INFO:tasks.ceph.osd.5.smithi154.stderr:2023-10-18T10:45:33.321+0000 7f593da29640 -1 Falling back to public interface 2023-10-18T10:45:33.323 INFO:tasks.ceph.osd.7.smithi154.stderr:2023-10-18T10:45:33.325+0000 7fa1bbc57640 -1 Falling back to public interface 2023-10-18T10:45:33.324 INFO:tasks.ceph.osd.7.smithi154.stderr:2023-10-18T10:45:33.325+0000 7fa1bac18640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6814/0: (98) Address already in use 2023-10-18T10:45:33.327 INFO:tasks.ceph.osd.6.smithi154.stderr:2023-10-18T10:45:33.329+0000 7f0c89961640 -1 Falling back to public interface 2023-10-18T10:45:33.389 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-18T10:45:33.389+0000 7f3f05055180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T10:45:33.461 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-18T10:45:33.461+0000 7f3f05055180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T10:45:33.597 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-18T10:45:33.597+0000 7f3f05055180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:45:33.660 DEBUG:teuthology.orchestra.run.smithi089:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-18T10:45:33.707 INFO:tasks.ceph.mgr.y.smithi089.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-10-18T10:45:33.708 INFO:tasks.ceph.mgr.y.smithi089.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-10-18T10:45:33.708 INFO:tasks.ceph.mgr.y.smithi089.stderr: from numpy import show_config as show_numpy_config 2023-10-18T10:45:33.759 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-18T10:45:33.756+0000 7fda4d0d5180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T10:45:33.788 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.788+0000 7f81d1140640 1 Processor -- start 2023-10-18T10:45:33.788 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.788+0000 7f81d1140640 1 -- start start 2023-10-18T10:45:33.790 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.788+0000 7f81d1140640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f81cc0cd880 0x7f81cc0d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:33.790 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.788+0000 7f81d1140640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f81cc0d2520 0x7f81cc0d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:33.790 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.788+0000 7f81d1140640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81cc0b1de0 con 0x7f81cc15cb40 2023-10-18T10:45:33.790 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.788+0000 7f81d1140640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81cc0d2f10 con 0x7f81cc0d2520 2023-10-18T10:45:33.791 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.788+0000 7f81d1140640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81cc15d0a0 con 0x7f81cc0cd880 2023-10-18T10:45:33.791 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.788+0000 7f81c3fff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f81cc15cb40 0x7f81cc0b16d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48198/0 (socket says 172.21.15.89:48198) 2023-10-18T10:45:33.791 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.788+0000 7f81c3fff640 1 -- 172.21.15.89:0/4020832385 learned_addr learned my addr 172.21.15.89:0/4020832385 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:45:33.791 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.788+0000 7f81cb59e640 1 --2- 172.21.15.89:0/4020832385 >> v2:172.21.15.89:3300/0 conn(0x7f81cc0d2520 0x7f81cc0d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:33.792 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.788+0000 7f81cad9d640 1 --2- 172.21.15.89:0/4020832385 >> v2:172.21.15.154:3300/0 conn(0x7f81cc0cd880 0x7f81cc0d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:33.792 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.788+0000 7f81ca59c640 1 -- 172.21.15.89:0/4020832385 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1040441038 0 0) 0x7f81cc0b1de0 con 0x7f81cc15cb40 2023-10-18T10:45:33.792 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.788+0000 7f81ca59c640 1 -- 172.21.15.89:0/4020832385 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81b8003530 con 0x7f81cc15cb40 2023-10-18T10:45:33.792 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.788+0000 7f81cb59e640 1 -- 172.21.15.89:0/4020832385 >> v2:172.21.15.154:3300/0 conn(0x7f81cc0cd880 msgr2=0x7f81cc0d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:33.792 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.788+0000 7f81cb59e640 1 --2- 172.21.15.89:0/4020832385 >> v2:172.21.15.154:3300/0 conn(0x7f81cc0cd880 0x7f81cc0d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:33.793 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.788+0000 7f81cb59e640 1 -- 172.21.15.89:0/4020832385 >> v1:172.21.15.89:6789/0 conn(0x7f81cc15cb40 legacy=0x7f81cc0b16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:33.793 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.788+0000 7f81cb59e640 1 -- 172.21.15.89:0/4020832385 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81cc15d380 con 0x7f81cc0d2520 2023-10-18T10:45:33.795 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.792+0000 7f81cb59e640 1 --2- 172.21.15.89:0/4020832385 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81cc0d2520 0x7f81cc0d29d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f81cc0ccb90 tx=0x7f81c4025120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb71333d1c2ecf4 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:33.796 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.792+0000 7f81ca59c640 1 -- 172.21.15.89:0/4020832385 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c40b4600 con 0x7f81cc0d2520 2023-10-18T10:45:33.796 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.792+0000 7f81ca59c640 1 -- 172.21.15.89:0/4020832385 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81c40b0050 con 0x7f81cc0d2520 2023-10-18T10:45:33.796 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.792+0000 7f81ca59c640 1 -- 172.21.15.89:0/4020832385 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c40b4980 con 0x7f81cc0d2520 2023-10-18T10:45:33.797 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.796+0000 7f81d1140640 1 -- 172.21.15.89:0/4020832385 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81cc0d2520 msgr2=0x7f81cc0d29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:33.797 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.796+0000 7f81d1140640 1 --2- 172.21.15.89:0/4020832385 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81cc0d2520 0x7f81cc0d29d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f81cc0ccb90 tx=0x7f81c4025120 comp rx=0 tx=0).stop 2023-10-18T10:45:33.797 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.796+0000 7f81d1140640 1 -- 172.21.15.89:0/4020832385 shutdown_connections 2023-10-18T10:45:33.798 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.796+0000 7f81d1140640 1 --2- 172.21.15.89:0/4020832385 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81cc0d2520 0x7f81cc0d29d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:33.798 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.796+0000 7f81d1140640 1 --2- 172.21.15.89:0/4020832385 >> v2:172.21.15.154:3300/0 conn(0x7f81cc0cd880 0x7f81cc0d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:33.798 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.796+0000 7f81d1140640 1 -- 172.21.15.89:0/4020832385 >> 172.21.15.89:0/4020832385 conn(0x7f81cc071430 msgr2=0x7f81cc071840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:33.798 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.796+0000 7f81d1140640 1 -- 172.21.15.89:0/4020832385 shutdown_connections 2023-10-18T10:45:33.798 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.796+0000 7f81d1140640 1 -- 172.21.15.89:0/4020832385 wait complete. 2023-10-18T10:45:33.800 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.796+0000 7f81d1140640 1 Processor -- start 2023-10-18T10:45:33.800 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.796+0000 7f81d1140640 1 -- start start 2023-10-18T10:45:33.801 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.796+0000 7f81d1140640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81cc0cd880 0x7f81cc155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:33.801 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.796+0000 7f81d1140640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81cc0d2520 0x7f81cc0c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:33.801 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.796+0000 7f81d1140640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81cc15cb40 0x7f81cc15b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:33.801 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.796+0000 7f81d1140640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc0c7f80 con 0x7f81cc15cb40 2023-10-18T10:45:33.801 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.796+0000 7f81d1140640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc0c60c0 con 0x7f81cc0d2520 2023-10-18T10:45:33.802 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.796+0000 7f81d1140640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc0c63a0 con 0x7f81cc0cd880 2023-10-18T10:45:33.802 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.796+0000 7f81cb59e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81cc15cb40 0x7f81cc15b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:33.802 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.796+0000 7f81cb59e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81cc15cb40 0x7f81cc15b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46656/0 (socket says 172.21.15.89:46656) 2023-10-18T10:45:33.802 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.796+0000 7f81cb59e640 1 -- 172.21.15.89:0/3683592255 learned_addr learned my addr 172.21.15.89:0/3683592255 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:45:33.802 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.796+0000 7f81cad9d640 1 --2- 172.21.15.89:0/3683592255 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81cc0cd880 0x7f81cc155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:33.803 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.796+0000 7f81c3fff640 1 --2- 172.21.15.89:0/3683592255 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81cc0d2520 0x7f81cc0c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:33.803 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.796+0000 7f81cb59e640 1 -- 172.21.15.89:0/3683592255 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81cc0cd880 msgr2=0x7f81cc155c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:33.803 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.796+0000 7f81cb59e640 1 --2- 172.21.15.89:0/3683592255 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81cc0cd880 0x7f81cc155c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:33.803 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.796+0000 7f81cb59e640 1 -- 172.21.15.89:0/3683592255 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81cc0d2520 msgr2=0x7f81cc0c75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:33.803 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.796+0000 7f81cb59e640 1 --2- 172.21.15.89:0/3683592255 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81cc0d2520 0x7f81cc0c75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:33.804 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.800+0000 7f81cb59e640 1 -- 172.21.15.89:0/3683592255 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c4009a70 con 0x7f81cc15cb40 2023-10-18T10:45:33.804 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.800+0000 7f81cad9d640 1 --2- 172.21.15.89:0/3683592255 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81cc0cd880 0x7f81cc155c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:45:33.804 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.800+0000 7f81cb59e640 1 --2- 172.21.15.89:0/3683592255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81cc15cb40 0x7f81cc15b1d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f81c40b7040 tx=0x7f81c4021ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:33.805 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.800+0000 7f81c37fe640 1 -- 172.21.15.89:0/3683592255 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c40b4e90 con 0x7f81cc15cb40 2023-10-18T10:45:33.805 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.800+0000 7f81c37fe640 1 -- 172.21.15.89:0/3683592255 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81c4009450 con 0x7f81cc15cb40 2023-10-18T10:45:33.805 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.800+0000 7f81d1140640 1 -- 172.21.15.89:0/3683592255 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81cc0c6680 con 0x7f81cc15cb40 2023-10-18T10:45:33.806 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.800+0000 7f81c37fe640 1 -- 172.21.15.89:0/3683592255 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c40be650 con 0x7f81cc15cb40 2023-10-18T10:45:33.806 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.800+0000 7f81d1140640 1 -- 172.21.15.89:0/3683592255 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81cc1562f0 con 0x7f81cc15cb40 2023-10-18T10:45:33.806 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.800+0000 7f81c37fe640 1 -- 172.21.15.89:0/3683592255 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f81c40bb050 con 0x7f81cc15cb40 2023-10-18T10:45:33.806 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.800+0000 7f81c37fe640 1 -- 172.21.15.89:0/3683592255 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f81c40046f0 con 0x7f81cc15cb40 2023-10-18T10:45:33.806 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.800+0000 7f81d1140640 1 -- 172.21.15.89:0/3683592255 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81cc073b50 con 0x7f81cc15cb40 2023-10-18T10:45:33.807 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.804+0000 7f81c37fe640 1 -- 172.21.15.89:0/3683592255 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f81c40b9070 con 0x7f81cc15cb40 2023-10-18T10:45:33.860 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.856+0000 7f81d1140640 1 -- 172.21.15.89:0/3683592255 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81cc0b8510 con 0x7f81cc15cb40 2023-10-18T10:45:33.861 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.860+0000 7f81c37fe640 1 -- 172.21.15.89:0/3683592255 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f81c4003ca0 con 0x7f81cc15cb40 2023-10-18T10:45:33.861 INFO:teuthology.misc.health.smithi089.stdout: 2023-10-18T10:45:33.861 INFO:teuthology.misc.health.smithi089.stdout:{"epoch":10,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:45:32.538820+0000","last_up_change":"0.000000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:45:33.862 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.860+0000 7f81d1140640 1 -- 172.21.15.89:0/3683592255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81cc15cb40 msgr2=0x7f81cc15b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:33.863 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.860+0000 7f81d1140640 1 --2- 172.21.15.89:0/3683592255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81cc15cb40 0x7f81cc15b1d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f81c40b7040 tx=0x7f81c4021ec0 comp rx=0 tx=0).stop 2023-10-18T10:45:33.863 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.860+0000 7f81d1140640 1 -- 172.21.15.89:0/3683592255 shutdown_connections 2023-10-18T10:45:33.863 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.860+0000 7f81d1140640 1 --2- 172.21.15.89:0/3683592255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81cc15cb40 0x7f81cc15b1d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:33.863 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.860+0000 7f81d1140640 1 --2- 172.21.15.89:0/3683592255 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81cc0d2520 0x7f81cc0c75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:33.863 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.860+0000 7f81d1140640 1 --2- 172.21.15.89:0/3683592255 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81cc0cd880 0x7f81cc155c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:33.863 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.860+0000 7f81d1140640 1 -- 172.21.15.89:0/3683592255 >> 172.21.15.89:0/3683592255 conn(0x7f81cc071430 msgr2=0x7f81cc15bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:33.864 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.860+0000 7f81d1140640 1 -- 172.21.15.89:0/3683592255 shutdown_connections 2023-10-18T10:45:33.864 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:33.860+0000 7f81d1140640 1 -- 172.21.15.89:0/3683592255 wait complete. 2023-10-18T10:45:33.878 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-18T10:45:33.903 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-18T10:45:33.900+0000 7fda4d0d5180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:45:34.090 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-18T10:45:34.088+0000 7fda4d0d5180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T10:45:34.163 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-18T10:45:34.160+0000 7fda4d0d5180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T10:45:34.278 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-18T10:45:34.277+0000 7f3f05055180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T10:45:34.301 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-18T10:45:34.300+0000 7fda4d0d5180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:45:34.364 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-18T10:45:34.365+0000 7f3f05055180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T10:45:34.553 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-18T10:45:34.553+0000 7f3f05055180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T10:45:34.637 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-18T10:45:34.637+0000 7f3f05055180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T10:45:34.963 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-18T10:45:34.965+0000 7f3f05055180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T10:45:34.983 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-18T10:45:34.980+0000 7fda4d0d5180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T10:45:35.030 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-18T10:45:35.033+0000 7f3f05055180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T10:45:35.072 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-18T10:45:35.072+0000 7fda4d0d5180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T10:45:35.265 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-18T10:45:35.264+0000 7fda4d0d5180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T10:45:35.350 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-18T10:45:35.348+0000 7fda4d0d5180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T10:45:35.676 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-18T10:45:35.676+0000 7fda4d0d5180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T10:45:35.746 INFO:tasks.ceph.mgr.y.smithi089.stderr:2023-10-18T10:45:35.744+0000 7fda4d0d5180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T10:45:36.053 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-18T10:45:36.053+0000 7f3eac8b9640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-18T10:45:36.054 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-18T10:45:36.053+0000 7f3eac8b9640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-18T10:45:37.050 INFO:tasks.ceph.osd.0.smithi089.stderr:2023-10-18T10:45:37.048+0000 7f5750cae640 -1 osd.0 0 log_to_monitors true 2023-10-18T10:45:37.072 INFO:tasks.ceph.osd.2.smithi089.stderr:2023-10-18T10:45:37.068+0000 7f687dd59640 -1 osd.2 0 log_to_monitors true 2023-10-18T10:45:37.079 INFO:tasks.ceph.osd.4.smithi154.stderr:2023-10-18T10:45:37.077+0000 7f9955107640 -1 osd.4 0 log_to_monitors true 2023-10-18T10:45:37.188 INFO:tasks.ceph.osd.3.smithi089.stderr:2023-10-18T10:45:37.188+0000 7fafe2719640 -1 osd.3 0 log_to_monitors true 2023-10-18T10:45:37.258 INFO:tasks.ceph.osd.5.smithi154.stderr:2023-10-18T10:45:37.257+0000 7f593da29640 -1 osd.5 0 log_to_monitors true 2023-10-18T10:45:37.260 INFO:tasks.ceph.osd.6.smithi154.stderr:2023-10-18T10:45:37.261+0000 7f0c89961640 -1 osd.6 0 log_to_monitors true 2023-10-18T10:45:37.279 INFO:tasks.ceph.osd.1.smithi089.stderr:2023-10-18T10:45:37.276+0000 7f4d18984640 -1 osd.1 0 log_to_monitors true 2023-10-18T10:45:37.289 INFO:tasks.ceph.osd.7.smithi154.stderr:2023-10-18T10:45:37.289+0000 7fa1bbc57640 -1 osd.7 0 log_to_monitors true 2023-10-18T10:45:40.684 DEBUG:teuthology.orchestra.run.smithi089:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-18T10:45:40.884 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.884+0000 7f8e8e967640 1 Processor -- start 2023-10-18T10:45:40.884 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.884+0000 7f8e8e967640 1 -- start start 2023-10-18T10:45:40.885 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.884+0000 7f8e8e967640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8e8815cb40 0x7f8e880b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:40.885 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.884+0000 7f8e8e967640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e880b1c10 0x7f8e880d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:40.885 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.884+0000 7f8e8e967640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e8815d0a0 con 0x7f8e880cd880 2023-10-18T10:45:40.885 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.884+0000 7f8e8e967640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e8815d380 con 0x7f8e8815cb40 2023-10-18T10:45:40.885 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.884+0000 7f8e8e967640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8815d660 con 0x7f8e880b1c10 2023-10-18T10:45:40.886 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.884+0000 7f8e877fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8e8815cb40 0x7f8e880b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:40.886 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.884+0000 7f8e877fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8e8815cb40 0x7f8e880b16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:59202/0 (socket says 172.21.15.89:59202) 2023-10-18T10:45:40.886 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.884+0000 7f8e877fe640 1 -- 172.21.15.89:0/3010431716 learned_addr learned my addr 172.21.15.89:0/3010431716 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:45:40.886 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.884+0000 7f8e877fe640 1 -- 172.21.15.89:0/3010431716 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e880b1c10 msgr2=0x7f8e880d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:40.887 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.884+0000 7f8e877fe640 1 --2- 172.21.15.89:0/3010431716 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e880b1c10 0x7f8e880d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:40.887 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.884+0000 7f8e877fe640 1 -- 172.21.15.89:0/3010431716 >> v1:172.21.15.89:6789/0 conn(0x7f8e880cd880 legacy=0x7f8e880d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:40.887 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.884+0000 7f8e877fe640 1 -- 172.21.15.89:0/3010431716 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e8815d940 con 0x7f8e8815cb40 2023-10-18T10:45:40.888 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.888+0000 7f8e877fe640 1 --2- 172.21.15.89:0/3010431716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e8815cb40 0x7f8e880b16d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009870 tx=0x7f8e7c01c360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44b0f56268bb6ea server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:40.889 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.892+0000 7f8e86ffd640 1 -- 172.21.15.89:0/3010431716 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c0ab600 con 0x7f8e8815cb40 2023-10-18T10:45:40.889 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.892+0000 7f8e86ffd640 1 -- 172.21.15.89:0/3010431716 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e7c0a7050 con 0x7f8e8815cb40 2023-10-18T10:45:40.890 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.892+0000 7f8e86ffd640 1 -- 172.21.15.89:0/3010431716 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c0ab980 con 0x7f8e8815cb40 2023-10-18T10:45:40.890 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.892+0000 7f8e8e967640 1 -- 172.21.15.89:0/3010431716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e8815cb40 msgr2=0x7f8e880b16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:40.890 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.892+0000 7f8e8e967640 1 --2- 172.21.15.89:0/3010431716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e8815cb40 0x7f8e880b16d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009870 tx=0x7f8e7c01c360 comp rx=0 tx=0).stop 2023-10-18T10:45:40.891 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.892+0000 7f8e8e967640 1 -- 172.21.15.89:0/3010431716 shutdown_connections 2023-10-18T10:45:40.892 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.892+0000 7f8e8e967640 1 --2- 172.21.15.89:0/3010431716 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e880b1c10 0x7f8e880d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:40.892 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.892+0000 7f8e8e967640 1 --2- 172.21.15.89:0/3010431716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e8815cb40 0x7f8e880b16d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:40.892 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.892+0000 7f8e8e967640 1 -- 172.21.15.89:0/3010431716 >> 172.21.15.89:0/3010431716 conn(0x7f8e88071430 msgr2=0x7f8e88071840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:40.892 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.892+0000 7f8e8e967640 1 -- 172.21.15.89:0/3010431716 shutdown_connections 2023-10-18T10:45:40.893 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.892+0000 7f8e8e967640 1 -- 172.21.15.89:0/3010431716 wait complete. 2023-10-18T10:45:40.895 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.896+0000 7f8e8e967640 1 Processor -- start 2023-10-18T10:45:40.895 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.896+0000 7f8e8e967640 1 -- start start 2023-10-18T10:45:40.896 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.896+0000 7f8e8e967640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e880b1c10 0x7f8e8814e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:40.896 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.896+0000 7f8e8e967640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e880cd880 0x7f8e8815ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:40.897 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.896+0000 7f8e8e967640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e8815cb40 0x7f8e880b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:40.897 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.896+0000 7f8e8e967640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8814e6f0 con 0x7f8e8815cb40 2023-10-18T10:45:40.897 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.896+0000 7f8e8e967640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e88151180 con 0x7f8e880b1c10 2023-10-18T10:45:40.897 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.896+0000 7f8e8e967640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e88151610 con 0x7f8e880cd880 2023-10-18T10:45:40.898 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.900+0000 7f8e877fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e880cd880 0x7f8e8815ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:40.898 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.900+0000 7f8e877fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e880cd880 0x7f8e8815ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:38696/0 (socket says 172.21.15.89:38696) 2023-10-18T10:45:40.899 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.900+0000 7f8e877fe640 1 -- 172.21.15.89:0/2916474221 learned_addr learned my addr 172.21.15.89:0/2916474221 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:45:40.899 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.900+0000 7f8e87fff640 1 --2- 172.21.15.89:0/2916474221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e880b1c10 0x7f8e8814e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:40.899 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.900+0000 7f8e87fff640 1 -- 172.21.15.89:0/2916474221 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e880cd880 msgr2=0x7f8e8815ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:40.899 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.900+0000 7f8e87fff640 1 --2- 172.21.15.89:0/2916474221 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e880cd880 0x7f8e8815ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:40.899 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.900+0000 7f8e87fff640 1 -- 172.21.15.89:0/2916474221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e8815cb40 msgr2=0x7f8e880b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:40.900 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.900+0000 7f8e87fff640 1 --2- 172.21.15.89:0/2916474221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e8815cb40 0x7f8e880b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:40.900 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.900+0000 7f8e87fff640 1 -- 172.21.15.89:0/2916474221 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e7c0094d0 con 0x7f8e880b1c10 2023-10-18T10:45:40.900 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.900+0000 7f8e87fff640 1 --2- 172.21.15.89:0/2916474221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e880b1c10 0x7f8e8814e1b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8e7800c3c0 tx=0x7f8e7800c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:40.901 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.900+0000 7f8e84ff9640 1 -- 172.21.15.89:0/2916474221 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e780052d0 con 0x7f8e880b1c10 2023-10-18T10:45:40.901 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.900+0000 7f8e84ff9640 1 -- 172.21.15.89:0/2916474221 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e78005430 con 0x7f8e880b1c10 2023-10-18T10:45:40.901 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.900+0000 7f8e84ff9640 1 -- 172.21.15.89:0/2916474221 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e78007260 con 0x7f8e880b1c10 2023-10-18T10:45:40.901 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.900+0000 7f8e8e967640 1 -- 172.21.15.89:0/2916474221 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e8815f250 con 0x7f8e880b1c10 2023-10-18T10:45:40.902 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.900+0000 7f8e8e967640 1 -- 172.21.15.89:0/2916474221 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e88150850 con 0x7f8e880b1c10 2023-10-18T10:45:40.902 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.904+0000 7f8e8e967640 1 -- 172.21.15.89:0/2916474221 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e68005260 con 0x7f8e880b1c10 2023-10-18T10:45:40.906 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.908+0000 7f8e84ff9640 1 -- 172.21.15.89:0/2916474221 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 4) v1 ==== 84906+0+0 (secure 0 0 0) 0x7f8e780073c0 con 0x7f8e880b1c10 2023-10-18T10:45:40.907 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.908+0000 7f8e84ff9640 1 --2- 172.21.15.89:0/2916474221 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8e740655f0 0x7f8e74067ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:40.908 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.908+0000 7f8e84ff9640 1 -- 172.21.15.89:0/2916474221 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f8e78081810 con 0x7f8e880b1c10 2023-10-18T10:45:40.908 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.908+0000 7f8e877fe640 1 --2- 172.21.15.89:0/2916474221 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8e740655f0 0x7f8e74067ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:40.910 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.912+0000 7f8e877fe640 1 --2- 172.21.15.89:0/2916474221 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8e740655f0 0x7f8e74067ab0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009870 tx=0x7f8e7c0a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:40.924 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:40.924+0000 7f8e84ff9640 1 -- 172.21.15.89:0/2916474221 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f8e78050430 con 0x7f8e880b1c10 2023-10-18T10:45:41.125 INFO:tasks.ceph.osd.7.smithi154.stderr:2023-10-18T10:45:41.125+0000 7fa1b8413640 -1 osd.7 0 waiting for initial osdmap 2023-10-18T10:45:41.164 INFO:tasks.ceph.osd.7.smithi154.stderr:2023-10-18T10:45:41.165+0000 7fa1afa34640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T10:45:41.171 INFO:tasks.ceph.osd.5.smithi154.stderr:2023-10-18T10:45:41.173+0000 7f59399d1640 -1 osd.5 0 waiting for initial osdmap 2023-10-18T10:45:41.202 INFO:tasks.ceph.osd.4.smithi154.stderr:2023-10-18T10:45:41.201+0000 7f99510af640 -1 osd.4 0 waiting for initial osdmap 2023-10-18T10:45:41.204 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:41.204+0000 7f8e8e967640 1 -- 172.21.15.89:0/2916474221 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e680057e0 con 0x7f8e880b1c10 2023-10-18T10:45:41.205 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:41.208+0000 7f8e84ff9640 1 -- 172.21.15.89:0/2916474221 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f8e78020020 con 0x7f8e880b1c10 2023-10-18T10:45:41.206 INFO:teuthology.misc.health.smithi089.stdout: 2023-10-18T10:45:41.206 INFO:teuthology.misc.health.smithi089.stdout:{"epoch":12,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:45:38.721760+0000","last_up_change":"0.000000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:45:41.207 INFO:tasks.ceph.osd.5.smithi154.stderr:2023-10-18T10:45:41.209+0000 7f5931806640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T10:45:41.209 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:41.212+0000 7f8e8e967640 1 -- 172.21.15.89:0/2916474221 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8e740655f0 msgr2=0x7f8e74067ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:41.209 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:41.212+0000 7f8e8e967640 1 --2- 172.21.15.89:0/2916474221 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8e740655f0 0x7f8e74067ab0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009870 tx=0x7f8e7c0a6000 comp rx=0 tx=0).stop 2023-10-18T10:45:41.209 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:41.212+0000 7f8e8e967640 1 -- 172.21.15.89:0/2916474221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e880b1c10 msgr2=0x7f8e8814e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:41.210 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:41.212+0000 7f8e8e967640 1 --2- 172.21.15.89:0/2916474221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e880b1c10 0x7f8e8814e1b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8e7800c3c0 tx=0x7f8e7800c880 comp rx=0 tx=0).stop 2023-10-18T10:45:41.210 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:41.212+0000 7f8e8e967640 1 -- 172.21.15.89:0/2916474221 shutdown_connections 2023-10-18T10:45:41.210 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:41.212+0000 7f8e8e967640 1 --2- 172.21.15.89:0/2916474221 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8e740655f0 0x7f8e74067ab0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:41.211 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:41.212+0000 7f8e8e967640 1 --2- 172.21.15.89:0/2916474221 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e880cd880 0x7f8e8815ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:41.211 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:41.212+0000 7f8e8e967640 1 --2- 172.21.15.89:0/2916474221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e8815cb40 0x7f8e880b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:41.211 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:41.212+0000 7f8e8e967640 1 --2- 172.21.15.89:0/2916474221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e880b1c10 0x7f8e8814e1b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:41.211 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:41.212+0000 7f8e8e967640 1 -- 172.21.15.89:0/2916474221 >> 172.21.15.89:0/2916474221 conn(0x7f8e88071430 msgr2=0x7f8e88071810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:41.212 INFO:tasks.ceph.osd.6.smithi154.stderr:2023-10-18T10:45:41.209+0000 7f0c85909640 -1 osd.6 0 waiting for initial osdmap 2023-10-18T10:45:41.217 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:41.212+0000 7f8e8e967640 1 -- 172.21.15.89:0/2916474221 shutdown_connections 2023-10-18T10:45:41.217 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:41.220+0000 7f8e8e967640 1 -- 172.21.15.89:0/2916474221 wait complete. 2023-10-18T10:45:41.227 INFO:tasks.ceph.osd.4.smithi154.stderr:2023-10-18T10:45:41.229+0000 7f9948ee4640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T10:45:41.243 INFO:tasks.ceph.osd.6.smithi154.stderr:2023-10-18T10:45:41.245+0000 7f0c7d73e640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T10:45:41.245 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-18T10:45:41.246 INFO:tasks.ceph.osd.0.smithi089.stderr:2023-10-18T10:45:41.248+0000 7f574d46a640 -1 osd.0 0 waiting for initial osdmap 2023-10-18T10:45:41.261 INFO:tasks.ceph.osd.2.smithi089.stderr:2023-10-18T10:45:41.260+0000 7f6879d01640 -1 osd.2 0 waiting for initial osdmap 2023-10-18T10:45:41.269 INFO:tasks.ceph.osd.1.smithi089.stderr:2023-10-18T10:45:41.272+0000 7f4d1492c640 -1 osd.1 0 waiting for initial osdmap 2023-10-18T10:45:41.293 INFO:tasks.ceph.osd.0.smithi089.stderr:2023-10-18T10:45:41.296+0000 7f5744a8b640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T10:45:41.299 INFO:tasks.ceph.osd.3.smithi089.stderr:2023-10-18T10:45:41.300+0000 7fafdeed7640 -1 osd.3 0 waiting for initial osdmap 2023-10-18T10:45:41.309 INFO:tasks.ceph.osd.1.smithi089.stderr:2023-10-18T10:45:41.312+0000 7f4d0c761640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T10:45:41.309 INFO:tasks.ceph.osd.2.smithi089.stderr:2023-10-18T10:45:41.312+0000 7f6871b36640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T10:45:41.325 INFO:tasks.ceph.osd.3.smithi089.stderr:2023-10-18T10:45:41.328+0000 7fafd64f8640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T10:45:47.779 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:45:47.784 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:45:48.051 DEBUG:teuthology.orchestra.run.smithi089:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-18T10:45:48.187 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.187+0000 7f66e61e8640 1 Processor -- start 2023-10-18T10:45:48.187 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.187+0000 7f66e61e8640 1 -- start start 2023-10-18T10:45:48.188 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.187+0000 7f66e61e8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66e00d25f0 0x7f66e00d2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:48.188 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.187+0000 7f66e61e8640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66e00d1900 con 0x7f66e00d0660 2023-10-18T10:45:48.189 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.187+0000 7f66e61e8640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66e00d0aa0 con 0x7f66e015fb30 2023-10-18T10:45:48.189 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.187+0000 7f66e61e8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66e00d2fc0 con 0x7f66e00d25f0 2023-10-18T10:45:48.189 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.187+0000 7f66deffd640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f66e00d0660 0x7f66e00d1d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37804/0 (socket says 172.21.15.89:37804) 2023-10-18T10:45:48.189 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.187+0000 7f66deffd640 1 -- 172.21.15.89:0/930960438 learned_addr learned my addr 172.21.15.89:0/930960438 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:45:48.189 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.187+0000 7f66dffff640 1 --2- 172.21.15.89:0/930960438 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66e00d25f0 0x7f66e00d2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:48.192 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.187+0000 7f66de7fc640 1 -- 172.21.15.89:0/930960438 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2035937376 0 0) 0x7f66e00d1900 con 0x7f66e00d0660 2023-10-18T10:45:48.192 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.187+0000 7f66de7fc640 1 -- 172.21.15.89:0/930960438 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66c8003530 con 0x7f66e00d0660 2023-10-18T10:45:48.192 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.187+0000 7f66de7fc640 1 -- 172.21.15.89:0/930960438 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1606475027 0 0) 0x7f66e00d0aa0 con 0x7f66e015fb30 2023-10-18T10:45:48.192 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.191+0000 7f66de7fc640 1 -- 172.21.15.89:0/930960438 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66e00d1900 con 0x7f66e015fb30 2023-10-18T10:45:48.193 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.191+0000 7f66de7fc640 1 -- 172.21.15.89:0/930960438 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 57631113 0 0) 0x7f66c8003530 con 0x7f66e00d0660 2023-10-18T10:45:48.193 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.191+0000 7f66de7fc640 1 -- 172.21.15.89:0/930960438 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66e00d25f0 msgr2=0x7f66e00d2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:48.193 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.191+0000 7f66de7fc640 1 --2- 172.21.15.89:0/930960438 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66e00d25f0 0x7f66e00d2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:48.193 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.191+0000 7f66de7fc640 1 -- 172.21.15.89:0/930960438 >> v1:172.21.15.154:6789/0 conn(0x7f66e015fb30 legacy=0x7f66e00b80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:48.194 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.191+0000 7f66de7fc640 1 -- 172.21.15.89:0/930960438 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e00d0cf0 con 0x7f66e00d0660 2023-10-18T10:45:48.194 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.191+0000 7f66de7fc640 1 -- 172.21.15.89:0/930960438 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7f66d4001680 con 0x7f66e00d0660 2023-10-18T10:45:48.194 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.191+0000 7f66de7fc640 1 -- 172.21.15.89:0/930960438 >> v1:172.21.15.89:6789/0 conn(0x7f66e00d0660 legacy=0x7f66e00d1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:48.194 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.191+0000 7f66de7fc640 1 --2- 172.21.15.89:0/930960438 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66c8003b40 0x7f66c8003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:48.194 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.191+0000 7f66de7fc640 1 --2- 172.21.15.89:0/930960438 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66e00d25f0 0x7f66c80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:48.195 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.191+0000 7f66de7fc640 1 --2- 172.21.15.89:0/930960438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66c8004d10 0x7f66c8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:48.195 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.191+0000 7f66de7fc640 1 -- 172.21.15.89:0/930960438 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e00d2fc0 con 0x7f66c8004d10 2023-10-18T10:45:48.195 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.191+0000 7f66de7fc640 1 -- 172.21.15.89:0/930960438 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c800ba40 con 0x7f66c8003b40 2023-10-18T10:45:48.195 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.191+0000 7f66de7fc640 1 -- 172.21.15.89:0/930960438 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c800bbb0 con 0x7f66e00d25f0 2023-10-18T10:45:48.196 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.191+0000 7f66dffff640 1 --2- 172.21.15.89:0/930960438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66c8004d10 0x7f66c8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:48.196 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.191+0000 7f66deffd640 1 --2- 172.21.15.89:0/930960438 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66e00d25f0 0x7f66c80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:48.196 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.191+0000 7f66df7fe640 1 --2- 172.21.15.89:0/930960438 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66c8003b40 0x7f66c8003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:48.196 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.191+0000 7f66dffff640 1 -- 172.21.15.89:0/930960438 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66e00d25f0 msgr2=0x7f66c80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:48.196 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.191+0000 7f66dffff640 1 --2- 172.21.15.89:0/930960438 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66e00d25f0 0x7f66c80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:48.197 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.191+0000 7f66dffff640 1 -- 172.21.15.89:0/930960438 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66c8003b40 msgr2=0x7f66c8003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:48.197 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.191+0000 7f66dffff640 1 --2- 172.21.15.89:0/930960438 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66c8003b40 0x7f66c8003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:48.197 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.191+0000 7f66dffff640 1 -- 172.21.15.89:0/930960438 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c8003530 con 0x7f66c8004d10 2023-10-18T10:45:48.197 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.191+0000 7f66df7fe640 1 --2- 172.21.15.89:0/930960438 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66c8003b40 0x7f66c8003f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:45:48.200 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.191+0000 7f66dffff640 1 --2- 172.21.15.89:0/930960438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66c8004d10 0x7f66c8005190 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009fa0 tx=0x7f66d801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23aef971da39d735 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:48.201 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.195+0000 7f66de7fc640 1 -- 172.21.15.89:0/930960438 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d80a5cf0 con 0x7f66c8004d10 2023-10-18T10:45:48.201 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.195+0000 7f66de7fc640 1 -- 172.21.15.89:0/930960438 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d800f040 con 0x7f66c8004d10 2023-10-18T10:45:48.201 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.195+0000 7f66de7fc640 1 -- 172.21.15.89:0/930960438 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d80ad660 con 0x7f66c8004d10 2023-10-18T10:45:48.201 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.195+0000 7f66e61e8640 1 -- 172.21.15.89:0/930960438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66c8004d10 msgr2=0x7f66c8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:48.202 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.195+0000 7f66e61e8640 1 --2- 172.21.15.89:0/930960438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66c8004d10 0x7f66c8005190 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009fa0 tx=0x7f66d801cc20 comp rx=0 tx=0).stop 2023-10-18T10:45:48.202 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.195+0000 7f66dffff640 1 -- 172.21.15.89:0/930960438 reap_dead start 2023-10-18T10:45:48.202 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.195+0000 7f66e61e8640 1 -- 172.21.15.89:0/930960438 shutdown_connections 2023-10-18T10:45:48.202 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.195+0000 7f66e61e8640 1 -- 172.21.15.89:0/930960438 >> 172.21.15.89:0/930960438 conn(0x7f66e0071430 msgr2=0x7f66e0071840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:48.202 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.195+0000 7f66e61e8640 1 -- 172.21.15.89:0/930960438 shutdown_connections 2023-10-18T10:45:48.203 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.195+0000 7f66e61e8640 1 -- 172.21.15.89:0/930960438 wait complete. 2023-10-18T10:45:48.203 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.195+0000 7f66e61e8640 1 Processor -- start 2023-10-18T10:45:48.203 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.195+0000 7f66e61e8640 1 -- start start 2023-10-18T10:45:48.203 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.195+0000 7f66e61e8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66c8004d10 0x7f66e0152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:48.203 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.195+0000 7f66e61e8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66e0153950 0x7f66e01f1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:48.204 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.195+0000 7f66e61e8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66e01f1890 0x7f66e015e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:48.204 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.195+0000 7f66e61e8640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e01543f0 con 0x7f66c8004d10 2023-10-18T10:45:48.204 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.195+0000 7f66e61e8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e0153e30 con 0x7f66e0153950 2023-10-18T10:45:48.204 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.195+0000 7f66e61e8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66e0154110 con 0x7f66e01f1890 2023-10-18T10:45:48.205 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.199+0000 7f66df7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66c8004d10 0x7f66e0152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:48.205 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.199+0000 7f66df7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66c8004d10 0x7f66e0152fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41738/0 (socket says 172.21.15.89:41738) 2023-10-18T10:45:48.205 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.199+0000 7f66df7fe640 1 -- 172.21.15.89:0/321620120 learned_addr learned my addr 172.21.15.89:0/321620120 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:45:48.205 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.199+0000 7f66deffd640 1 --2- 172.21.15.89:0/321620120 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66e0153950 0x7f66e01f1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:48.205 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.199+0000 7f66dffff640 1 --2- 172.21.15.89:0/321620120 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66e01f1890 0x7f66e015e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:48.206 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.199+0000 7f66df7fe640 1 -- 172.21.15.89:0/321620120 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66e01f1890 msgr2=0x7f66e015e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:48.206 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.199+0000 7f66df7fe640 1 --2- 172.21.15.89:0/321620120 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66e01f1890 0x7f66e015e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:48.206 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.199+0000 7f66df7fe640 1 -- 172.21.15.89:0/321620120 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66e0153950 msgr2=0x7f66e01f1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:48.206 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.199+0000 7f66df7fe640 1 --2- 172.21.15.89:0/321620120 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66e0153950 0x7f66e01f1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:48.206 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.199+0000 7f66df7fe640 1 -- 172.21.15.89:0/321620120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e014fc10 con 0x7f66c8004d10 2023-10-18T10:45:48.207 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.199+0000 7f66df7fe640 1 --2- 172.21.15.89:0/321620120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66c8004d10 0x7f66e0152fd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f66d000e780 tx=0x7f66d000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:48.207 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.199+0000 7f66c7fff640 1 -- 172.21.15.89:0/321620120 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d0017070 con 0x7f66c8004d10 2023-10-18T10:45:48.207 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.199+0000 7f66c7fff640 1 -- 172.21.15.89:0/321620120 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d00050d0 con 0x7f66c8004d10 2023-10-18T10:45:48.207 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.199+0000 7f66c7fff640 1 -- 172.21.15.89:0/321620120 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d0005240 con 0x7f66c8004d10 2023-10-18T10:45:48.208 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.199+0000 7f66e61e8640 1 -- 172.21.15.89:0/321620120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66e014fef0 con 0x7f66c8004d10 2023-10-18T10:45:48.208 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.199+0000 7f66e61e8640 1 -- 172.21.15.89:0/321620120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e015eca0 con 0x7f66c8004d10 2023-10-18T10:45:48.208 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.199+0000 7f66e61e8640 1 -- 172.21.15.89:0/321620120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66e0073b50 con 0x7f66c8004d10 2023-10-18T10:45:48.208 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.199+0000 7f66c7fff640 1 -- 172.21.15.89:0/321620120 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f66d0007020 con 0x7f66c8004d10 2023-10-18T10:45:48.209 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.203+0000 7f66c7fff640 1 --2- 172.21.15.89:0/321620120 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f66cc065680 0x7f66cc067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:48.209 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.203+0000 7f66c7fff640 1 -- 172.21.15.89:0/321620120 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f66d0082100 con 0x7f66c8004d10 2023-10-18T10:45:48.210 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.203+0000 7f66deffd640 1 --2- 172.21.15.89:0/321620120 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f66cc065680 0x7f66cc067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:48.210 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.203+0000 7f66deffd640 1 --2- 172.21.15.89:0/321620120 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f66cc065680 0x7f66cc067b40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f66d4001000 tx=0x7f66d4001e20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:48.210 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.207+0000 7f66c7fff640 1 -- 172.21.15.89:0/321620120 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f66d0050060 con 0x7f66c8004d10 2023-10-18T10:45:48.364 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.363+0000 7f66e61e8640 1 -- 172.21.15.89:0/321620120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66e00c6920 con 0x7f66c8004d10 2023-10-18T10:45:48.365 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.367+0000 7f66c7fff640 1 -- 172.21.15.89:0/321620120 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12826 (secure 0 0 0) 0x7f66d0053680 con 0x7f66c8004d10 2023-10-18T10:45:48.366 INFO:teuthology.misc.health.smithi089.stdout: 2023-10-18T10:45:48.366 INFO:teuthology.misc.health.smithi089.stdout:{"epoch":16,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:45:45.203485+0000","last_up_change":"2023-10-18T10:45:42.164688+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6824","nonce":773078364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6827","nonce":773078364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6831","nonce":773078364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6829","nonce":773078364}]},"public_addr":"172.21.15.154:6824/773078364","cluster_addr":"172.21.15.154:6827/773078364","heartbeat_back_addr":"172.21.15.154:6831/773078364","heartbeat_front_addr":"172.21.15.154:6829/773078364","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:45:48.367 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.367+0000 7f66e61e8640 1 -- 172.21.15.89:0/321620120 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f66cc065680 msgr2=0x7f66cc067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:48.368 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.367+0000 7f66e61e8640 1 --2- 172.21.15.89:0/321620120 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f66cc065680 0x7f66cc067b40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f66d4001000 tx=0x7f66d4001e20 comp rx=0 tx=0).stop 2023-10-18T10:45:48.368 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.367+0000 7f66e61e8640 1 -- 172.21.15.89:0/321620120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66c8004d10 msgr2=0x7f66e0152fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:48.368 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.367+0000 7f66e61e8640 1 --2- 172.21.15.89:0/321620120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66c8004d10 0x7f66e0152fd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f66d000e780 tx=0x7f66d000eb80 comp rx=0 tx=0).stop 2023-10-18T10:45:48.368 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.367+0000 7f66e61e8640 1 -- 172.21.15.89:0/321620120 shutdown_connections 2023-10-18T10:45:48.368 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.367+0000 7f66e61e8640 1 --2- 172.21.15.89:0/321620120 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f66cc065680 0x7f66cc067b40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:48.369 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.367+0000 7f66e61e8640 1 --2- 172.21.15.89:0/321620120 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66e01f1890 0x7f66e015e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:48.369 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.367+0000 7f66e61e8640 1 --2- 172.21.15.89:0/321620120 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66e0153950 0x7f66e01f1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:48.369 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.367+0000 7f66e61e8640 1 --2- 172.21.15.89:0/321620120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66c8004d10 0x7f66e0152fd0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:48.369 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.367+0000 7f66e61e8640 1 -- 172.21.15.89:0/321620120 >> 172.21.15.89:0/321620120 conn(0x7f66e0071430 msgr2=0x7f66e006a960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:48.369 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.367+0000 7f66e61e8640 1 -- 172.21.15.89:0/321620120 shutdown_connections 2023-10-18T10:45:48.370 INFO:teuthology.misc.health.smithi089.stderr:2023-10-18T10:45:48.367+0000 7f66e61e8640 1 -- 172.21.15.89:0/321620120 wait complete. 2023-10-18T10:45:48.385 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-18T10:45:48.385 INFO:tasks.ceph:Creating RBD pool 2023-10-18T10:45:48.385 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --cluster ceph osd pool create rbd 8 2023-10-18T10:45:48.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.519+0000 7f7b0dfa3640 1 Processor -- start 2023-10-18T10:45:48.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.519+0000 7f7b0dfa3640 1 -- start start 2023-10-18T10:45:48.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.519+0000 7f7b0dfa3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7b08070600 0x7f7b0806b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:48.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.519+0000 7f7b0dfa3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7b0806bd10 0x7f7b080c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:48.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.519+0000 7f7b0dfa3640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b08070b90 con 0x7f7b08159490 2023-10-18T10:45:48.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.523+0000 7f7b0dfa3640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b0806c1a0 con 0x7f7b0806bd10 2023-10-18T10:45:48.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.523+0000 7f7b0dfa3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b08070e70 con 0x7f7b08070600 2023-10-18T10:45:48.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.523+0000 7f7b06ffd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7b08070600 0x7f7b0806b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:48.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.523+0000 7f7b07fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7b0806bd10 0x7f7b080c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:48.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.523+0000 7f7b06ffd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7b08070600 0x7f7b0806b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:57268/0 (socket says 172.21.15.89:57268) 2023-10-18T10:45:48.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.523+0000 7f7b06ffd640 1 -- 172.21.15.89:0/2529411126 learned_addr learned my addr 172.21.15.89:0/2529411126 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:45:48.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.523+0000 7f7b06ffd640 1 -- 172.21.15.89:0/2529411126 >> v2:172.21.15.154:3300/0 conn(0x7f7b0806bd10 msgr2=0x7f7b080c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:48.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.523+0000 7f7b06ffd640 1 --2- 172.21.15.89:0/2529411126 >> v2:172.21.15.154:3300/0 conn(0x7f7b0806bd10 0x7f7b080c8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:48.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.523+0000 7f7b06ffd640 1 -- 172.21.15.89:0/2529411126 >> v1:172.21.15.154:6789/0 conn(0x7f7b08159490 legacy=0x7f7b0815d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:48.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.523+0000 7f7b06ffd640 1 -- 172.21.15.89:0/2529411126 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b08071150 con 0x7f7b08070600 2023-10-18T10:45:48.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.523+0000 7f7b067fc640 1 -- 172.21.15.89:0/2529411126 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 395847413 0 0) 0x7f7b08070b90 con 0x7f7b08159490 2023-10-18T10:45:48.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.523+0000 7f7b07fff640 1 --2- 172.21.15.89:0/2529411126 >> v2:172.21.15.154:3300/0 conn(0x7f7b0806bd10 0x7f7b080c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:45:48.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.527+0000 7f7b06ffd640 1 --2- 172.21.15.89:0/2529411126 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7b08070600 0x7f7b0806b7d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009870 tx=0x7f7afc01c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f694280f1759e29 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:48.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.527+0000 7f7b067fc640 1 -- 172.21.15.89:0/2529411126 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc0ab680 con 0x7f7b08070600 2023-10-18T10:45:48.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.527+0000 7f7b067fc640 1 -- 172.21.15.89:0/2529411126 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7afc0ab7e0 con 0x7f7b08070600 2023-10-18T10:45:48.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.527+0000 7f7b067fc640 1 -- 172.21.15.89:0/2529411126 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc0abb70 con 0x7f7b08070600 2023-10-18T10:45:48.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.527+0000 7f7b0dfa3640 1 -- 172.21.15.89:0/2529411126 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7b08070600 msgr2=0x7f7b0806b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:48.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.527+0000 7f7b0dfa3640 1 --2- 172.21.15.89:0/2529411126 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7b08070600 0x7f7b0806b7d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009870 tx=0x7f7afc01c480 comp rx=0 tx=0).stop 2023-10-18T10:45:48.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.527+0000 7f7b0dfa3640 1 -- 172.21.15.89:0/2529411126 shutdown_connections 2023-10-18T10:45:48.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.527+0000 7f7b0dfa3640 1 --2- 172.21.15.89:0/2529411126 >> v2:172.21.15.154:3300/0 conn(0x7f7b0806bd10 0x7f7b080c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:48.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.527+0000 7f7b0dfa3640 1 --2- 172.21.15.89:0/2529411126 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7b08070600 0x7f7b0806b7d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:48.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.527+0000 7f7b0dfa3640 1 -- 172.21.15.89:0/2529411126 >> 172.21.15.89:0/2529411126 conn(0x7f7b08069f00 msgr2=0x7f7b080cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:48.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.527+0000 7f7b0dfa3640 1 -- 172.21.15.89:0/2529411126 shutdown_connections 2023-10-18T10:45:48.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.527+0000 7f7b0dfa3640 1 -- 172.21.15.89:0/2529411126 wait complete. 2023-10-18T10:45:48.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.527+0000 7f7b0dfa3640 1 Processor -- start 2023-10-18T10:45:48.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.527+0000 7f7b0dfa3640 1 -- start start 2023-10-18T10:45:48.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.527+0000 7f7b0dfa3640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b0806bd10 0x7f7b0814e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:48.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.527+0000 7f7b0dfa3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b08070600 0x7f7b08165a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:48.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.531+0000 7f7b0dfa3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7b08159490 0x7f7b08165fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:48.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.531+0000 7f7b0dfa3640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b08167230 con 0x7f7b0806bd10 2023-10-18T10:45:48.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.531+0000 7f7b0dfa3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b081665b0 con 0x7f7b08070600 2023-10-18T10:45:48.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.531+0000 7f7b0dfa3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b08166890 con 0x7f7b08159490 2023-10-18T10:45:48.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.531+0000 7f7b077fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b0806bd10 0x7f7b0814e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:48.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.531+0000 7f7b077fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b0806bd10 0x7f7b0814e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41750/0 (socket says 172.21.15.89:41750) 2023-10-18T10:45:48.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.531+0000 7f7b077fe640 1 -- 172.21.15.89:0/3249729550 learned_addr learned my addr 172.21.15.89:0/3249729550 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:45:48.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.531+0000 7f7b06ffd640 1 --2- 172.21.15.89:0/3249729550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b08070600 0x7f7b08165a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:48.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.531+0000 7f7b07fff640 1 --2- 172.21.15.89:0/3249729550 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7b08159490 0x7f7b08165fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:48.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.531+0000 7f7b077fe640 1 -- 172.21.15.89:0/3249729550 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7b08159490 msgr2=0x7f7b08165fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:48.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.531+0000 7f7b077fe640 1 --2- 172.21.15.89:0/3249729550 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7b08159490 0x7f7b08165fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:48.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.531+0000 7f7b077fe640 1 -- 172.21.15.89:0/3249729550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b08070600 msgr2=0x7f7b08165a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:48.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.531+0000 7f7b077fe640 1 --2- 172.21.15.89:0/3249729550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b08070600 0x7f7b08165a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:48.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.531+0000 7f7b077fe640 1 -- 172.21.15.89:0/3249729550 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b00009a70 con 0x7f7b0806bd10 2023-10-18T10:45:48.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.531+0000 7f7b077fe640 1 --2- 172.21.15.89:0/3249729550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b0806bd10 0x7f7b0814e7d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7af8001ec0 tx=0x7f7af800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:48.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.531+0000 7f7aeffff640 1 -- 172.21.15.89:0/3249729550 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af80054e0 con 0x7f7b0806bd10 2023-10-18T10:45:48.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.531+0000 7f7aeffff640 1 -- 172.21.15.89:0/3249729550 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af8003600 con 0x7f7b0806bd10 2023-10-18T10:45:48.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.531+0000 7f7b0dfa3640 1 -- 172.21.15.89:0/3249729550 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7afc0094d0 con 0x7f7b0806bd10 2023-10-18T10:45:48.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.531+0000 7f7aeffff640 1 -- 172.21.15.89:0/3249729550 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af80039b0 con 0x7f7b0806bd10 2023-10-18T10:45:48.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.531+0000 7f7b0dfa3640 1 -- 172.21.15.89:0/3249729550 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b08166e50 con 0x7f7b0806bd10 2023-10-18T10:45:48.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.531+0000 7f7b0dfa3640 1 -- 172.21.15.89:0/3249729550 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b0804dbe0 con 0x7f7b0806bd10 2023-10-18T10:45:48.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.531+0000 7f7aeffff640 1 -- 172.21.15.89:0/3249729550 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f7af8003b10 con 0x7f7b0806bd10 2023-10-18T10:45:48.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.535+0000 7f7aeffff640 1 --2- 172.21.15.89:0/3249729550 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7af4065680 0x7f7af4067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:48.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.535+0000 7f7aeffff640 1 -- 172.21.15.89:0/3249729550 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7af8081eb0 con 0x7f7b0806bd10 2023-10-18T10:45:48.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.535+0000 7f7b06ffd640 1 --2- 172.21.15.89:0/3249729550 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7af4065680 0x7f7af4067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:48.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.535+0000 7f7b06ffd640 1 --2- 172.21.15.89:0/3249729550 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7af4065680 0x7f7af4067b40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7afc0af000 tx=0x7f7afc0a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:48.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.535+0000 7f7aeffff640 1 -- 172.21.15.89:0/3249729550 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f7af804fec0 con 0x7f7b0806bd10 2023-10-18T10:45:48.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:48.687+0000 7f7b0dfa3640 1 -- 172.21.15.89:0/3249729550 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f7b080cbb60 con 0x7f7b0806bd10 2023-10-18T10:45:49.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.235+0000 7f7aeffff640 1 -- 172.21.15.89:0/3249729550 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v17) v1 ==== 109+0+0 (secure 0 0 0) 0x7f7af8017020 con 0x7f7b0806bd10 2023-10-18T10:45:49.234 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'rbd' created 2023-10-18T10:45:49.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.235+0000 7f7b0dfa3640 1 -- 172.21.15.89:0/3249729550 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7af4065680 msgr2=0x7f7af4067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:49.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.235+0000 7f7b0dfa3640 1 --2- 172.21.15.89:0/3249729550 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7af4065680 0x7f7af4067b40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7afc0af000 tx=0x7f7afc0a7000 comp rx=0 tx=0).stop 2023-10-18T10:45:49.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.235+0000 7f7b0dfa3640 1 -- 172.21.15.89:0/3249729550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b0806bd10 msgr2=0x7f7b0814e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:49.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.235+0000 7f7b0dfa3640 1 --2- 172.21.15.89:0/3249729550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b0806bd10 0x7f7b0814e7d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7af8001ec0 tx=0x7f7af800c8e0 comp rx=0 tx=0).stop 2023-10-18T10:45:49.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.235+0000 7f7b0dfa3640 1 -- 172.21.15.89:0/3249729550 shutdown_connections 2023-10-18T10:45:49.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.235+0000 7f7b0dfa3640 1 --2- 172.21.15.89:0/3249729550 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7af4065680 0x7f7af4067b40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:49.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.235+0000 7f7b0dfa3640 1 --2- 172.21.15.89:0/3249729550 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7b08159490 0x7f7b08165fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:49.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.235+0000 7f7b0dfa3640 1 --2- 172.21.15.89:0/3249729550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b08070600 0x7f7b08165a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:49.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.235+0000 7f7b0dfa3640 1 --2- 172.21.15.89:0/3249729550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b0806bd10 0x7f7b0814e7d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:49.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.235+0000 7f7b0dfa3640 1 -- 172.21.15.89:0/3249729550 >> 172.21.15.89:0/3249729550 conn(0x7f7b08069f00 msgr2=0x7f7b080c5470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:49.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.235+0000 7f7b0dfa3640 1 -- 172.21.15.89:0/3249729550 shutdown_connections 2023-10-18T10:45:49.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.235+0000 7f7b0dfa3640 1 -- 172.21.15.89:0/3249729550 wait complete. 2023-10-18T10:45:49.258 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-10-18T10:45:49.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.395+0000 7f8a055e1640 1 Processor -- start 2023-10-18T10:45:49.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.395+0000 7f8a055e1640 1 -- start start 2023-10-18T10:45:49.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.395+0000 7f8a055e1640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8a001600b0 0x7f8a000b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:49.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.395+0000 7f8a055e1640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a0006a4c0 con 0x7f8a000d0c70 2023-10-18T10:45:49.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.395+0000 7f8a055e1640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a00160490 con 0x7f8a00069900 2023-10-18T10:45:49.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.395+0000 7f8a055e1640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a000b1a80 con 0x7f8a001600b0 2023-10-18T10:45:49.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.399+0000 7f89feffd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8a001600b0 0x7f8a000b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:49.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.399+0000 7f89feffd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8a001600b0 0x7f8a000b12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41760/0 (socket says 172.21.15.89:41760) 2023-10-18T10:45:49.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.399+0000 7f89fe7fc640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f8a000d0c70 0x7f8a0006a950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37806/0 (socket says 172.21.15.89:37806) 2023-10-18T10:45:49.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.399+0000 7f89feffd640 1 -- 172.21.15.89:0/3899548587 learned_addr learned my addr 172.21.15.89:0/3899548587 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:45:49.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.399+0000 7f89feffd640 1 -- 172.21.15.89:0/3899548587 >> v1:172.21.15.154:6789/0 conn(0x7f8a00069900 legacy=0x7f8a00069d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:49.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.399+0000 7f89feffd640 1 -- 172.21.15.89:0/3899548587 >> v1:172.21.15.89:6789/0 conn(0x7f8a000d0c70 legacy=0x7f8a0006a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:49.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.399+0000 7f89feffd640 1 -- 172.21.15.89:0/3899548587 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a000d1120 con 0x7f8a001600b0 2023-10-18T10:45:49.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.399+0000 7f89fdffb640 1 -- 172.21.15.89:0/3899548587 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2028757622 0 0) 0x7f8a0006a4c0 con 0x7f8a000d0c70 2023-10-18T10:45:49.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.399+0000 7f89feffd640 1 --2- 172.21.15.89:0/3899548587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a001600b0 0x7f8a000b12b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f89f0009870 tx=0x7f89f001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d89239456e061b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:49.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.403+0000 7f89fdffb640 1 -- 172.21.15.89:0/3899548587 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f00aa670 con 0x7f8a001600b0 2023-10-18T10:45:49.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.403+0000 7f89fdffb640 1 -- 172.21.15.89:0/3899548587 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f00aa7d0 con 0x7f8a001600b0 2023-10-18T10:45:49.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.403+0000 7f89fdffb640 1 -- 172.21.15.89:0/3899548587 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f00aab60 con 0x7f8a001600b0 2023-10-18T10:45:49.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.403+0000 7f8a055e1640 1 -- 172.21.15.89:0/3899548587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a001600b0 msgr2=0x7f8a000b12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:49.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.403+0000 7f8a055e1640 1 --2- 172.21.15.89:0/3899548587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a001600b0 0x7f8a000b12b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f89f0009870 tx=0x7f89f001c510 comp rx=0 tx=0).stop 2023-10-18T10:45:49.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.403+0000 7f8a055e1640 1 -- 172.21.15.89:0/3899548587 shutdown_connections 2023-10-18T10:45:49.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.403+0000 7f8a055e1640 1 --2- 172.21.15.89:0/3899548587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a001600b0 0x7f8a000b12b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:49.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.403+0000 7f8a055e1640 1 -- 172.21.15.89:0/3899548587 >> 172.21.15.89:0/3899548587 conn(0x7f8a0009ba00 msgr2=0x7f8a000d2690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:49.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.403+0000 7f8a055e1640 1 -- 172.21.15.89:0/3899548587 shutdown_connections 2023-10-18T10:45:49.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.403+0000 7f8a055e1640 1 -- 172.21.15.89:0/3899548587 wait complete. 2023-10-18T10:45:49.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.403+0000 7f8a055e1640 1 Processor -- start 2023-10-18T10:45:49.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.403+0000 7f8a055e1640 1 -- start start 2023-10-18T10:45:49.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.403+0000 7f8a055e1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a00069900 0x7f8a00157d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:49.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.403+0000 7f8a055e1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a000d0c70 0x7f8a00157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:49.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.403+0000 7f8a055e1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a001600b0 0x7f8a001575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:49.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.403+0000 7f8a055e1640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a000c8d10 con 0x7f8a000d0c70 2023-10-18T10:45:49.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.403+0000 7f8a055e1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a000c89e0 con 0x7f8a00069900 2023-10-18T10:45:49.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.403+0000 7f8a055e1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a000c62f0 con 0x7f8a001600b0 2023-10-18T10:45:49.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.403+0000 7f89fe7fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a000d0c70 0x7f8a00157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:49.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.403+0000 7f89fe7fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a000d0c70 0x7f8a00157090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41766/0 (socket says 172.21.15.89:41766) 2023-10-18T10:45:49.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.403+0000 7f89fe7fc640 1 -- 172.21.15.89:0/541630413 learned_addr learned my addr 172.21.15.89:0/541630413 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:45:49.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.407+0000 7f89feffd640 1 --2- 172.21.15.89:0/541630413 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a00069900 0x7f8a00157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:49.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.407+0000 7f89ff7fe640 1 --2- 172.21.15.89:0/541630413 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a001600b0 0x7f8a001575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:49.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.407+0000 7f89fe7fc640 1 -- 172.21.15.89:0/541630413 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a001600b0 msgr2=0x7f8a001575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:49.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.407+0000 7f89fe7fc640 1 --2- 172.21.15.89:0/541630413 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a001600b0 0x7f8a001575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:49.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.407+0000 7f89fe7fc640 1 -- 172.21.15.89:0/541630413 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a00069900 msgr2=0x7f8a00157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:49.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.407+0000 7f89fe7fc640 1 --2- 172.21.15.89:0/541630413 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a00069900 0x7f8a00157d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:49.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.407+0000 7f89fe7fc640 1 -- 172.21.15.89:0/541630413 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f00094d0 con 0x7f8a000d0c70 2023-10-18T10:45:49.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.407+0000 7f89fe7fc640 1 --2- 172.21.15.89:0/541630413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a000d0c70 0x7f8a00157090 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f89f400c4a0 tx=0x7f89f400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:49.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.407+0000 7f89e77fe640 1 -- 172.21.15.89:0/541630413 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f40074e0 con 0x7f8a000d0c70 2023-10-18T10:45:49.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.407+0000 7f89e77fe640 1 -- 172.21.15.89:0/541630413 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f4007640 con 0x7f8a000d0c70 2023-10-18T10:45:49.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.407+0000 7f89e77fe640 1 -- 172.21.15.89:0/541630413 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f4004a30 con 0x7f8a000d0c70 2023-10-18T10:45:49.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.407+0000 7f8a055e1640 1 -- 172.21.15.89:0/541630413 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a000c65d0 con 0x7f8a000d0c70 2023-10-18T10:45:49.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.407+0000 7f8a055e1640 1 -- 172.21.15.89:0/541630413 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a000c68b0 con 0x7f8a000d0c70 2023-10-18T10:45:49.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.407+0000 7f8a055e1640 1 -- 172.21.15.89:0/541630413 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a000c8e80 con 0x7f8a000d0c70 2023-10-18T10:45:49.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.407+0000 7f89e77fe640 1 -- 172.21.15.89:0/541630413 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f89f4004b90 con 0x7f8a000d0c70 2023-10-18T10:45:49.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.407+0000 7f89e77fe640 1 --2- 172.21.15.89:0/541630413 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f89ec0656b0 0x7f89ec067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:49.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.407+0000 7f89e77fe640 1 -- 172.21.15.89:0/541630413 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5199+0+0 (secure 0 0 0) 0x7f89f4081bb0 con 0x7f8a000d0c70 2023-10-18T10:45:49.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.411+0000 7f89feffd640 1 --2- 172.21.15.89:0/541630413 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f89ec0656b0 0x7f89ec067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:49.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.411+0000 7f89feffd640 1 --2- 172.21.15.89:0/541630413 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f89ec0656b0 0x7f89ec067b70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f89f00ae000 tx=0x7f89f00a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:49.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.415+0000 7f89e77fe640 1 -- 172.21.15.89:0/541630413 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f89f404fa70 con 0x7f8a000d0c70 2023-10-18T10:45:49.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:49.571+0000 7f8a055e1640 1 -- 172.21.15.89:0/541630413 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8a0004dbe0 con 0x7f8a000d0c70 2023-10-18T10:45:50.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.243+0000 7f89e77fe640 1 -- 172.21.15.89:0/541630413 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v18) v1 ==== 173+0+0 (secure 0 0 0) 0x7f89f40180c0 con 0x7f8a000d0c70 2023-10-18T10:45:50.242 INFO:teuthology.orchestra.run.smithi089.stderr:enabled application 'rbd' on pool 'rbd' 2023-10-18T10:45:50.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.243+0000 7f8a055e1640 1 -- 172.21.15.89:0/541630413 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f89ec0656b0 msgr2=0x7f89ec067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:50.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.243+0000 7f8a055e1640 1 --2- 172.21.15.89:0/541630413 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f89ec0656b0 0x7f89ec067b70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f89f00ae000 tx=0x7f89f00a7000 comp rx=0 tx=0).stop 2023-10-18T10:45:50.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.243+0000 7f8a055e1640 1 -- 172.21.15.89:0/541630413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a000d0c70 msgr2=0x7f8a00157090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:50.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.243+0000 7f8a055e1640 1 --2- 172.21.15.89:0/541630413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a000d0c70 0x7f8a00157090 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f89f400c4a0 tx=0x7f89f400c960 comp rx=0 tx=0).stop 2023-10-18T10:45:50.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.243+0000 7f8a055e1640 1 -- 172.21.15.89:0/541630413 shutdown_connections 2023-10-18T10:45:50.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.243+0000 7f8a055e1640 1 --2- 172.21.15.89:0/541630413 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f89ec0656b0 0x7f89ec067b70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:50.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.243+0000 7f8a055e1640 1 --2- 172.21.15.89:0/541630413 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a001600b0 0x7f8a001575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:50.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.243+0000 7f8a055e1640 1 --2- 172.21.15.89:0/541630413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a000d0c70 0x7f8a00157090 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:50.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.243+0000 7f8a055e1640 1 --2- 172.21.15.89:0/541630413 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a00069900 0x7f8a00157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:50.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.243+0000 7f8a055e1640 1 -- 172.21.15.89:0/541630413 >> 172.21.15.89:0/541630413 conn(0x7f8a0009ba00 msgr2=0x7f8a000cd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:50.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.247+0000 7f8a055e1640 1 -- 172.21.15.89:0/541630413 shutdown_connections 2023-10-18T10:45:50.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.247+0000 7f8a055e1640 1 -- 172.21.15.89:0/541630413 wait complete. 2023-10-18T10:45:50.271 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-18T10:45:50.271 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-18T10:45:50.272 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-18T10:45:50.275 INFO:tasks.ceph.mds.a:Started 2023-10-18T10:45:50.276 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-18T10:45:50.276 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-18T10:45:50.279 INFO:tasks.ceph.mds.c:Started 2023-10-18T10:45:50.280 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-18T10:45:50.280 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-18T10:45:50.284 INFO:tasks.ceph.mds.b:Started 2023-10-18T10:45:50.284 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-18T10:45:50.285 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-18T10:45:50.290 INFO:tasks.ceph.mds.d:Started 2023-10-18T10:45:50.291 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-18T10:45:50.291 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:45:50.382 INFO:tasks.ceph.mds.d.smithi154.stdout:starting mds.d at 2023-10-18T10:45:50.383 INFO:tasks.ceph.mds.b.smithi154.stdout:starting mds.b at 2023-10-18T10:45:50.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.435+0000 7f35629ba640 1 Processor -- start 2023-10-18T10:45:50.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.435+0000 7f35629ba640 1 -- start start 2023-10-18T10:45:50.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.435+0000 7f35629ba640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f355c15c940 0x7f355c0d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:50.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.435+0000 7f35629ba640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f355c15da50 con 0x7f355c0d2f40 2023-10-18T10:45:50.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.435+0000 7f35629ba640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f355c15ce40 con 0x7f355c0d0e70 2023-10-18T10:45:50.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.435+0000 7f35629ba640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f355c15d120 con 0x7f355c15c940 2023-10-18T10:45:50.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.435+0000 7f355bfff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f355c15c940 0x7f355c0d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:50.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.435+0000 7f355b7fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f355c0d2f40 0x7f355c0d0760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37808/0 (socket says 172.21.15.89:37808) 2023-10-18T10:45:50.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.435+0000 7f355b7fe640 1 -- 172.21.15.89:0/4186443670 learned_addr learned my addr 172.21.15.89:0/4186443670 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:45:50.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.435+0000 7f355bfff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f355c15c940 0x7f355c0d2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41782/0 (socket says 172.21.15.89:41782) 2023-10-18T10:45:50.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.435+0000 7f355bfff640 1 -- 172.21.15.89:0/4186443670 >> v1:172.21.15.154:6789/0 conn(0x7f355c0d0e70 legacy=0x7f355c0cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:50.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.435+0000 7f355bfff640 1 -- 172.21.15.89:0/4186443670 >> v1:172.21.15.89:6789/0 conn(0x7f355c0d2f40 legacy=0x7f355c0d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:50.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.435+0000 7f355bfff640 1 -- 172.21.15.89:0/4186443670 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f355c15d400 con 0x7f355c15c940 2023-10-18T10:45:50.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.439+0000 7f355bfff640 1 --2- 172.21.15.89:0/4186443670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f355c15c940 0x7f355c0d2a00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f354c0097c0 tx=0x7f354c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a601b93a5e4af03c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:50.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.443+0000 7f355affd640 1 -- 172.21.15.89:0/4186443670 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c0a7070 con 0x7f355c15c940 2023-10-18T10:45:50.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.443+0000 7f355affd640 1 -- 172.21.15.89:0/4186443670 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354c0ab430 con 0x7f355c15c940 2023-10-18T10:45:50.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.443+0000 7f355affd640 1 -- 172.21.15.89:0/4186443670 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c0ab7c0 con 0x7f355c15c940 2023-10-18T10:45:50.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.443+0000 7f35629ba640 1 -- 172.21.15.89:0/4186443670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f355c15c940 msgr2=0x7f355c0d2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:50.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.443+0000 7f35629ba640 1 --2- 172.21.15.89:0/4186443670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f355c15c940 0x7f355c0d2a00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f354c0097c0 tx=0x7f354c01c410 comp rx=0 tx=0).stop 2023-10-18T10:45:50.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.443+0000 7f35629ba640 1 -- 172.21.15.89:0/4186443670 shutdown_connections 2023-10-18T10:45:50.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.443+0000 7f35629ba640 1 --2- 172.21.15.89:0/4186443670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f355c15c940 0x7f355c0d2a00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:50.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.443+0000 7f35629ba640 1 -- 172.21.15.89:0/4186443670 >> 172.21.15.89:0/4186443670 conn(0x7f355c089fd0 msgr2=0x7f355c09b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:50.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.443+0000 7f35629ba640 1 -- 172.21.15.89:0/4186443670 shutdown_connections 2023-10-18T10:45:50.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.443+0000 7f35629ba640 1 -- 172.21.15.89:0/4186443670 wait complete. 2023-10-18T10:45:50.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.443+0000 7f35629ba640 1 Processor -- start 2023-10-18T10:45:50.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.443+0000 7f35629ba640 1 -- start start 2023-10-18T10:45:50.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.447+0000 7f35629ba640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f355c0d0e70 0x7f355c1534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:50.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.447+0000 7f35629ba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f355c153dc0 0x7f355c0c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:50.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.447+0000 7f35629ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f355c0c79e0 0x7f355c0b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:50.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.447+0000 7f35629ba640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c0cc340 con 0x7f355c0d0e70 2023-10-18T10:45:50.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.447+0000 7f35629ba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c0cc060 con 0x7f355c0c79e0 2023-10-18T10:45:50.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.447+0000 7f35629ba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f355c0b53a0 con 0x7f355c153dc0 2023-10-18T10:45:50.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.447+0000 7f355bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f355c0d0e70 0x7f355c1534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:50.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.447+0000 7f355bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f355c0d0e70 0x7f355c1534f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41798/0 (socket says 172.21.15.89:41798) 2023-10-18T10:45:50.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.447+0000 7f355bfff640 1 -- 172.21.15.89:0/68767165 learned_addr learned my addr 172.21.15.89:0/68767165 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:45:50.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.447+0000 7f355b7fe640 1 --2- 172.21.15.89:0/68767165 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f355c153dc0 0x7f355c0c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:50.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.447+0000 7f3560f57640 1 --2- 172.21.15.89:0/68767165 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f355c0c79e0 0x7f355c0b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:50.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.447+0000 7f355bfff640 1 -- 172.21.15.89:0/68767165 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f355c153dc0 msgr2=0x7f355c0c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:50.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.447+0000 7f355bfff640 1 --2- 172.21.15.89:0/68767165 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f355c153dc0 0x7f355c0c74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:50.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.447+0000 7f355bfff640 1 -- 172.21.15.89:0/68767165 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f355c0c79e0 msgr2=0x7f355c0b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:50.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.447+0000 7f355bfff640 1 --2- 172.21.15.89:0/68767165 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f355c0c79e0 0x7f355c0b5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:50.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.447+0000 7f355bfff640 1 -- 172.21.15.89:0/68767165 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f354c009420 con 0x7f355c0d0e70 2023-10-18T10:45:50.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.447+0000 7f355bfff640 1 --2- 172.21.15.89:0/68767165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f355c0d0e70 0x7f355c1534f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f354c0ad040 tx=0x7f354c01ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:50.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.447+0000 7f3558ff9640 1 -- 172.21.15.89:0/68767165 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c0b5810 con 0x7f355c0d0e70 2023-10-18T10:45:50.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.447+0000 7f3558ff9640 1 -- 172.21.15.89:0/68767165 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354c0abe90 con 0x7f355c0d0e70 2023-10-18T10:45:50.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.447+0000 7f35629ba640 1 -- 172.21.15.89:0/68767165 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f355c0b5680 con 0x7f355c0d0e70 2023-10-18T10:45:50.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.447+0000 7f35629ba640 1 -- 172.21.15.89:0/68767165 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f355c0c7e60 con 0x7f355c0d0e70 2023-10-18T10:45:50.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.447+0000 7f3558ff9640 1 -- 172.21.15.89:0/68767165 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c0b5bf0 con 0x7f355c0d0e70 2023-10-18T10:45:50.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.451+0000 7f35629ba640 1 -- 172.21.15.89:0/68767165 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f355c0c9590 con 0x7f355c0d0e70 2023-10-18T10:45:50.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.451+0000 7f3558ff9640 1 -- 172.21.15.89:0/68767165 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f354c0b5d50 con 0x7f355c0d0e70 2023-10-18T10:45:50.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.451+0000 7f3558ff9640 1 --2- 172.21.15.89:0/68767165 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3548065680 0x7f3548067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:50.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.451+0000 7f3558ff9640 1 -- 172.21.15.89:0/68767165 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f354c11d750 con 0x7f355c0d0e70 2023-10-18T10:45:50.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.455+0000 7f355b7fe640 1 --2- 172.21.15.89:0/68767165 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3548065680 0x7f3548067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:50.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.455+0000 7f355b7fe640 1 --2- 172.21.15.89:0/68767165 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3548065680 0x7f3548067b40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f355c1f8910 tx=0x7f3550009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:50.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.459+0000 7f3558ff9640 1 -- 172.21.15.89:0/68767165 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f354c0eb600 con 0x7f355c0d0e70 2023-10-18T10:45:50.479 INFO:tasks.ceph.mds.c.smithi089.stdout:starting mds.c at 2023-10-18T10:45:50.480 INFO:tasks.ceph.mds.a.smithi089.stdout:starting mds.a at 2023-10-18T10:45:50.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.615+0000 7f35629ba640 1 -- 172.21.15.89:0/68767165 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f355c0b54e0 con 0x7f355c0d0e70 2023-10-18T10:45:50.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.615+0000 7f3558ff9640 1 -- 172.21.15.89:0/68767165 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+14625 (secure 0 0 0) 0x7f355c0b54e0 con 0x7f355c0d0e70 2023-10-18T10:45:50.615 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:45:50.615 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":18,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:45:50.234423+0000","last_up_change":"2023-10-18T10:45:42.164688+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6824","nonce":773078364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6827","nonce":773078364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6831","nonce":773078364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6829","nonce":773078364}]},"public_addr":"172.21.15.154:6824/773078364","cluster_addr":"172.21.15.154:6827/773078364","heartbeat_back_addr":"172.21.15.154:6831/773078364","heartbeat_front_addr":"172.21.15.154:6829/773078364","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:45:50.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.615+0000 7f35629ba640 1 -- 172.21.15.89:0/68767165 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3548065680 msgr2=0x7f3548067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:50.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.615+0000 7f35629ba640 1 --2- 172.21.15.89:0/68767165 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3548065680 0x7f3548067b40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f355c1f8910 tx=0x7f3550009040 comp rx=0 tx=0).stop 2023-10-18T10:45:50.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.615+0000 7f35629ba640 1 -- 172.21.15.89:0/68767165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f355c0d0e70 msgr2=0x7f355c1534f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:50.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.615+0000 7f35629ba640 1 --2- 172.21.15.89:0/68767165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f355c0d0e70 0x7f355c1534f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f354c0ad040 tx=0x7f354c01ac50 comp rx=0 tx=0).stop 2023-10-18T10:45:50.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.615+0000 7f35629ba640 1 -- 172.21.15.89:0/68767165 shutdown_connections 2023-10-18T10:45:50.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.615+0000 7f35629ba640 1 --2- 172.21.15.89:0/68767165 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3548065680 0x7f3548067b40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:50.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.615+0000 7f35629ba640 1 --2- 172.21.15.89:0/68767165 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f355c153dc0 0x7f355c0c74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:50.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.615+0000 7f35629ba640 1 --2- 172.21.15.89:0/68767165 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f355c0c79e0 0x7f355c0b5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:50.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.615+0000 7f35629ba640 1 --2- 172.21.15.89:0/68767165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f355c0d0e70 0x7f355c1534f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:50.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.615+0000 7f35629ba640 1 -- 172.21.15.89:0/68767165 >> 172.21.15.89:0/68767165 conn(0x7f355c089fd0 msgr2=0x7f355c0cf040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:50.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.615+0000 7f35629ba640 1 -- 172.21.15.89:0/68767165 shutdown_connections 2023-10-18T10:45:50.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.615+0000 7f35629ba640 1 -- 172.21.15.89:0/68767165 wait complete. 2023-10-18T10:45:50.636 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:45:43.114184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-18T10:45:48.730968+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T10:45:50.636 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:45:50.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.775+0000 7f7de28a1640 1 Processor -- start 2023-10-18T10:45:50.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.775+0000 7f7de28a1640 1 -- start start 2023-10-18T10:45:50.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.779+0000 7f7de28a1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ddc0d20f0 0x7f7ddc0d1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:50.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.779+0000 7f7de28a1640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ddc0d0070 con 0x7f7ddc15c330 2023-10-18T10:45:50.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.779+0000 7f7de28a1640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ddc0d0350 con 0x7f7ddc0b46c0 2023-10-18T10:45:50.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.779+0000 7f7de28a1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc0d1650 con 0x7f7ddc0d20f0 2023-10-18T10:45:50.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.779+0000 7f7de189f640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7ddc15c330 0x7f7ddc0b3fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37822/0 (socket says 172.21.15.89:37822) 2023-10-18T10:45:50.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.779+0000 7f7de189f640 1 -- 172.21.15.89:0/1311750073 learned_addr learned my addr 172.21.15.89:0/1311750073 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:45:50.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.779+0000 7f7de20a0640 1 --2- 172.21.15.89:0/1311750073 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ddc0d20f0 0x7f7ddc0d1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:50.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.779+0000 7f7de089d640 1 -- 172.21.15.89:0/1311750073 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 724158708 0 0) 0x7f7ddc0d0070 con 0x7f7ddc15c330 2023-10-18T10:45:50.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.779+0000 7f7de089d640 1 -- 172.21.15.89:0/1311750073 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7dc8003530 con 0x7f7ddc15c330 2023-10-18T10:45:50.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.779+0000 7f7de089d640 1 -- 172.21.15.89:0/1311750073 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4120859760 0 0) 0x7f7ddc0d0350 con 0x7f7ddc0b46c0 2023-10-18T10:45:50.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.779+0000 7f7de089d640 1 -- 172.21.15.89:0/1311750073 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ddc0d0070 con 0x7f7ddc0b46c0 2023-10-18T10:45:50.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.779+0000 7f7de20a0640 1 -- 172.21.15.89:0/1311750073 >> v1:172.21.15.154:6789/0 conn(0x7f7ddc0b46c0 legacy=0x7f7ddc0d19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:50.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.779+0000 7f7de20a0640 1 -- 172.21.15.89:0/1311750073 >> v1:172.21.15.89:6789/0 conn(0x7f7ddc15c330 legacy=0x7f7ddc0b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:50.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.779+0000 7f7de20a0640 1 -- 172.21.15.89:0/1311750073 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ddc0d0630 con 0x7f7ddc0d20f0 2023-10-18T10:45:50.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.783+0000 7f7de20a0640 1 --2- 172.21.15.89:0/1311750073 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ddc0d20f0 0x7f7ddc0d1110 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4009dc0 tx=0x7f7dd401c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53adbbb3974678d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:50.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.783+0000 7f7de089d640 1 -- 172.21.15.89:0/1311750073 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd40a5a00 con 0x7f7ddc0d20f0 2023-10-18T10:45:50.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.783+0000 7f7de089d640 1 -- 172.21.15.89:0/1311750073 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd40a5b60 con 0x7f7ddc0d20f0 2023-10-18T10:45:50.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.783+0000 7f7de089d640 1 -- 172.21.15.89:0/1311750073 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd40ad440 con 0x7f7ddc0d20f0 2023-10-18T10:45:50.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.783+0000 7f7de28a1640 1 -- 172.21.15.89:0/1311750073 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ddc0d20f0 msgr2=0x7f7ddc0d1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:50.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.783+0000 7f7de28a1640 1 --2- 172.21.15.89:0/1311750073 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ddc0d20f0 0x7f7ddc0d1110 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4009dc0 tx=0x7f7dd401c940 comp rx=0 tx=0).stop 2023-10-18T10:45:50.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.783+0000 7f7de28a1640 1 -- 172.21.15.89:0/1311750073 shutdown_connections 2023-10-18T10:45:50.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.783+0000 7f7de28a1640 1 --2- 172.21.15.89:0/1311750073 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ddc0d20f0 0x7f7ddc0d1110 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:50.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.783+0000 7f7de28a1640 1 -- 172.21.15.89:0/1311750073 >> 172.21.15.89:0/1311750073 conn(0x7f7ddc070320 msgr2=0x7f7ddc070730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:50.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.783+0000 7f7de28a1640 1 -- 172.21.15.89:0/1311750073 shutdown_connections 2023-10-18T10:45:50.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.783+0000 7f7de28a1640 1 -- 172.21.15.89:0/1311750073 wait complete. 2023-10-18T10:45:50.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.783+0000 7f7de28a1640 1 Processor -- start 2023-10-18T10:45:50.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.783+0000 7f7de28a1640 1 -- start start 2023-10-18T10:45:50.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.783+0000 7f7de28a1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ddc0d20f0 0x7f7ddc0c4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:50.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.783+0000 7f7de28a1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ddc15c330 0x7f7ddc152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:50.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.783+0000 7f7de28a1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ddc149100 0x7f7ddc145ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:50.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.783+0000 7f7de28a1640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc0b67e0 con 0x7f7ddc15c330 2023-10-18T10:45:50.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.783+0000 7f7de28a1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc149590 con 0x7f7ddc0d20f0 2023-10-18T10:45:50.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.783+0000 7f7de28a1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc0b6a30 con 0x7f7ddc149100 2023-10-18T10:45:50.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.787+0000 7f7de109e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ddc15c330 0x7f7ddc152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:50.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.787+0000 7f7de189f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ddc0d20f0 0x7f7ddc0c4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:50.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.787+0000 7f7de189f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ddc0d20f0 0x7f7ddc0c4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:50932/0 (socket says 172.21.15.89:50932) 2023-10-18T10:45:50.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.787+0000 7f7de189f640 1 -- 172.21.15.89:0/460513966 learned_addr learned my addr 172.21.15.89:0/460513966 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:45:50.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.787+0000 7f7de109e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ddc15c330 0x7f7ddc152250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41850/0 (socket says 172.21.15.89:41850) 2023-10-18T10:45:50.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.787+0000 7f7de20a0640 1 --2- 172.21.15.89:0/460513966 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ddc149100 0x7f7ddc145ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:50.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.787+0000 7f7de189f640 1 -- 172.21.15.89:0/460513966 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ddc149100 msgr2=0x7f7ddc145ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:50.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.787+0000 7f7de189f640 1 --2- 172.21.15.89:0/460513966 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ddc149100 0x7f7ddc145ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:50.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.787+0000 7f7de189f640 1 -- 172.21.15.89:0/460513966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ddc15c330 msgr2=0x7f7ddc152250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:50.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.787+0000 7f7de189f640 1 --2- 172.21.15.89:0/460513966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ddc15c330 0x7f7ddc152250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:50.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.787+0000 7f7de189f640 1 -- 172.21.15.89:0/460513966 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd4009a70 con 0x7f7ddc0d20f0 2023-10-18T10:45:50.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.787+0000 7f7de189f640 1 --2- 172.21.15.89:0/460513966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ddc0d20f0 0x7f7ddc0c4ad0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7dd800c420 tx=0x7f7dd800c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:50.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.787+0000 7f7dd27fc640 1 -- 172.21.15.89:0/460513966 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd8005cf0 con 0x7f7ddc0d20f0 2023-10-18T10:45:50.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.787+0000 7f7dd27fc640 1 -- 172.21.15.89:0/460513966 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd8005e50 con 0x7f7ddc0d20f0 2023-10-18T10:45:50.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.787+0000 7f7dd27fc640 1 -- 172.21.15.89:0/460513966 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd80072d0 con 0x7f7ddc0d20f0 2023-10-18T10:45:50.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.787+0000 7f7de28a1640 1 -- 172.21.15.89:0/460513966 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ddc0b6d10 con 0x7f7ddc0d20f0 2023-10-18T10:45:50.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.787+0000 7f7de28a1640 1 -- 172.21.15.89:0/460513966 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ddc148140 con 0x7f7ddc0d20f0 2023-10-18T10:45:50.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.787+0000 7f7de28a1640 1 -- 172.21.15.89:0/460513966 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ddc073b50 con 0x7f7ddc0d20f0 2023-10-18T10:45:50.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.787+0000 7f7dd27fc640 1 -- 172.21.15.89:0/460513966 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f7dd8007430 con 0x7f7ddc0d20f0 2023-10-18T10:45:50.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.791+0000 7f7dd27fc640 1 --2- 172.21.15.89:0/460513966 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7dc0065680 0x7f7dc0067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:50.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.791+0000 7f7dd27fc640 1 -- 172.21.15.89:0/460513966 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7dd80824f0 con 0x7f7ddc0d20f0 2023-10-18T10:45:50.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.791+0000 7f7de109e640 1 --2- 172.21.15.89:0/460513966 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7dc0065680 0x7f7dc0067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:50.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.791+0000 7f7de109e640 1 --2- 172.21.15.89:0/460513966 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7dc0065680 0x7f7dc0067b40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc14bda0 tx=0x7f7dcc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:50.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.795+0000 7f7dd27fc640 1 -- 172.21.15.89:0/460513966 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f7dd8050390 con 0x7f7ddc0d20f0 2023-10-18T10:45:50.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.939+0000 7f7de28a1640 1 -- 172.21.15.89:0/460513966 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7ddc151e70 con 0x7f7ddc0d20f0 2023-10-18T10:45:50.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.943+0000 7f7dd27fc640 1 -- 172.21.15.89:0/460513966 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7dd80539b0 con 0x7f7ddc0d20f0 2023-10-18T10:45:50.942 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-18T10:45:50.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.943+0000 7f7de28a1640 1 -- 172.21.15.89:0/460513966 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7dc0065680 msgr2=0x7f7dc0067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:50.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.943+0000 7f7de28a1640 1 --2- 172.21.15.89:0/460513966 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7dc0065680 0x7f7dc0067b40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc14bda0 tx=0x7f7dcc009040 comp rx=0 tx=0).stop 2023-10-18T10:45:50.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.943+0000 7f7de28a1640 1 -- 172.21.15.89:0/460513966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ddc0d20f0 msgr2=0x7f7ddc0c4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:50.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.943+0000 7f7de28a1640 1 --2- 172.21.15.89:0/460513966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ddc0d20f0 0x7f7ddc0c4ad0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7dd800c420 tx=0x7f7dd800c8e0 comp rx=0 tx=0).stop 2023-10-18T10:45:50.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.943+0000 7f7de28a1640 1 -- 172.21.15.89:0/460513966 shutdown_connections 2023-10-18T10:45:50.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.943+0000 7f7de28a1640 1 --2- 172.21.15.89:0/460513966 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7dc0065680 0x7f7dc0067b40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:50.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.943+0000 7f7de28a1640 1 --2- 172.21.15.89:0/460513966 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ddc149100 0x7f7ddc145ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:50.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.943+0000 7f7de28a1640 1 --2- 172.21.15.89:0/460513966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ddc15c330 0x7f7ddc152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:50.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.943+0000 7f7de28a1640 1 --2- 172.21.15.89:0/460513966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ddc0d20f0 0x7f7ddc0c4ad0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:50.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.943+0000 7f7de28a1640 1 -- 172.21.15.89:0/460513966 >> 172.21.15.89:0/460513966 conn(0x7f7ddc070320 msgr2=0x7f7ddc0ccad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:50.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.943+0000 7f7de28a1640 1 -- 172.21.15.89:0/460513966 shutdown_connections 2023-10-18T10:45:50.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:50.943+0000 7f7de28a1640 1 -- 172.21.15.89:0/460513966 wait complete. 2023-10-18T10:45:50.962 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-18T10:45:51.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.103+0000 7fcf145e1640 1 Processor -- start 2023-10-18T10:45:51.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.103+0000 7fcf145e1640 1 -- start start 2023-10-18T10:45:51.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.103+0000 7fcf145e1640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcf0c070600 0x7fcf0c06b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:51.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.103+0000 7fcf145e1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcf0c06bd10 0x7fcf0c0c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:51.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.103+0000 7fcf145e1640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf0c070b90 con 0x7fcf0c159490 2023-10-18T10:45:51.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.103+0000 7fcf145e1640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf0c06c1a0 con 0x7fcf0c070600 2023-10-18T10:45:51.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.103+0000 7fcf145e1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c070e70 con 0x7fcf0c06bd10 2023-10-18T10:45:51.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.103+0000 7fcf1237d640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fcf0c159490 0x7fcf0c15d8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37834/0 (socket says 172.21.15.89:37834) 2023-10-18T10:45:51.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.103+0000 7fcf1237d640 1 -- 172.21.15.89:0/2458957673 learned_addr learned my addr 172.21.15.89:0/2458957673 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:45:51.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.103+0000 7fcf11b7c640 1 --2- 172.21.15.89:0/2458957673 >> v2:172.21.15.89:3300/0 conn(0x7fcf0c070600 0x7fcf0c06b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:51.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.103+0000 7fcf12b7e640 1 --2- 172.21.15.89:0/2458957673 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcf0c06bd10 0x7fcf0c0c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:51.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.103+0000 7fcf11b7c640 1 -- 172.21.15.89:0/2458957673 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcf0c06bd10 msgr2=0x7fcf0c0c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:51.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.103+0000 7fcf11b7c640 1 --2- 172.21.15.89:0/2458957673 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcf0c06bd10 0x7fcf0c0c8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:51.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.103+0000 7fcf11b7c640 1 -- 172.21.15.89:0/2458957673 >> v1:172.21.15.89:6789/0 conn(0x7fcf0c159490 legacy=0x7fcf0c15d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:51.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.103+0000 7fcf11b7c640 1 -- 172.21.15.89:0/2458957673 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf0c071150 con 0x7fcf0c070600 2023-10-18T10:45:51.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.103+0000 7fcf12b7e640 1 --2- 172.21.15.89:0/2458957673 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcf0c06bd10 0x7fcf0c0c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:45:51.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.107+0000 7fcf11b7c640 1 --2- 172.21.15.89:0/2458957673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf0c070600 0x7fcf0c06b7d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcefc0097c0 tx=0x7fcefc01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c60f1dd2fca4f3a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:51.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.107+0000 7fcf1137b640 1 -- 172.21.15.89:0/2458957673 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcefc0ac620 con 0x7fcf0c070600 2023-10-18T10:45:51.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.107+0000 7fcf1137b640 1 -- 172.21.15.89:0/2458957673 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcefc0ac780 con 0x7fcf0c070600 2023-10-18T10:45:51.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.107+0000 7fcf1137b640 1 -- 172.21.15.89:0/2458957673 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcefc0acb10 con 0x7fcf0c070600 2023-10-18T10:45:51.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.107+0000 7fcf145e1640 1 -- 172.21.15.89:0/2458957673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf0c070600 msgr2=0x7fcf0c06b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:51.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.107+0000 7fcf145e1640 1 --2- 172.21.15.89:0/2458957673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf0c070600 0x7fcf0c06b7d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcefc0097c0 tx=0x7fcefc01c3c0 comp rx=0 tx=0).stop 2023-10-18T10:45:51.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.107+0000 7fcf145e1640 1 -- 172.21.15.89:0/2458957673 shutdown_connections 2023-10-18T10:45:51.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.107+0000 7fcf145e1640 1 --2- 172.21.15.89:0/2458957673 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcf0c06bd10 0x7fcf0c0c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:51.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.107+0000 7fcf145e1640 1 --2- 172.21.15.89:0/2458957673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf0c070600 0x7fcf0c06b7d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:51.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.107+0000 7fcf145e1640 1 -- 172.21.15.89:0/2458957673 >> 172.21.15.89:0/2458957673 conn(0x7fcf0c069f00 msgr2=0x7fcf0c0cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:51.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.107+0000 7fcf145e1640 1 -- 172.21.15.89:0/2458957673 shutdown_connections 2023-10-18T10:45:51.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.107+0000 7fcf145e1640 1 -- 172.21.15.89:0/2458957673 wait complete. 2023-10-18T10:45:51.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.111+0000 7fcf145e1640 1 Processor -- start 2023-10-18T10:45:51.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.111+0000 7fcf145e1640 1 -- start start 2023-10-18T10:45:51.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.111+0000 7fcf145e1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf0c06bd10 0x7fcf0c0c5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:51.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.111+0000 7fcf145e1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf0c070600 0x7fcf0c159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:51.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.111+0000 7fcf145e1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcf0c159490 0x7fcf0c0cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:51.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.111+0000 7fcf145e1640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c0c62c0 con 0x7fcf0c070600 2023-10-18T10:45:51.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.111+0000 7fcf145e1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c159db0 con 0x7fcf0c06bd10 2023-10-18T10:45:51.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.111+0000 7fcf145e1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c14b160 con 0x7fcf0c159490 2023-10-18T10:45:51.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.111+0000 7fcf11b7c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf0c070600 0x7fcf0c159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:51.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.111+0000 7fcf11b7c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf0c070600 0x7fcf0c159870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41864/0 (socket says 172.21.15.89:41864) 2023-10-18T10:45:51.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.111+0000 7fcf11b7c640 1 -- 172.21.15.89:0/1630722755 learned_addr learned my addr 172.21.15.89:0/1630722755 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:45:51.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.111+0000 7fcf12b7e640 1 --2- 172.21.15.89:0/1630722755 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcf0c159490 0x7fcf0c0cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:51.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.111+0000 7fcf1237d640 1 --2- 172.21.15.89:0/1630722755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf0c06bd10 0x7fcf0c0c5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:51.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.111+0000 7fcf11b7c640 1 -- 172.21.15.89:0/1630722755 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcf0c159490 msgr2=0x7fcf0c0cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:51.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.111+0000 7fcf11b7c640 1 --2- 172.21.15.89:0/1630722755 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcf0c159490 0x7fcf0c0cc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:51.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.111+0000 7fcf11b7c640 1 -- 172.21.15.89:0/1630722755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf0c06bd10 msgr2=0x7fcf0c0c5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:51.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.111+0000 7fcf11b7c640 1 --2- 172.21.15.89:0/1630722755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf0c06bd10 0x7fcf0c0c5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:51.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.111+0000 7fcf11b7c640 1 -- 172.21.15.89:0/1630722755 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcefc009420 con 0x7fcf0c070600 2023-10-18T10:45:51.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.111+0000 7fcf1237d640 1 --2- 172.21.15.89:0/1630722755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf0c06bd10 0x7fcf0c0c5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:45:51.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.111+0000 7fcf11b7c640 1 --2- 172.21.15.89:0/1630722755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf0c070600 0x7fcf0c159870 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcefc019ff0 tx=0x7fcefc019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:51.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.111+0000 7fcf02ffd640 1 -- 172.21.15.89:0/1630722755 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcefc0b5410 con 0x7fcf0c070600 2023-10-18T10:45:51.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.111+0000 7fcf02ffd640 1 -- 172.21.15.89:0/1630722755 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcefc0b5570 con 0x7fcf0c070600 2023-10-18T10:45:51.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.111+0000 7fcf02ffd640 1 -- 172.21.15.89:0/1630722755 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcefc0b5920 con 0x7fcf0c070600 2023-10-18T10:45:51.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.111+0000 7fcf145e1640 1 -- 172.21.15.89:0/1630722755 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf0c0c8230 con 0x7fcf0c070600 2023-10-18T10:45:51.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.111+0000 7fcf145e1640 1 -- 172.21.15.89:0/1630722755 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf0c15b520 con 0x7fcf0c070600 2023-10-18T10:45:51.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.111+0000 7fcf145e1640 1 -- 172.21.15.89:0/1630722755 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf0c15b710 con 0x7fcf0c070600 2023-10-18T10:45:51.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.115+0000 7fcf02ffd640 1 -- 172.21.15.89:0/1630722755 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fcefc0b3040 con 0x7fcf0c070600 2023-10-18T10:45:51.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.115+0000 7fcf02ffd640 1 --2- 172.21.15.89:0/1630722755 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fcef0065680 0x7fcef0067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:51.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.115+0000 7fcf02ffd640 1 -- 172.21.15.89:0/1630722755 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fcefc11ccc0 con 0x7fcf0c070600 2023-10-18T10:45:51.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.115+0000 7fcf1237d640 1 --2- 172.21.15.89:0/1630722755 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fcef0065680 0x7fcef0067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:51.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.115+0000 7fcf1237d640 1 --2- 172.21.15.89:0/1630722755 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fcef0065680 0x7fcef0067b40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcf08005890 tx=0x7fcf08009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:51.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.119+0000 7fcf02ffd640 1 -- 172.21.15.89:0/1630722755 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fcefc123080 con 0x7fcf0c070600 2023-10-18T10:45:51.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.267+0000 7fcf145e1640 1 -- 172.21.15.89:0/1630722755 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fcf0c04dbe0 con 0x7fcf0c070600 2023-10-18T10:45:51.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.267+0000 7fcf02ffd640 1 -- 172.21.15.89:0/1630722755 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7fcefc0b1020 con 0x7fcf0c070600 2023-10-18T10:45:51.269 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-10-18T10:45:51.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.271+0000 7fcf145e1640 1 -- 172.21.15.89:0/1630722755 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fcef0065680 msgr2=0x7fcef0067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:51.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.271+0000 7fcf145e1640 1 --2- 172.21.15.89:0/1630722755 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fcef0065680 0x7fcef0067b40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcf08005890 tx=0x7fcf08009040 comp rx=0 tx=0).stop 2023-10-18T10:45:51.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.271+0000 7fcf145e1640 1 -- 172.21.15.89:0/1630722755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf0c070600 msgr2=0x7fcf0c159870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:51.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.271+0000 7fcf145e1640 1 --2- 172.21.15.89:0/1630722755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf0c070600 0x7fcf0c159870 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcefc019ff0 tx=0x7fcefc019160 comp rx=0 tx=0).stop 2023-10-18T10:45:51.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.271+0000 7fcf145e1640 1 -- 172.21.15.89:0/1630722755 shutdown_connections 2023-10-18T10:45:51.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.271+0000 7fcf145e1640 1 --2- 172.21.15.89:0/1630722755 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fcef0065680 0x7fcef0067b40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:51.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.271+0000 7fcf145e1640 1 --2- 172.21.15.89:0/1630722755 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcf0c159490 0x7fcf0c0cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:51.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.271+0000 7fcf145e1640 1 --2- 172.21.15.89:0/1630722755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf0c070600 0x7fcf0c159870 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:51.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.271+0000 7fcf145e1640 1 --2- 172.21.15.89:0/1630722755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf0c06bd10 0x7fcf0c0c5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:51.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.271+0000 7fcf145e1640 1 -- 172.21.15.89:0/1630722755 >> 172.21.15.89:0/1630722755 conn(0x7fcf0c069f00 msgr2=0x7fcf0c0af540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:51.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.271+0000 7fcf145e1640 1 -- 172.21.15.89:0/1630722755 shutdown_connections 2023-10-18T10:45:51.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.271+0000 7fcf145e1640 1 -- 172.21.15.89:0/1630722755 wait complete. 2023-10-18T10:45:51.289 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:45:51.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.427+0000 7f0e0529c640 1 Processor -- start 2023-10-18T10:45:51.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.427+0000 7f0e0529c640 1 -- start start 2023-10-18T10:45:51.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.427+0000 7f0e0529c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0e000d0760 0x7f0e000d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:51.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.427+0000 7f0e0529c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0e000cd160 0x7f0e000cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:51.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.427+0000 7f0e0529c640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e000d1100 con 0x7f0e0015c940 2023-10-18T10:45:51.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.427+0000 7f0e0529c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e0015da50 con 0x7f0e000cd160 2023-10-18T10:45:51.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.427+0000 7f0e0529c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e000d3110 con 0x7f0e000d0760 2023-10-18T10:45:51.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.427+0000 7f0dfeffd640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0e0015c940 0x7f0e000d2a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37842/0 (socket says 172.21.15.89:37842) 2023-10-18T10:45:51.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.427+0000 7f0dfeffd640 1 -- 172.21.15.89:0/1115863302 learned_addr learned my addr 172.21.15.89:0/1115863302 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:45:51.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.427+0000 7f0dff7fe640 1 --2- 172.21.15.89:0/1115863302 >> v2:172.21.15.154:3300/0 conn(0x7f0e000cd160 0x7f0e000cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:51.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.427+0000 7f0dfe7fc640 1 --2- 172.21.15.89:0/1115863302 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0e000d0760 0x7f0e000d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:51.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.431+0000 7f0dfdffb640 1 -- 172.21.15.89:0/1115863302 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3597892813 0 0) 0x7f0e000d1100 con 0x7f0e0015c940 2023-10-18T10:45:51.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.431+0000 7f0dfdffb640 1 -- 172.21.15.89:0/1115863302 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0de8003530 con 0x7f0e0015c940 2023-10-18T10:45:51.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.431+0000 7f0dfdffb640 1 -- 172.21.15.89:0/1115863302 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1128587966 0 0) 0x7f0de8003530 con 0x7f0e0015c940 2023-10-18T10:45:51.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.431+0000 7f0dfdffb640 1 -- 172.21.15.89:0/1115863302 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0e000d0760 msgr2=0x7f0e000d0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:51.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.431+0000 7f0dfdffb640 1 --2- 172.21.15.89:0/1115863302 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0e000d0760 0x7f0e000d0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:51.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.431+0000 7f0dfdffb640 1 -- 172.21.15.89:0/1115863302 >> v2:172.21.15.154:3300/0 conn(0x7f0e000cd160 msgr2=0x7f0e000cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:51.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.431+0000 7f0dfdffb640 1 --2- 172.21.15.89:0/1115863302 >> v2:172.21.15.154:3300/0 conn(0x7f0e000cd160 0x7f0e000cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:51.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.431+0000 7f0dfdffb640 1 -- 172.21.15.89:0/1115863302 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0015ce40 con 0x7f0e0015c940 2023-10-18T10:45:51.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.431+0000 7f0dfdffb640 1 -- 172.21.15.89:0/1115863302 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7f0df0001a00 con 0x7f0e0015c940 2023-10-18T10:45:51.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.431+0000 7f0dfdffb640 1 -- 172.21.15.89:0/1115863302 >> v1:172.21.15.89:6789/0 conn(0x7f0e0015c940 legacy=0x7f0e000d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:51.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.431+0000 7f0dfdffb640 1 --2- 172.21.15.89:0/1115863302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0de8003980 0x7f0de8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:51.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.431+0000 7f0dfdffb640 1 --2- 172.21.15.89:0/1115863302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0de80047d0 0x7f0de8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:51.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.431+0000 7f0dfdffb640 1 --2- 172.21.15.89:0/1115863302 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0e000d0760 0x7f0de8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:51.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.431+0000 7f0dfdffb640 1 -- 172.21.15.89:0/1115863302 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0015da50 con 0x7f0de8003980 2023-10-18T10:45:51.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.431+0000 7f0dfdffb640 1 -- 172.21.15.89:0/1115863302 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e000d3110 con 0x7f0de80047d0 2023-10-18T10:45:51.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.431+0000 7f0dfdffb640 1 -- 172.21.15.89:0/1115863302 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0de800b990 con 0x7f0e000d0760 2023-10-18T10:45:51.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.431+0000 7f0dfeffd640 1 --2- 172.21.15.89:0/1115863302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0de8003980 0x7f0de8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:51.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.431+0000 7f0dfe7fc640 1 --2- 172.21.15.89:0/1115863302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0de80047d0 0x7f0de8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:51.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.431+0000 7f0dff7fe640 1 --2- 172.21.15.89:0/1115863302 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0e000d0760 0x7f0de8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:51.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.431+0000 7f0dfeffd640 1 -- 172.21.15.89:0/1115863302 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0e000d0760 msgr2=0x7f0de8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:51.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.431+0000 7f0dfeffd640 1 --2- 172.21.15.89:0/1115863302 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0e000d0760 0x7f0de8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:51.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.431+0000 7f0dfeffd640 1 -- 172.21.15.89:0/1115863302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0de80047d0 msgr2=0x7f0de8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:51.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.431+0000 7f0dfeffd640 1 --2- 172.21.15.89:0/1115863302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0de80047d0 0x7f0de8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:51.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.431+0000 7f0dfeffd640 1 -- 172.21.15.89:0/1115863302 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de8003530 con 0x7f0de8003980 2023-10-18T10:45:51.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.431+0000 7f0dfeffd640 1 --2- 172.21.15.89:0/1115863302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0de8003980 0x7f0de8003da0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0df000a860 tx=0x7f0df001d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c533fcce9b70906 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:51.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.435+0000 7f0dfdffb640 1 -- 172.21.15.89:0/1115863302 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df00ad630 con 0x7f0de8003980 2023-10-18T10:45:51.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.435+0000 7f0dfdffb640 1 -- 172.21.15.89:0/1115863302 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df00a8070 con 0x7f0de8003980 2023-10-18T10:45:51.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.435+0000 7f0dfdffb640 1 -- 172.21.15.89:0/1115863302 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df00ad9f0 con 0x7f0de8003980 2023-10-18T10:45:51.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.435+0000 7f0e0529c640 1 -- 172.21.15.89:0/1115863302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0de8003980 msgr2=0x7f0de8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:51.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.435+0000 7f0e0529c640 1 --2- 172.21.15.89:0/1115863302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0de8003980 0x7f0de8003da0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0df000a860 tx=0x7f0df001d330 comp rx=0 tx=0).stop 2023-10-18T10:45:51.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.435+0000 7f0dff7fe640 1 -- 172.21.15.89:0/1115863302 reap_dead start 2023-10-18T10:45:51.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.435+0000 7f0e0529c640 1 -- 172.21.15.89:0/1115863302 shutdown_connections 2023-10-18T10:45:51.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.435+0000 7f0e0529c640 1 -- 172.21.15.89:0/1115863302 >> 172.21.15.89:0/1115863302 conn(0x7f0e00089fd0 msgr2=0x7f0e0009b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:51.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.435+0000 7f0e0529c640 1 -- 172.21.15.89:0/1115863302 shutdown_connections 2023-10-18T10:45:51.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.435+0000 7f0e0529c640 1 -- 172.21.15.89:0/1115863302 wait complete. 2023-10-18T10:45:51.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.435+0000 7f0e0529c640 1 Processor -- start 2023-10-18T10:45:51.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.435+0000 7f0e0529c640 1 -- start start 2023-10-18T10:45:51.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.435+0000 7f0e0529c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0de8003980 0x7f0e0015f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:51.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.435+0000 7f0e0529c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e000c8780 0x7f0e000c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:51.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.435+0000 7f0e0529c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e000c6650 0x7f0e000c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:51.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.435+0000 7f0e0529c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e00156c90 con 0x7f0e000c8780 2023-10-18T10:45:51.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.435+0000 7f0e0529c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e00152860 con 0x7f0e000c6650 2023-10-18T10:45:51.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.435+0000 7f0e0529c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e000c7010 con 0x7f0de8003980 2023-10-18T10:45:51.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.435+0000 7f0dfe7fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e000c8780 0x7f0e000c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:51.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.439+0000 7f0dfe7fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e000c8780 0x7f0e000c8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41886/0 (socket says 172.21.15.89:41886) 2023-10-18T10:45:51.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.439+0000 7f0dfe7fc640 1 -- 172.21.15.89:0/2791066755 learned_addr learned my addr 172.21.15.89:0/2791066755 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:45:51.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.439+0000 7f0dfeffd640 1 --2- 172.21.15.89:0/2791066755 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0de8003980 0x7f0e0015f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:51.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.439+0000 7f0dff7fe640 1 --2- 172.21.15.89:0/2791066755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e000c6650 0x7f0e000c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:51.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.439+0000 7f0dfe7fc640 1 -- 172.21.15.89:0/2791066755 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0de8003980 msgr2=0x7f0e0015f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:51.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.439+0000 7f0dfe7fc640 1 --2- 172.21.15.89:0/2791066755 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0de8003980 0x7f0e0015f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:51.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.439+0000 7f0dfe7fc640 1 -- 172.21.15.89:0/2791066755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e000c6650 msgr2=0x7f0e000c6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:51.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.439+0000 7f0dfe7fc640 1 --2- 172.21.15.89:0/2791066755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e000c6650 0x7f0e000c6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:51.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.439+0000 7f0dfe7fc640 1 -- 172.21.15.89:0/2791066755 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0015fd80 con 0x7f0e000c8780 2023-10-18T10:45:51.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.439+0000 7f0dfe7fc640 1 --2- 172.21.15.89:0/2791066755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e000c8780 0x7f0e000c8c00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0df4011960 tx=0x7f0df4011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:51.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.439+0000 7f0de77fe640 1 -- 172.21.15.89:0/2791066755 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df400f9d0 con 0x7f0e000c8780 2023-10-18T10:45:51.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.439+0000 7f0de77fe640 1 -- 172.21.15.89:0/2791066755 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df400fb50 con 0x7f0e000c8780 2023-10-18T10:45:51.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.439+0000 7f0de77fe640 1 -- 172.21.15.89:0/2791066755 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df4014440 con 0x7f0e000c8780 2023-10-18T10:45:51.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.439+0000 7f0e0529c640 1 -- 172.21.15.89:0/2791066755 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e00156010 con 0x7f0e000c8780 2023-10-18T10:45:51.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.439+0000 7f0e0529c640 1 -- 172.21.15.89:0/2791066755 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e001565d0 con 0x7f0e000c8780 2023-10-18T10:45:51.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.439+0000 7f0e0529c640 1 -- 172.21.15.89:0/2791066755 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e00156dd0 con 0x7f0e000c8780 2023-10-18T10:45:51.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.439+0000 7f0de77fe640 1 -- 172.21.15.89:0/2791066755 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f0df40145a0 con 0x7f0e000c8780 2023-10-18T10:45:51.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.439+0000 7f0de77fe640 1 --2- 172.21.15.89:0/2791066755 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0dec065680 0x7f0dec067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:51.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.439+0000 7f0de77fe640 1 -- 172.21.15.89:0/2791066755 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0df40831c0 con 0x7f0e000c8780 2023-10-18T10:45:51.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.443+0000 7f0dfeffd640 1 --2- 172.21.15.89:0/2791066755 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0dec065680 0x7f0dec067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:51.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.443+0000 7f0dfeffd640 1 --2- 172.21.15.89:0/2791066755 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0dec065680 0x7f0dec067b40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0df00ad000 tx=0x7f0df0001e00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:51.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.443+0000 7f0de77fe640 1 -- 172.21.15.89:0/2791066755 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f0e00156dd0 con 0x7f0e000c8780 2023-10-18T10:45:51.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.627+0000 7f0e0529c640 1 -- 172.21.15.89:0/2791066755 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e00156dd0 con 0x7f0e000c8780 2023-10-18T10:45:51.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.631+0000 7f0de77fe640 1 -- 172.21.15.89:0/2791066755 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3697 (secure 0 0 0) 0x7f0df4022e90 con 0x7f0e000c8780 2023-10-18T10:45:51.630 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:45:51.630 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/1247973498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1247973498},{"type":"v1","addr":"172.21.15.154:6836","nonce":1247973498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4284,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/516057316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":516057316},{"type":"v1","addr":"172.21.15.154:6837","nonce":516057316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4299,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6834/3957051407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3957051407},{"type":"v1","addr":"172.21.15.89:6834","nonce":3957051407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4312,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2906113039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6833","nonce":2906113039},{"type":"v1","addr":"172.21.15.89:6835","nonce":2906113039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-10-18T10:45:51.631 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 3 2023-10-18T10:45:51.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.631+0000 7f0e0529c640 1 -- 172.21.15.89:0/2791066755 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0dec065680 msgr2=0x7f0dec067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:51.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.631+0000 7f0e0529c640 1 --2- 172.21.15.89:0/2791066755 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0dec065680 0x7f0dec067b40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0df00ad000 tx=0x7f0df0001e00 comp rx=0 tx=0).stop 2023-10-18T10:45:51.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.631+0000 7f0e0529c640 1 -- 172.21.15.89:0/2791066755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e000c8780 msgr2=0x7f0e000c8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:51.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.631+0000 7f0e0529c640 1 --2- 172.21.15.89:0/2791066755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e000c8780 0x7f0e000c8c00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0df4011960 tx=0x7f0df4011d60 comp rx=0 tx=0).stop 2023-10-18T10:45:51.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.631+0000 7f0e0529c640 1 -- 172.21.15.89:0/2791066755 shutdown_connections 2023-10-18T10:45:51.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.631+0000 7f0e0529c640 1 --2- 172.21.15.89:0/2791066755 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0dec065680 0x7f0dec067b40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:51.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.631+0000 7f0e0529c640 1 --2- 172.21.15.89:0/2791066755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e000c6650 0x7f0e000c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:51.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.631+0000 7f0e0529c640 1 --2- 172.21.15.89:0/2791066755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e000c8780 0x7f0e000c8c00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:51.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.631+0000 7f0e0529c640 1 --2- 172.21.15.89:0/2791066755 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0de8003980 0x7f0e0015f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:51.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.631+0000 7f0e0529c640 1 -- 172.21.15.89:0/2791066755 >> 172.21.15.89:0/2791066755 conn(0x7f0e00089fd0 msgr2=0x7f0e00162540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:51.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.631+0000 7f0e0529c640 1 -- 172.21.15.89:0/2791066755 shutdown_connections 2023-10-18T10:45:51.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.631+0000 7f0e0529c640 1 -- 172.21.15.89:0/2791066755 wait complete. 2023-10-18T10:45:51.650 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:45:51.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.787+0000 7fb2f71c8640 1 Processor -- start 2023-10-18T10:45:51.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.787+0000 7fb2f71c8640 1 -- start start 2023-10-18T10:45:51.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.791+0000 7fb2f71c8640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb2f0159490 0x7fb2f015d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:51.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.791+0000 7fb2f71c8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb2f006bee0 0x7fb2f00c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:51.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.791+0000 7fb2f71c8640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2f015ddf0 con 0x7fb2f0070600 2023-10-18T10:45:51.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.791+0000 7fb2f71c8640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2f0070b90 con 0x7fb2f0159490 2023-10-18T10:45:51.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.791+0000 7fb2f71c8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f0070e70 con 0x7fb2f006bee0 2023-10-18T10:45:51.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.791+0000 7fb2f4f64640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb2f0159490 0x7fb2f015d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:51.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.791+0000 7fb2f4f64640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb2f0159490 0x7fb2f015d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41894/0 (socket says 172.21.15.89:41894) 2023-10-18T10:45:51.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.791+0000 7fb2f4f64640 1 -- 172.21.15.89:0/3882521948 learned_addr learned my addr 172.21.15.89:0/3882521948 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:45:51.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.791+0000 7fb2f4f64640 1 -- 172.21.15.89:0/3882521948 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb2f006bee0 msgr2=0x7fb2f00c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:51.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.791+0000 7fb2f5765640 1 --2- 172.21.15.89:0/3882521948 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb2f006bee0 0x7fb2f00c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:51.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.791+0000 7fb2f4f64640 1 --2- 172.21.15.89:0/3882521948 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb2f006bee0 0x7fb2f00c8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:51.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.791+0000 7fb2f4f64640 1 -- 172.21.15.89:0/3882521948 >> v1:172.21.15.89:6789/0 conn(0x7fb2f0070600 legacy=0x7fb2f006b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:51.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.791+0000 7fb2f4f64640 1 -- 172.21.15.89:0/3882521948 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f0071150 con 0x7fb2f0159490 2023-10-18T10:45:51.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.791+0000 7fb2f5765640 1 --2- 172.21.15.89:0/3882521948 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb2f006bee0 0x7fb2f00c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:45:51.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.795+0000 7fb2f4f64640 1 --2- 172.21.15.89:0/3882521948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2f0159490 0x7fb2f015d8b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec009870 tx=0x7fb2ec01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a1d76f3209d0c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:51.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.795+0000 7fb2e77fe640 1 -- 172.21.15.89:0/3882521948 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec0ab680 con 0x7fb2f0159490 2023-10-18T10:45:51.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.795+0000 7fb2e77fe640 1 -- 172.21.15.89:0/3882521948 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2ec0ab7e0 con 0x7fb2f0159490 2023-10-18T10:45:51.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.795+0000 7fb2e77fe640 1 -- 172.21.15.89:0/3882521948 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec0abb70 con 0x7fb2f0159490 2023-10-18T10:45:51.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.795+0000 7fb2f71c8640 1 -- 172.21.15.89:0/3882521948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2f0159490 msgr2=0x7fb2f015d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:51.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.795+0000 7fb2f71c8640 1 --2- 172.21.15.89:0/3882521948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2f0159490 0x7fb2f015d8b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec009870 tx=0x7fb2ec01c480 comp rx=0 tx=0).stop 2023-10-18T10:45:51.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.795+0000 7fb2f71c8640 1 -- 172.21.15.89:0/3882521948 shutdown_connections 2023-10-18T10:45:51.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.795+0000 7fb2f71c8640 1 --2- 172.21.15.89:0/3882521948 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb2f006bee0 0x7fb2f00c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:51.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.795+0000 7fb2f71c8640 1 --2- 172.21.15.89:0/3882521948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2f0159490 0x7fb2f015d8b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:51.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.795+0000 7fb2f71c8640 1 -- 172.21.15.89:0/3882521948 >> 172.21.15.89:0/3882521948 conn(0x7fb2f0069f00 msgr2=0x7fb2f00cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:51.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.795+0000 7fb2f71c8640 1 -- 172.21.15.89:0/3882521948 shutdown_connections 2023-10-18T10:45:51.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.795+0000 7fb2f71c8640 1 -- 172.21.15.89:0/3882521948 wait complete. 2023-10-18T10:45:51.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.795+0000 7fb2f71c8640 1 Processor -- start 2023-10-18T10:45:51.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.795+0000 7fb2f71c8640 1 -- start start 2023-10-18T10:45:51.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.799+0000 7fb2f71c8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb2f006bee0 0x7fb2f014ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:51.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.799+0000 7fb2f71c8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2f0070600 0x7fb2f014b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:51.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.799+0000 7fb2f71c8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2f0159490 0x7fb2f00cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:51.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.799+0000 7fb2f71c8640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f01f6ce0 con 0x7fb2f0159490 2023-10-18T10:45:51.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.799+0000 7fb2f71c8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f00c28a0 con 0x7fb2f0070600 2023-10-18T10:45:51.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.799+0000 7fb2f71c8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f00c1f70 con 0x7fb2f006bee0 2023-10-18T10:45:51.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.799+0000 7fb2f5765640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2f0159490 0x7fb2f00cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:51.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.799+0000 7fb2f5765640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2f0159490 0x7fb2f00cbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41910/0 (socket says 172.21.15.89:41910) 2023-10-18T10:45:51.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.799+0000 7fb2f5765640 1 -- 172.21.15.89:0/865404698 learned_addr learned my addr 172.21.15.89:0/865404698 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:45:51.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.799+0000 7fb2f5765640 1 -- 172.21.15.89:0/865404698 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb2f006bee0 msgr2=0x7fb2f014ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:51.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.799+0000 7fb2f4f64640 1 --2- 172.21.15.89:0/865404698 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb2f006bee0 0x7fb2f014ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:51.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.799+0000 7fb2e7fff640 1 --2- 172.21.15.89:0/865404698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2f0070600 0x7fb2f014b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:51.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.799+0000 7fb2f5765640 1 --2- 172.21.15.89:0/865404698 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb2f006bee0 0x7fb2f014ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:51.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.799+0000 7fb2f5765640 1 -- 172.21.15.89:0/865404698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2f0070600 msgr2=0x7fb2f014b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:51.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.799+0000 7fb2f5765640 1 --2- 172.21.15.89:0/865404698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2f0070600 0x7fb2f014b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:51.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.799+0000 7fb2f5765640 1 -- 172.21.15.89:0/865404698 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ec0094d0 con 0x7fb2f0159490 2023-10-18T10:45:51.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.799+0000 7fb2f4f64640 1 --2- 172.21.15.89:0/865404698 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb2f006bee0 0x7fb2f014ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:45:51.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.799+0000 7fb2e7fff640 1 --2- 172.21.15.89:0/865404698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2f0070600 0x7fb2f014b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:45:51.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.799+0000 7fb2f5765640 1 --2- 172.21.15.89:0/865404698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2f0159490 0x7fb2f00cbd80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb2e800ce40 tx=0x7fb2e800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:51.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.799+0000 7fb2e57fa640 1 -- 172.21.15.89:0/865404698 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e800d840 con 0x7fb2f0159490 2023-10-18T10:45:51.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.799+0000 7fb2e57fa640 1 -- 172.21.15.89:0/865404698 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e800d9a0 con 0x7fb2f0159490 2023-10-18T10:45:51.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.799+0000 7fb2e57fa640 1 -- 172.21.15.89:0/865404698 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e800dd50 con 0x7fb2f0159490 2023-10-18T10:45:51.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.799+0000 7fb2f71c8640 1 -- 172.21.15.89:0/865404698 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2f00c2250 con 0x7fb2f0159490 2023-10-18T10:45:51.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.799+0000 7fb2f71c8640 1 -- 172.21.15.89:0/865404698 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2f00c2d30 con 0x7fb2f0159490 2023-10-18T10:45:51.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.799+0000 7fb2f71c8640 1 -- 172.21.15.89:0/865404698 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2f004dbe0 con 0x7fb2f0159490 2023-10-18T10:45:51.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.799+0000 7fb2e57fa640 1 -- 172.21.15.89:0/865404698 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fb2e801d360 con 0x7fb2f0159490 2023-10-18T10:45:51.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.803+0000 7fb2e57fa640 1 --2- 172.21.15.89:0/865404698 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb2dc065680 0x7fb2dc067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:51.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.803+0000 7fb2e57fa640 1 -- 172.21.15.89:0/865404698 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb2e80833e0 con 0x7fb2f0159490 2023-10-18T10:45:51.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.803+0000 7fb2f4f64640 1 --2- 172.21.15.89:0/865404698 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb2dc065680 0x7fb2dc067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:51.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.803+0000 7fb2f4f64640 1 --2- 172.21.15.89:0/865404698 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb2dc065680 0x7fb2dc067b40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0af000 tx=0x7fb2ec008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:51.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.807+0000 7fb2e57fa640 1 -- 172.21.15.89:0/865404698 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fb2e8051280 con 0x7fb2f0159490 2023-10-18T10:45:51.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.955+0000 7fb2f71c8640 1 -- 172.21.15.89:0/865404698 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2f00c29e0 con 0x7fb2f0159490 2023-10-18T10:45:51.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.955+0000 7fb2e57fa640 1 -- 172.21.15.89:0/865404698 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+14612 (secure 0 0 0) 0x7fb2e80548a0 con 0x7fb2f0159490 2023-10-18T10:45:51.955 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:45:51.956 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":19,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:45:51.242199+0000","last_up_change":"2023-10-18T10:45:42.164688+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6824","nonce":773078364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6827","nonce":773078364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6831","nonce":773078364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6829","nonce":773078364}]},"public_addr":"172.21.15.154:6824/773078364","cluster_addr":"172.21.15.154:6827/773078364","heartbeat_back_addr":"172.21.15.154:6831/773078364","heartbeat_front_addr":"172.21.15.154:6829/773078364","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:45:51.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.955+0000 7fb2f71c8640 1 -- 172.21.15.89:0/865404698 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb2dc065680 msgr2=0x7fb2dc067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:51.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.955+0000 7fb2f71c8640 1 --2- 172.21.15.89:0/865404698 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb2dc065680 0x7fb2dc067b40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0af000 tx=0x7fb2ec008040 comp rx=0 tx=0).stop 2023-10-18T10:45:51.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.955+0000 7fb2f71c8640 1 -- 172.21.15.89:0/865404698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2f0159490 msgr2=0x7fb2f00cbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:51.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.955+0000 7fb2f71c8640 1 --2- 172.21.15.89:0/865404698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2f0159490 0x7fb2f00cbd80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb2e800ce40 tx=0x7fb2e800a710 comp rx=0 tx=0).stop 2023-10-18T10:45:51.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.959+0000 7fb2f71c8640 1 -- 172.21.15.89:0/865404698 shutdown_connections 2023-10-18T10:45:51.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.959+0000 7fb2f71c8640 1 --2- 172.21.15.89:0/865404698 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb2dc065680 0x7fb2dc067b40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:51.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.959+0000 7fb2f71c8640 1 --2- 172.21.15.89:0/865404698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2f0159490 0x7fb2f00cbd80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:51.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.959+0000 7fb2f71c8640 1 --2- 172.21.15.89:0/865404698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2f0070600 0x7fb2f014b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:51.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.959+0000 7fb2f71c8640 1 --2- 172.21.15.89:0/865404698 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb2f006bee0 0x7fb2f014ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:51.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.959+0000 7fb2f71c8640 1 -- 172.21.15.89:0/865404698 >> 172.21.15.89:0/865404698 conn(0x7fb2f0069f00 msgr2=0x7fb2f006f750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:51.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.959+0000 7fb2f71c8640 1 -- 172.21.15.89:0/865404698 shutdown_connections 2023-10-18T10:45:51.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:51.959+0000 7fb2f71c8640 1 -- 172.21.15.89:0/865404698 wait complete. 2023-10-18T10:45:51.977 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:45:43.114184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-18T10:45:48.730968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T10:45:51.977 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:45:52.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.115+0000 7faa33703640 1 Processor -- start 2023-10-18T10:45:52.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.115+0000 7faa33703640 1 -- start start 2023-10-18T10:45:52.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.119+0000 7faa33703640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7faa2c159490 0x7faa2c15d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:52.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.119+0000 7faa33703640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa2c070600 0x7faa2c06b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:52.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.119+0000 7faa33703640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faa2c06bd10 0x7faa2c0cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:52.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.119+0000 7faa33703640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa2c06c1a0 con 0x7faa2c06bd10 2023-10-18T10:45:52.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.119+0000 7faa33703640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa2c15ddf0 con 0x7faa2c159490 2023-10-18T10:45:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.119+0000 7faa33703640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c070b90 con 0x7faa2c070600 2023-10-18T10:45:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.119+0000 7faa31ca0640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faa2c06bd10 0x7faa2c0cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.119+0000 7faa31ca0640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faa2c06bd10 0x7faa2c0cbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41924/0 (socket says 172.21.15.89:41924) 2023-10-18T10:45:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.119+0000 7faa31ca0640 1 -- 172.21.15.89:0/4114750102 learned_addr learned my addr 172.21.15.89:0/4114750102 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:45:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.119+0000 7faa30c9e640 1 --2- 172.21.15.89:0/4114750102 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa2c070600 0x7faa2c06b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.119+0000 7faa3149f640 1 --2- 172.21.15.89:0/4114750102 >> v2:172.21.15.154:3300/0 conn(0x7faa2c159490 0x7faa2c15d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.119+0000 7faa31ca0640 1 -- 172.21.15.89:0/4114750102 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa2c070600 msgr2=0x7faa2c06b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.119+0000 7faa31ca0640 1 --2- 172.21.15.89:0/4114750102 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa2c070600 0x7faa2c06b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.119+0000 7faa31ca0640 1 -- 172.21.15.89:0/4114750102 >> v2:172.21.15.154:3300/0 conn(0x7faa2c159490 msgr2=0x7faa2c15d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.119+0000 7faa31ca0640 1 --2- 172.21.15.89:0/4114750102 >> v2:172.21.15.154:3300/0 conn(0x7faa2c159490 0x7faa2c15d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.119+0000 7faa31ca0640 1 -- 172.21.15.89:0/4114750102 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2c070e70 con 0x7faa2c06bd10 2023-10-18T10:45:52.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.119+0000 7faa3149f640 1 --2- 172.21.15.89:0/4114750102 >> v2:172.21.15.154:3300/0 conn(0x7faa2c159490 0x7faa2c15d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:45:52.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.123+0000 7faa31ca0640 1 --2- 172.21.15.89:0/4114750102 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa2c06bd10 0x7faa2c0cbee0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faa2c06c560 tx=0x7faa2401c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea1f7917c81c81a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:52.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.123+0000 7faa1ffff640 1 -- 172.21.15.89:0/4114750102 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa240a64e0 con 0x7faa2c06bd10 2023-10-18T10:45:52.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.123+0000 7faa1ffff640 1 -- 172.21.15.89:0/4114750102 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa240a6640 con 0x7faa2c06bd10 2023-10-18T10:45:52.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.123+0000 7faa1ffff640 1 -- 172.21.15.89:0/4114750102 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa240a69d0 con 0x7faa2c06bd10 2023-10-18T10:45:52.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.123+0000 7faa33703640 1 -- 172.21.15.89:0/4114750102 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa2c06bd10 msgr2=0x7faa2c0cbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:52.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.123+0000 7faa33703640 1 --2- 172.21.15.89:0/4114750102 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa2c06bd10 0x7faa2c0cbee0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faa2c06c560 tx=0x7faa2401c650 comp rx=0 tx=0).stop 2023-10-18T10:45:52.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.123+0000 7faa33703640 1 -- 172.21.15.89:0/4114750102 shutdown_connections 2023-10-18T10:45:52.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.123+0000 7faa33703640 1 --2- 172.21.15.89:0/4114750102 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa2c06bd10 0x7faa2c0cbee0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:52.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.123+0000 7faa33703640 1 --2- 172.21.15.89:0/4114750102 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa2c070600 0x7faa2c06b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:52.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.123+0000 7faa33703640 1 --2- 172.21.15.89:0/4114750102 >> v2:172.21.15.154:3300/0 conn(0x7faa2c159490 0x7faa2c15d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:52.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.123+0000 7faa33703640 1 -- 172.21.15.89:0/4114750102 >> 172.21.15.89:0/4114750102 conn(0x7faa2c069f00 msgr2=0x7faa2c0cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:52.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.123+0000 7faa33703640 1 -- 172.21.15.89:0/4114750102 shutdown_connections 2023-10-18T10:45:52.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.123+0000 7faa33703640 1 -- 172.21.15.89:0/4114750102 wait complete. 2023-10-18T10:45:52.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.123+0000 7faa33703640 1 Processor -- start 2023-10-18T10:45:52.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.123+0000 7faa33703640 1 -- start start 2023-10-18T10:45:52.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.127+0000 7faa33703640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa2c06bd10 0x7faa2c0c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:52.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.127+0000 7faa33703640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa2c070600 0x7faa2c1f8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:52.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.127+0000 7faa33703640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa2c159490 0x7faa2c15adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:52.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.127+0000 7faa33703640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c152090 con 0x7faa2c06bd10 2023-10-18T10:45:52.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.127+0000 7faa33703640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c0c4df0 con 0x7faa2c070600 2023-10-18T10:45:52.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.127+0000 7faa33703640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c0c38c0 con 0x7faa2c159490 2023-10-18T10:45:52.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.127+0000 7faa3149f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa2c06bd10 0x7faa2c0c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:52.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.127+0000 7faa3149f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa2c06bd10 0x7faa2c0c48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41932/0 (socket says 172.21.15.89:41932) 2023-10-18T10:45:52.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.127+0000 7faa3149f640 1 -- 172.21.15.89:0/3767998132 learned_addr learned my addr 172.21.15.89:0/3767998132 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:45:52.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.127+0000 7faa31ca0640 1 --2- 172.21.15.89:0/3767998132 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa2c159490 0x7faa2c15adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:52.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.127+0000 7faa30c9e640 1 --2- 172.21.15.89:0/3767998132 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa2c070600 0x7faa2c1f8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:52.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.127+0000 7faa3149f640 1 -- 172.21.15.89:0/3767998132 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa2c159490 msgr2=0x7faa2c15adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:52.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.127+0000 7faa3149f640 1 --2- 172.21.15.89:0/3767998132 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa2c159490 0x7faa2c15adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:52.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.127+0000 7faa3149f640 1 -- 172.21.15.89:0/3767998132 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa2c070600 msgr2=0x7faa2c1f8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:52.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.127+0000 7faa3149f640 1 --2- 172.21.15.89:0/3767998132 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa2c070600 0x7faa2c1f8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:52.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.127+0000 7faa3149f640 1 -- 172.21.15.89:0/3767998132 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa24009b30 con 0x7faa2c06bd10 2023-10-18T10:45:52.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.127+0000 7faa30c9e640 1 --2- 172.21.15.89:0/3767998132 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa2c070600 0x7faa2c1f8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:45:52.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.127+0000 7faa31ca0640 1 --2- 172.21.15.89:0/3767998132 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa2c159490 0x7faa2c15adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:45:52.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.127+0000 7faa3149f640 1 --2- 172.21.15.89:0/3767998132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa2c06bd10 0x7faa2c0c48b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faa2000c850 tx=0x7faa2000cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:52.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.127+0000 7faa1dffb640 1 -- 172.21.15.89:0/3767998132 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa20007830 con 0x7faa2c06bd10 2023-10-18T10:45:52.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.127+0000 7faa1dffb640 1 -- 172.21.15.89:0/3767998132 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa20007990 con 0x7faa2c06bd10 2023-10-18T10:45:52.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.127+0000 7faa33703640 1 -- 172.21.15.89:0/3767998132 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa2c1f90e0 con 0x7faa2c06bd10 2023-10-18T10:45:52.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.127+0000 7faa1dffb640 1 -- 172.21.15.89:0/3767998132 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa20007d60 con 0x7faa2c06bd10 2023-10-18T10:45:52.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.127+0000 7faa33703640 1 -- 172.21.15.89:0/3767998132 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa2c0c40b0 con 0x7faa2c06bd10 2023-10-18T10:45:52.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.127+0000 7faa33703640 1 -- 172.21.15.89:0/3767998132 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa2c0c3c00 con 0x7faa2c06bd10 2023-10-18T10:45:52.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.127+0000 7faa1dffb640 1 -- 172.21.15.89:0/3767998132 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7faa200113f0 con 0x7faa2c06bd10 2023-10-18T10:45:52.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.131+0000 7faa1dffb640 1 --2- 172.21.15.89:0/3767998132 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7faa10065680 0x7faa10067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:52.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.131+0000 7faa1dffb640 1 -- 172.21.15.89:0/3767998132 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7faa20082c10 con 0x7faa2c06bd10 2023-10-18T10:45:52.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.131+0000 7faa30c9e640 1 --2- 172.21.15.89:0/3767998132 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7faa10065680 0x7faa10067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:52.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.131+0000 7faa30c9e640 1 --2- 172.21.15.89:0/3767998132 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7faa10065680 0x7faa10067b40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7faa2c1548e0 tx=0x7faa28008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:52.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.135+0000 7faa1dffb640 1 -- 172.21.15.89:0/3767998132 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7faa20050980 con 0x7faa2c06bd10 2023-10-18T10:45:52.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.283+0000 7faa33703640 1 -- 172.21.15.89:0/3767998132 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faa2c04dbe0 con 0x7faa2c06bd10 2023-10-18T10:45:52.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.283+0000 7faa1dffb640 1 -- 172.21.15.89:0/3767998132 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7faa20053fa0 con 0x7faa2c06bd10 2023-10-18T10:45:52.284 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-18T10:45:52.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.283+0000 7faa33703640 1 -- 172.21.15.89:0/3767998132 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7faa10065680 msgr2=0x7faa10067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:52.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.283+0000 7faa33703640 1 --2- 172.21.15.89:0/3767998132 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7faa10065680 0x7faa10067b40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7faa2c1548e0 tx=0x7faa28008040 comp rx=0 tx=0).stop 2023-10-18T10:45:52.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.283+0000 7faa33703640 1 -- 172.21.15.89:0/3767998132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa2c06bd10 msgr2=0x7faa2c0c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:52.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.283+0000 7faa33703640 1 --2- 172.21.15.89:0/3767998132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa2c06bd10 0x7faa2c0c48b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faa2000c850 tx=0x7faa2000cd10 comp rx=0 tx=0).stop 2023-10-18T10:45:52.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.283+0000 7faa33703640 1 -- 172.21.15.89:0/3767998132 shutdown_connections 2023-10-18T10:45:52.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.283+0000 7faa33703640 1 --2- 172.21.15.89:0/3767998132 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7faa10065680 0x7faa10067b40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:52.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.283+0000 7faa33703640 1 --2- 172.21.15.89:0/3767998132 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa2c159490 0x7faa2c15adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:52.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.283+0000 7faa33703640 1 --2- 172.21.15.89:0/3767998132 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa2c070600 0x7faa2c1f8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:52.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.283+0000 7faa33703640 1 --2- 172.21.15.89:0/3767998132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa2c06bd10 0x7faa2c0c48b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:52.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.283+0000 7faa33703640 1 -- 172.21.15.89:0/3767998132 >> 172.21.15.89:0/3767998132 conn(0x7faa2c069f00 msgr2=0x7faa2c0ac260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:52.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.283+0000 7faa33703640 1 -- 172.21.15.89:0/3767998132 shutdown_connections 2023-10-18T10:45:52.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.283+0000 7faa33703640 1 -- 172.21.15.89:0/3767998132 wait complete. 2023-10-18T10:45:52.304 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-18T10:45:52.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.443+0000 7f0d406cb640 1 Processor -- start 2023-10-18T10:45:52.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.443+0000 7f0d406cb640 1 -- start start 2023-10-18T10:45:52.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.443+0000 7f0d406cb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d3815c660 0x7f0d380cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:52.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.443+0000 7f0d406cb640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0d380d20c0 0x7f0d380cfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:52.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.443+0000 7f0d406cb640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0d380d1a90 0x7f0d380d0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:52.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.443+0000 7f0d406cb640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d3815ca40 con 0x7f0d380d1a90 2023-10-18T10:45:52.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.443+0000 7f0d406cb640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d380d3490 con 0x7f0d380d20c0 2023-10-18T10:45:52.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.443+0000 7f0d406cb640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d380d2630 con 0x7f0d3815c660 2023-10-18T10:45:52.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.443+0000 7f0d3ec68640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0d380d1a90 0x7f0d380d0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:52.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.447+0000 7f0d3ec68640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0d380d1a90 0x7f0d380d0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48724/0 (socket says 172.21.15.89:48724) 2023-10-18T10:45:52.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.447+0000 7f0d3ec68640 1 -- 172.21.15.89:0/2460858652 learned_addr learned my addr 172.21.15.89:0/2460858652 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:45:52.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.447+0000 7f0d3e467640 1 --2- 172.21.15.89:0/2460858652 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d3815c660 0x7f0d380cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:52.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.447+0000 7f0d3dc66640 1 --2- 172.21.15.89:0/2460858652 >> v2:172.21.15.154:3300/0 conn(0x7f0d380d20c0 0x7f0d380cfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:52.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.447+0000 7f0d3ec68640 1 -- 172.21.15.89:0/2460858652 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d3815c660 msgr2=0x7f0d380cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:52.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.447+0000 7f0d3ec68640 1 --2- 172.21.15.89:0/2460858652 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d3815c660 0x7f0d380cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:52.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.447+0000 7f0d3ec68640 1 -- 172.21.15.89:0/2460858652 >> v2:172.21.15.154:3300/0 conn(0x7f0d380d20c0 msgr2=0x7f0d380cfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:52.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.447+0000 7f0d3ec68640 1 --2- 172.21.15.89:0/2460858652 >> v2:172.21.15.154:3300/0 conn(0x7f0d380d20c0 0x7f0d380cfb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:52.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.447+0000 7f0d3ec68640 1 -- 172.21.15.89:0/2460858652 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d380d2910 con 0x7f0d380d1a90 2023-10-18T10:45:52.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.447+0000 7f0d3ec68640 1 --2- 172.21.15.89:0/2460858652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d380d1a90 0x7f0d380d0830 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0d3806b3b0 tx=0x7f0d3001c1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d715bc204f4aa6 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:52.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.447+0000 7f0d3d465640 1 -- 172.21.15.89:0/2460858652 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d300ab600 con 0x7f0d380d1a90 2023-10-18T10:45:52.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.447+0000 7f0d3d465640 1 -- 172.21.15.89:0/2460858652 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d300a7050 con 0x7f0d380d1a90 2023-10-18T10:45:52.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.447+0000 7f0d3d465640 1 -- 172.21.15.89:0/2460858652 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d300ab980 con 0x7f0d380d1a90 2023-10-18T10:45:52.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.451+0000 7f0d406cb640 1 -- 172.21.15.89:0/2460858652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d380d1a90 msgr2=0x7f0d380d0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:52.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.451+0000 7f0d406cb640 1 --2- 172.21.15.89:0/2460858652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d380d1a90 0x7f0d380d0830 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0d3806b3b0 tx=0x7f0d3001c1b0 comp rx=0 tx=0).stop 2023-10-18T10:45:52.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.451+0000 7f0d406cb640 1 -- 172.21.15.89:0/2460858652 shutdown_connections 2023-10-18T10:45:52.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.451+0000 7f0d406cb640 1 --2- 172.21.15.89:0/2460858652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d380d1a90 0x7f0d380d0830 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:52.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.451+0000 7f0d406cb640 1 --2- 172.21.15.89:0/2460858652 >> v2:172.21.15.154:3300/0 conn(0x7f0d380d20c0 0x7f0d380cfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:52.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.451+0000 7f0d406cb640 1 --2- 172.21.15.89:0/2460858652 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d3815c660 0x7f0d380cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:52.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.451+0000 7f0d406cb640 1 -- 172.21.15.89:0/2460858652 >> 172.21.15.89:0/2460858652 conn(0x7f0d38069d00 msgr2=0x7f0d3806a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:52.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.451+0000 7f0d406cb640 1 -- 172.21.15.89:0/2460858652 shutdown_connections 2023-10-18T10:45:52.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.451+0000 7f0d406cb640 1 -- 172.21.15.89:0/2460858652 wait complete. 2023-10-18T10:45:52.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.451+0000 7f0d406cb640 1 Processor -- start 2023-10-18T10:45:52.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.451+0000 7f0d406cb640 1 -- start start 2023-10-18T10:45:52.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.451+0000 7f0d406cb640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d380d1a90 0x7f0d380c4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:52.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.451+0000 7f0d406cb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d380d20c0 0x7f0d3815cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:52.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.451+0000 7f0d406cb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d3815c660 0x7f0d3815d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:52.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.451+0000 7f0d406cb640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3815f020 con 0x7f0d380d1a90 2023-10-18T10:45:52.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.451+0000 7f0d406cb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3815f300 con 0x7f0d380d20c0 2023-10-18T10:45:52.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.451+0000 7f0d406cb640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3815f5e0 con 0x7f0d3815c660 2023-10-18T10:45:52.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.451+0000 7f0d3e467640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d380d1a90 0x7f0d380c4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:52.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.451+0000 7f0d3e467640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d380d1a90 0x7f0d380c4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48732/0 (socket says 172.21.15.89:48732) 2023-10-18T10:45:52.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.451+0000 7f0d3e467640 1 -- 172.21.15.89:0/3288248436 learned_addr learned my addr 172.21.15.89:0/3288248436 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:45:52.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.451+0000 7f0d3e467640 1 -- 172.21.15.89:0/3288248436 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d3815c660 msgr2=0x7f0d3815d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:52.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.451+0000 7f0d3e467640 1 --2- 172.21.15.89:0/3288248436 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d3815c660 0x7f0d3815d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:52.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.451+0000 7f0d3e467640 1 -- 172.21.15.89:0/3288248436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d380d20c0 msgr2=0x7f0d3815cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:52.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.451+0000 7f0d3e467640 1 --2- 172.21.15.89:0/3288248436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d380d20c0 0x7f0d3815cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:52.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.451+0000 7f0d3e467640 1 -- 172.21.15.89:0/3288248436 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d300094d0 con 0x7f0d380d1a90 2023-10-18T10:45:52.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.451+0000 7f0d3e467640 1 --2- 172.21.15.89:0/3288248436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d380d1a90 0x7f0d380c4700 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400efc0 tx=0x7f0d3400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:52.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.455+0000 7f0d2effd640 1 -- 172.21.15.89:0/3288248436 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d34009b80 con 0x7f0d380d1a90 2023-10-18T10:45:52.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.455+0000 7f0d2effd640 1 -- 172.21.15.89:0/3288248436 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d3400f040 con 0x7f0d380d1a90 2023-10-18T10:45:52.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.455+0000 7f0d406cb640 1 -- 172.21.15.89:0/3288248436 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d3815f8c0 con 0x7f0d380d1a90 2023-10-18T10:45:52.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.455+0000 7f0d2effd640 1 -- 172.21.15.89:0/3288248436 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d340146a0 con 0x7f0d380d1a90 2023-10-18T10:45:52.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.455+0000 7f0d406cb640 1 -- 172.21.15.89:0/3288248436 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d380c4c40 con 0x7f0d380d1a90 2023-10-18T10:45:52.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.455+0000 7f0d406cb640 1 -- 172.21.15.89:0/3288248436 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d3804dbe0 con 0x7f0d380d1a90 2023-10-18T10:45:52.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.455+0000 7f0d2effd640 1 -- 172.21.15.89:0/3288248436 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f0d34014800 con 0x7f0d380d1a90 2023-10-18T10:45:52.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.455+0000 7f0d2effd640 1 --2- 172.21.15.89:0/3288248436 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0d1c065680 0x7f0d1c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:52.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.455+0000 7f0d2effd640 1 -- 172.21.15.89:0/3288248436 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0d340835e0 con 0x7f0d380d1a90 2023-10-18T10:45:52.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.455+0000 7f0d3dc66640 1 --2- 172.21.15.89:0/3288248436 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0d1c065680 0x7f0d1c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:52.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.459+0000 7f0d3dc66640 1 --2- 172.21.15.89:0/3288248436 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0d1c065680 0x7f0d1c067b40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0d38154cd0 tx=0x7f0d28009520 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:52.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.459+0000 7f0d2effd640 1 -- 172.21.15.89:0/3288248436 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f0d34054f10 con 0x7f0d380d1a90 2023-10-18T10:45:52.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.607+0000 7f0d406cb640 1 -- 172.21.15.89:0/3288248436 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0d38150490 con 0x7f0d380d1a90 2023-10-18T10:45:52.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.607+0000 7f0d2effd640 1 -- 172.21.15.89:0/3288248436 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0d34054aa0 con 0x7f0d380d1a90 2023-10-18T10:45:52.609 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-10-18T10:45:52.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.611+0000 7f0d406cb640 1 -- 172.21.15.89:0/3288248436 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0d1c065680 msgr2=0x7f0d1c067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:52.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.611+0000 7f0d406cb640 1 --2- 172.21.15.89:0/3288248436 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0d1c065680 0x7f0d1c067b40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0d38154cd0 tx=0x7f0d28009520 comp rx=0 tx=0).stop 2023-10-18T10:45:52.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.611+0000 7f0d406cb640 1 -- 172.21.15.89:0/3288248436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d380d1a90 msgr2=0x7f0d380c4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:52.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.611+0000 7f0d406cb640 1 --2- 172.21.15.89:0/3288248436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d380d1a90 0x7f0d380c4700 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400efc0 tx=0x7f0d3400c490 comp rx=0 tx=0).stop 2023-10-18T10:45:52.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.611+0000 7f0d406cb640 1 -- 172.21.15.89:0/3288248436 shutdown_connections 2023-10-18T10:45:52.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.611+0000 7f0d406cb640 1 --2- 172.21.15.89:0/3288248436 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0d1c065680 0x7f0d1c067b40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:52.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.611+0000 7f0d406cb640 1 --2- 172.21.15.89:0/3288248436 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d3815c660 0x7f0d3815d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:52.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.611+0000 7f0d406cb640 1 --2- 172.21.15.89:0/3288248436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d380d20c0 0x7f0d3815cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:52.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.611+0000 7f0d406cb640 1 --2- 172.21.15.89:0/3288248436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d380d1a90 0x7f0d380c4700 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:52.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.611+0000 7f0d406cb640 1 -- 172.21.15.89:0/3288248436 >> 172.21.15.89:0/3288248436 conn(0x7f0d38069d00 msgr2=0x7f0d38153e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:52.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.611+0000 7f0d406cb640 1 -- 172.21.15.89:0/3288248436 shutdown_connections 2023-10-18T10:45:52.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.611+0000 7f0d406cb640 1 -- 172.21.15.89:0/3288248436 wait complete. 2023-10-18T10:45:52.629 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-18T10:45:52.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.767+0000 7f81f105f640 1 Processor -- start 2023-10-18T10:45:52.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.767+0000 7f81f105f640 1 -- start start 2023-10-18T10:45:52.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.767+0000 7f81f105f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f81ec0d0760 0x7f81ec0d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:52.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.767+0000 7f81f105f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f81ec0cd160 0x7f81ec0cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:52.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.767+0000 7f81f105f640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ec0d1100 con 0x7f81ec15c940 2023-10-18T10:45:52.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.767+0000 7f81f105f640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81ec15da50 con 0x7f81ec0d0760 2023-10-18T10:45:52.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.767+0000 7f81f105f640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81ec0d3110 con 0x7f81ec0cd160 2023-10-18T10:45:52.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.767+0000 7f81e259c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f81ec0d0760 0x7f81ec0d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:52.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.767+0000 7f81e259c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f81ec0d0760 0x7f81ec0d0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48740/0 (socket says 172.21.15.89:48740) 2023-10-18T10:45:52.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.767+0000 7f81e259c640 1 -- 172.21.15.89:0/2246422000 learned_addr learned my addr 172.21.15.89:0/2246422000 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:45:52.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.767+0000 7f81eb59e640 1 --2- 172.21.15.89:0/2246422000 >> v2:172.21.15.154:3300/0 conn(0x7f81ec0cd160 0x7f81ec0cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:52.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.767+0000 7f81e259c640 1 -- 172.21.15.89:0/2246422000 >> v2:172.21.15.154:3300/0 conn(0x7f81ec0cd160 msgr2=0x7f81ec0cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:52.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.767+0000 7f81e259c640 1 --2- 172.21.15.89:0/2246422000 >> v2:172.21.15.154:3300/0 conn(0x7f81ec0cd160 0x7f81ec0cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:52.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.767+0000 7f81e259c640 1 -- 172.21.15.89:0/2246422000 >> v1:172.21.15.154:6789/0 conn(0x7f81ec15c940 legacy=0x7f81ec0d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:52.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.767+0000 7f81e259c640 1 -- 172.21.15.89:0/2246422000 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ec15ce40 con 0x7f81ec0d0760 2023-10-18T10:45:52.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.771+0000 7f81e259c640 1 --2- 172.21.15.89:0/2246422000 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81ec0d0760 0x7f81ec0d0bc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f81ec15cd30 tx=0x7f81d401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f639c5a2bcefa84 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:52.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.771+0000 7f81ea59c640 1 -- 172.21.15.89:0/2246422000 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d40aa430 con 0x7f81ec0d0760 2023-10-18T10:45:52.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.771+0000 7f81ea59c640 1 -- 172.21.15.89:0/2246422000 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d40aa590 con 0x7f81ec0d0760 2023-10-18T10:45:52.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.771+0000 7f81ea59c640 1 -- 172.21.15.89:0/2246422000 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d40aa920 con 0x7f81ec0d0760 2023-10-18T10:45:52.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.771+0000 7f81f105f640 1 -- 172.21.15.89:0/2246422000 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81ec0d0760 msgr2=0x7f81ec0d0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:52.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.771+0000 7f81f105f640 1 --2- 172.21.15.89:0/2246422000 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81ec0d0760 0x7f81ec0d0bc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f81ec15cd30 tx=0x7f81d401c4d0 comp rx=0 tx=0).stop 2023-10-18T10:45:52.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.771+0000 7f81f105f640 1 -- 172.21.15.89:0/2246422000 shutdown_connections 2023-10-18T10:45:52.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.771+0000 7f81f105f640 1 --2- 172.21.15.89:0/2246422000 >> v2:172.21.15.154:3300/0 conn(0x7f81ec0cd160 0x7f81ec0cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:52.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.771+0000 7f81f105f640 1 --2- 172.21.15.89:0/2246422000 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81ec0d0760 0x7f81ec0d0bc0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:52.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.771+0000 7f81f105f640 1 -- 172.21.15.89:0/2246422000 >> 172.21.15.89:0/2246422000 conn(0x7f81ec089fd0 msgr2=0x7f81ec09b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:52.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.771+0000 7f81f105f640 1 -- 172.21.15.89:0/2246422000 shutdown_connections 2023-10-18T10:45:52.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.771+0000 7f81f105f640 1 -- 172.21.15.89:0/2246422000 wait complete. 2023-10-18T10:45:52.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.775+0000 7f81f105f640 1 Processor -- start 2023-10-18T10:45:52.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.775+0000 7f81f105f640 1 -- start start 2023-10-18T10:45:52.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.775+0000 7f81f105f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81ec0cd160 0x7f81ec15ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:52.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.775+0000 7f81f105f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81ec0d0760 0x7f81ec0b3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:52.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.775+0000 7f81f105f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81ec1f94e0 0x7f81ec154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:52.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.775+0000 7f81f105f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec1f90c0 con 0x7f81ec0cd160 2023-10-18T10:45:52.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.775+0000 7f81f105f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec0cc650 con 0x7f81ec1f94e0 2023-10-18T10:45:52.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.775+0000 7f81f105f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec0cc090 con 0x7f81ec0d0760 2023-10-18T10:45:52.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.775+0000 7f81ead9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81ec0cd160 0x7f81ec15ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:52.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.775+0000 7f81ead9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81ec0cd160 0x7f81ec15ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48748/0 (socket says 172.21.15.89:48748) 2023-10-18T10:45:52.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.775+0000 7f81ead9d640 1 -- 172.21.15.89:0/3783718231 learned_addr learned my addr 172.21.15.89:0/3783718231 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:45:52.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.775+0000 7f81eb59e640 1 --2- 172.21.15.89:0/3783718231 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81ec1f94e0 0x7f81ec154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:52.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.775+0000 7f81e259c640 1 --2- 172.21.15.89:0/3783718231 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81ec0d0760 0x7f81ec0b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:52.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.775+0000 7f81ead9d640 1 -- 172.21.15.89:0/3783718231 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81ec0d0760 msgr2=0x7f81ec0b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:52.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.775+0000 7f81ead9d640 1 --2- 172.21.15.89:0/3783718231 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81ec0d0760 0x7f81ec0b3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:52.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.775+0000 7f81ead9d640 1 -- 172.21.15.89:0/3783718231 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81ec1f94e0 msgr2=0x7f81ec154bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:52.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.775+0000 7f81ead9d640 1 --2- 172.21.15.89:0/3783718231 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81ec1f94e0 0x7f81ec154bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:52.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.775+0000 7f81ead9d640 1 -- 172.21.15.89:0/3783718231 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d40094d0 con 0x7f81ec0cd160 2023-10-18T10:45:52.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.775+0000 7f81ead9d640 1 --2- 172.21.15.89:0/3783718231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81ec0cd160 0x7f81ec15ac70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00c7d0 tx=0x7f81dc00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:52.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.775+0000 7f81e3fff640 1 -- 172.21.15.89:0/3783718231 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc003b80 con 0x7f81ec0cd160 2023-10-18T10:45:52.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.775+0000 7f81e3fff640 1 -- 172.21.15.89:0/3783718231 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81dc003ce0 con 0x7f81ec0cd160 2023-10-18T10:45:52.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.775+0000 7f81e3fff640 1 -- 172.21.15.89:0/3783718231 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc00aa30 con 0x7f81ec0cd160 2023-10-18T10:45:52.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.775+0000 7f81f105f640 1 -- 172.21.15.89:0/3783718231 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81ec0cc370 con 0x7f81ec0cd160 2023-10-18T10:45:52.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.775+0000 7f81f105f640 1 -- 172.21.15.89:0/3783718231 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81ec1f8ce0 con 0x7f81ec0cd160 2023-10-18T10:45:52.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.779+0000 7f81f105f640 1 -- 172.21.15.89:0/3783718231 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81ec04dbe0 con 0x7f81ec0cd160 2023-10-18T10:45:52.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.779+0000 7f81e3fff640 1 -- 172.21.15.89:0/3783718231 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f81dc00ab90 con 0x7f81ec0cd160 2023-10-18T10:45:52.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.779+0000 7f81e3fff640 1 --2- 172.21.15.89:0/3783718231 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f81cc065680 0x7f81cc067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:52.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.779+0000 7f81e3fff640 1 -- 172.21.15.89:0/3783718231 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f81dc082790 con 0x7f81ec0cd160 2023-10-18T10:45:52.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.779+0000 7f81e259c640 1 --2- 172.21.15.89:0/3783718231 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f81cc065680 0x7f81cc067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:52.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.779+0000 7f81e259c640 1 --2- 172.21.15.89:0/3783718231 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f81cc065680 0x7f81cc067b40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f81d40ae000 tx=0x7f81d40b5040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:52.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.783+0000 7f81e3fff640 1 -- 172.21.15.89:0/3783718231 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f81dc0505b0 con 0x7f81ec0cd160 2023-10-18T10:45:52.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.931+0000 7f81f105f640 1 -- 172.21.15.89:0/3783718231 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f81ec0c58c0 con 0x7f81ec0cd160 2023-10-18T10:45:52.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.931+0000 7f81e3fff640 1 -- 172.21.15.89:0/3783718231 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+126 (secure 0 0 0) 0x7f81dc053bd0 con 0x7f81ec0cd160 2023-10-18T10:45:52.933 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:45:52.934 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-10-18T10:45:52.934 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-18T10:45:52.934 INFO:teuthology.orchestra.run.smithi089.stdout: "poolnum": 1, 2023-10-18T10:45:52.934 INFO:teuthology.orchestra.run.smithi089.stdout: "poolname": ".mgr" 2023-10-18T10:45:52.935 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-18T10:45:52.935 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-18T10:45:52.935 INFO:teuthology.orchestra.run.smithi089.stdout: "poolnum": 2, 2023-10-18T10:45:52.935 INFO:teuthology.orchestra.run.smithi089.stdout: "poolname": "rbd" 2023-10-18T10:45:52.935 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-18T10:45:52.936 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-10-18T10:45:52.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.935+0000 7f81f105f640 1 -- 172.21.15.89:0/3783718231 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f81cc065680 msgr2=0x7f81cc067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:52.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.935+0000 7f81f105f640 1 --2- 172.21.15.89:0/3783718231 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f81cc065680 0x7f81cc067b40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f81d40ae000 tx=0x7f81d40b5040 comp rx=0 tx=0).stop 2023-10-18T10:45:52.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.935+0000 7f81f105f640 1 -- 172.21.15.89:0/3783718231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81ec0cd160 msgr2=0x7f81ec15ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:52.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.935+0000 7f81f105f640 1 --2- 172.21.15.89:0/3783718231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81ec0cd160 0x7f81ec15ac70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00c7d0 tx=0x7f81dc00cc90 comp rx=0 tx=0).stop 2023-10-18T10:45:52.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.935+0000 7f81f105f640 1 -- 172.21.15.89:0/3783718231 shutdown_connections 2023-10-18T10:45:52.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.935+0000 7f81f105f640 1 --2- 172.21.15.89:0/3783718231 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f81cc065680 0x7f81cc067b40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:52.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.935+0000 7f81f105f640 1 --2- 172.21.15.89:0/3783718231 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81ec0d0760 0x7f81ec0b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:52.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.935+0000 7f81f105f640 1 --2- 172.21.15.89:0/3783718231 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81ec1f94e0 0x7f81ec154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:52.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.935+0000 7f81f105f640 1 --2- 172.21.15.89:0/3783718231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81ec0cd160 0x7f81ec15ac70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:52.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.935+0000 7f81f105f640 1 -- 172.21.15.89:0/3783718231 >> 172.21.15.89:0/3783718231 conn(0x7f81ec089fd0 msgr2=0x7f81ec159720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:52.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.935+0000 7f81f105f640 1 -- 172.21.15.89:0/3783718231 shutdown_connections 2023-10-18T10:45:52.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:52.935+0000 7f81f105f640 1 -- 172.21.15.89:0/3783718231 wait complete. 2023-10-18T10:45:52.954 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-18T10:45:52.954 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-18T10:45:53.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.095+0000 7f97f6e9c640 1 Processor -- start 2023-10-18T10:45:53.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.095+0000 7f97f6e9c640 1 -- start start 2023-10-18T10:45:53.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.095+0000 7f97f6e9c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f97f00d2d80 0x7f97f00cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:53.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.095+0000 7f97f6e9c640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f00b1b80 con 0x7f97f00d0bc0 2023-10-18T10:45:53.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.095+0000 7f97f6e9c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f00d01b0 con 0x7f97f00cd630 2023-10-18T10:45:53.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.095+0000 7f97f6e9c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97f00b1dd0 con 0x7f97f00d2d80 2023-10-18T10:45:53.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.099+0000 7f97f4c38640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f97f00d0bc0 0x7f97f00d0fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55120/0 (socket says 172.21.15.89:55120) 2023-10-18T10:45:53.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.099+0000 7f97f4c38640 1 -- 172.21.15.89:0/944478365 learned_addr learned my addr 172.21.15.89:0/944478365 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:45:53.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.099+0000 7f97effff640 1 --2- 172.21.15.89:0/944478365 >> v2:172.21.15.154:3300/0 conn(0x7f97f00d2d80 0x7f97f00cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:53.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.099+0000 7f97ef7fe640 1 -- 172.21.15.89:0/944478365 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 556484375 0 0) 0x7f97f00b1b80 con 0x7f97f00d0bc0 2023-10-18T10:45:53.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.099+0000 7f97ef7fe640 1 -- 172.21.15.89:0/944478365 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97dc003530 con 0x7f97f00d0bc0 2023-10-18T10:45:53.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.099+0000 7f97ef7fe640 1 -- 172.21.15.89:0/944478365 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4044341875 0 0) 0x7f97dc003530 con 0x7f97f00d0bc0 2023-10-18T10:45:53.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.099+0000 7f97ef7fe640 1 -- 172.21.15.89:0/944478365 >> v2:172.21.15.154:3300/0 conn(0x7f97f00d2d80 msgr2=0x7f97f00cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:53.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.099+0000 7f97ef7fe640 1 --2- 172.21.15.89:0/944478365 >> v2:172.21.15.154:3300/0 conn(0x7f97f00d2d80 0x7f97f00cd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:53.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.099+0000 7f97ef7fe640 1 -- 172.21.15.89:0/944478365 >> v1:172.21.15.154:6789/0 conn(0x7f97f00cd630 legacy=0x7f97f015d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:53.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.099+0000 7f97ef7fe640 1 -- 172.21.15.89:0/944478365 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f00d0470 con 0x7f97f00d0bc0 2023-10-18T10:45:53.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.099+0000 7f97ef7fe640 1 -- 172.21.15.89:0/944478365 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7f97d8001800 con 0x7f97f00d0bc0 2023-10-18T10:45:53.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.099+0000 7f97ef7fe640 1 -- 172.21.15.89:0/944478365 >> v1:172.21.15.89:6789/0 conn(0x7f97f00d0bc0 legacy=0x7f97f00d0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:53.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.099+0000 7f97ef7fe640 1 --2- 172.21.15.89:0/944478365 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97dc003980 0x7f97dc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:53.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.099+0000 7f97ef7fe640 1 --2- 172.21.15.89:0/944478365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97dc0047d0 0x7f97dc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:53.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.099+0000 7f97ef7fe640 1 --2- 172.21.15.89:0/944478365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97dc005190 0x7f97dc00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:53.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.099+0000 7f97ef7fe640 1 -- 172.21.15.89:0/944478365 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f00b1dd0 con 0x7f97dc0047d0 2023-10-18T10:45:53.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.099+0000 7f97ef7fe640 1 -- 172.21.15.89:0/944478365 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc00c130 con 0x7f97dc005190 2023-10-18T10:45:53.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.099+0000 7f97ef7fe640 1 -- 172.21.15.89:0/944478365 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc00c2f0 con 0x7f97dc003980 2023-10-18T10:45:53.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.099+0000 7f97effff640 1 --2- 172.21.15.89:0/944478365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97dc0047d0 0x7f97dc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:53.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.099+0000 7f97f5439640 1 --2- 172.21.15.89:0/944478365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97dc005190 0x7f97dc00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:53.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.099+0000 7f97f4c38640 1 --2- 172.21.15.89:0/944478365 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97dc003980 0x7f97dc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:53.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.099+0000 7f97effff640 1 -- 172.21.15.89:0/944478365 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97dc003980 msgr2=0x7f97dc003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:53.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.099+0000 7f97effff640 1 --2- 172.21.15.89:0/944478365 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97dc003980 0x7f97dc003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:53.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.099+0000 7f97effff640 1 -- 172.21.15.89:0/944478365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97dc005190 msgr2=0x7f97dc00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:53.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.099+0000 7f97effff640 1 --2- 172.21.15.89:0/944478365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97dc005190 0x7f97dc00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:53.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.099+0000 7f97effff640 1 -- 172.21.15.89:0/944478365 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97dc003530 con 0x7f97dc0047d0 2023-10-18T10:45:53.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.099+0000 7f97f5439640 1 -- 172.21.15.89:0/944478365 reap_dead start 2023-10-18T10:45:53.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.103+0000 7f97effff640 1 --2- 172.21.15.89:0/944478365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97dc0047d0 0x7f97dc004c50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f97e0009ae0 tx=0x7f97e001ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e813b43caaf0cf2d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:53.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.103+0000 7f97ef7fe640 1 -- 172.21.15.89:0/944478365 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e00a55d0 con 0x7f97dc0047d0 2023-10-18T10:45:53.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.103+0000 7f97ef7fe640 1 -- 172.21.15.89:0/944478365 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e00a5730 con 0x7f97dc0047d0 2023-10-18T10:45:53.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.103+0000 7f97ef7fe640 1 -- 172.21.15.89:0/944478365 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e00a5ae0 con 0x7f97dc0047d0 2023-10-18T10:45:53.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.103+0000 7f97f6e9c640 1 -- 172.21.15.89:0/944478365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97dc0047d0 msgr2=0x7f97dc004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:53.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.103+0000 7f97f6e9c640 1 --2- 172.21.15.89:0/944478365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97dc0047d0 0x7f97dc004c50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f97e0009ae0 tx=0x7f97e001ca20 comp rx=0 tx=0).stop 2023-10-18T10:45:53.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.103+0000 7f97f6e9c640 1 -- 172.21.15.89:0/944478365 shutdown_connections 2023-10-18T10:45:53.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.103+0000 7f97f6e9c640 1 --2- 172.21.15.89:0/944478365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97dc0047d0 0x7f97dc004c50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:53.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.103+0000 7f97f6e9c640 1 -- 172.21.15.89:0/944478365 >> 172.21.15.89:0/944478365 conn(0x7f97f009b9d0 msgr2=0x7f97f00d2670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:53.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.103+0000 7f97f6e9c640 1 -- 172.21.15.89:0/944478365 shutdown_connections 2023-10-18T10:45:53.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.103+0000 7f97f6e9c640 1 -- 172.21.15.89:0/944478365 wait complete. 2023-10-18T10:45:53.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.103+0000 7f97f6e9c640 1 Processor -- start 2023-10-18T10:45:53.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.103+0000 7f97f6e9c640 1 -- start start 2023-10-18T10:45:53.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.107+0000 7f97f6e9c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97dc0047d0 0x7f97f01f5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:53.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.107+0000 7f97f6e9c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97f00c96b0 0x7f97f00c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:53.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.107+0000 7f97f6e9c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97f00b62e0 0x7f97f00b6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:53.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.107+0000 7f97f6e9c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f00b6ca0 con 0x7f97f00b62e0 2023-10-18T10:45:53.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.107+0000 7f97f6e9c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f00ca050 con 0x7f97f00c96b0 2023-10-18T10:45:53.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.107+0000 7f97f6e9c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f00b7ad0 con 0x7f97dc0047d0 2023-10-18T10:45:53.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.107+0000 7f97f5439640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97f00b62e0 0x7f97f00b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:53.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.107+0000 7f97effff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97f00c96b0 0x7f97f00c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:53.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.107+0000 7f97f5439640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97f00b62e0 0x7f97f00b6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48772/0 (socket says 172.21.15.89:48772) 2023-10-18T10:45:53.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.107+0000 7f97f5439640 1 -- 172.21.15.89:0/1842264145 learned_addr learned my addr 172.21.15.89:0/1842264145 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:45:53.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.107+0000 7f97f4c38640 1 --2- 172.21.15.89:0/1842264145 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97dc0047d0 0x7f97f01f5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:53.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.107+0000 7f97effff640 1 -- 172.21.15.89:0/1842264145 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97dc0047d0 msgr2=0x7f97f01f5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:53.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.107+0000 7f97effff640 1 --2- 172.21.15.89:0/1842264145 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97dc0047d0 0x7f97f01f5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:53.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.107+0000 7f97effff640 1 -- 172.21.15.89:0/1842264145 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97f00b62e0 msgr2=0x7f97f00b6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:53.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.107+0000 7f97effff640 1 --2- 172.21.15.89:0/1842264145 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97f00b62e0 0x7f97f00b6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:53.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.107+0000 7f97effff640 1 -- 172.21.15.89:0/1842264145 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f00cb950 con 0x7f97f00c96b0 2023-10-18T10:45:53.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.107+0000 7f97f5439640 1 --2- 172.21.15.89:0/1842264145 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97f00b62e0 0x7f97f00b6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:45:53.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.107+0000 7f97effff640 1 --2- 172.21.15.89:0/1842264145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97f00c96b0 0x7f97f00c9b10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f97dc00c100 tx=0x7f97dc005800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:53.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.107+0000 7f97ed7fa640 1 -- 172.21.15.89:0/1842264145 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e0004020 con 0x7f97f00c96b0 2023-10-18T10:45:53.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.107+0000 7f97ed7fa640 1 -- 172.21.15.89:0/1842264145 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e00041a0 con 0x7f97f00c96b0 2023-10-18T10:45:53.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.107+0000 7f97ed7fa640 1 -- 172.21.15.89:0/1842264145 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e0003620 con 0x7f97f00c96b0 2023-10-18T10:45:53.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.107+0000 7f97f6e9c640 1 -- 172.21.15.89:0/1842264145 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f00b6ec0 con 0x7f97f00c96b0 2023-10-18T10:45:53.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.107+0000 7f97f6e9c640 1 -- 172.21.15.89:0/1842264145 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f00b7480 con 0x7f97f00c96b0 2023-10-18T10:45:53.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.107+0000 7f97f6e9c640 1 -- 172.21.15.89:0/1842264145 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f004dbe0 con 0x7f97f00c96b0 2023-10-18T10:45:53.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.111+0000 7f97ed7fa640 1 -- 172.21.15.89:0/1842264145 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f97e00b4040 con 0x7f97f00c96b0 2023-10-18T10:45:53.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.111+0000 7f97ed7fa640 1 --2- 172.21.15.89:0/1842264145 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f97d00656b0 0x7f97d0067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:53.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.111+0000 7f97ed7fa640 1 -- 172.21.15.89:0/1842264145 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f97e011c480 con 0x7f97f00c96b0 2023-10-18T10:45:53.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.111+0000 7f97f4c38640 1 --2- 172.21.15.89:0/1842264145 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f97d00656b0 0x7f97d0067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:53.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.111+0000 7f97f4c38640 1 --2- 172.21.15.89:0/1842264145 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f97d00656b0 0x7f97d0067b70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f97d8000c10 tx=0x7f97d8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:53.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.115+0000 7f97ed7fa640 1 -- 172.21.15.89:0/1842264145 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f97e00ea210 con 0x7f97f00c96b0 2023-10-18T10:45:53.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:53.263+0000 7f97f6e9c640 1 -- 172.21.15.89:0/1842264145 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f97f00b0f00 con 0x7f97f00c96b0 2023-10-18T10:45:54.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.267+0000 7f97ed7fa640 1 -- 172.21.15.89:0/1842264145 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) v1 ==== 138+0+0 (secure 0 0 0) 0x7f97e00ed830 con 0x7f97f00c96b0 2023-10-18T10:45:54.269 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' created 2023-10-18T10:45:54.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.271+0000 7f97f6e9c640 1 -- 172.21.15.89:0/1842264145 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f97d00656b0 msgr2=0x7f97d0067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:54.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.271+0000 7f97f6e9c640 1 --2- 172.21.15.89:0/1842264145 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f97d00656b0 0x7f97d0067b70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f97d8000c10 tx=0x7f97d8009040 comp rx=0 tx=0).stop 2023-10-18T10:45:54.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.271+0000 7f97f6e9c640 1 -- 172.21.15.89:0/1842264145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97f00c96b0 msgr2=0x7f97f00c9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:54.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.271+0000 7f97f6e9c640 1 --2- 172.21.15.89:0/1842264145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97f00c96b0 0x7f97f00c9b10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f97dc00c100 tx=0x7f97dc005800 comp rx=0 tx=0).stop 2023-10-18T10:45:54.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.275+0000 7f97f6e9c640 1 -- 172.21.15.89:0/1842264145 shutdown_connections 2023-10-18T10:45:54.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.275+0000 7f97f6e9c640 1 --2- 172.21.15.89:0/1842264145 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f97d00656b0 0x7f97d0067b70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:54.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.275+0000 7f97f6e9c640 1 --2- 172.21.15.89:0/1842264145 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97f00b62e0 0x7f97f00b6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:54.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.275+0000 7f97f6e9c640 1 --2- 172.21.15.89:0/1842264145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97f00c96b0 0x7f97f00c9b10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:54.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.275+0000 7f97f6e9c640 1 --2- 172.21.15.89:0/1842264145 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97dc0047d0 0x7f97f01f5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:54.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.275+0000 7f97f6e9c640 1 -- 172.21.15.89:0/1842264145 >> 172.21.15.89:0/1842264145 conn(0x7f97f009b9d0 msgr2=0x7f97f00b38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:54.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.275+0000 7f97f6e9c640 1 -- 172.21.15.89:0/1842264145 shutdown_connections 2023-10-18T10:45:54.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.275+0000 7f97f6e9c640 1 -- 172.21.15.89:0/1842264145 wait complete. 2023-10-18T10:45:54.297 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-10-18T10:45:54.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.443+0000 7fd322e93640 1 Processor -- start 2023-10-18T10:45:54.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.443+0000 7fd322e93640 1 -- start start 2023-10-18T10:45:54.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.447+0000 7fd322e93640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd31c159490 0x7fd31c15d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:54.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.447+0000 7fd322e93640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd31c15ddf0 con 0x7fd31c070600 2023-10-18T10:45:54.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.447+0000 7fd322e93640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd31c070b90 con 0x7fd31c06bee0 2023-10-18T10:45:54.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.447+0000 7fd322e93640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd31c070e70 con 0x7fd31c159490 2023-10-18T10:45:54.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.447+0000 7fd313fff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd31c070600 0x7fd31c06b7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55136/0 (socket says 172.21.15.89:55136) 2023-10-18T10:45:54.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.447+0000 7fd313fff640 1 -- 172.21.15.89:0/2478602617 learned_addr learned my addr 172.21.15.89:0/2478602617 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:45:54.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.447+0000 7fd320c2f640 1 --2- 172.21.15.89:0/2478602617 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd31c159490 0x7fd31c15d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:54.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.447+0000 7fd3137fe640 1 -- 172.21.15.89:0/2478602617 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3131049006 0 0) 0x7fd31c15ddf0 con 0x7fd31c070600 2023-10-18T10:45:54.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.447+0000 7fd3137fe640 1 -- 172.21.15.89:0/2478602617 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd304005740 con 0x7fd31c070600 2023-10-18T10:45:54.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.447+0000 7fd3137fe640 1 -- 172.21.15.89:0/2478602617 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2048424398 0 0) 0x7fd31c070b90 con 0x7fd31c06bee0 2023-10-18T10:45:54.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.447+0000 7fd3137fe640 1 -- 172.21.15.89:0/2478602617 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd31c15ddf0 con 0x7fd31c06bee0 2023-10-18T10:45:54.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.447+0000 7fd320c2f640 1 -- 172.21.15.89:0/2478602617 >> v1:172.21.15.154:6789/0 conn(0x7fd31c06bee0 legacy=0x7fd31c0c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:54.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.447+0000 7fd3137fe640 1 -- 172.21.15.89:0/2478602617 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1671527907 0 0) 0x7fd304005740 con 0x7fd31c070600 2023-10-18T10:45:54.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.447+0000 7fd320c2f640 1 -- 172.21.15.89:0/2478602617 >> v1:172.21.15.89:6789/0 conn(0x7fd31c070600 legacy=0x7fd31c06b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:54.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.447+0000 7fd320c2f640 1 -- 172.21.15.89:0/2478602617 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd31c071150 con 0x7fd31c159490 2023-10-18T10:45:54.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.451+0000 7fd320c2f640 1 --2- 172.21.15.89:0/2478602617 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd31c159490 0x7fd31c15d8b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd3180097f0 tx=0x7fd31801c320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d214f41bf2fc4c13 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:54.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.451+0000 7fd3137fe640 1 -- 172.21.15.89:0/2478602617 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30c001680 con 0x7fd31c159490 2023-10-18T10:45:54.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.451+0000 7fd3137fe640 1 -- 172.21.15.89:0/2478602617 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3180a7050 con 0x7fd31c159490 2023-10-18T10:45:54.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.451+0000 7fd3137fe640 1 -- 172.21.15.89:0/2478602617 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3180ab600 con 0x7fd31c159490 2023-10-18T10:45:54.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.451+0000 7fd322e93640 1 -- 172.21.15.89:0/2478602617 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd31c159490 msgr2=0x7fd31c15d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:54.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.451+0000 7fd322e93640 1 --2- 172.21.15.89:0/2478602617 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd31c159490 0x7fd31c15d8b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd3180097f0 tx=0x7fd31801c320 comp rx=0 tx=0).stop 2023-10-18T10:45:54.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.451+0000 7fd322e93640 1 -- 172.21.15.89:0/2478602617 shutdown_connections 2023-10-18T10:45:54.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.451+0000 7fd322e93640 1 --2- 172.21.15.89:0/2478602617 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd31c159490 0x7fd31c15d8b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:54.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.451+0000 7fd322e93640 1 -- 172.21.15.89:0/2478602617 >> 172.21.15.89:0/2478602617 conn(0x7fd31c069f00 msgr2=0x7fd31c0cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:54.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.451+0000 7fd322e93640 1 -- 172.21.15.89:0/2478602617 shutdown_connections 2023-10-18T10:45:54.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.451+0000 7fd322e93640 1 -- 172.21.15.89:0/2478602617 wait complete. 2023-10-18T10:45:54.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.451+0000 7fd322e93640 1 Processor -- start 2023-10-18T10:45:54.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.451+0000 7fd322e93640 1 -- start start 2023-10-18T10:45:54.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.455+0000 7fd322e93640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd31c06bee0 0x7fd31c153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:54.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.455+0000 7fd322e93640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd31c070600 0x7fd31c157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:54.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.455+0000 7fd322e93640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd31c159490 0x7fd31c0afde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:54.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.455+0000 7fd322e93640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd31c0b37a0 con 0x7fd31c159490 2023-10-18T10:45:54.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.455+0000 7fd322e93640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd31c0b1d40 con 0x7fd31c070600 2023-10-18T10:45:54.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.455+0000 7fd322e93640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd31c0b1a10 con 0x7fd31c06bee0 2023-10-18T10:45:54.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.455+0000 7fd321430640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd31c159490 0x7fd31c0afde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:54.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.455+0000 7fd321430640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd31c159490 0x7fd31c0afde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48776/0 (socket says 172.21.15.89:48776) 2023-10-18T10:45:54.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.455+0000 7fd313fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd31c070600 0x7fd31c157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:54.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.455+0000 7fd321430640 1 -- 172.21.15.89:0/1551557550 learned_addr learned my addr 172.21.15.89:0/1551557550 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:45:54.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.455+0000 7fd320c2f640 1 --2- 172.21.15.89:0/1551557550 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd31c06bee0 0x7fd31c153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:54.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.455+0000 7fd321430640 1 -- 172.21.15.89:0/1551557550 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd31c06bee0 msgr2=0x7fd31c153860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:54.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.455+0000 7fd321430640 1 --2- 172.21.15.89:0/1551557550 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd31c06bee0 0x7fd31c153860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:54.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.455+0000 7fd321430640 1 -- 172.21.15.89:0/1551557550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd31c070600 msgr2=0x7fd31c157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:54.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.455+0000 7fd321430640 1 --2- 172.21.15.89:0/1551557550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd31c070600 0x7fd31c157070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:54.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.455+0000 7fd321430640 1 -- 172.21.15.89:0/1551557550 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd318009420 con 0x7fd31c159490 2023-10-18T10:45:54.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.455+0000 7fd321430640 1 --2- 172.21.15.89:0/1551557550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd31c159490 0x7fd31c0afde0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd31400c730 tx=0x7fd31400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:54.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.455+0000 7fd3117fa640 1 -- 172.21.15.89:0/1551557550 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd314003b80 con 0x7fd31c159490 2023-10-18T10:45:54.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.455+0000 7fd3117fa640 1 -- 172.21.15.89:0/1551557550 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd314003ce0 con 0x7fd31c159490 2023-10-18T10:45:54.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.455+0000 7fd322e93640 1 -- 172.21.15.89:0/1551557550 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd31c0b39b0 con 0x7fd31c159490 2023-10-18T10:45:54.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.455+0000 7fd3117fa640 1 -- 172.21.15.89:0/1551557550 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd314005310 con 0x7fd31c159490 2023-10-18T10:45:54.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.455+0000 7fd322e93640 1 -- 172.21.15.89:0/1551557550 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd31c0b3f70 con 0x7fd31c159490 2023-10-18T10:45:54.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.455+0000 7fd322e93640 1 -- 172.21.15.89:0/1551557550 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd31c04dbe0 con 0x7fd31c159490 2023-10-18T10:45:54.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.459+0000 7fd3117fa640 1 -- 172.21.15.89:0/1551557550 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fd314005680 con 0x7fd31c159490 2023-10-18T10:45:54.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.459+0000 7fd3117fa640 1 --2- 172.21.15.89:0/1551557550 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd308065680 0x7fd308067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:54.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.459+0000 7fd3117fa640 1 -- 172.21.15.89:0/1551557550 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fd314082b70 con 0x7fd31c159490 2023-10-18T10:45:54.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.459+0000 7fd320c2f640 1 --2- 172.21.15.89:0/1551557550 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd308065680 0x7fd308067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:54.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.459+0000 7fd320c2f640 1 --2- 172.21.15.89:0/1551557550 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd308065680 0x7fd308067b40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd3180ad000 tx=0x7fd318008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:54.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.463+0000 7fd3117fa640 1 -- 172.21.15.89:0/1551557550 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fd314050820 con 0x7fd31c159490 2023-10-18T10:45:54.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:54.615+0000 7fd322e93640 1 -- 172.21.15.89:0/1551557550 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fd31c0b1e80 con 0x7fd31c159490 2023-10-18T10:45:55.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.271+0000 7fd3117fa640 1 -- 172.21.15.89:0/1551557550 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v21) v1 ==== 171+0+0 (secure 0 0 0) 0x7fd314053e40 con 0x7fd31c159490 2023-10-18T10:45:55.272 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' created 2023-10-18T10:45:55.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.275+0000 7fd322e93640 1 -- 172.21.15.89:0/1551557550 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd308065680 msgr2=0x7fd308067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:55.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.275+0000 7fd322e93640 1 --2- 172.21.15.89:0/1551557550 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd308065680 0x7fd308067b40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd3180ad000 tx=0x7fd318008040 comp rx=0 tx=0).stop 2023-10-18T10:45:55.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.275+0000 7fd322e93640 1 -- 172.21.15.89:0/1551557550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd31c159490 msgr2=0x7fd31c0afde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:55.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.275+0000 7fd322e93640 1 --2- 172.21.15.89:0/1551557550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd31c159490 0x7fd31c0afde0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd31400c730 tx=0x7fd31400cbf0 comp rx=0 tx=0).stop 2023-10-18T10:45:55.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.275+0000 7fd322e93640 1 -- 172.21.15.89:0/1551557550 shutdown_connections 2023-10-18T10:45:55.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.275+0000 7fd322e93640 1 --2- 172.21.15.89:0/1551557550 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd308065680 0x7fd308067b40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:55.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.275+0000 7fd322e93640 1 --2- 172.21.15.89:0/1551557550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd31c159490 0x7fd31c0afde0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:55.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.275+0000 7fd322e93640 1 --2- 172.21.15.89:0/1551557550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd31c070600 0x7fd31c157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:55.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.275+0000 7fd322e93640 1 --2- 172.21.15.89:0/1551557550 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd31c06bee0 0x7fd31c153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:55.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.275+0000 7fd322e93640 1 -- 172.21.15.89:0/1551557550 >> 172.21.15.89:0/1551557550 conn(0x7fd31c069f00 msgr2=0x7fd31c0c6000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:55.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.279+0000 7fd322e93640 1 -- 172.21.15.89:0/1551557550 shutdown_connections 2023-10-18T10:45:55.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.279+0000 7fd322e93640 1 -- 172.21.15.89:0/1551557550 wait complete. 2023-10-18T10:45:55.314 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-18T10:45:55.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.455+0000 7f444cb59640 1 Processor -- start 2023-10-18T10:45:55.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.455+0000 7f444cb59640 1 -- start start 2023-10-18T10:45:55.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.455+0000 7f444cb59640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f444815c550 0x7f44480d2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:55.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.455+0000 7f444cb59640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44480d1e50 con 0x7f44480cfd20 2023-10-18T10:45:55.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.455+0000 7f444cb59640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44480d2130 con 0x7f44480b1860 2023-10-18T10:45:55.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.455+0000 7f444cb59640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44480d2410 con 0x7f444815c550 2023-10-18T10:45:55.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.455+0000 7f444659c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f444815c550 0x7f44480d2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:55.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.455+0000 7f444659c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f444815c550 0x7f44480d2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48780/0 (socket says 172.21.15.89:48780) 2023-10-18T10:45:55.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.455+0000 7f444659c640 1 -- 172.21.15.89:0/3324853634 learned_addr learned my addr 172.21.15.89:0/3324853634 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:45:55.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.455+0000 7f444659c640 1 -- 172.21.15.89:0/3324853634 >> v1:172.21.15.154:6789/0 conn(0x7f44480b1860 legacy=0x7f44480cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:55.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.455+0000 7f444659c640 1 -- 172.21.15.89:0/3324853634 >> v1:172.21.15.89:6789/0 conn(0x7f44480cfd20 legacy=0x7f44480b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:55.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.455+0000 7f444659c640 1 -- 172.21.15.89:0/3324853634 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44480d26f0 con 0x7f444815c550 2023-10-18T10:45:55.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.459+0000 7f444659c640 1 --2- 172.21.15.89:0/3324853634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f444815c550 0x7f44480d2ef0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4438009870 tx=0x7f443801cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7d3742e37aa9e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:55.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.459+0000 7f444559a640 1 -- 172.21.15.89:0/3324853634 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44380a95f0 con 0x7f444815c550 2023-10-18T10:45:55.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.459+0000 7f444559a640 1 -- 172.21.15.89:0/3324853634 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44380a9750 con 0x7f444815c550 2023-10-18T10:45:55.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.459+0000 7f444559a640 1 -- 172.21.15.89:0/3324853634 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44380a9ae0 con 0x7f444815c550 2023-10-18T10:45:55.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.459+0000 7f444cb59640 1 -- 172.21.15.89:0/3324853634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f444815c550 msgr2=0x7f44480d2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:55.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.459+0000 7f444cb59640 1 --2- 172.21.15.89:0/3324853634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f444815c550 0x7f44480d2ef0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4438009870 tx=0x7f443801cf70 comp rx=0 tx=0).stop 2023-10-18T10:45:55.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.459+0000 7f444cb59640 1 -- 172.21.15.89:0/3324853634 shutdown_connections 2023-10-18T10:45:55.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.459+0000 7f444cb59640 1 --2- 172.21.15.89:0/3324853634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f444815c550 0x7f44480d2ef0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:55.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.459+0000 7f444cb59640 1 -- 172.21.15.89:0/3324853634 >> 172.21.15.89:0/3324853634 conn(0x7f4448069d00 msgr2=0x7f444806a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:55.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.459+0000 7f444cb59640 1 -- 172.21.15.89:0/3324853634 shutdown_connections 2023-10-18T10:45:55.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.459+0000 7f444cb59640 1 -- 172.21.15.89:0/3324853634 wait complete. 2023-10-18T10:45:55.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.463+0000 7f444cb59640 1 Processor -- start 2023-10-18T10:45:55.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.463+0000 7f444cb59640 1 -- start start 2023-10-18T10:45:55.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.463+0000 7f444cb59640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44480b1860 0x7f444815d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:55.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.463+0000 7f444cb59640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44480cfd20 0x7f444814da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:55.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.463+0000 7f444cb59640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f444815c550 0x7f4448158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:55.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.463+0000 7f444cb59640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444815ceb0 con 0x7f44480cfd20 2023-10-18T10:45:55.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.463+0000 7f444cb59640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444815dc90 con 0x7f444815c550 2023-10-18T10:45:55.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.463+0000 7f444cb59640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4448155f30 con 0x7f44480b1860 2023-10-18T10:45:55.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.463+0000 7f4445d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44480cfd20 0x7f444814da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:55.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.463+0000 7f4445d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44480cfd20 0x7f444814da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48794/0 (socket says 172.21.15.89:48794) 2023-10-18T10:45:55.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.463+0000 7f4445d9b640 1 -- 172.21.15.89:0/3847050680 learned_addr learned my addr 172.21.15.89:0/3847050680 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:45:55.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.463+0000 7f444659c640 1 --2- 172.21.15.89:0/3847050680 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44480b1860 0x7f444815d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:55.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.463+0000 7f4446d9d640 1 --2- 172.21.15.89:0/3847050680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f444815c550 0x7f4448158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:55.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.463+0000 7f4445d9b640 1 -- 172.21.15.89:0/3847050680 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44480b1860 msgr2=0x7f444815d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:55.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.463+0000 7f4445d9b640 1 --2- 172.21.15.89:0/3847050680 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44480b1860 0x7f444815d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:55.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.463+0000 7f4445d9b640 1 -- 172.21.15.89:0/3847050680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f444815c550 msgr2=0x7f4448158690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:55.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.463+0000 7f4445d9b640 1 --2- 172.21.15.89:0/3847050680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f444815c550 0x7f4448158690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:55.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.463+0000 7f4445d9b640 1 -- 172.21.15.89:0/3847050680 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44380094d0 con 0x7f44480cfd20 2023-10-18T10:45:55.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.463+0000 7f4445d9b640 1 --2- 172.21.15.89:0/3847050680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44480cfd20 0x7f444814da50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f443c00c500 tx=0x7f443c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:55.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.463+0000 7f4436ffd640 1 -- 172.21.15.89:0/3847050680 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c005220 con 0x7f44480cfd20 2023-10-18T10:45:55.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.463+0000 7f4436ffd640 1 -- 172.21.15.89:0/3847050680 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f443c005380 con 0x7f44480cfd20 2023-10-18T10:45:55.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.463+0000 7f4436ffd640 1 -- 172.21.15.89:0/3847050680 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c007270 con 0x7f44480cfd20 2023-10-18T10:45:55.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.463+0000 7f444cb59640 1 -- 172.21.15.89:0/3847050680 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4448156210 con 0x7f44480cfd20 2023-10-18T10:45:55.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.463+0000 7f444cb59640 1 -- 172.21.15.89:0/3847050680 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4448155c20 con 0x7f44480cfd20 2023-10-18T10:45:55.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.467+0000 7f444cb59640 1 -- 172.21.15.89:0/3847050680 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4448156730 con 0x7f44480cfd20 2023-10-18T10:45:55.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.467+0000 7f4436ffd640 1 -- 172.21.15.89:0/3847050680 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f443c0073d0 con 0x7f44480cfd20 2023-10-18T10:45:55.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.467+0000 7f4436ffd640 1 --2- 172.21.15.89:0/3847050680 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4428065680 0x7f4428067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:55.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.467+0000 7f4436ffd640 1 -- 172.21.15.89:0/3847050680 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f443c082600 con 0x7f44480cfd20 2023-10-18T10:45:55.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.467+0000 7f444659c640 1 --2- 172.21.15.89:0/3847050680 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4428065680 0x7f4428067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:55.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.467+0000 7f444659c640 1 --2- 172.21.15.89:0/3847050680 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4428065680 0x7f4428067b40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f44380ae000 tx=0x7f44380b3040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:55.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.471+0000 7f4436ffd640 1 -- 172.21.15.89:0/3847050680 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f4448156730 con 0x7f44480cfd20 2023-10-18T10:45:55.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:55.651+0000 7f444cb59640 1 -- 172.21.15.89:0/3847050680 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4448156730 con 0x7f44480cfd20 2023-10-18T10:45:56.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.271+0000 7f4436ffd640 1 -- 172.21.15.89:0/3847050680 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f4448156730 con 0x7f44480cfd20 2023-10-18T10:45:56.273 INFO:teuthology.orchestra.run.smithi089.stderr:new fs with metadata pool 3 and data pool 4 2023-10-18T10:45:56.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.275+0000 7f444cb59640 1 -- 172.21.15.89:0/3847050680 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4428065680 msgr2=0x7f4428067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:56.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.275+0000 7f444cb59640 1 --2- 172.21.15.89:0/3847050680 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4428065680 0x7f4428067b40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f44380ae000 tx=0x7f44380b3040 comp rx=0 tx=0).stop 2023-10-18T10:45:56.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.275+0000 7f444cb59640 1 -- 172.21.15.89:0/3847050680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44480cfd20 msgr2=0x7f444814da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:56.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.275+0000 7f444cb59640 1 --2- 172.21.15.89:0/3847050680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44480cfd20 0x7f444814da50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f443c00c500 tx=0x7f443c00c9c0 comp rx=0 tx=0).stop 2023-10-18T10:45:56.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.275+0000 7f444cb59640 1 -- 172.21.15.89:0/3847050680 shutdown_connections 2023-10-18T10:45:56.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.275+0000 7f444cb59640 1 --2- 172.21.15.89:0/3847050680 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4428065680 0x7f4428067b40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:56.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.275+0000 7f444cb59640 1 --2- 172.21.15.89:0/3847050680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f444815c550 0x7f4448158690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:56.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.275+0000 7f444cb59640 1 --2- 172.21.15.89:0/3847050680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44480cfd20 0x7f444814da50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:56.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.275+0000 7f444cb59640 1 --2- 172.21.15.89:0/3847050680 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44480b1860 0x7f444815d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:56.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.275+0000 7f444cb59640 1 -- 172.21.15.89:0/3847050680 >> 172.21.15.89:0/3847050680 conn(0x7f4448069d00 msgr2=0x7f44480d0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:56.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.275+0000 7f444cb59640 1 -- 172.21.15.89:0/3847050680 shutdown_connections 2023-10-18T10:45:56.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.275+0000 7f444cb59640 1 -- 172.21.15.89:0/3847050680 wait complete. 2023-10-18T10:45:56.303 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-10-18T10:45:56.304 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-10-18T10:45:56.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.447+0000 7fa90cf8f640 1 Processor -- start 2023-10-18T10:45:56.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.447+0000 7fa90cf8f640 1 -- start start 2023-10-18T10:45:56.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.447+0000 7fa90cf8f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa9080d0bc0 0x7fa9080d0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:56.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.447+0000 7fa90cf8f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa9080d2d80 0x7fa9080cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:56.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.447+0000 7fa90cf8f640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9080b1b80 con 0x7fa9080cd630 2023-10-18T10:45:56.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.447+0000 7fa90cf8f640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9080d14e0 con 0x7fa9080d2d80 2023-10-18T10:45:56.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.447+0000 7fa90cf8f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9080d01b0 con 0x7fa9080d0bc0 2023-10-18T10:45:56.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.451+0000 7fa905d9b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa9080d2d80 0x7fa9080cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:56.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.451+0000 7fa905d9b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa9080d2d80 0x7fa9080cd0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48802/0 (socket says 172.21.15.89:48802) 2023-10-18T10:45:56.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.451+0000 7fa905d9b640 1 -- 172.21.15.89:0/1944865185 learned_addr learned my addr 172.21.15.89:0/1944865185 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:45:56.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.451+0000 7fa90659c640 1 --2- 172.21.15.89:0/1944865185 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa9080d0bc0 0x7fa9080d0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:56.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.451+0000 7fa905d9b640 1 -- 172.21.15.89:0/1944865185 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa9080d0bc0 msgr2=0x7fa9080d0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:56.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.451+0000 7fa905d9b640 1 --2- 172.21.15.89:0/1944865185 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa9080d0bc0 0x7fa9080d0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:56.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.451+0000 7fa905d9b640 1 -- 172.21.15.89:0/1944865185 >> v1:172.21.15.154:6789/0 conn(0x7fa9080cd630 legacy=0x7fa90815d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:56.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.451+0000 7fa905d9b640 1 -- 172.21.15.89:0/1944865185 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9080b1dd0 con 0x7fa9080d2d80 2023-10-18T10:45:56.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.451+0000 7fa905d9b640 1 --2- 172.21.15.89:0/1944865185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9080d2d80 0x7fa9080cd0f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc009870 tx=0x7fa8fc01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42cc35604e58d8dd server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:56.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.451+0000 7fa90559a640 1 -- 172.21.15.89:0/1944865185 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc0aa670 con 0x7fa9080d2d80 2023-10-18T10:45:56.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.451+0000 7fa90559a640 1 -- 172.21.15.89:0/1944865185 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8fc0aa7d0 con 0x7fa9080d2d80 2023-10-18T10:45:56.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.451+0000 7fa90559a640 1 -- 172.21.15.89:0/1944865185 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc0aab60 con 0x7fa9080d2d80 2023-10-18T10:45:56.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.455+0000 7fa90cf8f640 1 -- 172.21.15.89:0/1944865185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9080d2d80 msgr2=0x7fa9080cd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:56.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.455+0000 7fa90cf8f640 1 --2- 172.21.15.89:0/1944865185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9080d2d80 0x7fa9080cd0f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc009870 tx=0x7fa8fc01c510 comp rx=0 tx=0).stop 2023-10-18T10:45:56.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.455+0000 7fa90cf8f640 1 -- 172.21.15.89:0/1944865185 shutdown_connections 2023-10-18T10:45:56.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.455+0000 7fa90cf8f640 1 --2- 172.21.15.89:0/1944865185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9080d2d80 0x7fa9080cd0f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:56.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.455+0000 7fa90cf8f640 1 --2- 172.21.15.89:0/1944865185 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa9080d0bc0 0x7fa9080d0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:56.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.455+0000 7fa90cf8f640 1 -- 172.21.15.89:0/1944865185 >> 172.21.15.89:0/1944865185 conn(0x7fa90809b9d0 msgr2=0x7fa9080d2670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:56.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.455+0000 7fa90cf8f640 1 -- 172.21.15.89:0/1944865185 shutdown_connections 2023-10-18T10:45:56.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.455+0000 7fa90cf8f640 1 -- 172.21.15.89:0/1944865185 wait complete. 2023-10-18T10:45:56.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.455+0000 7fa90cf8f640 1 Processor -- start 2023-10-18T10:45:56.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.455+0000 7fa90cf8f640 1 -- start start 2023-10-18T10:45:56.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.455+0000 7fa90cf8f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9080cd630 0x7fa908154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:56.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.455+0000 7fa90cf8f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa9080d0bc0 0x7fa9080ca610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:56.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.455+0000 7fa90cf8f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9080d2d80 0x7fa9080b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:56.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.455+0000 7fa90cf8f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9081548e0 con 0x7fa9080cd630 2023-10-18T10:45:56.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.455+0000 7fa90cf8f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa908155240 con 0x7fa9080d2d80 2023-10-18T10:45:56.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.455+0000 7fa90cf8f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9081593d0 con 0x7fa9080d0bc0 2023-10-18T10:45:56.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.455+0000 7fa90659c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9080cd630 0x7fa908154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:56.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.455+0000 7fa90659c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9080cd630 0x7fa908154d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48806/0 (socket says 172.21.15.89:48806) 2023-10-18T10:45:56.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.455+0000 7fa90659c640 1 -- 172.21.15.89:0/3158974094 learned_addr learned my addr 172.21.15.89:0/3158974094 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:45:56.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.455+0000 7fa906d9d640 1 --2- 172.21.15.89:0/3158974094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9080d2d80 0x7fa9080b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:56.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.455+0000 7fa905d9b640 1 --2- 172.21.15.89:0/3158974094 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa9080d0bc0 0x7fa9080ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:56.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.459+0000 7fa90659c640 1 -- 172.21.15.89:0/3158974094 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa9080d0bc0 msgr2=0x7fa9080ca610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:56.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.459+0000 7fa90659c640 1 --2- 172.21.15.89:0/3158974094 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa9080d0bc0 0x7fa9080ca610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:56.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.459+0000 7fa90659c640 1 -- 172.21.15.89:0/3158974094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9080d2d80 msgr2=0x7fa9080b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:56.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.459+0000 7fa90659c640 1 --2- 172.21.15.89:0/3158974094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9080d2d80 0x7fa9080b06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:56.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.459+0000 7fa90659c640 1 -- 172.21.15.89:0/3158974094 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8fc0094d0 con 0x7fa9080cd630 2023-10-18T10:45:56.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.459+0000 7fa90659c640 1 --2- 172.21.15.89:0/3158974094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9080cd630 0x7fa908154d00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa8f800e9b0 tx=0x7fa8f800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:56.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.459+0000 7fa8f6ffd640 1 -- 172.21.15.89:0/3158974094 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8009670 con 0x7fa9080cd630 2023-10-18T10:45:56.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.459+0000 7fa8f6ffd640 1 -- 172.21.15.89:0/3158974094 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f800c9a0 con 0x7fa9080cd630 2023-10-18T10:45:56.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.459+0000 7fa90cf8f640 1 -- 172.21.15.89:0/3158974094 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa908154220 con 0x7fa9080cd630 2023-10-18T10:45:56.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.459+0000 7fa8f6ffd640 1 -- 172.21.15.89:0/3158974094 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f800cd10 con 0x7fa9080cd630 2023-10-18T10:45:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.459+0000 7fa90cf8f640 1 -- 172.21.15.89:0/3158974094 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9080cab50 con 0x7fa9080cd630 2023-10-18T10:45:56.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.459+0000 7fa90cf8f640 1 -- 172.21.15.89:0/3158974094 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa90804dbe0 con 0x7fa9080cd630 2023-10-18T10:45:56.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.459+0000 7fa8f6ffd640 1 -- 172.21.15.89:0/3158974094 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fa8f8013470 con 0x7fa9080cd630 2023-10-18T10:45:56.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.459+0000 7fa8f6ffd640 1 --2- 172.21.15.89:0/3158974094 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa8e8065680 0x7fa8e8067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:56.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.459+0000 7fa8f6ffd640 1 -- 172.21.15.89:0/3158974094 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa8f8083410 con 0x7fa9080cd630 2023-10-18T10:45:56.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.459+0000 7fa905d9b640 1 --2- 172.21.15.89:0/3158974094 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa8e8065680 0x7fa8e8067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:56.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.463+0000 7fa905d9b640 1 --2- 172.21.15.89:0/3158974094 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa8e8065680 0x7fa8e8067b40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc0ae000 tx=0x7fa8fc0a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:56.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.463+0000 7fa8f6ffd640 1 -- 172.21.15.89:0/3158974094 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fa8f8050ec0 con 0x7fa9080cd630 2023-10-18T10:45:56.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.619+0000 7fa90cf8f640 1 -- 172.21.15.89:0/3158974094 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fa9080b4d60 con 0x7fa9080cd630 2023-10-18T10:45:56.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.743+0000 7fa8f6ffd640 1 -- 172.21.15.89:0/3158974094 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v23) v1 ==== 157+0+0 (secure 0 0 0) 0x7fa8f80544e0 con 0x7fa9080cd630 2023-10-18T10:45:56.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.747+0000 7fa90cf8f640 1 -- 172.21.15.89:0/3158974094 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa8e8065680 msgr2=0x7fa8e8067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:56.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.747+0000 7fa90cf8f640 1 --2- 172.21.15.89:0/3158974094 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa8e8065680 0x7fa8e8067b40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc0ae000 tx=0x7fa8fc0a7000 comp rx=0 tx=0).stop 2023-10-18T10:45:56.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.747+0000 7fa90cf8f640 1 -- 172.21.15.89:0/3158974094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9080cd630 msgr2=0x7fa908154d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:56.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.747+0000 7fa90cf8f640 1 --2- 172.21.15.89:0/3158974094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9080cd630 0x7fa908154d00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa8f800e9b0 tx=0x7fa8f800ee70 comp rx=0 tx=0).stop 2023-10-18T10:45:56.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.747+0000 7fa90cf8f640 1 -- 172.21.15.89:0/3158974094 shutdown_connections 2023-10-18T10:45:56.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.747+0000 7fa90cf8f640 1 --2- 172.21.15.89:0/3158974094 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa8e8065680 0x7fa8e8067b40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:56.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.747+0000 7fa90cf8f640 1 --2- 172.21.15.89:0/3158974094 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa9080d0bc0 0x7fa9080ca610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:56.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.747+0000 7fa90cf8f640 1 --2- 172.21.15.89:0/3158974094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9080d2d80 0x7fa9080b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:56.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.747+0000 7fa90cf8f640 1 --2- 172.21.15.89:0/3158974094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9080cd630 0x7fa908154d00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:56.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.747+0000 7fa90cf8f640 1 -- 172.21.15.89:0/3158974094 >> 172.21.15.89:0/3158974094 conn(0x7fa90809b9d0 msgr2=0x7fa9080d2670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:56.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.747+0000 7fa90cf8f640 1 -- 172.21.15.89:0/3158974094 shutdown_connections 2023-10-18T10:45:56.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.747+0000 7fa90cf8f640 1 -- 172.21.15.89:0/3158974094 wait complete. 2023-10-18T10:45:56.766 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-10-18T10:45:56.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.911+0000 7f1f96c93640 1 Processor -- start 2023-10-18T10:45:56.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.911+0000 7f1f96c93640 1 -- start start 2023-10-18T10:45:56.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.911+0000 7f1f96c93640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1f900ceec0 0x7f1f90069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:56.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.911+0000 7f1f96c93640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f900d0c60 0x7f1f900d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:56.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.911+0000 7f1f96c93640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f900cd530 con 0x7f1f9015c880 2023-10-18T10:45:56.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.911+0000 7f1f96c93640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f900d1630 con 0x7f1f900ceec0 2023-10-18T10:45:56.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.911+0000 7f1f96c93640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9015cc60 con 0x7f1f900d0c60 2023-10-18T10:45:56.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.911+0000 7f1f87fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1f900ceec0 0x7f1f90069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:56.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.911+0000 7f1f95230640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f900d0c60 0x7f1f900d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:56.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.911+0000 7f1f94a2f640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f1f9015c880 0x7f1f901602e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.89:43904/0 (socket says 172.21.15.89:43904) 2023-10-18T10:45:56.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.911+0000 7f1f87fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1f900ceec0 0x7f1f90069bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:40514/0 (socket says 172.21.15.89:40514) 2023-10-18T10:45:56.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.911+0000 7f1f94a2f640 1 -- 172.21.15.89:0/2913332500 learned_addr learned my addr 172.21.15.89:0/2913332500 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:45:56.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.915+0000 7f1f95230640 1 -- 172.21.15.89:0/2913332500 >> v2:172.21.15.154:3300/0 conn(0x7f1f900ceec0 msgr2=0x7f1f90069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:56.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.915+0000 7f1f95230640 1 --2- 172.21.15.89:0/2913332500 >> v2:172.21.15.154:3300/0 conn(0x7f1f900ceec0 0x7f1f90069bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:56.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.915+0000 7f1f95230640 1 -- 172.21.15.89:0/2913332500 >> v1:172.21.15.154:6789/0 conn(0x7f1f9015c880 legacy=0x7f1f901602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:56.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.915+0000 7f1f95230640 1 -- 172.21.15.89:0/2913332500 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f90160ac0 con 0x7f1f900d0c60 2023-10-18T10:45:56.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.915+0000 7f1f87fff640 1 --2- 172.21.15.89:0/2913332500 >> v2:172.21.15.154:3300/0 conn(0x7f1f900ceec0 0x7f1f90069bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:45:56.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.915+0000 7f1f877fe640 1 -- 172.21.15.89:0/2913332500 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1074872513 0 0) 0x7f1f900cd530 con 0x7f1f9015c880 2023-10-18T10:45:56.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.915+0000 7f1f95230640 1 --2- 172.21.15.89:0/2913332500 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f900d0c60 0x7f1f900d10f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1f88009d90 tx=0x7f1f8801c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac6542fe26901572 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:56.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.915+0000 7f1f877fe640 1 -- 172.21.15.89:0/2913332500 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f880a59e0 con 0x7f1f900d0c60 2023-10-18T10:45:56.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.915+0000 7f1f877fe640 1 -- 172.21.15.89:0/2913332500 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f880a5b40 con 0x7f1f900d0c60 2023-10-18T10:45:56.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.915+0000 7f1f877fe640 1 -- 172.21.15.89:0/2913332500 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f880ad440 con 0x7f1f900d0c60 2023-10-18T10:45:56.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.919+0000 7f1f96c93640 1 -- 172.21.15.89:0/2913332500 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f900d0c60 msgr2=0x7f1f900d10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:56.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.919+0000 7f1f96c93640 1 --2- 172.21.15.89:0/2913332500 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f900d0c60 0x7f1f900d10f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1f88009d90 tx=0x7f1f8801c920 comp rx=0 tx=0).stop 2023-10-18T10:45:56.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.919+0000 7f1f96c93640 1 -- 172.21.15.89:0/2913332500 shutdown_connections 2023-10-18T10:45:56.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.919+0000 7f1f96c93640 1 --2- 172.21.15.89:0/2913332500 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f900d0c60 0x7f1f900d10f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:56.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.919+0000 7f1f96c93640 1 --2- 172.21.15.89:0/2913332500 >> v2:172.21.15.154:3300/0 conn(0x7f1f900ceec0 0x7f1f90069bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:56.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.919+0000 7f1f96c93640 1 -- 172.21.15.89:0/2913332500 >> 172.21.15.89:0/2913332500 conn(0x7f1f9008c0a0 msgr2=0x7f1f9009b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:56.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.919+0000 7f1f96c93640 1 -- 172.21.15.89:0/2913332500 shutdown_connections 2023-10-18T10:45:56.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.919+0000 7f1f96c93640 1 -- 172.21.15.89:0/2913332500 wait complete. 2023-10-18T10:45:56.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.919+0000 7f1f96c93640 1 Processor -- start 2023-10-18T10:45:56.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.919+0000 7f1f96c93640 1 -- start start 2023-10-18T10:45:56.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.919+0000 7f1f96c93640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f900ceec0 0x7f1f9014fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:56.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.919+0000 7f1f96c93640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f900d0c60 0x7f1f900ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:56.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.919+0000 7f1f96c93640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f9015c880 0x7f1f900af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:56.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.919+0000 7f1f96c93640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f90151300 con 0x7f1f9015c880 2023-10-18T10:45:56.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.919+0000 7f1f96c93640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f90151790 con 0x7f1f900ceec0 2023-10-18T10:45:56.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.919+0000 7f1f96c93640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f90150410 con 0x7f1f900d0c60 2023-10-18T10:45:56.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.919+0000 7f1f95230640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f9015c880 0x7f1f900af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:56.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.919+0000 7f1f95230640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f9015c880 0x7f1f900af7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48808/0 (socket says 172.21.15.89:48808) 2023-10-18T10:45:56.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.919+0000 7f1f95230640 1 -- 172.21.15.89:0/2269773551 learned_addr learned my addr 172.21.15.89:0/2269773551 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:45:56.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.919+0000 7f1f94a2f640 1 --2- 172.21.15.89:0/2269773551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f900ceec0 0x7f1f9014fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:56.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.919+0000 7f1f87fff640 1 --2- 172.21.15.89:0/2269773551 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f900d0c60 0x7f1f900ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:56.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.919+0000 7f1f95230640 1 -- 172.21.15.89:0/2269773551 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f900d0c60 msgr2=0x7f1f900ca8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:56.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.919+0000 7f1f95230640 1 --2- 172.21.15.89:0/2269773551 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f900d0c60 0x7f1f900ca8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:56.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.919+0000 7f1f95230640 1 -- 172.21.15.89:0/2269773551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f900ceec0 msgr2=0x7f1f9014fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:56.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.919+0000 7f1f95230640 1 --2- 172.21.15.89:0/2269773551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f900ceec0 0x7f1f9014fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:56.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.919+0000 7f1f95230640 1 -- 172.21.15.89:0/2269773551 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f80009420 con 0x7f1f9015c880 2023-10-18T10:45:56.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.919+0000 7f1f87fff640 1 --2- 172.21.15.89:0/2269773551 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f900d0c60 0x7f1f900ca8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:45:56.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.919+0000 7f1f94a2f640 1 --2- 172.21.15.89:0/2269773551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f900ceec0 0x7f1f9014fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:45:56.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.923+0000 7f1f95230640 1 --2- 172.21.15.89:0/2269773551 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f9015c880 0x7f1f900af7c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1f8801afd0 tx=0x7f1f880196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:56.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.923+0000 7f1f857fa640 1 -- 172.21.15.89:0/2269773551 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f880adcc0 con 0x7f1f9015c880 2023-10-18T10:45:56.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.923+0000 7f1f857fa640 1 -- 172.21.15.89:0/2269773551 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f880ade20 con 0x7f1f9015c880 2023-10-18T10:45:56.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.923+0000 7f1f857fa640 1 -- 172.21.15.89:0/2269773551 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f880b9690 con 0x7f1f9015c880 2023-10-18T10:45:56.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.923+0000 7f1f96c93640 1 -- 172.21.15.89:0/2269773551 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f88009a70 con 0x7f1f9015c880 2023-10-18T10:45:56.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.923+0000 7f1f96c93640 1 -- 172.21.15.89:0/2269773551 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f901509d0 con 0x7f1f9015c880 2023-10-18T10:45:56.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.923+0000 7f1f96c93640 1 -- 172.21.15.89:0/2269773551 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f901514b0 con 0x7f1f9015c880 2023-10-18T10:45:56.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.923+0000 7f1f857fa640 1 -- 172.21.15.89:0/2269773551 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f1f880b97f0 con 0x7f1f9015c880 2023-10-18T10:45:56.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.923+0000 7f1f857fa640 1 --2- 172.21.15.89:0/2269773551 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1f7c065680 0x7f1f7c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:56.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.923+0000 7f1f857fa640 1 -- 172.21.15.89:0/2269773551 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f1f8811e5b0 con 0x7f1f9015c880 2023-10-18T10:45:56.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.923+0000 7f1f94a2f640 1 --2- 172.21.15.89:0/2269773551 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1f7c065680 0x7f1f7c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:56.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.927+0000 7f1f94a2f640 1 --2- 172.21.15.89:0/2269773551 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1f7c065680 0x7f1f7c067b40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c000f70 tx=0x7f1f8c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:56.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:56.927+0000 7f1f857fa640 1 -- 172.21.15.89:0/2269773551 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f1f901514b0 con 0x7f1f9015c880 2023-10-18T10:45:57.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:57.083+0000 7f1f96c93640 1 -- 172.21.15.89:0/2269773551 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1f9006a100 con 0x7f1f9015c880 2023-10-18T10:45:57.455 INFO:tasks.ceph.mds.a.smithi089.stderr:2023-10-18T10:45:57.455+0000 7fc15bd82640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T10:45:58.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.755+0000 7f1f857fa640 1 -- 172.21.15.89:0/2269773551 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v25) v1 ==== 228+0+0 (secure 0 0 0) 0x7f1f880ec020 con 0x7f1f9015c880 2023-10-18T10:45:58.754 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data_ec' created 2023-10-18T10:45:58.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.759+0000 7f1f96c93640 1 -- 172.21.15.89:0/2269773551 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1f7c065680 msgr2=0x7f1f7c067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:58.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.759+0000 7f1f96c93640 1 --2- 172.21.15.89:0/2269773551 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1f7c065680 0x7f1f7c067b40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c000f70 tx=0x7f1f8c009040 comp rx=0 tx=0).stop 2023-10-18T10:45:58.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.759+0000 7f1f96c93640 1 -- 172.21.15.89:0/2269773551 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f9015c880 msgr2=0x7f1f900af7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:58.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.759+0000 7f1f96c93640 1 --2- 172.21.15.89:0/2269773551 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f9015c880 0x7f1f900af7c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1f8801afd0 tx=0x7f1f880196c0 comp rx=0 tx=0).stop 2023-10-18T10:45:58.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.759+0000 7f1f96c93640 1 -- 172.21.15.89:0/2269773551 shutdown_connections 2023-10-18T10:45:58.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.759+0000 7f1f96c93640 1 --2- 172.21.15.89:0/2269773551 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1f7c065680 0x7f1f7c067b40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:58.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.759+0000 7f1f96c93640 1 --2- 172.21.15.89:0/2269773551 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f900d0c60 0x7f1f900ca8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:58.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.759+0000 7f1f96c93640 1 --2- 172.21.15.89:0/2269773551 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f9015c880 0x7f1f900af7c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:58.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.759+0000 7f1f96c93640 1 --2- 172.21.15.89:0/2269773551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f900ceec0 0x7f1f9014fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:58.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.759+0000 7f1f96c93640 1 -- 172.21.15.89:0/2269773551 >> 172.21.15.89:0/2269773551 conn(0x7f1f9008c0a0 msgr2=0x7f1f900b0850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:58.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.759+0000 7f1f96c93640 1 -- 172.21.15.89:0/2269773551 shutdown_connections 2023-10-18T10:45:58.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.759+0000 7f1f96c93640 1 -- 172.21.15.89:0/2269773551 wait complete. 2023-10-18T10:45:58.789 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-10-18T10:45:58.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.935+0000 7f6ae9a34640 1 Processor -- start 2023-10-18T10:45:58.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.935+0000 7f6ae9a34640 1 -- start start 2023-10-18T10:45:58.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.935+0000 7f6ae9a34640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6ae40cd570 0x7f6ae40cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:58.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.935+0000 7f6ae9a34640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ae40cd950 0x7f6ae406adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:58.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.935+0000 7f6ae9a34640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ae41604e0 con 0x7f6ae40d02c0 2023-10-18T10:45:58.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.935+0000 7f6ae9a34640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ae415cb50 con 0x7f6ae40cd570 2023-10-18T10:45:58.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.935+0000 7f6ae9a34640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae41607c0 con 0x7f6ae40cd950 2023-10-18T10:45:58.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.935+0000 7f6ae2ffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6ae40cd570 0x7f6ae40cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:58.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.935+0000 7f6ae37fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ae40cd950 0x7f6ae406adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:58.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.935+0000 7f6ae27fc640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f6ae40d02c0 0x7f6ae40ce140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.89:43914/0 (socket says 172.21.15.89:43914) 2023-10-18T10:45:58.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.935+0000 7f6ae2ffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6ae40cd570 0x7f6ae40cfd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:40532/0 (socket says 172.21.15.89:40532) 2023-10-18T10:45:58.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.935+0000 7f6ae27fc640 1 -- 172.21.15.89:0/3780323593 learned_addr learned my addr 172.21.15.89:0/3780323593 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:45:58.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.935+0000 7f6ae2ffd640 1 -- 172.21.15.89:0/3780323593 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ae40cd950 msgr2=0x7f6ae406adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:58.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.935+0000 7f6ae2ffd640 1 --2- 172.21.15.89:0/3780323593 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ae40cd950 0x7f6ae406adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:58.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.935+0000 7f6ae2ffd640 1 -- 172.21.15.89:0/3780323593 >> v1:172.21.15.154:6789/0 conn(0x7f6ae40d02c0 legacy=0x7f6ae40ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:58.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.935+0000 7f6ae2ffd640 1 -- 172.21.15.89:0/3780323593 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae4160aa0 con 0x7f6ae40cd570 2023-10-18T10:45:58.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.935+0000 7f6ae1ffb640 1 -- 172.21.15.89:0/3780323593 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1738476912 0 0) 0x7f6ae41604e0 con 0x7f6ae40d02c0 2023-10-18T10:45:58.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.939+0000 7f6ae2ffd640 1 --2- 172.21.15.89:0/3780323593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ae40cd570 0x7f6ae40cfd80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6ad40097c0 tx=0x7f6ad401c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a5409e71d5ebd44 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:58.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.939+0000 7f6ae1ffb640 1 -- 172.21.15.89:0/3780323593 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad40ab400 con 0x7f6ae40cd570 2023-10-18T10:45:58.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.939+0000 7f6ae1ffb640 1 -- 172.21.15.89:0/3780323593 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad40a7050 con 0x7f6ae40cd570 2023-10-18T10:45:58.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.939+0000 7f6ae1ffb640 1 -- 172.21.15.89:0/3780323593 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad40ab780 con 0x7f6ae40cd570 2023-10-18T10:45:58.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.939+0000 7f6ae9a34640 1 -- 172.21.15.89:0/3780323593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ae40cd570 msgr2=0x7f6ae40cfd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:58.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.939+0000 7f6ae9a34640 1 --2- 172.21.15.89:0/3780323593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ae40cd570 0x7f6ae40cfd80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6ad40097c0 tx=0x7f6ad401c480 comp rx=0 tx=0).stop 2023-10-18T10:45:58.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.939+0000 7f6ae9a34640 1 -- 172.21.15.89:0/3780323593 shutdown_connections 2023-10-18T10:45:58.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.939+0000 7f6ae9a34640 1 --2- 172.21.15.89:0/3780323593 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ae40cd950 0x7f6ae406adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:58.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.939+0000 7f6ae9a34640 1 --2- 172.21.15.89:0/3780323593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ae40cd570 0x7f6ae40cfd80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:58.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.939+0000 7f6ae9a34640 1 -- 172.21.15.89:0/3780323593 >> 172.21.15.89:0/3780323593 conn(0x7f6ae406bb10 msgr2=0x7f6ae406bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:58.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.939+0000 7f6ae9a34640 1 -- 172.21.15.89:0/3780323593 shutdown_connections 2023-10-18T10:45:58.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.939+0000 7f6ae9a34640 1 -- 172.21.15.89:0/3780323593 wait complete. 2023-10-18T10:45:58.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.943+0000 7f6ae9a34640 1 Processor -- start 2023-10-18T10:45:58.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.943+0000 7f6ae9a34640 1 -- start start 2023-10-18T10:45:58.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.943+0000 7f6ae9a34640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ae40cd570 0x7f6ae414d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:58.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.943+0000 7f6ae9a34640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ae40cd950 0x7f6ae414a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:58.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.943+0000 7f6ae9a34640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ae414aff0 0x7f6ae4156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:58.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.943+0000 7f6ae9a34640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae414dce0 con 0x7f6ae414aff0 2023-10-18T10:45:58.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.943+0000 7f6ae9a34640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae4155490 con 0x7f6ae40cd570 2023-10-18T10:45:58.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.943+0000 7f6ae9a34640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae4155770 con 0x7f6ae40cd950 2023-10-18T10:45:58.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.943+0000 7f6ae37fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ae414aff0 0x7f6ae4156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:58.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.943+0000 7f6ae37fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ae414aff0 0x7f6ae4156530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48816/0 (socket says 172.21.15.89:48816) 2023-10-18T10:45:58.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.943+0000 7f6ae37fe640 1 -- 172.21.15.89:0/1660968494 learned_addr learned my addr 172.21.15.89:0/1660968494 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:45:58.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.943+0000 7f6ae2ffd640 1 --2- 172.21.15.89:0/1660968494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ae40cd570 0x7f6ae414d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:58.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.943+0000 7f6ae27fc640 1 --2- 172.21.15.89:0/1660968494 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ae40cd950 0x7f6ae414a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:58.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.943+0000 7f6ae37fe640 1 -- 172.21.15.89:0/1660968494 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ae40cd950 msgr2=0x7f6ae414a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:58.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.943+0000 7f6ae37fe640 1 --2- 172.21.15.89:0/1660968494 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ae40cd950 0x7f6ae414a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:58.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.943+0000 7f6ae37fe640 1 -- 172.21.15.89:0/1660968494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ae40cd570 msgr2=0x7f6ae414d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:58.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.943+0000 7f6ae37fe640 1 --2- 172.21.15.89:0/1660968494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ae40cd570 0x7f6ae414d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:58.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.943+0000 7f6ae37fe640 1 -- 172.21.15.89:0/1660968494 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad4009420 con 0x7f6ae414aff0 2023-10-18T10:45:58.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.943+0000 7f6ae37fe640 1 --2- 172.21.15.89:0/1660968494 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ae414aff0 0x7f6ae4156530 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6adc00efc0 tx=0x7f6adc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:58.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.943+0000 7f6acbfff640 1 -- 172.21.15.89:0/1660968494 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc009ca0 con 0x7f6ae414aff0 2023-10-18T10:45:58.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.943+0000 7f6acbfff640 1 -- 172.21.15.89:0/1660968494 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6adc00f040 con 0x7f6ae414aff0 2023-10-18T10:45:58.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.943+0000 7f6acbfff640 1 -- 172.21.15.89:0/1660968494 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc0146c0 con 0x7f6ae414aff0 2023-10-18T10:45:58.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.943+0000 7f6ae9a34640 1 -- 172.21.15.89:0/1660968494 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ae4155a50 con 0x7f6ae414aff0 2023-10-18T10:45:58.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.943+0000 7f6ae9a34640 1 -- 172.21.15.89:0/1660968494 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ae4156010 con 0x7f6ae414aff0 2023-10-18T10:45:58.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.947+0000 7f6ae9a34640 1 -- 172.21.15.89:0/1660968494 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ae4073b50 con 0x7f6ae414aff0 2023-10-18T10:45:58.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.947+0000 7f6acbfff640 1 -- 172.21.15.89:0/1660968494 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f6adc014820 con 0x7f6ae414aff0 2023-10-18T10:45:58.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.947+0000 7f6acbfff640 1 --2- 172.21.15.89:0/1660968494 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6ad0065680 0x7f6ad0067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:58.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.947+0000 7f6acbfff640 1 -- 172.21.15.89:0/1660968494 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6781+0+0 (secure 0 0 0) 0x7f6adc083fc0 con 0x7f6ae414aff0 2023-10-18T10:45:58.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.947+0000 7f6ae2ffd640 1 --2- 172.21.15.89:0/1660968494 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6ad0065680 0x7f6ad0067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:58.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.951+0000 7f6ae2ffd640 1 --2- 172.21.15.89:0/1660968494 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6ad0065680 0x7f6ad0067b40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6ad40ad000 tx=0x7f6ad4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:58.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:58.951+0000 7f6acbfff640 1 -- 172.21.15.89:0/1660968494 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f6adc051740 con 0x7f6ae414aff0 2023-10-18T10:45:59.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.103+0000 7f6ae9a34640 1 -- 172.21.15.89:0/1660968494 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f6ae415f6b0 con 0x7f6ae414aff0 2023-10-18T10:45:59.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.763+0000 7f6acbfff640 1 -- 172.21.15.89:0/1660968494 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 5 allow_ec_overwrites to true v26) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6adc0180c0 con 0x7f6ae414aff0 2023-10-18T10:45:59.763 INFO:teuthology.orchestra.run.smithi089.stderr:set pool 5 allow_ec_overwrites to true 2023-10-18T10:45:59.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.767+0000 7f6ae9a34640 1 -- 172.21.15.89:0/1660968494 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6ad0065680 msgr2=0x7f6ad0067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:59.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.767+0000 7f6ae9a34640 1 --2- 172.21.15.89:0/1660968494 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6ad0065680 0x7f6ad0067b40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6ad40ad000 tx=0x7f6ad4008040 comp rx=0 tx=0).stop 2023-10-18T10:45:59.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.767+0000 7f6ae9a34640 1 -- 172.21.15.89:0/1660968494 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ae414aff0 msgr2=0x7f6ae4156530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:59.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.767+0000 7f6ae9a34640 1 --2- 172.21.15.89:0/1660968494 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ae414aff0 0x7f6ae4156530 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6adc00efc0 tx=0x7f6adc00c490 comp rx=0 tx=0).stop 2023-10-18T10:45:59.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.771+0000 7f6ae9a34640 1 -- 172.21.15.89:0/1660968494 shutdown_connections 2023-10-18T10:45:59.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.771+0000 7f6ae9a34640 1 --2- 172.21.15.89:0/1660968494 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6ad0065680 0x7f6ad0067b40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:59.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.771+0000 7f6ae9a34640 1 --2- 172.21.15.89:0/1660968494 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ae40cd950 0x7f6ae414a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:59.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.771+0000 7f6ae9a34640 1 --2- 172.21.15.89:0/1660968494 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ae414aff0 0x7f6ae4156530 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:59.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.771+0000 7f6ae9a34640 1 --2- 172.21.15.89:0/1660968494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ae40cd570 0x7f6ae414d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:59.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.771+0000 7f6ae9a34640 1 -- 172.21.15.89:0/1660968494 >> 172.21.15.89:0/1660968494 conn(0x7f6ae406bb10 msgr2=0x7f6ae406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:59.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.775+0000 7f6ae9a34640 1 -- 172.21.15.89:0/1660968494 shutdown_connections 2023-10-18T10:45:59.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.775+0000 7f6ae9a34640 1 -- 172.21.15.89:0/1660968494 wait complete. 2023-10-18T10:45:59.819 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-10-18T10:45:59.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.963+0000 7f068166a640 1 Processor -- start 2023-10-18T10:45:59.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.963+0000 7f068166a640 1 -- start start 2023-10-18T10:45:59.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.963+0000 7f068166a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f067c15c550 0x7f067c0d2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:59.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.963+0000 7f068166a640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f067c0cfd20 0x7f067c0b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:59.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.963+0000 7f068166a640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f067c0d1e50 con 0x7f067c0b1860 2023-10-18T10:45:59.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.963+0000 7f068166a640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f067c0d2130 con 0x7f067c0cfd20 2023-10-18T10:45:59.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.963+0000 7f068166a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f067c0d2410 con 0x7f067c15c550 2023-10-18T10:45:59.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.963+0000 7f067b7fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f067c0cfd20 0x7f067c0b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:59.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.963+0000 7f067b7fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f067c0cfd20 0x7f067c0b55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48832/0 (socket says 172.21.15.89:48832) 2023-10-18T10:45:59.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.963+0000 7f067b7fe640 1 -- 172.21.15.89:0/1714415281 learned_addr learned my addr 172.21.15.89:0/1714415281 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:45:59.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.963+0000 7f067affd640 1 --2- 172.21.15.89:0/1714415281 >> v2:172.21.15.154:3300/0 conn(0x7f067c15c550 0x7f067c0d2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:59.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.963+0000 7f067b7fe640 1 -- 172.21.15.89:0/1714415281 >> v2:172.21.15.154:3300/0 conn(0x7f067c15c550 msgr2=0x7f067c0d2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:59.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.963+0000 7f067b7fe640 1 --2- 172.21.15.89:0/1714415281 >> v2:172.21.15.154:3300/0 conn(0x7f067c15c550 0x7f067c0d2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:59.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.963+0000 7f067b7fe640 1 -- 172.21.15.89:0/1714415281 >> v1:172.21.15.89:6789/0 conn(0x7f067c0b1860 legacy=0x7f067c0cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:59.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.963+0000 7f067b7fe640 1 -- 172.21.15.89:0/1714415281 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f067c0d26f0 con 0x7f067c0cfd20 2023-10-18T10:45:59.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.967+0000 7f067b7fe640 1 --2- 172.21.15.89:0/1714415281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f067c0cfd20 0x7f067c0b55f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f067c06b3b0 tx=0x7f067401cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6100511533dbd7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:59.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.967+0000 7f0679ffb640 1 -- 172.21.15.89:0/1714415281 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06740a7070 con 0x7f067c0cfd20 2023-10-18T10:45:59.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.967+0000 7f0679ffb640 1 -- 172.21.15.89:0/1714415281 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06740ab430 con 0x7f067c0cfd20 2023-10-18T10:45:59.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.967+0000 7f0679ffb640 1 -- 172.21.15.89:0/1714415281 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06740ab7c0 con 0x7f067c0cfd20 2023-10-18T10:45:59.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.967+0000 7f068166a640 1 -- 172.21.15.89:0/1714415281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f067c0cfd20 msgr2=0x7f067c0b55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:59.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.967+0000 7f068166a640 1 --2- 172.21.15.89:0/1714415281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f067c0cfd20 0x7f067c0b55f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f067c06b3b0 tx=0x7f067401cf30 comp rx=0 tx=0).stop 2023-10-18T10:45:59.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.967+0000 7f068166a640 1 -- 172.21.15.89:0/1714415281 shutdown_connections 2023-10-18T10:45:59.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.967+0000 7f068166a640 1 --2- 172.21.15.89:0/1714415281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f067c0cfd20 0x7f067c0b55f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:59.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.967+0000 7f068166a640 1 --2- 172.21.15.89:0/1714415281 >> v2:172.21.15.154:3300/0 conn(0x7f067c15c550 0x7f067c0d2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:59.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.967+0000 7f068166a640 1 -- 172.21.15.89:0/1714415281 >> 172.21.15.89:0/1714415281 conn(0x7f067c069d00 msgr2=0x7f067c06a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:59.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.967+0000 7f068166a640 1 -- 172.21.15.89:0/1714415281 shutdown_connections 2023-10-18T10:45:59.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.967+0000 7f068166a640 1 -- 172.21.15.89:0/1714415281 wait complete. 2023-10-18T10:45:59.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.971+0000 7f068166a640 1 Processor -- start 2023-10-18T10:45:59.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.971+0000 7f068166a640 1 -- start start 2023-10-18T10:45:59.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.971+0000 7f068166a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f067c0b1860 0x7f067c1f52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:59.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.971+0000 7f068166a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f067c0cfd20 0x7f067c0b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:59.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.971+0000 7f068166a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f067c15c550 0x7f067c152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:59.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.971+0000 7f068166a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067c15e210 con 0x7f067c15c550 2023-10-18T10:45:59.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.971+0000 7f068166a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067c1f5800 con 0x7f067c0cfd20 2023-10-18T10:45:59.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.971+0000 7f068166a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f067c15d170 con 0x7f067c0b1860 2023-10-18T10:45:59.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.971+0000 7f067b7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f067c15c550 0x7f067c152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:59.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.971+0000 7f067b7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f067c15c550 0x7f067c152250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48842/0 (socket says 172.21.15.89:48842) 2023-10-18T10:45:59.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.971+0000 7f067b7fe640 1 -- 172.21.15.89:0/2716315098 learned_addr learned my addr 172.21.15.89:0/2716315098 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:45:59.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.971+0000 7f067affd640 1 --2- 172.21.15.89:0/2716315098 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f067c0b1860 0x7f067c1f52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:59.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.971+0000 7f067a7fc640 1 --2- 172.21.15.89:0/2716315098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f067c0cfd20 0x7f067c0b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:59.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.971+0000 7f067b7fe640 1 -- 172.21.15.89:0/2716315098 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f067c0b1860 msgr2=0x7f067c1f52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:59.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.971+0000 7f067b7fe640 1 --2- 172.21.15.89:0/2716315098 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f067c0b1860 0x7f067c1f52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:59.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.971+0000 7f067b7fe640 1 -- 172.21.15.89:0/2716315098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f067c0cfd20 msgr2=0x7f067c0b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:59.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.971+0000 7f067b7fe640 1 --2- 172.21.15.89:0/2716315098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f067c0cfd20 0x7f067c0b2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:59.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.971+0000 7f067b7fe640 1 -- 172.21.15.89:0/2716315098 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06740094d0 con 0x7f067c15c550 2023-10-18T10:45:59.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.971+0000 7f067b7fe640 1 --2- 172.21.15.89:0/2716315098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f067c15c550 0x7f067c152250 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f067401a100 tx=0x7f067401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:59.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.971+0000 7f06637fe640 1 -- 172.21.15.89:0/2716315098 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06740b4800 con 0x7f067c15c550 2023-10-18T10:45:59.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.971+0000 7f06637fe640 1 -- 172.21.15.89:0/2716315098 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06740b4960 con 0x7f067c15c550 2023-10-18T10:45:59.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.971+0000 7f068166a640 1 -- 172.21.15.89:0/2716315098 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f067c15d450 con 0x7f067c15c550 2023-10-18T10:45:59.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.971+0000 7f06637fe640 1 -- 172.21.15.89:0/2716315098 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06740b4d10 con 0x7f067c15c550 2023-10-18T10:45:59.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.971+0000 7f068166a640 1 -- 172.21.15.89:0/2716315098 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f067c15da10 con 0x7f067c15c550 2023-10-18T10:45:59.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.975+0000 7f068166a640 1 -- 172.21.15.89:0/2716315098 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f067c159660 con 0x7f067c15c550 2023-10-18T10:45:59.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.975+0000 7f06637fe640 1 -- 172.21.15.89:0/2716315098 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f06740b3380 con 0x7f067c15c550 2023-10-18T10:45:59.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.975+0000 7f06637fe640 1 --2- 172.21.15.89:0/2716315098 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0668065680 0x7f0668067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:59.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.975+0000 7f06637fe640 1 -- 172.21.15.89:0/2716315098 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6781+0+0 (secure 0 0 0) 0x7f067411cfd0 con 0x7f067c15c550 2023-10-18T10:45:59.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.975+0000 7f067affd640 1 --2- 172.21.15.89:0/2716315098 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0668065680 0x7f0668067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:59.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.975+0000 7f067affd640 1 --2- 172.21.15.89:0/2716315098 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0668065680 0x7f0668067b40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f067c166e70 tx=0x7f066c009830 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:59.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:45:59.979+0000 7f06637fe640 1 -- 172.21.15.89:0/2716315098 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f06740ea7d0 con 0x7f067c15c550 2023-10-18T10:46:00.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.163+0000 7f068166a640 1 -- 172.21.15.89:0/2716315098 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f067c0cb780 con 0x7f067c15c550 2023-10-18T10:46:00.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.762+0000 7f06637fe640 1 -- 172.21.15.89:0/2716315098 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 5 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7f06740eddf0 con 0x7f067c15c550 2023-10-18T10:46:00.765 INFO:teuthology.orchestra.run.smithi089.stderr:added data pool 5 to fsmap 2023-10-18T10:46:00.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.766+0000 7f068166a640 1 -- 172.21.15.89:0/2716315098 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0668065680 msgr2=0x7f0668067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:00.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.766+0000 7f068166a640 1 --2- 172.21.15.89:0/2716315098 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0668065680 0x7f0668067b40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f067c166e70 tx=0x7f066c009830 comp rx=0 tx=0).stop 2023-10-18T10:46:00.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.766+0000 7f068166a640 1 -- 172.21.15.89:0/2716315098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f067c15c550 msgr2=0x7f067c152250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:00.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.766+0000 7f068166a640 1 --2- 172.21.15.89:0/2716315098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f067c15c550 0x7f067c152250 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f067401a100 tx=0x7f067401ad10 comp rx=0 tx=0).stop 2023-10-18T10:46:00.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.766+0000 7f068166a640 1 -- 172.21.15.89:0/2716315098 shutdown_connections 2023-10-18T10:46:00.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.766+0000 7f068166a640 1 --2- 172.21.15.89:0/2716315098 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0668065680 0x7f0668067b40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:00.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.766+0000 7f068166a640 1 --2- 172.21.15.89:0/2716315098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f067c15c550 0x7f067c152250 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:00.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.766+0000 7f068166a640 1 --2- 172.21.15.89:0/2716315098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f067c0cfd20 0x7f067c0b2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:00.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.766+0000 7f068166a640 1 --2- 172.21.15.89:0/2716315098 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f067c0b1860 0x7f067c1f52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:00.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.766+0000 7f068166a640 1 -- 172.21.15.89:0/2716315098 >> 172.21.15.89:0/2716315098 conn(0x7f067c069d00 msgr2=0x7f067c06a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:00.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.766+0000 7f068166a640 1 -- 172.21.15.89:0/2716315098 shutdown_connections 2023-10-18T10:46:00.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.766+0000 7f068166a640 1 -- 172.21.15.89:0/2716315098 wait complete. 2023-10-18T10:46:00.787 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:46:00.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.922+0000 7f4d83f96640 1 Processor -- start 2023-10-18T10:46:00.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.922+0000 7f4d83f96640 1 -- start start 2023-10-18T10:46:00.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.926+0000 7f4d83f96640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d7c0c8960 0x7f4d7c06c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:00.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.926+0000 7f4d83f96640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d7c1614e0 con 0x7f4d7c15e600 2023-10-18T10:46:00.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.926+0000 7f4d83f96640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d7c0c8dc0 con 0x7f4d7c0afd90 2023-10-18T10:46:00.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.926+0000 7f4d83f96640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c06c6f0 con 0x7f4d7c0c8960 2023-10-18T10:46:00.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.926+0000 7f4d81d32640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f4d7c15e600 0x7f4d7c161970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55152/0 (socket says 172.21.15.89:55152) 2023-10-18T10:46:00.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.926+0000 7f4d81d32640 1 -- 172.21.15.89:0/2060902020 learned_addr learned my addr 172.21.15.89:0/2060902020 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:00.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.926+0000 7f4d81531640 1 --2- 172.21.15.89:0/2060902020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d7c0c8960 0x7f4d7c06c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:00.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.926+0000 7f4d80d30640 1 -- 172.21.15.89:0/2060902020 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2072013494 0 0) 0x7f4d7c1614e0 con 0x7f4d7c15e600 2023-10-18T10:46:00.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.926+0000 7f4d80d30640 1 -- 172.21.15.89:0/2060902020 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d68003530 con 0x7f4d7c15e600 2023-10-18T10:46:00.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.926+0000 7f4d80d30640 1 -- 172.21.15.89:0/2060902020 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3975959012 0 0) 0x7f4d7c0c8dc0 con 0x7f4d7c0afd90 2023-10-18T10:46:00.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.926+0000 7f4d80d30640 1 -- 172.21.15.89:0/2060902020 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d7c1614e0 con 0x7f4d7c0afd90 2023-10-18T10:46:00.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.926+0000 7f4d81531640 1 -- 172.21.15.89:0/2060902020 >> v1:172.21.15.154:6789/0 conn(0x7f4d7c0afd90 legacy=0x7f4d7c0c1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:00.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.926+0000 7f4d81531640 1 -- 172.21.15.89:0/2060902020 >> v1:172.21.15.89:6789/0 conn(0x7f4d7c15e600 legacy=0x7f4d7c161970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:00.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.926+0000 7f4d81531640 1 -- 172.21.15.89:0/2060902020 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d7c15e9e0 con 0x7f4d7c0c8960 2023-10-18T10:46:00.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.930+0000 7f4d81531640 1 --2- 172.21.15.89:0/2060902020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d7c0c8960 0x7f4d7c06c1b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c0097f0 tx=0x7f4d6c01c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d100775527366471 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:00.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.930+0000 7f4d80d30640 1 -- 172.21.15.89:0/2060902020 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c0ab600 con 0x7f4d7c0c8960 2023-10-18T10:46:00.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.930+0000 7f4d80d30640 1 -- 172.21.15.89:0/2060902020 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6c0a7050 con 0x7f4d7c0c8960 2023-10-18T10:46:00.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.930+0000 7f4d80d30640 1 -- 172.21.15.89:0/2060902020 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c0ab980 con 0x7f4d7c0c8960 2023-10-18T10:46:00.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.930+0000 7f4d83f96640 1 -- 172.21.15.89:0/2060902020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d7c0c8960 msgr2=0x7f4d7c06c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:00.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.930+0000 7f4d83f96640 1 --2- 172.21.15.89:0/2060902020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d7c0c8960 0x7f4d7c06c1b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c0097f0 tx=0x7f4d6c01c4c0 comp rx=0 tx=0).stop 2023-10-18T10:46:00.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.930+0000 7f4d83f96640 1 -- 172.21.15.89:0/2060902020 shutdown_connections 2023-10-18T10:46:00.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.930+0000 7f4d83f96640 1 --2- 172.21.15.89:0/2060902020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d7c0c8960 0x7f4d7c06c1b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:00.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.930+0000 7f4d83f96640 1 -- 172.21.15.89:0/2060902020 >> 172.21.15.89:0/2060902020 conn(0x7f4d7c069f00 msgr2=0x7f4d7c0cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:00.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.930+0000 7f4d83f96640 1 -- 172.21.15.89:0/2060902020 shutdown_connections 2023-10-18T10:46:00.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.930+0000 7f4d83f96640 1 -- 172.21.15.89:0/2060902020 wait complete. 2023-10-18T10:46:00.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.930+0000 7f4d83f96640 1 Processor -- start 2023-10-18T10:46:00.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.930+0000 7f4d83f96640 1 -- start start 2023-10-18T10:46:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.934+0000 7f4d83f96640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d7c0afd90 0x7f4d7c156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.934+0000 7f4d83f96640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d7c0c8960 0x7f4d7c0c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.934+0000 7f4d83f96640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d7c15e600 0x7f4d7c0c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.934+0000 7f4d83f96640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c156550 con 0x7f4d7c0afd90 2023-10-18T10:46:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.934+0000 7f4d83f96640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c0c5dc0 con 0x7f4d7c15e600 2023-10-18T10:46:00.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.934+0000 7f4d83f96640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c0c60a0 con 0x7f4d7c0c8960 2023-10-18T10:46:00.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.934+0000 7f4d81d32640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d7c0afd90 0x7f4d7c156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:00.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.934+0000 7f4d81d32640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d7c0afd90 0x7f4d7c156010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48846/0 (socket says 172.21.15.89:48846) 2023-10-18T10:46:00.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.934+0000 7f4d81d32640 1 -- 172.21.15.89:0/3824776959 learned_addr learned my addr 172.21.15.89:0/3824776959 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:00.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.934+0000 7f4d81531640 1 --2- 172.21.15.89:0/3824776959 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d7c0c8960 0x7f4d7c0c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:00.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.934+0000 7f4d82533640 1 --2- 172.21.15.89:0/3824776959 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d7c15e600 0x7f4d7c0c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:00.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.934+0000 7f4d81d32640 1 -- 172.21.15.89:0/3824776959 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d7c0c8960 msgr2=0x7f4d7c0c53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:00.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.934+0000 7f4d81d32640 1 --2- 172.21.15.89:0/3824776959 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d7c0c8960 0x7f4d7c0c53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:00.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.934+0000 7f4d81d32640 1 -- 172.21.15.89:0/3824776959 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d7c15e600 msgr2=0x7f4d7c0c6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:00.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.934+0000 7f4d81d32640 1 --2- 172.21.15.89:0/3824776959 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d7c15e600 0x7f4d7c0c6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:00.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.934+0000 7f4d81d32640 1 -- 172.21.15.89:0/3824776959 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6c009420 con 0x7f4d7c0afd90 2023-10-18T10:46:00.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.934+0000 7f4d81d32640 1 --2- 172.21.15.89:0/3824776959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d7c0afd90 0x7f4d7c156010 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800c4d0 tx=0x7f4d7800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:00.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.934+0000 7f4d727fc640 1 -- 172.21.15.89:0/3824776959 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d78005cf0 con 0x7f4d7c0afd90 2023-10-18T10:46:00.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.934+0000 7f4d727fc640 1 -- 172.21.15.89:0/3824776959 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d78005e50 con 0x7f4d7c0afd90 2023-10-18T10:46:00.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.934+0000 7f4d727fc640 1 -- 172.21.15.89:0/3824776959 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d78007290 con 0x7f4d7c0afd90 2023-10-18T10:46:00.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.934+0000 7f4d83f96640 1 -- 172.21.15.89:0/3824776959 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d7c0c6380 con 0x7f4d7c0afd90 2023-10-18T10:46:00.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.934+0000 7f4d83f96640 1 -- 172.21.15.89:0/3824776959 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d7c0c6940 con 0x7f4d7c0afd90 2023-10-18T10:46:00.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.934+0000 7f4d83f96640 1 -- 172.21.15.89:0/3824776959 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d7c0c6b30 con 0x7f4d7c0afd90 2023-10-18T10:46:00.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.934+0000 7f4d727fc640 1 -- 172.21.15.89:0/3824776959 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f4d780073f0 con 0x7f4d7c0afd90 2023-10-18T10:46:00.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.938+0000 7f4d727fc640 1 --2- 172.21.15.89:0/3824776959 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4d60065680 0x7f4d60067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:00.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.938+0000 7f4d727fc640 1 -- 172.21.15.89:0/3824776959 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4d78082680 con 0x7f4d7c0afd90 2023-10-18T10:46:00.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.938+0000 7f4d81531640 1 --2- 172.21.15.89:0/3824776959 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4d60065680 0x7f4d60067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:00.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.938+0000 7f4d81531640 1 --2- 172.21.15.89:0/3824776959 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4d60065680 0x7f4d60067b40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c0ad000 tx=0x7f4d6c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:00.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:00.942+0000 7f4d727fc640 1 -- 172.21.15.89:0/3824776959 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f4d7804fee0 con 0x7f4d7c0afd90 2023-10-18T10:46:01.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.122+0000 7f4d83f96640 1 -- 172.21.15.89:0/3824776959 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d7c04dbe0 con 0x7f4d7c0afd90 2023-10-18T10:46:01.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.122+0000 7f4d727fc640 1 -- 172.21.15.89:0/3824776959 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4874 (secure 0 0 0) 0x7f4d78053500 con 0x7f4d7c0afd90 2023-10-18T10:46:01.124 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:01.125 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/1247973498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1247973498},{"type":"v1","addr":"172.21.15.154:6836","nonce":1247973498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4284,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/516057316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":516057316},{"type":"v1","addr":"172.21.15.154:6837","nonce":516057316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4299,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6834/3957051407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3957051407},{"type":"v1","addr":"172.21.15.89:6834","nonce":3957051407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:45:55.655016+0000","modified":"2023-10-18T10:46:00.759759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4312":{"gid":4312,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6835/2906113039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6833","nonce":2906113039},{"type":"v1","addr":"172.21.15.89:6835","nonce":2906113039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-10-18T10:46:01.125 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 7 2023-10-18T10:46:01.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.126+0000 7f4d83f96640 1 -- 172.21.15.89:0/3824776959 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4d60065680 msgr2=0x7f4d60067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:01.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.126+0000 7f4d83f96640 1 --2- 172.21.15.89:0/3824776959 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4d60065680 0x7f4d60067b40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c0ad000 tx=0x7f4d6c008040 comp rx=0 tx=0).stop 2023-10-18T10:46:01.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.126+0000 7f4d83f96640 1 -- 172.21.15.89:0/3824776959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d7c0afd90 msgr2=0x7f4d7c156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:01.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.126+0000 7f4d83f96640 1 --2- 172.21.15.89:0/3824776959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d7c0afd90 0x7f4d7c156010 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800c4d0 tx=0x7f4d7800c990 comp rx=0 tx=0).stop 2023-10-18T10:46:01.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.126+0000 7f4d83f96640 1 -- 172.21.15.89:0/3824776959 shutdown_connections 2023-10-18T10:46:01.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.126+0000 7f4d83f96640 1 --2- 172.21.15.89:0/3824776959 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4d60065680 0x7f4d60067b40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:01.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.126+0000 7f4d83f96640 1 --2- 172.21.15.89:0/3824776959 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4d7c0c8960 0x7f4d7c0c53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:01.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.126+0000 7f4d83f96640 1 --2- 172.21.15.89:0/3824776959 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d7c15e600 0x7f4d7c0c6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:01.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.126+0000 7f4d83f96640 1 --2- 172.21.15.89:0/3824776959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d7c0afd90 0x7f4d7c156010 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:01.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.126+0000 7f4d83f96640 1 -- 172.21.15.89:0/3824776959 >> 172.21.15.89:0/3824776959 conn(0x7f4d7c069f00 msgr2=0x7f4d7c1605b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:01.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.126+0000 7f4d83f96640 1 -- 172.21.15.89:0/3824776959 shutdown_connections 2023-10-18T10:46:01.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.126+0000 7f4d83f96640 1 -- 172.21.15.89:0/3824776959 wait complete. 2023-10-18T10:46:01.145 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:46:01.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.282+0000 7efe95a6d640 1 Processor -- start 2023-10-18T10:46:01.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.282+0000 7efe95a6d640 1 -- start start 2023-10-18T10:46:01.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.282+0000 7efe95a6d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efe9015c550 0x7efe900d2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:01.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.282+0000 7efe95a6d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe900b1860 0x7efe900cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:01.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.282+0000 7efe95a6d640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe900d1e50 con 0x7efe900b1cc0 2023-10-18T10:46:01.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.282+0000 7efe95a6d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe900d2130 con 0x7efe9015c550 2023-10-18T10:46:01.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.282+0000 7efe95a6d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe900d2410 con 0x7efe900b1860 2023-10-18T10:46:01.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.282+0000 7efe8ffff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7efe900b1cc0 0x7efe900b55f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55168/0 (socket says 172.21.15.89:55168) 2023-10-18T10:46:01.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.282+0000 7efe8ffff640 1 -- 172.21.15.89:0/3479155254 learned_addr learned my addr 172.21.15.89:0/3479155254 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:01.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.282+0000 7efe8f7fe640 1 --2- 172.21.15.89:0/3479155254 >> v2:172.21.15.154:3300/0 conn(0x7efe9015c550 0x7efe900d2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:01.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.282+0000 7efe8effd640 1 --2- 172.21.15.89:0/3479155254 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe900b1860 0x7efe900cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:01.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.282+0000 7efe8e7fc640 1 -- 172.21.15.89:0/3479155254 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2075351671 0 0) 0x7efe900d1e50 con 0x7efe900b1cc0 2023-10-18T10:46:01.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.282+0000 7efe8e7fc640 1 -- 172.21.15.89:0/3479155254 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe78003530 con 0x7efe900b1cc0 2023-10-18T10:46:01.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.282+0000 7efe8effd640 1 -- 172.21.15.89:0/3479155254 >> v2:172.21.15.154:3300/0 conn(0x7efe9015c550 msgr2=0x7efe900d2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:01.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.282+0000 7efe8effd640 1 --2- 172.21.15.89:0/3479155254 >> v2:172.21.15.154:3300/0 conn(0x7efe9015c550 0x7efe900d2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:01.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.282+0000 7efe8effd640 1 -- 172.21.15.89:0/3479155254 >> v1:172.21.15.89:6789/0 conn(0x7efe900b1cc0 legacy=0x7efe900b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:01.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.282+0000 7efe8effd640 1 -- 172.21.15.89:0/3479155254 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe900d26f0 con 0x7efe900b1860 2023-10-18T10:46:01.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.282+0000 7efe8f7fe640 1 --2- 172.21.15.89:0/3479155254 >> v2:172.21.15.154:3300/0 conn(0x7efe9015c550 0x7efe900d2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:46:01.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.286+0000 7efe8effd640 1 --2- 172.21.15.89:0/3479155254 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe900b1860 0x7efe900cf610 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7efe840097c0 tx=0x7efe8401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31906ba3bad4bf8c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:01.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.286+0000 7efe8e7fc640 1 -- 172.21.15.89:0/3479155254 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe840ad660 con 0x7efe900b1860 2023-10-18T10:46:01.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.286+0000 7efe8e7fc640 1 -- 172.21.15.89:0/3479155254 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe840ad7c0 con 0x7efe900b1860 2023-10-18T10:46:01.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.286+0000 7efe8e7fc640 1 -- 172.21.15.89:0/3479155254 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe840adb50 con 0x7efe900b1860 2023-10-18T10:46:01.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.286+0000 7efe95a6d640 1 -- 172.21.15.89:0/3479155254 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe900b1860 msgr2=0x7efe900cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:01.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.286+0000 7efe95a6d640 1 --2- 172.21.15.89:0/3479155254 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe900b1860 0x7efe900cf610 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7efe840097c0 tx=0x7efe8401c450 comp rx=0 tx=0).stop 2023-10-18T10:46:01.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.286+0000 7efe95a6d640 1 -- 172.21.15.89:0/3479155254 shutdown_connections 2023-10-18T10:46:01.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.286+0000 7efe95a6d640 1 --2- 172.21.15.89:0/3479155254 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe900b1860 0x7efe900cf610 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:01.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.286+0000 7efe95a6d640 1 --2- 172.21.15.89:0/3479155254 >> v2:172.21.15.154:3300/0 conn(0x7efe9015c550 0x7efe900d2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:01.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.286+0000 7efe95a6d640 1 -- 172.21.15.89:0/3479155254 >> 172.21.15.89:0/3479155254 conn(0x7efe90069d00 msgr2=0x7efe9006a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:01.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.286+0000 7efe95a6d640 1 -- 172.21.15.89:0/3479155254 shutdown_connections 2023-10-18T10:46:01.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.286+0000 7efe95a6d640 1 -- 172.21.15.89:0/3479155254 wait complete. 2023-10-18T10:46:01.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.290+0000 7efe95a6d640 1 Processor -- start 2023-10-18T10:46:01.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.290+0000 7efe95a6d640 1 -- start start 2023-10-18T10:46:01.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.290+0000 7efe95a6d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe900b1860 0x7efe901539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:01.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.290+0000 7efe95a6d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe900b1cc0 0x7efe9015e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:01.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.290+0000 7efe95a6d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe9015c550 0x7efe90156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:01.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.290+0000 7efe95a6d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe901fa150 con 0x7efe9015c550 2023-10-18T10:46:01.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.290+0000 7efe95a6d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe90154390 con 0x7efe900b1cc0 2023-10-18T10:46:01.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.290+0000 7efe95a6d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe90154670 con 0x7efe900b1860 2023-10-18T10:46:01.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.290+0000 7efe8ffff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe9015c550 0x7efe90156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:01.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.290+0000 7efe8ffff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe9015c550 0x7efe90156cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48858/0 (socket says 172.21.15.89:48858) 2023-10-18T10:46:01.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.290+0000 7efe8ffff640 1 -- 172.21.15.89:0/234418120 learned_addr learned my addr 172.21.15.89:0/234418120 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:01.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.290+0000 7efe8f7fe640 1 --2- 172.21.15.89:0/234418120 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe900b1860 0x7efe901539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:01.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.290+0000 7efe8effd640 1 --2- 172.21.15.89:0/234418120 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe900b1cc0 0x7efe9015e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:01.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.290+0000 7efe8ffff640 1 -- 172.21.15.89:0/234418120 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe900b1860 msgr2=0x7efe901539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:01.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.290+0000 7efe8ffff640 1 --2- 172.21.15.89:0/234418120 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe900b1860 0x7efe901539a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:01.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.290+0000 7efe8ffff640 1 -- 172.21.15.89:0/234418120 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe900b1cc0 msgr2=0x7efe9015e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:01.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.290+0000 7efe8ffff640 1 --2- 172.21.15.89:0/234418120 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe900b1cc0 0x7efe9015e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:01.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.290+0000 7efe8ffff640 1 -- 172.21.15.89:0/234418120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe80009420 con 0x7efe9015c550 2023-10-18T10:46:01.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.290+0000 7efe8ffff640 1 --2- 172.21.15.89:0/234418120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe9015c550 0x7efe90156cf0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efe8800cae0 tx=0x7efe8800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:01.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.290+0000 7efe77fff640 1 -- 172.21.15.89:0/234418120 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8800ac40 con 0x7efe9015c550 2023-10-18T10:46:01.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.290+0000 7efe77fff640 1 -- 172.21.15.89:0/234418120 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe8800ada0 con 0x7efe9015c550 2023-10-18T10:46:01.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.290+0000 7efe77fff640 1 -- 172.21.15.89:0/234418120 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe88012640 con 0x7efe9015c550 2023-10-18T10:46:01.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.290+0000 7efe95a6d640 1 -- 172.21.15.89:0/234418120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe84009420 con 0x7efe9015c550 2023-10-18T10:46:01.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.290+0000 7efe95a6d640 1 -- 172.21.15.89:0/234418120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe90154c30 con 0x7efe9015c550 2023-10-18T10:46:01.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.290+0000 7efe95a6d640 1 -- 172.21.15.89:0/234418120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe9004dbe0 con 0x7efe9015c550 2023-10-18T10:46:01.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.294+0000 7efe77fff640 1 -- 172.21.15.89:0/234418120 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7efe88007070 con 0x7efe9015c550 2023-10-18T10:46:01.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.294+0000 7efe77fff640 1 --2- 172.21.15.89:0/234418120 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efe7c065680 0x7efe7c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:01.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.294+0000 7efe77fff640 1 -- 172.21.15.89:0/234418120 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7efe880830b0 con 0x7efe9015c550 2023-10-18T10:46:01.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.294+0000 7efe8f7fe640 1 --2- 172.21.15.89:0/234418120 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efe7c065680 0x7efe7c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:01.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.294+0000 7efe8f7fe640 1 --2- 172.21.15.89:0/234418120 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efe7c065680 0x7efe7c067b40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efe9006fef0 tx=0x7efe80009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:01.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.298+0000 7efe77fff640 1 -- 172.21.15.89:0/234418120 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7efe88050890 con 0x7efe9015c550 2023-10-18T10:46:01.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.446+0000 7efe95a6d640 1 -- 172.21.15.89:0/234418120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe900b8040 con 0x7efe9015c550 2023-10-18T10:46:01.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.446+0000 7efe77fff640 1 -- 172.21.15.89:0/234418120 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20096 (secure 0 0 0) 0x7efe88053eb0 con 0x7efe9015c550 2023-10-18T10:46:01.449 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:01.449 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":27,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:46:00.166702+0000","last_up_change":"2023-10-18T10:45:42.164688+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:45:53.309122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-18T10:45:54.659763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:45:57.796514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6824","nonce":773078364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6827","nonce":773078364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6831","nonce":773078364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6829","nonce":773078364}]},"public_addr":"172.21.15.154:6824/773078364","cluster_addr":"172.21.15.154:6827/773078364","heartbeat_back_addr":"172.21.15.154:6831/773078364","heartbeat_front_addr":"172.21.15.154:6829/773078364","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:46:01.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.450+0000 7efe95a6d640 1 -- 172.21.15.89:0/234418120 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efe7c065680 msgr2=0x7efe7c067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:01.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.450+0000 7efe95a6d640 1 --2- 172.21.15.89:0/234418120 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efe7c065680 0x7efe7c067b40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efe9006fef0 tx=0x7efe80009190 comp rx=0 tx=0).stop 2023-10-18T10:46:01.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.450+0000 7efe95a6d640 1 -- 172.21.15.89:0/234418120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe9015c550 msgr2=0x7efe90156cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:01.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.450+0000 7efe95a6d640 1 --2- 172.21.15.89:0/234418120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe9015c550 0x7efe90156cf0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efe8800cae0 tx=0x7efe8800a060 comp rx=0 tx=0).stop 2023-10-18T10:46:01.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.450+0000 7efe95a6d640 1 -- 172.21.15.89:0/234418120 shutdown_connections 2023-10-18T10:46:01.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.450+0000 7efe95a6d640 1 --2- 172.21.15.89:0/234418120 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efe7c065680 0x7efe7c067b40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:01.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.450+0000 7efe95a6d640 1 --2- 172.21.15.89:0/234418120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe9015c550 0x7efe90156cf0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:01.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.450+0000 7efe95a6d640 1 --2- 172.21.15.89:0/234418120 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe900b1cc0 0x7efe9015e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:01.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.450+0000 7efe95a6d640 1 --2- 172.21.15.89:0/234418120 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe900b1860 0x7efe901539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:01.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.450+0000 7efe95a6d640 1 -- 172.21.15.89:0/234418120 >> 172.21.15.89:0/234418120 conn(0x7efe90069d00 msgr2=0x7efe900d0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:01.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.450+0000 7efe95a6d640 1 -- 172.21.15.89:0/234418120 shutdown_connections 2023-10-18T10:46:01.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.450+0000 7efe95a6d640 1 -- 172.21.15.89:0/234418120 wait complete. 2023-10-18T10:46:01.470 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:46:01.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.606+0000 7f95fc888640 1 Processor -- start 2023-10-18T10:46:01.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.606+0000 7f95fc888640 1 -- start start 2023-10-18T10:46:01.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.606+0000 7f95fc888640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95f815c2f0 0x7f95f80b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:01.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.606+0000 7f95fc888640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f95f80d06d0 0x7f95f8160780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:01.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.606+0000 7f95fc888640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95f815d1b0 con 0x7f95f80d0270 2023-10-18T10:46:01.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.606+0000 7f95fc888640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95f80d0b20 con 0x7f95f80d06d0 2023-10-18T10:46:01.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.606+0000 7f95fc888640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95f815d490 con 0x7f95f815c2f0 2023-10-18T10:46:01.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.606+0000 7f95f6d9d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f95f80d06d0 0x7f95f8160780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:01.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.606+0000 7f95f6d9d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f95f80d06d0 0x7f95f8160780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48874/0 (socket says 172.21.15.89:48874) 2023-10-18T10:46:01.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.606+0000 7f95f6d9d640 1 -- 172.21.15.89:0/2915771784 learned_addr learned my addr 172.21.15.89:0/2915771784 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:01.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.606+0000 7f95f659c640 1 --2- 172.21.15.89:0/2915771784 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95f815c2f0 0x7f95f80b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:01.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.606+0000 7f95f6d9d640 1 -- 172.21.15.89:0/2915771784 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95f815c2f0 msgr2=0x7f95f80b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:01.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.606+0000 7f95f6d9d640 1 --2- 172.21.15.89:0/2915771784 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95f815c2f0 0x7f95f80b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:01.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.606+0000 7f95f6d9d640 1 -- 172.21.15.89:0/2915771784 >> v1:172.21.15.154:6789/0 conn(0x7f95f80d0270 legacy=0x7f95f815c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:01.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.606+0000 7f95f6d9d640 1 -- 172.21.15.89:0/2915771784 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f815d770 con 0x7f95f80d06d0 2023-10-18T10:46:01.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.606+0000 7f95f659c640 1 --2- 172.21.15.89:0/2915771784 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95f815c2f0 0x7f95f80b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:01.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.610+0000 7f95f6d9d640 1 --2- 172.21.15.89:0/2915771784 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95f80d06d0 0x7f95f8160780 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f95f0009870 tx=0x7f95f001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0144a492801031 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:01.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.610+0000 7f95f5d9b640 1 -- 172.21.15.89:0/2915771784 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f00ac600 con 0x7f95f80d06d0 2023-10-18T10:46:01.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.610+0000 7f95f5d9b640 1 -- 172.21.15.89:0/2915771784 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f00a8050 con 0x7f95f80d06d0 2023-10-18T10:46:01.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.610+0000 7f95f5d9b640 1 -- 172.21.15.89:0/2915771784 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f00ac980 con 0x7f95f80d06d0 2023-10-18T10:46:01.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.610+0000 7f95fc888640 1 -- 172.21.15.89:0/2915771784 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95f80d06d0 msgr2=0x7f95f8160780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:01.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.610+0000 7f95fc888640 1 --2- 172.21.15.89:0/2915771784 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95f80d06d0 0x7f95f8160780 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f95f0009870 tx=0x7f95f001c4d0 comp rx=0 tx=0).stop 2023-10-18T10:46:01.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.610+0000 7f95fc888640 1 -- 172.21.15.89:0/2915771784 shutdown_connections 2023-10-18T10:46:01.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.610+0000 7f95fc888640 1 --2- 172.21.15.89:0/2915771784 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95f80d06d0 0x7f95f8160780 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:01.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.610+0000 7f95fc888640 1 --2- 172.21.15.89:0/2915771784 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95f815c2f0 0x7f95f80b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:01.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.610+0000 7f95fc888640 1 -- 172.21.15.89:0/2915771784 >> 172.21.15.89:0/2915771784 conn(0x7f95f8069ad0 msgr2=0x7f95f8069eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:01.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.610+0000 7f95fc888640 1 -- 172.21.15.89:0/2915771784 shutdown_connections 2023-10-18T10:46:01.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.610+0000 7f95fc888640 1 -- 172.21.15.89:0/2915771784 wait complete. 2023-10-18T10:46:01.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.614+0000 7f95fc888640 1 Processor -- start 2023-10-18T10:46:01.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.614+0000 7f95fc888640 1 -- start start 2023-10-18T10:46:01.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.614+0000 7f95fc888640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95f80d0270 0x7f95f8159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:01.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.614+0000 7f95fc888640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95f80d06d0 0x7f95f80c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:01.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.614+0000 7f95fc888640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95f815c2f0 0x7f95f80c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:01.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.614+0000 7f95fc888640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f815a300 con 0x7f95f815c2f0 2023-10-18T10:46:01.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.614+0000 7f95fc888640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f80cc000 con 0x7f95f80d0270 2023-10-18T10:46:01.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.614+0000 7f95fc888640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95f80cc5c0 con 0x7f95f80d06d0 2023-10-18T10:46:01.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.614+0000 7f95f6d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95f815c2f0 0x7f95f80c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:01.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.614+0000 7f95f659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95f80d0270 0x7f95f8159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:01.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.614+0000 7f95edd9b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95f80d06d0 0x7f95f80c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:01.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.614+0000 7f95f659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95f80d0270 0x7f95f8159dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:40600/0 (socket says 172.21.15.89:40600) 2023-10-18T10:46:01.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.614+0000 7f95edd9b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95f80d06d0 0x7f95f80c83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:54142/0 (socket says 172.21.15.89:54142) 2023-10-18T10:46:01.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.614+0000 7f95f659c640 1 -- 172.21.15.89:0/4262429240 learned_addr learned my addr 172.21.15.89:0/4262429240 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:01.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.614+0000 7f95f6d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95f815c2f0 0x7f95f80c88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48886/0 (socket says 172.21.15.89:48886) 2023-10-18T10:46:01.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.614+0000 7f95f6d9d640 1 -- 172.21.15.89:0/4262429240 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95f80d06d0 msgr2=0x7f95f80c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:01.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.614+0000 7f95f6d9d640 1 --2- 172.21.15.89:0/4262429240 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95f80d06d0 0x7f95f80c83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:01.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.614+0000 7f95f6d9d640 1 -- 172.21.15.89:0/4262429240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95f80d0270 msgr2=0x7f95f8159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:01.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.614+0000 7f95f6d9d640 1 --2- 172.21.15.89:0/4262429240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95f80d0270 0x7f95f8159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:01.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.614+0000 7f95f6d9d640 1 -- 172.21.15.89:0/4262429240 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f00094d0 con 0x7f95f815c2f0 2023-10-18T10:46:01.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.614+0000 7f95f6d9d640 1 --2- 172.21.15.89:0/4262429240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95f815c2f0 0x7f95f80c88e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f95f00ac000 tx=0x7f95f001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:01.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.614+0000 7f95ef7fe640 1 -- 172.21.15.89:0/4262429240 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f00b7650 con 0x7f95f815c2f0 2023-10-18T10:46:01.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.614+0000 7f95ef7fe640 1 -- 172.21.15.89:0/4262429240 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f00b77f0 con 0x7f95f815c2f0 2023-10-18T10:46:01.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.614+0000 7f95fc888640 1 -- 172.21.15.89:0/4262429240 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95f81667a0 con 0x7f95f815c2f0 2023-10-18T10:46:01.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.614+0000 7f95ef7fe640 1 -- 172.21.15.89:0/4262429240 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f00b7ba0 con 0x7f95f815c2f0 2023-10-18T10:46:01.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.614+0000 7f95fc888640 1 -- 172.21.15.89:0/4262429240 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95f80cc250 con 0x7f95f815c2f0 2023-10-18T10:46:01.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.618+0000 7f95fc888640 1 -- 172.21.15.89:0/4262429240 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95f80cc700 con 0x7f95f815c2f0 2023-10-18T10:46:01.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.618+0000 7f95ef7fe640 1 -- 172.21.15.89:0/4262429240 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f95f00b3070 con 0x7f95f815c2f0 2023-10-18T10:46:01.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.618+0000 7f95ef7fe640 1 --2- 172.21.15.89:0/4262429240 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f95d80656b0 0x7f95d8067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:01.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.618+0000 7f95ef7fe640 1 -- 172.21.15.89:0/4262429240 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f95f011d5d0 con 0x7f95f815c2f0 2023-10-18T10:46:01.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.618+0000 7f95f659c640 1 --2- 172.21.15.89:0/4262429240 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f95d80656b0 0x7f95d8067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:01.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.618+0000 7f95f659c640 1 --2- 172.21.15.89:0/4262429240 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f95d80656b0 0x7f95d8067b70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f95e800f5e0 tx=0x7f95e8007830 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:01.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.622+0000 7f95ef7fe640 1 -- 172.21.15.89:0/4262429240 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f95f80cc700 con 0x7f95f815c2f0 2023-10-18T10:46:01.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.774+0000 7f95fc888640 1 -- 172.21.15.89:0/4262429240 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95f80cc700 con 0x7f95f815c2f0 2023-10-18T10:46:01.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.774+0000 7f95ef7fe640 1 -- 172.21.15.89:0/4262429240 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20096 (secure 0 0 0) 0x7f95f80cc700 con 0x7f95f815c2f0 2023-10-18T10:46:01.777 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:01.777 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":27,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:46:00.166702+0000","last_up_change":"2023-10-18T10:45:42.164688+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:45:53.309122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-18T10:45:54.659763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:45:57.796514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6824","nonce":773078364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6827","nonce":773078364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6831","nonce":773078364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6829","nonce":773078364}]},"public_addr":"172.21.15.154:6824/773078364","cluster_addr":"172.21.15.154:6827/773078364","heartbeat_back_addr":"172.21.15.154:6831/773078364","heartbeat_front_addr":"172.21.15.154:6829/773078364","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:46:01.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.778+0000 7f95fc888640 1 -- 172.21.15.89:0/4262429240 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f95d80656b0 msgr2=0x7f95d8067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:01.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.778+0000 7f95fc888640 1 --2- 172.21.15.89:0/4262429240 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f95d80656b0 0x7f95d8067b70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f95e800f5e0 tx=0x7f95e8007830 comp rx=0 tx=0).stop 2023-10-18T10:46:01.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.778+0000 7f95fc888640 1 -- 172.21.15.89:0/4262429240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95f815c2f0 msgr2=0x7f95f80c88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:01.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.778+0000 7f95fc888640 1 --2- 172.21.15.89:0/4262429240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95f815c2f0 0x7f95f80c88e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f95f00ac000 tx=0x7f95f001ad10 comp rx=0 tx=0).stop 2023-10-18T10:46:01.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.778+0000 7f95fc888640 1 -- 172.21.15.89:0/4262429240 shutdown_connections 2023-10-18T10:46:01.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.778+0000 7f95fc888640 1 --2- 172.21.15.89:0/4262429240 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f95d80656b0 0x7f95d8067b70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:01.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.778+0000 7f95fc888640 1 --2- 172.21.15.89:0/4262429240 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95f80d06d0 0x7f95f80c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:01.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.778+0000 7f95fc888640 1 --2- 172.21.15.89:0/4262429240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95f815c2f0 0x7f95f80c88e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:01.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.778+0000 7f95fc888640 1 --2- 172.21.15.89:0/4262429240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95f80d0270 0x7f95f8159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:01.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.778+0000 7f95fc888640 1 -- 172.21.15.89:0/4262429240 >> 172.21.15.89:0/4262429240 conn(0x7f95f8069ad0 msgr2=0x7f95f815f200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:01.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.778+0000 7f95fc888640 1 -- 172.21.15.89:0/4262429240 shutdown_connections 2023-10-18T10:46:01.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.778+0000 7f95fc888640 1 -- 172.21.15.89:0/4262429240 wait complete. 2023-10-18T10:46:01.812 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:46:01.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.946+0000 7f60cd69c640 1 Processor -- start 2023-10-18T10:46:01.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.950+0000 7f60cd69c640 1 -- start start 2023-10-18T10:46:01.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.950+0000 7f60cd69c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f60c815c2f0 0x7f60c80b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:01.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.950+0000 7f60cd69c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60c80d0270 0x7f60c80d06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:01.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.950+0000 7f60cd69c640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60c815d1b0 con 0x7f60c815c6d0 2023-10-18T10:46:01.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.950+0000 7f60cd69c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60c815d490 con 0x7f60c815c2f0 2023-10-18T10:46:01.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.950+0000 7f60cd69c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c815d770 con 0x7f60c80d0270 2023-10-18T10:46:01.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.950+0000 7f60c77fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f60c815c6d0 0x7f60c8160780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55174/0 (socket says 172.21.15.89:55174) 2023-10-18T10:46:01.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.950+0000 7f60c77fe640 1 -- 172.21.15.89:0/977258019 learned_addr learned my addr 172.21.15.89:0/977258019 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:01.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.950+0000 7f60c6ffd640 1 --2- 172.21.15.89:0/977258019 >> v2:172.21.15.154:3300/0 conn(0x7f60c815c2f0 0x7f60c80b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:01.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.950+0000 7f60bffff640 1 --2- 172.21.15.89:0/977258019 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60c80d0270 0x7f60c80d06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:01.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.950+0000 7f60c67fc640 1 -- 172.21.15.89:0/977258019 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1506735440 0 0) 0x7f60c815d1b0 con 0x7f60c815c6d0 2023-10-18T10:46:01.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.950+0000 7f60c67fc640 1 -- 172.21.15.89:0/977258019 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60b4003530 con 0x7f60c815c6d0 2023-10-18T10:46:01.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.950+0000 7f60c6ffd640 1 -- 172.21.15.89:0/977258019 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60c80d0270 msgr2=0x7f60c80d06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:01.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.950+0000 7f60c6ffd640 1 --2- 172.21.15.89:0/977258019 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60c80d0270 0x7f60c80d06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:01.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.950+0000 7f60c6ffd640 1 -- 172.21.15.89:0/977258019 >> v1:172.21.15.89:6789/0 conn(0x7f60c815c6d0 legacy=0x7f60c8160780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:01.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.950+0000 7f60c6ffd640 1 -- 172.21.15.89:0/977258019 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c815da50 con 0x7f60c815c2f0 2023-10-18T10:46:01.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.954+0000 7f60c6ffd640 1 --2- 172.21.15.89:0/977258019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60c815c2f0 0x7f60c80b18e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f60b80097c0 tx=0x7f60b801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27f4d5a7d61b2533 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:01.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.954+0000 7f60c67fc640 1 -- 172.21.15.89:0/977258019 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b80ab670 con 0x7f60c815c2f0 2023-10-18T10:46:01.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.954+0000 7f60c67fc640 1 -- 172.21.15.89:0/977258019 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60b80ab7d0 con 0x7f60c815c2f0 2023-10-18T10:46:01.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.954+0000 7f60c67fc640 1 -- 172.21.15.89:0/977258019 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b80abb60 con 0x7f60c815c2f0 2023-10-18T10:46:01.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.954+0000 7f60cd69c640 1 -- 172.21.15.89:0/977258019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60c815c2f0 msgr2=0x7f60c80b18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:01.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.954+0000 7f60cd69c640 1 --2- 172.21.15.89:0/977258019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60c815c2f0 0x7f60c80b18e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f60b80097c0 tx=0x7f60b801c450 comp rx=0 tx=0).stop 2023-10-18T10:46:01.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.954+0000 7f60cd69c640 1 -- 172.21.15.89:0/977258019 shutdown_connections 2023-10-18T10:46:01.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.954+0000 7f60cd69c640 1 --2- 172.21.15.89:0/977258019 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60c80d0270 0x7f60c80d06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:01.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.954+0000 7f60cd69c640 1 --2- 172.21.15.89:0/977258019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60c815c2f0 0x7f60c80b18e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:01.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.954+0000 7f60cd69c640 1 -- 172.21.15.89:0/977258019 >> 172.21.15.89:0/977258019 conn(0x7f60c8069ad0 msgr2=0x7f60c8069eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:01.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.954+0000 7f60cd69c640 1 -- 172.21.15.89:0/977258019 shutdown_connections 2023-10-18T10:46:01.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.954+0000 7f60cd69c640 1 -- 172.21.15.89:0/977258019 wait complete. 2023-10-18T10:46:01.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.954+0000 7f60cd69c640 1 Processor -- start 2023-10-18T10:46:01.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.954+0000 7f60cd69c640 1 -- start start 2023-10-18T10:46:01.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.958+0000 7f60cd69c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60c80d0270 0x7f60c815b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:01.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.958+0000 7f60cd69c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60c815c2f0 0x7f60c80c8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:01.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.958+0000 7f60cd69c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60c815c6d0 0x7f60c8159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:01.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.958+0000 7f60cd69c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c80ca3b0 con 0x7f60c815c6d0 2023-10-18T10:46:01.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.958+0000 7f60cd69c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c815f770 con 0x7f60c80d0270 2023-10-18T10:46:01.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.958+0000 7f60cd69c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c815eae0 con 0x7f60c815c2f0 2023-10-18T10:46:01.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.958+0000 7f60c77fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60c815c6d0 0x7f60c8159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:01.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.958+0000 7f60c77fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60c815c6d0 0x7f60c8159810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48896/0 (socket says 172.21.15.89:48896) 2023-10-18T10:46:01.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.958+0000 7f60c77fe640 1 -- 172.21.15.89:0/3727388225 learned_addr learned my addr 172.21.15.89:0/3727388225 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:01.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.958+0000 7f60bffff640 1 --2- 172.21.15.89:0/3727388225 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60c815c2f0 0x7f60c80c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:01.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.958+0000 7f60c6ffd640 1 --2- 172.21.15.89:0/3727388225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60c80d0270 0x7f60c815b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:01.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.958+0000 7f60c77fe640 1 -- 172.21.15.89:0/3727388225 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60c815c2f0 msgr2=0x7f60c80c8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:01.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.958+0000 7f60c77fe640 1 --2- 172.21.15.89:0/3727388225 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60c815c2f0 0x7f60c80c8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:01.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.958+0000 7f60c77fe640 1 -- 172.21.15.89:0/3727388225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60c80d0270 msgr2=0x7f60c815b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:01.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.958+0000 7f60c77fe640 1 --2- 172.21.15.89:0/3727388225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60c80d0270 0x7f60c815b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:01.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.958+0000 7f60c77fe640 1 -- 172.21.15.89:0/3727388225 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b8009420 con 0x7f60c815c6d0 2023-10-18T10:46:01.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.958+0000 7f60c77fe640 1 --2- 172.21.15.89:0/3727388225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60c815c6d0 0x7f60c8159810 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f60c000caa0 tx=0x7f60c000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:01.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.958+0000 7f60bf7fe640 1 -- 172.21.15.89:0/3727388225 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c000ac00 con 0x7f60c815c6d0 2023-10-18T10:46:01.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.958+0000 7f60bf7fe640 1 -- 172.21.15.89:0/3727388225 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c000ad60 con 0x7f60c815c6d0 2023-10-18T10:46:01.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.958+0000 7f60cd69c640 1 -- 172.21.15.89:0/3727388225 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c815edc0 con 0x7f60c815c6d0 2023-10-18T10:46:01.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.958+0000 7f60bf7fe640 1 -- 172.21.15.89:0/3727388225 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c0012620 con 0x7f60c815c6d0 2023-10-18T10:46:01.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.958+0000 7f60cd69c640 1 -- 172.21.15.89:0/3727388225 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c80c8b40 con 0x7f60c815c6d0 2023-10-18T10:46:01.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.958+0000 7f60cd69c640 1 -- 172.21.15.89:0/3727388225 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60c815f000 con 0x7f60c815c6d0 2023-10-18T10:46:01.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.958+0000 7f60bf7fe640 1 -- 172.21.15.89:0/3727388225 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f60c0007070 con 0x7f60c815c6d0 2023-10-18T10:46:01.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.962+0000 7f60bf7fe640 1 --2- 172.21.15.89:0/3727388225 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f60a80656b0 0x7f60a8067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:01.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.962+0000 7f60bf7fe640 1 -- 172.21.15.89:0/3727388225 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f60c0082ff0 con 0x7f60c815c6d0 2023-10-18T10:46:01.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.962+0000 7f60c6ffd640 1 --2- 172.21.15.89:0/3727388225 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f60a80656b0 0x7f60a8067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:01.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.962+0000 7f60c6ffd640 1 --2- 172.21.15.89:0/3727388225 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f60a80656b0 0x7f60a8067b70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f60b80af000 tx=0x7f60b80a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:01.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:01.966+0000 7f60bf7fe640 1 -- 172.21.15.89:0/3727388225 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f60c00b3d40 con 0x7f60c815c6d0 2023-10-18T10:46:02.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.142+0000 7f60cd69c640 1 -- 172.21.15.89:0/3727388225 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60c80ca180 con 0x7f60c815c6d0 2023-10-18T10:46:02.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.142+0000 7f60bf7fe640 1 -- 172.21.15.89:0/3727388225 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4874 (secure 0 0 0) 0x7f60c80ca180 con 0x7f60c815c6d0 2023-10-18T10:46:02.146 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:02.146 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/1247973498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1247973498},{"type":"v1","addr":"172.21.15.154:6836","nonce":1247973498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4284,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/516057316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":516057316},{"type":"v1","addr":"172.21.15.154:6837","nonce":516057316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4299,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6834/3957051407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3957051407},{"type":"v1","addr":"172.21.15.89:6834","nonce":3957051407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:45:55.655016+0000","modified":"2023-10-18T10:46:00.759759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4312":{"gid":4312,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6835/2906113039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6833","nonce":2906113039},{"type":"v1","addr":"172.21.15.89:6835","nonce":2906113039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-10-18T10:46:02.146 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 7 2023-10-18T10:46:02.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.146+0000 7f60cd69c640 1 -- 172.21.15.89:0/3727388225 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f60a80656b0 msgr2=0x7f60a8067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:02.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.146+0000 7f60cd69c640 1 --2- 172.21.15.89:0/3727388225 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f60a80656b0 0x7f60a8067b70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f60b80af000 tx=0x7f60b80a7000 comp rx=0 tx=0).stop 2023-10-18T10:46:02.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.146+0000 7f60cd69c640 1 -- 172.21.15.89:0/3727388225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60c815c6d0 msgr2=0x7f60c8159810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:02.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.146+0000 7f60cd69c640 1 --2- 172.21.15.89:0/3727388225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60c815c6d0 0x7f60c8159810 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f60c000caa0 tx=0x7f60c000cf60 comp rx=0 tx=0).stop 2023-10-18T10:46:02.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.146+0000 7f60cd69c640 1 -- 172.21.15.89:0/3727388225 shutdown_connections 2023-10-18T10:46:02.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.146+0000 7f60cd69c640 1 --2- 172.21.15.89:0/3727388225 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f60a80656b0 0x7f60a8067b70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:02.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.146+0000 7f60cd69c640 1 --2- 172.21.15.89:0/3727388225 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60c815c2f0 0x7f60c80c8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:02.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.146+0000 7f60cd69c640 1 --2- 172.21.15.89:0/3727388225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60c815c6d0 0x7f60c8159810 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:02.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.146+0000 7f60cd69c640 1 --2- 172.21.15.89:0/3727388225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60c80d0270 0x7f60c815b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:02.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.146+0000 7f60cd69c640 1 -- 172.21.15.89:0/3727388225 >> 172.21.15.89:0/3727388225 conn(0x7f60c8069ad0 msgr2=0x7f60c80b16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:02.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.146+0000 7f60cd69c640 1 -- 172.21.15.89:0/3727388225 shutdown_connections 2023-10-18T10:46:02.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.146+0000 7f60cd69c640 1 -- 172.21.15.89:0/3727388225 wait complete. 2023-10-18T10:46:02.166 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:46:02.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.306+0000 7f2d67167640 1 Processor -- start 2023-10-18T10:46:02.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.306+0000 7f2d67167640 1 -- start start 2023-10-18T10:46:02.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.306+0000 7f2d67167640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d600d0700 0x7f2d60160780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:02.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.306+0000 7f2d67167640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d6015d1b0 con 0x7f2d600d0270 2023-10-18T10:46:02.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.306+0000 7f2d67167640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d6015d490 con 0x7f2d6015c2f0 2023-10-18T10:46:02.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.306+0000 7f2d67167640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d600d0b50 con 0x7f2d600d0700 2023-10-18T10:46:02.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.306+0000 7f2d5ffff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2d600d0270 0x7f2d6015c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55562/0 (socket says 172.21.15.89:55562) 2023-10-18T10:46:02.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.306+0000 7f2d5ffff640 1 -- 172.21.15.89:0/793116826 learned_addr learned my addr 172.21.15.89:0/793116826 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:02.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.306+0000 7f2d65704640 1 --2- 172.21.15.89:0/793116826 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d600d0700 0x7f2d60160780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:02.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.306+0000 7f2d5f7fe640 1 -- 172.21.15.89:0/793116826 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4248971799 0 0) 0x7f2d6015d1b0 con 0x7f2d600d0270 2023-10-18T10:46:02.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.306+0000 7f2d5f7fe640 1 -- 172.21.15.89:0/793116826 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d48003530 con 0x7f2d600d0270 2023-10-18T10:46:02.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.306+0000 7f2d5f7fe640 1 -- 172.21.15.89:0/793116826 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3431230276 0 0) 0x7f2d6015d490 con 0x7f2d6015c2f0 2023-10-18T10:46:02.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.306+0000 7f2d5f7fe640 1 -- 172.21.15.89:0/793116826 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d6015d1b0 con 0x7f2d6015c2f0 2023-10-18T10:46:02.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.306+0000 7f2d65704640 1 -- 172.21.15.89:0/793116826 >> v1:172.21.15.154:6789/0 conn(0x7f2d6015c2f0 legacy=0x7f2d600b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:02.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.306+0000 7f2d65704640 1 -- 172.21.15.89:0/793116826 >> v1:172.21.15.89:6789/0 conn(0x7f2d600d0270 legacy=0x7f2d6015c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:02.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.306+0000 7f2d65704640 1 -- 172.21.15.89:0/793116826 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6015d770 con 0x7f2d600d0700 2023-10-18T10:46:02.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.310+0000 7f2d65704640 1 --2- 172.21.15.89:0/793116826 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d600d0700 0x7f2d60160780 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2d58009df0 tx=0x7f2d5801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb2194266687e77b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:02.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.310+0000 7f2d5f7fe640 1 -- 172.21.15.89:0/793116826 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d580a5830 con 0x7f2d600d0700 2023-10-18T10:46:02.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.310+0000 7f2d5f7fe640 1 -- 172.21.15.89:0/793116826 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d580a5990 con 0x7f2d600d0700 2023-10-18T10:46:02.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.310+0000 7f2d5f7fe640 1 -- 172.21.15.89:0/793116826 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d580a5d20 con 0x7f2d600d0700 2023-10-18T10:46:02.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.310+0000 7f2d67167640 1 -- 172.21.15.89:0/793116826 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d600d0700 msgr2=0x7f2d60160780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:02.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.310+0000 7f2d67167640 1 --2- 172.21.15.89:0/793116826 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d600d0700 0x7f2d60160780 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2d58009df0 tx=0x7f2d5801c970 comp rx=0 tx=0).stop 2023-10-18T10:46:02.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.310+0000 7f2d67167640 1 -- 172.21.15.89:0/793116826 shutdown_connections 2023-10-18T10:46:02.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.310+0000 7f2d67167640 1 --2- 172.21.15.89:0/793116826 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d600d0700 0x7f2d60160780 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:02.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.310+0000 7f2d67167640 1 -- 172.21.15.89:0/793116826 >> 172.21.15.89:0/793116826 conn(0x7f2d60069ad0 msgr2=0x7f2d60069eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:02.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.314+0000 7f2d67167640 1 -- 172.21.15.89:0/793116826 shutdown_connections 2023-10-18T10:46:02.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.314+0000 7f2d67167640 1 -- 172.21.15.89:0/793116826 wait complete. 2023-10-18T10:46:02.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.314+0000 7f2d67167640 1 Processor -- start 2023-10-18T10:46:02.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.314+0000 7f2d67167640 1 -- start start 2023-10-18T10:46:02.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.314+0000 7f2d67167640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d600d0270 0x7f2d600cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:02.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.314+0000 7f2d67167640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d600d0700 0x7f2d601612d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:02.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.314+0000 7f2d67167640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d6015c2f0 0x7f2d6015eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:02.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.314+0000 7f2d67167640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d601619d0 con 0x7f2d6015c2f0 2023-10-18T10:46:02.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.314+0000 7f2d67167640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d60161cb0 con 0x7f2d600d0700 2023-10-18T10:46:02.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.314+0000 7f2d67167640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d60161f90 con 0x7f2d600d0270 2023-10-18T10:46:02.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.314+0000 7f2d65704640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d6015c2f0 0x7f2d6015eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:02.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.314+0000 7f2d65704640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d6015c2f0 0x7f2d6015eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44900/0 (socket says 172.21.15.89:44900) 2023-10-18T10:46:02.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.314+0000 7f2d65704640 1 -- 172.21.15.89:0/2726354185 learned_addr learned my addr 172.21.15.89:0/2726354185 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:02.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.314+0000 7f2d5ffff640 1 --2- 172.21.15.89:0/2726354185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d600d0700 0x7f2d601612d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:02.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.314+0000 7f2d64f03640 1 --2- 172.21.15.89:0/2726354185 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d600d0270 0x7f2d600cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:02.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.314+0000 7f2d65704640 1 -- 172.21.15.89:0/2726354185 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d600d0270 msgr2=0x7f2d600cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:02.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.314+0000 7f2d65704640 1 --2- 172.21.15.89:0/2726354185 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d600d0270 0x7f2d600cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:02.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.314+0000 7f2d65704640 1 -- 172.21.15.89:0/2726354185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d600d0700 msgr2=0x7f2d601612d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:02.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.314+0000 7f2d65704640 1 --2- 172.21.15.89:0/2726354185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d600d0700 0x7f2d601612d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:02.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.314+0000 7f2d65704640 1 -- 172.21.15.89:0/2726354185 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d58009a70 con 0x7f2d6015c2f0 2023-10-18T10:46:02.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.314+0000 7f2d5ffff640 1 --2- 172.21.15.89:0/2726354185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d600d0700 0x7f2d601612d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:02.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.314+0000 7f2d65704640 1 --2- 172.21.15.89:0/2726354185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d6015c2f0 0x7f2d6015eb90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2d5801a5a0 tx=0x7f2d58019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:02.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.314+0000 7f2d5d7fa640 1 -- 172.21.15.89:0/2726354185 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d580ad640 con 0x7f2d6015c2f0 2023-10-18T10:46:02.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.318+0000 7f2d5d7fa640 1 -- 172.21.15.89:0/2726354185 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d580ad7a0 con 0x7f2d6015c2f0 2023-10-18T10:46:02.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.318+0000 7f2d67167640 1 -- 172.21.15.89:0/2726354185 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d60162270 con 0x7f2d6015c2f0 2023-10-18T10:46:02.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.318+0000 7f2d5d7fa640 1 -- 172.21.15.89:0/2726354185 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d580adb10 con 0x7f2d6015c2f0 2023-10-18T10:46:02.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.318+0000 7f2d67167640 1 -- 172.21.15.89:0/2726354185 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d6015f0d0 con 0x7f2d6015c2f0 2023-10-18T10:46:02.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.318+0000 7f2d67167640 1 -- 172.21.15.89:0/2726354185 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d60066ca0 con 0x7f2d6015c2f0 2023-10-18T10:46:02.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.318+0000 7f2d5d7fa640 1 -- 172.21.15.89:0/2726354185 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f2d580b4070 con 0x7f2d6015c2f0 2023-10-18T10:46:02.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.318+0000 7f2d5d7fa640 1 --2- 172.21.15.89:0/2726354185 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2d4c065680 0x7f2d4c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:02.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.318+0000 7f2d5d7fa640 1 -- 172.21.15.89:0/2726354185 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2d5811dc50 con 0x7f2d6015c2f0 2023-10-18T10:46:02.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.318+0000 7f2d64f03640 1 --2- 172.21.15.89:0/2726354185 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2d4c065680 0x7f2d4c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:02.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.318+0000 7f2d64f03640 1 --2- 172.21.15.89:0/2726354185 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2d4c065680 0x7f2d4c067b40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2d500015e0 tx=0x7f2d50009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:02.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.322+0000 7f2d5d7fa640 1 -- 172.21.15.89:0/2726354185 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f2d580eb300 con 0x7f2d6015c2f0 2023-10-18T10:46:02.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.474+0000 7f2d67167640 1 -- 172.21.15.89:0/2726354185 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d600b7eb0 con 0x7f2d6015c2f0 2023-10-18T10:46:02.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.474+0000 7f2d5d7fa640 1 -- 172.21.15.89:0/2726354185 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20096 (secure 0 0 0) 0x7f2d580ee920 con 0x7f2d6015c2f0 2023-10-18T10:46:02.476 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:02.477 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":27,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:46:00.166702+0000","last_up_change":"2023-10-18T10:45:42.164688+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:45:53.309122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-18T10:45:54.659763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:45:57.796514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6824","nonce":773078364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6827","nonce":773078364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6831","nonce":773078364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6829","nonce":773078364}]},"public_addr":"172.21.15.154:6824/773078364","cluster_addr":"172.21.15.154:6827/773078364","heartbeat_back_addr":"172.21.15.154:6831/773078364","heartbeat_front_addr":"172.21.15.154:6829/773078364","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:46:02.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.478+0000 7f2d67167640 1 -- 172.21.15.89:0/2726354185 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2d4c065680 msgr2=0x7f2d4c067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:02.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.478+0000 7f2d67167640 1 --2- 172.21.15.89:0/2726354185 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2d4c065680 0x7f2d4c067b40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2d500015e0 tx=0x7f2d50009040 comp rx=0 tx=0).stop 2023-10-18T10:46:02.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.478+0000 7f2d67167640 1 -- 172.21.15.89:0/2726354185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d6015c2f0 msgr2=0x7f2d6015eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:02.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.478+0000 7f2d67167640 1 --2- 172.21.15.89:0/2726354185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d6015c2f0 0x7f2d6015eb90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2d5801a5a0 tx=0x7f2d58019710 comp rx=0 tx=0).stop 2023-10-18T10:46:02.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.478+0000 7f2d67167640 1 -- 172.21.15.89:0/2726354185 shutdown_connections 2023-10-18T10:46:02.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.478+0000 7f2d67167640 1 --2- 172.21.15.89:0/2726354185 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2d4c065680 0x7f2d4c067b40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:02.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.478+0000 7f2d67167640 1 --2- 172.21.15.89:0/2726354185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d6015c2f0 0x7f2d6015eb90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:02.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.478+0000 7f2d67167640 1 --2- 172.21.15.89:0/2726354185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d600d0700 0x7f2d601612d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:02.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.478+0000 7f2d67167640 1 --2- 172.21.15.89:0/2726354185 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d600d0270 0x7f2d600cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:02.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.478+0000 7f2d67167640 1 -- 172.21.15.89:0/2726354185 >> 172.21.15.89:0/2726354185 conn(0x7f2d60069ad0 msgr2=0x7f2d600b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:02.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.478+0000 7f2d67167640 1 -- 172.21.15.89:0/2726354185 shutdown_connections 2023-10-18T10:46:02.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:02.478+0000 7f2d67167640 1 -- 172.21.15.89:0/2726354185 wait complete. 2023-10-18T10:46:02.900 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-18T10:45:55.655016+0000', 'modified': '2023-10-18T10:46:00.759759+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4312}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4312': {'gid': 4312, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.89:6835/2906113039', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6833', 'nonce': 2906113039}, {'type': 'v1', 'addr': '172.21.15.89:6835', 'nonce': 2906113039}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-10-18T10:46:02.900 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T10:46:02.901 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-18T10:46:03.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.038+0000 7f8d7c1e1640 1 Processor -- start 2023-10-18T10:46:03.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.038+0000 7f8d7c1e1640 1 -- start start 2023-10-18T10:46:03.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.042+0000 7f8d7c1e1640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8d740cd4f0 0x7f8d74143470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:03.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.042+0000 7f8d7c1e1640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8d74145b70 0x7f8d74160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:03.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.042+0000 7f8d7c1e1640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d740cd8d0 con 0x7f8d740ce070 2023-10-18T10:46:03.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.042+0000 7f8d7c1e1640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d74069900 con 0x7f8d740cd4f0 2023-10-18T10:46:03.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.042+0000 7f8d7c1e1640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d7415cbf0 con 0x7f8d74145b70 2023-10-18T10:46:03.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.042+0000 7f8d79f7d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8d740cd4f0 0x7f8d74143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:03.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.042+0000 7f8d79f7d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8d740cd4f0 0x7f8d74143470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44902/0 (socket says 172.21.15.89:44902) 2023-10-18T10:46:03.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.042+0000 7f8d79f7d640 1 -- 172.21.15.89:0/3268399868 learned_addr learned my addr 172.21.15.89:0/3268399868 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:03.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.042+0000 7f8d7977c640 1 --2- 172.21.15.89:0/3268399868 >> v2:172.21.15.154:3300/0 conn(0x7f8d74145b70 0x7f8d74160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:03.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.042+0000 7f8d79f7d640 1 -- 172.21.15.89:0/3268399868 >> v2:172.21.15.154:3300/0 conn(0x7f8d74145b70 msgr2=0x7f8d74160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:03.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.042+0000 7f8d79f7d640 1 --2- 172.21.15.89:0/3268399868 >> v2:172.21.15.154:3300/0 conn(0x7f8d74145b70 0x7f8d74160cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.042+0000 7f8d79f7d640 1 -- 172.21.15.89:0/3268399868 >> v1:172.21.15.89:6789/0 conn(0x7f8d740ce070 legacy=0x7f8d7406abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:03.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.042+0000 7f8d79f7d640 1 -- 172.21.15.89:0/3268399868 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d74161230 con 0x7f8d740cd4f0 2023-10-18T10:46:03.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.042+0000 7f8d79f7d640 1 --2- 172.21.15.89:0/3268399868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d740cd4f0 0x7f8d74143470 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8d740cdbf0 tx=0x7f8d7001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1014f6ef606c41f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:03.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.046+0000 7f8d78f7b640 1 -- 172.21.15.89:0/3268399868 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d700a5860 con 0x7f8d740cd4f0 2023-10-18T10:46:03.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.046+0000 7f8d78f7b640 1 -- 172.21.15.89:0/3268399868 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d700a59c0 con 0x7f8d740cd4f0 2023-10-18T10:46:03.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.046+0000 7f8d78f7b640 1 -- 172.21.15.89:0/3268399868 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d700a5d50 con 0x7f8d740cd4f0 2023-10-18T10:46:03.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.046+0000 7f8d7c1e1640 1 -- 172.21.15.89:0/3268399868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d740cd4f0 msgr2=0x7f8d74143470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:03.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.046+0000 7f8d7c1e1640 1 --2- 172.21.15.89:0/3268399868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d740cd4f0 0x7f8d74143470 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8d740cdbf0 tx=0x7f8d7001c9a0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.046+0000 7f8d7c1e1640 1 -- 172.21.15.89:0/3268399868 shutdown_connections 2023-10-18T10:46:03.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.046+0000 7f8d7c1e1640 1 --2- 172.21.15.89:0/3268399868 >> v2:172.21.15.154:3300/0 conn(0x7f8d74145b70 0x7f8d74160cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.046+0000 7f8d7c1e1640 1 --2- 172.21.15.89:0/3268399868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d740cd4f0 0x7f8d74143470 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.046+0000 7f8d7c1e1640 1 -- 172.21.15.89:0/3268399868 >> 172.21.15.89:0/3268399868 conn(0x7f8d7406dad0 msgr2=0x7f8d7406dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:03.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.046+0000 7f8d7c1e1640 1 -- 172.21.15.89:0/3268399868 shutdown_connections 2023-10-18T10:46:03.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.046+0000 7f8d7c1e1640 1 -- 172.21.15.89:0/3268399868 wait complete. 2023-10-18T10:46:03.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.046+0000 7f8d7c1e1640 1 Processor -- start 2023-10-18T10:46:03.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.046+0000 7f8d7c1e1640 1 -- start start 2023-10-18T10:46:03.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.046+0000 7f8d7c1e1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d740cd4f0 0x7f8d740c3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:03.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.046+0000 7f8d7c1e1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d740ce070 0x7f8d740c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:03.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.046+0000 7f8d7c1e1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d74145b70 0x7f8d740c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:03.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.046+0000 7f8d7c1e1640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d740b2bf0 con 0x7f8d740ce070 2023-10-18T10:46:03.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.046+0000 7f8d7c1e1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d741551f0 con 0x7f8d740cd4f0 2023-10-18T10:46:03.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.046+0000 7f8d7c1e1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d74157c70 con 0x7f8d74145b70 2023-10-18T10:46:03.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.046+0000 7f8d7977c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d740ce070 0x7f8d740c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:03.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.046+0000 7f8d7977c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d740ce070 0x7f8d740c9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44910/0 (socket says 172.21.15.89:44910) 2023-10-18T10:46:03.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.046+0000 7f8d7977c640 1 -- 172.21.15.89:0/2122281303 learned_addr learned my addr 172.21.15.89:0/2122281303 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:03.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.050+0000 7f8d79f7d640 1 --2- 172.21.15.89:0/2122281303 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d740cd4f0 0x7f8d740c3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:03.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.050+0000 7f8d7a77e640 1 --2- 172.21.15.89:0/2122281303 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d74145b70 0x7f8d740c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:03.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.050+0000 7f8d7977c640 1 -- 172.21.15.89:0/2122281303 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d74145b70 msgr2=0x7f8d740c9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:03.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.050+0000 7f8d7977c640 1 --2- 172.21.15.89:0/2122281303 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d74145b70 0x7f8d740c9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.050+0000 7f8d7977c640 1 -- 172.21.15.89:0/2122281303 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d740cd4f0 msgr2=0x7f8d740c3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:03.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.050+0000 7f8d7977c640 1 --2- 172.21.15.89:0/2122281303 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d740cd4f0 0x7f8d740c3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.050+0000 7f8d7977c640 1 -- 172.21.15.89:0/2122281303 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d70009b30 con 0x7f8d740ce070 2023-10-18T10:46:03.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.050+0000 7f8d79f7d640 1 --2- 172.21.15.89:0/2122281303 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d740cd4f0 0x7f8d740c3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:03.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.050+0000 7f8d7977c640 1 --2- 172.21.15.89:0/2122281303 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d740ce070 0x7f8d740c9580 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400d800 tx=0x7f8d6400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:03.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.050+0000 7f8d6a7fc640 1 -- 172.21.15.89:0/2122281303 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6400b6c0 con 0x7f8d740ce070 2023-10-18T10:46:03.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.050+0000 7f8d6a7fc640 1 -- 172.21.15.89:0/2122281303 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6400b820 con 0x7f8d740ce070 2023-10-18T10:46:03.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.050+0000 7f8d6a7fc640 1 -- 172.21.15.89:0/2122281303 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6400bb90 con 0x7f8d740ce070 2023-10-18T10:46:03.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.050+0000 7f8d7c1e1640 1 -- 172.21.15.89:0/2122281303 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d740b1fe0 con 0x7f8d740ce070 2023-10-18T10:46:03.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.050+0000 7f8d7c1e1640 1 -- 172.21.15.89:0/2122281303 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d740b25a0 con 0x7f8d740ce070 2023-10-18T10:46:03.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.050+0000 7f8d6a7fc640 1 -- 172.21.15.89:0/2122281303 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f8d6400bcf0 con 0x7f8d740ce070 2023-10-18T10:46:03.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.054+0000 7f8d6a7fc640 1 --2- 172.21.15.89:0/2122281303 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8d58065680 0x7f8d58067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:03.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.054+0000 7f8d6a7fc640 1 -- 172.21.15.89:0/2122281303 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8d640836c0 con 0x7f8d740ce070 2023-10-18T10:46:03.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.054+0000 7f8d79f7d640 1 --2- 172.21.15.89:0/2122281303 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8d58065680 0x7f8d58067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:03.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.054+0000 7f8d7f542000 1 Processor -- start 2023-10-18T10:46:03.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.054+0000 7f8d7f542000 1 -- start start 2023-10-18T10:46:03.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.054+0000 7f8d79f7d640 1 --2- 172.21.15.89:0/2122281303 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8d58065680 0x7f8d58067b40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8d700ae000 tx=0x7f8d700a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:03.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.054+0000 7f8d7f542000 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b35097da70 0x55b35097de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:03.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.054+0000 7f8d7f542000 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b35097e390 0x55b350933930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:03.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.054+0000 7f8d7f542000 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b350933e70 0x55b350934250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:03.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.054+0000 7f8d7f542000 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b35079d5a0 con 0x55b350933e70 2023-10-18T10:46:03.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.054+0000 7f8d7f542000 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b35078b320 con 0x55b35097e390 2023-10-18T10:46:03.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.054+0000 7f8d7f542000 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b350788850 con 0x55b35097da70 2023-10-18T10:46:03.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.054+0000 7f8d7a77e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b350933e70 0x55b350934250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:03.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.054+0000 7f8d7a77e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b350933e70 0x55b350934250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44926/0 (socket says 172.21.15.89:44926) 2023-10-18T10:46:03.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.054+0000 7f8d7a77e640 1 -- 172.21.15.89:0/1910922634 learned_addr learned my addr 172.21.15.89:0/1910922634 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:03.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.054+0000 7f8d79f7d640 1 --2- 172.21.15.89:0/1910922634 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b35097da70 0x55b35097de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:03.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.054+0000 7f8d7977c640 1 --2- 172.21.15.89:0/1910922634 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b35097e390 0x55b350933930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:03.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.054+0000 7f8d7a77e640 1 -- 172.21.15.89:0/1910922634 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b35097da70 msgr2=0x55b35097de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:03.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.054+0000 7f8d7a77e640 1 --2- 172.21.15.89:0/1910922634 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b35097da70 0x55b35097de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.054+0000 7f8d7a77e640 1 -- 172.21.15.89:0/1910922634 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b35097e390 msgr2=0x55b350933930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:03.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.054+0000 7f8d7a77e640 1 --2- 172.21.15.89:0/1910922634 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b35097e390 0x55b350933930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.054+0000 7f8d7a77e640 1 -- 172.21.15.89:0/1910922634 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b35047e890 con 0x55b350933e70 2023-10-18T10:46:03.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.054+0000 7f8d7a77e640 1 --2- 172.21.15.89:0/1910922634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b350933e70 0x55b350934250 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c001560 tx=0x7f8d6c001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f16a88f94faa51 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:03.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.054+0000 7f8d577fe640 1 -- 172.21.15.89:0/1910922634 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c0050d0 con 0x55b350933e70 2023-10-18T10:46:03.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.054+0000 7f8d577fe640 1 -- 172.21.15.89:0/1910922634 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6c005230 con 0x55b350933e70 2023-10-18T10:46:03.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.054+0000 7f8d577fe640 1 -- 172.21.15.89:0/1910922634 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c00e740 con 0x55b350933e70 2023-10-18T10:46:03.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.054+0000 7f8d7f542000 1 -- 172.21.15.89:0/1910922634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b350933e70 msgr2=0x55b350934250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:03.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.054+0000 7f8d7f542000 1 --2- 172.21.15.89:0/1910922634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b350933e70 0x55b350934250 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c001560 tx=0x7f8d6c001a20 comp rx=0 tx=0).stop 2023-10-18T10:46:03.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.054+0000 7f8d7f542000 1 -- 172.21.15.89:0/1910922634 shutdown_connections 2023-10-18T10:46:03.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.054+0000 7f8d7f542000 1 --2- 172.21.15.89:0/1910922634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b350933e70 0x55b350934250 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.054+0000 7f8d7f542000 1 --2- 172.21.15.89:0/1910922634 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b35097e390 0x55b350933930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.054+0000 7f8d7f542000 1 --2- 172.21.15.89:0/1910922634 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b35097da70 0x55b35097de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.054+0000 7f8d7f542000 1 -- 172.21.15.89:0/1910922634 >> 172.21.15.89:0/1910922634 conn(0x55b35092b1e0 msgr2=0x55b350932210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:03.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.054+0000 7f8d7f542000 1 -- 172.21.15.89:0/1910922634 shutdown_connections 2023-10-18T10:46:03.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.054+0000 7f8d7f542000 1 -- 172.21.15.89:0/1910922634 wait complete. 2023-10-18T10:46:03.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d7f542000 1 -- start start 2023-10-18T10:46:03.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d7f542000 1 Processor -- start 2023-10-18T10:46:03.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d7f542000 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b350933e70 0x55b35093f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:03.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d7f542000 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b35097da70 0x55b35093fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:03.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d7f542000 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b35097e390 0x55b3509402f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:03.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d7f542000 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b350784140 con 0x55b350933e70 2023-10-18T10:46:03.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d7f542000 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b35073ef60 con 0x55b35097da70 2023-10-18T10:46:03.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d7f542000 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b35073ec00 con 0x55b35097e390 2023-10-18T10:46:03.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d7f542000 10 client.0 _ll_register_callbacks cb 0x55b35091f350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T10:46:03.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d79f7d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b350933e70 0x55b35093f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:03.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d79f7d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b350933e70 0x55b35093f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44942/0 (socket says 172.21.15.89:44942) 2023-10-18T10:46:03.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d79f7d640 1 -- 172.21.15.89:0/2093532527 learned_addr learned my addr 172.21.15.89:0/2093532527 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:03.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d7a77e640 1 --2- 172.21.15.89:0/2093532527 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b35097e390 0x55b3509402f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:03.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d7977c640 1 --2- 172.21.15.89:0/2093532527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b35097da70 0x55b35093fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:03.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d79f7d640 1 -- 172.21.15.89:0/2093532527 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b35097e390 msgr2=0x55b3509402f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:03.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d79f7d640 1 --2- 172.21.15.89:0/2093532527 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b35097e390 0x55b3509402f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d79f7d640 1 -- 172.21.15.89:0/2093532527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b35097da70 msgr2=0x55b35093fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:03.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d79f7d640 1 --2- 172.21.15.89:0/2093532527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b35097da70 0x55b35093fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d79f7d640 1 -- 172.21.15.89:0/2093532527 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d6c00f040 con 0x55b350933e70 2023-10-18T10:46:03.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d79f7d640 1 --2- 172.21.15.89:0/2093532527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b350933e70 0x55b35093f870 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8d70098580 tx=0x7f8d7001b1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f8f541d77a6ed3 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:03.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d54ff9640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-18T10:46:03.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d54ff9640 1 -- 172.21.15.89:0/2093532527 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d700b6cb0 con 0x55b350933e70 2023-10-18T10:46:03.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d54ff9640 1 -- 172.21.15.89:0/2093532527 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8d700b6e10 con 0x55b350933e70 2023-10-18T10:46:03.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d7f542000 1 -- 172.21.15.89:0/2093532527 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b35078a120 con 0x55b350933e70 2023-10-18T10:46:03.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d54ff9640 1 -- 172.21.15.89:0/2093532527 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d700c3480 con 0x55b350933e70 2023-10-18T10:46:03.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d7f542000 10 client.4434 fetch_fsmap learned FSMap version 7 2023-10-18T10:46:03.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d7f542000 1 -- 172.21.15.89:0/2093532527 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55b350747190 con 0x55b350933e70 2023-10-18T10:46:03.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d54ff9640 1 -- 172.21.15.89:0/2093532527 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b35078a120 con 0x55b350933e70 2023-10-18T10:46:03.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d54ff9640 1 -- 172.21.15.89:0/2093532527 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f8d70005d20 con 0x55b350933e70 2023-10-18T10:46:03.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d7f542000 10 client.4434 fetch_fsmap finished waiting for FSMap version 7 2023-10-18T10:46:03.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d7f542000 10 client.4434 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-18T10:46:03.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d7f542000 20 client.4434 populate_metadata read hostname 'smithi089' 2023-10-18T10:46:03.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d7f542000 1 --2- 172.21.15.89:0/2093532527 >> [v2:172.21.15.89:6833/2906113039,v1:172.21.15.89:6835/2906113039] conn(0x55b350940d00 0x55b35098fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:03.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d7f542000 4 client.4434 mds_command: new command op to 4312 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-18T10:46:03.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d7f542000 1 -- 172.21.15.89:0/2093532527 --> [v2:172.21.15.89:6833/2906113039,v1:172.21.15.89:6835/2906113039] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b350782a90 con 0x55b350940d00 2023-10-18T10:46:03.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.058+0000 7f8d7977c640 1 --2- 172.21.15.89:0/2093532527 >> [v2:172.21.15.89:6833/2906113039,v1:172.21.15.89:6835/2906113039] conn(0x55b350940d00 0x55b35098fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:03.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.062+0000 7f8d7977c640 1 --2- 172.21.15.89:0/2093532527 >> [v2:172.21.15.89:6833/2906113039,v1:172.21.15.89:6835/2906113039] conn(0x55b350940d00 0x55b35098fb20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68560c6a313b87e8 server_cookie=6c46cfd708d4d075 in_seq=0 out_seq=0 2023-10-18T10:46:03.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.062+0000 7f8d54ff9640 10 client.4434 ms_handle_connect on v2:172.21.15.89:6833/2906113039 2023-10-18T10:46:03.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.062+0000 7f8d54ff9640 1 -- 172.21.15.89:0/2093532527 <== mds.0 v2:172.21.15.89:6833/2906113039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b350782a90 con 0x55b350940d00 2023-10-18T10:46:03.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.062+0000 7f8d54ff9640 10 client.4434 handle_command_reply: tid=0 2023-10-18T10:46:03.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.062+0000 7f8d7f542000 1 client.4434 shutdown 2023-10-18T10:46:03.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.062+0000 7f8d7f542000 1 -- 172.21.15.89:0/2093532527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b350933e70 msgr2=0x55b35093f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:03.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.062+0000 7f8d7f542000 1 --2- 172.21.15.89:0/2093532527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b350933e70 0x55b35093f870 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8d70098580 tx=0x7f8d7001b1e0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.062+0000 7f8d7f542000 1 -- 172.21.15.89:0/2093532527 shutdown_connections 2023-10-18T10:46:03.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.062+0000 7f8d7f542000 1 --2- 172.21.15.89:0/2093532527 >> [v2:172.21.15.89:6833/2906113039,v1:172.21.15.89:6835/2906113039] conn(0x55b350940d00 0x55b35098fb20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.062+0000 7f8d7f542000 1 --2- 172.21.15.89:0/2093532527 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b35097e390 0x55b3509402f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.062+0000 7f8d54ff9640 0 client.4434 ms_handle_reset on v2:172.21.15.89:6833/2906113039 2023-10-18T10:46:03.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.062+0000 7f8d7f542000 1 --2- 172.21.15.89:0/2093532527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b35097da70 0x55b35093fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.062+0000 7f8d7f542000 1 --2- 172.21.15.89:0/2093532527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b350933e70 0x55b35093f870 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.062+0000 7f8d7f542000 1 -- 172.21.15.89:0/2093532527 >> 172.21.15.89:0/2093532527 conn(0x55b35092b1e0 msgr2=0x55b350932910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:03.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.062+0000 7f8d7f542000 1 -- 172.21.15.89:0/2093532527 shutdown_connections 2023-10-18T10:46:03.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.062+0000 7f8d7f542000 1 -- 172.21.15.89:0/2093532527 wait complete. 2023-10-18T10:46:03.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.062+0000 7f8d7f542000 20 client.4434 trim_cache size 0 max 16384 2023-10-18T10:46:03.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.078+0000 7f8d7f542000 1 Processor -- start 2023-10-18T10:46:03.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.078+0000 7f8d7f542000 1 -- start start 2023-10-18T10:46:03.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.078+0000 7f8d7f542000 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b350933e70 0x55b35097c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:03.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.078+0000 7f8d7f542000 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b35097da70 0x55b35097c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:03.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.078+0000 7f8d7f542000 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b35097e390 0x55b35098db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:03.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.078+0000 7f8d7f542000 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b35043f940 con 0x55b350933e70 2023-10-18T10:46:03.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.078+0000 7f8d7f542000 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b350786ca0 con 0x55b35097e390 2023-10-18T10:46:03.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.078+0000 7f8d7f542000 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b35097d930 con 0x55b35097da70 2023-10-18T10:46:03.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.078+0000 7f8d79f7d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b350933e70 0x55b35097c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:03.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.078+0000 7f8d79f7d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b350933e70 0x55b35097c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44952/0 (socket says 172.21.15.89:44952) 2023-10-18T10:46:03.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.078+0000 7f8d79f7d640 1 -- 172.21.15.89:0/2171412180 learned_addr learned my addr 172.21.15.89:0/2171412180 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:03.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.078+0000 7f8d7977c640 1 --2- 172.21.15.89:0/2171412180 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b35097da70 0x55b35097c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:03.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.078+0000 7f8d7a77e640 1 --2- 172.21.15.89:0/2171412180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b35097e390 0x55b35098db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:03.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.078+0000 7f8d79f7d640 1 -- 172.21.15.89:0/2171412180 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b35097da70 msgr2=0x55b35097c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:03.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.078+0000 7f8d79f7d640 1 --2- 172.21.15.89:0/2171412180 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b35097da70 0x55b35097c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.078+0000 7f8d79f7d640 1 -- 172.21.15.89:0/2171412180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b35097e390 msgr2=0x55b35098db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:03.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.078+0000 7f8d79f7d640 1 --2- 172.21.15.89:0/2171412180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b35097e390 0x55b35098db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.078+0000 7f8d79f7d640 1 -- 172.21.15.89:0/2171412180 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b35072d420 con 0x55b350933e70 2023-10-18T10:46:03.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.078+0000 7f8d79f7d640 1 --2- 172.21.15.89:0/2171412180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b350933e70 0x55b35097c2c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8d7001b220 tx=0x7f8d700c1000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f9383489cd78f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:03.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.078+0000 7f8d57fff640 1 -- 172.21.15.89:0/2171412180 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d70009510 con 0x55b350933e70 2023-10-18T10:46:03.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.078+0000 7f8d57fff640 1 -- 172.21.15.89:0/2171412180 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d700bd040 con 0x55b350933e70 2023-10-18T10:46:03.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.078+0000 7f8d57fff640 1 -- 172.21.15.89:0/2171412180 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d700b6ab0 con 0x55b350933e70 2023-10-18T10:46:03.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.078+0000 7f8d7f542000 1 -- 172.21.15.89:0/2171412180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b350933e70 msgr2=0x55b35097c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:03.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.078+0000 7f8d7f542000 1 --2- 172.21.15.89:0/2171412180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b350933e70 0x55b35097c2c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8d7001b220 tx=0x7f8d700c1000 comp rx=0 tx=0).stop 2023-10-18T10:46:03.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.078+0000 7f8d7f542000 1 -- 172.21.15.89:0/2171412180 shutdown_connections 2023-10-18T10:46:03.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.078+0000 7f8d7f542000 1 --2- 172.21.15.89:0/2171412180 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b35097da70 0x55b35097c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.078+0000 7f8d7f542000 1 --2- 172.21.15.89:0/2171412180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b35097e390 0x55b35098db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.078+0000 7f8d7f542000 1 --2- 172.21.15.89:0/2171412180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b350933e70 0x55b35097c2c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.078+0000 7f8d7f542000 1 -- 172.21.15.89:0/2171412180 >> 172.21.15.89:0/2171412180 conn(0x55b35092b1e0 msgr2=0x55b35093f730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:03.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.078+0000 7f8d7f542000 1 -- 172.21.15.89:0/2171412180 shutdown_connections 2023-10-18T10:46:03.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.078+0000 7f8d7f542000 1 -- 172.21.15.89:0/2171412180 wait complete. 2023-10-18T10:46:03.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.078+0000 7f8d7f542000 1 -- start start 2023-10-18T10:46:03.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.078+0000 7f8d7f542000 1 Processor -- start 2023-10-18T10:46:03.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d7f542000 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b350933e70 0x55b350990b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:03.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d7f542000 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b35097da70 0x55b3509a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:03.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d7f542000 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b35097e390 0x55b3509a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:03.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d7f542000 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3507517a0 con 0x55b350933e70 2023-10-18T10:46:03.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d7f542000 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b350782fd0 con 0x55b35097e390 2023-10-18T10:46:03.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d7f542000 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b35092cc90 con 0x55b35097da70 2023-10-18T10:46:03.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d7f542000 10 client.0 _ll_register_callbacks cb 0x7f8d6c007da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T10:46:03.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d79f7d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b350933e70 0x55b350990b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:03.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d79f7d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b350933e70 0x55b350990b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44960/0 (socket says 172.21.15.89:44960) 2023-10-18T10:46:03.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d79f7d640 1 -- 172.21.15.89:0/3634326745 learned_addr learned my addr 172.21.15.89:0/3634326745 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:03.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d7977c640 1 --2- 172.21.15.89:0/3634326745 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b35097da70 0x55b3509a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:03.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d7a77e640 1 --2- 172.21.15.89:0/3634326745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b35097e390 0x55b3509a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:03.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d79f7d640 1 -- 172.21.15.89:0/3634326745 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b35097da70 msgr2=0x55b3509a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:03.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d79f7d640 1 --2- 172.21.15.89:0/3634326745 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b35097da70 0x55b3509a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d79f7d640 1 -- 172.21.15.89:0/3634326745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b35097e390 msgr2=0x55b3509a1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:03.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d79f7d640 1 --2- 172.21.15.89:0/3634326745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b35097e390 0x55b3509a1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d79f7d640 1 -- 172.21.15.89:0/3634326745 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b350747190 con 0x55b350933e70 2023-10-18T10:46:03.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d7a77e640 1 --2- 172.21.15.89:0/3634326745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b35097e390 0x55b3509a1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:03.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d79f7d640 1 --2- 172.21.15.89:0/3634326745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b350933e70 0x55b350990b40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8d7001b220 tx=0x7f8d70009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3211b6608972d00d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:03.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d54ff9640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-18T10:46:03.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d54ff9640 1 -- 172.21.15.89:0/3634326745 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d700c3dc0 con 0x55b350933e70 2023-10-18T10:46:03.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d54ff9640 1 -- 172.21.15.89:0/3634326745 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8d700d3430 con 0x55b350933e70 2023-10-18T10:46:03.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d7f542000 1 -- 172.21.15.89:0/3634326745 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b350789030 con 0x55b350933e70 2023-10-18T10:46:03.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d54ff9640 1 -- 172.21.15.89:0/3634326745 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d70009510 con 0x55b350933e70 2023-10-18T10:46:03.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d7f542000 10 client.4440 fetch_fsmap learned FSMap version 7 2023-10-18T10:46:03.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d7f542000 1 -- 172.21.15.89:0/3634326745 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55b35047ecc0 con 0x55b350933e70 2023-10-18T10:46:03.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d54ff9640 1 -- 172.21.15.89:0/3634326745 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b350789030 con 0x55b350933e70 2023-10-18T10:46:03.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d54ff9640 1 -- 172.21.15.89:0/3634326745 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f8d70005d20 con 0x55b350933e70 2023-10-18T10:46:03.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d7f542000 10 client.4440 fetch_fsmap finished waiting for FSMap version 7 2023-10-18T10:46:03.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d7f542000 10 client.4440 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-18T10:46:03.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d7f542000 20 client.4440 populate_metadata read hostname 'smithi089' 2023-10-18T10:46:03.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d7f542000 1 --2- 172.21.15.89:0/3634326745 >> [v2:172.21.15.89:6833/2906113039,v1:172.21.15.89:6835/2906113039] conn(0x55b3509e4e80 0x55b3509e5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:03.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d7f542000 4 client.4440 mds_command: new command op to 4312 tid=0[{"prefix": "status"}] 2023-10-18T10:46:03.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d7f542000 1 -- 172.21.15.89:0/3634326745 --> [v2:172.21.15.89:6833/2906113039,v1:172.21.15.89:6835/2906113039] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b35077d120 con 0x55b3509e4e80 2023-10-18T10:46:03.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d7977c640 1 --2- 172.21.15.89:0/3634326745 >> [v2:172.21.15.89:6833/2906113039,v1:172.21.15.89:6835/2906113039] conn(0x55b3509e4e80 0x55b3509e5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:03.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d7977c640 1 --2- 172.21.15.89:0/3634326745 >> [v2:172.21.15.89:6833/2906113039,v1:172.21.15.89:6835/2906113039] conn(0x55b3509e4e80 0x55b3509e5260 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9faab61b828b676b server_cookie=91f74cb03dbcddca in_seq=0 out_seq=0 2023-10-18T10:46:03.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d54ff9640 10 client.4440 ms_handle_connect on v2:172.21.15.89:6833/2906113039 2023-10-18T10:46:03.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d54ff9640 1 -- 172.21.15.89:0/3634326745 <== mds.0 v2:172.21.15.89:6833/2906113039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55b35077d120 con 0x55b3509e4e80 2023-10-18T10:46:03.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d54ff9640 10 client.4440 handle_command_reply: tid=0 2023-10-18T10:46:03.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d7f542000 1 client.4440 shutdown 2023-10-18T10:46:03.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d7f542000 1 -- 172.21.15.89:0/3634326745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b350933e70 msgr2=0x55b350990b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:03.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d7f542000 1 --2- 172.21.15.89:0/3634326745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b350933e70 0x55b350990b40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8d7001b220 tx=0x7f8d70009810 comp rx=0 tx=0).stop 2023-10-18T10:46:03.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d7f542000 1 -- 172.21.15.89:0/3634326745 shutdown_connections 2023-10-18T10:46:03.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d7f542000 1 --2- 172.21.15.89:0/3634326745 >> [v2:172.21.15.89:6833/2906113039,v1:172.21.15.89:6835/2906113039] conn(0x55b3509e4e80 0x55b3509e5260 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d7f542000 1 --2- 172.21.15.89:0/3634326745 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b35097da70 0x55b3509a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d54ff9640 0 client.4440 ms_handle_reset on v2:172.21.15.89:6833/2906113039 2023-10-18T10:46:03.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d7f542000 1 --2- 172.21.15.89:0/3634326745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b35097e390 0x55b3509a1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d7f542000 1 --2- 172.21.15.89:0/3634326745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b350933e70 0x55b350990b40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.082+0000 7f8d7f542000 1 -- 172.21.15.89:0/3634326745 >> 172.21.15.89:0/3634326745 conn(0x55b35092b1e0 msgr2=0x55b35097b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:03.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.086+0000 7f8d7f542000 1 -- 172.21.15.89:0/3634326745 shutdown_connections 2023-10-18T10:46:03.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.086+0000 7f8d7f542000 1 -- 172.21.15.89:0/3634326745 wait complete. 2023-10-18T10:46:03.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.086+0000 7f8d7f542000 20 client.4440 trim_cache size 0 max 16384 2023-10-18T10:46:03.108 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-18T10:46:03.108 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "1352f62a-2330-4e96-afcd-b8e87cd82667", 2023-10-18T10:46:03.109 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 0, 2023-10-18T10:46:03.109 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 4312, 2023-10-18T10:46:03.109 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2023-10-18T10:46:03.109 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2023-10-18T10:46:03.109 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2023-10-18T10:46:03.110 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 6.8123432949999998, 2023-10-18T10:46:03.110 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 7, 2023-10-18T10:46:03.110 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 27, 2023-10-18T10:46:03.110 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 26, 2023-10-18T10:46:03.110 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 12.601253791 2023-10-18T10:46:03.111 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-18T10:46:03.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.086+0000 7f8d7c1e1640 1 -- 172.21.15.89:0/2122281303 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8d58065680 msgr2=0x7f8d58067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:03.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.086+0000 7f8d7c1e1640 1 --2- 172.21.15.89:0/2122281303 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8d58065680 0x7f8d58067b40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8d700ae000 tx=0x7f8d700a7070 comp rx=0 tx=0).stop 2023-10-18T10:46:03.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.086+0000 7f8d7c1e1640 1 -- 172.21.15.89:0/2122281303 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d740ce070 msgr2=0x7f8d740c9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:03.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.086+0000 7f8d7c1e1640 1 --2- 172.21.15.89:0/2122281303 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d740ce070 0x7f8d740c9580 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400d800 tx=0x7f8d6400dcc0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.086+0000 7f8d7c1e1640 1 -- 172.21.15.89:0/2122281303 shutdown_connections 2023-10-18T10:46:03.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.086+0000 7f8d7c1e1640 1 --2- 172.21.15.89:0/2122281303 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8d58065680 0x7f8d58067b40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.086+0000 7f8d7c1e1640 1 --2- 172.21.15.89:0/2122281303 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d74145b70 0x7f8d740c9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.086+0000 7f8d7c1e1640 1 --2- 172.21.15.89:0/2122281303 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d740ce070 0x7f8d740c9580 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.086+0000 7f8d7c1e1640 1 --2- 172.21.15.89:0/2122281303 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d740cd4f0 0x7f8d740c3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.086+0000 7f8d7c1e1640 1 -- 172.21.15.89:0/2122281303 >> 172.21.15.89:0/2122281303 conn(0x7f8d7406dad0 msgr2=0x7f8d74160470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:03.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.086+0000 7f8d7c1e1640 1 -- 172.21.15.89:0/2122281303 shutdown_connections 2023-10-18T10:46:03.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.086+0000 7f8d7c1e1640 1 -- 172.21.15.89:0/2122281303 wait complete. 2023-10-18T10:46:03.115 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-18T10:46:03.115 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-10-18T10:46:03.116 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T10:46:03.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.250+0000 7f23c99fa640 1 Processor -- start 2023-10-18T10:46:03.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.250+0000 7f23c99fa640 1 -- start start 2023-10-18T10:46:03.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.254+0000 7f23c99fa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f23c40d0760 0x7f23c40d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:03.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.254+0000 7f23c99fa640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f23c40cd160 0x7f23c40cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:03.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.254+0000 7f23c99fa640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23c40d1100 con 0x7f23c415c940 2023-10-18T10:46:03.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.254+0000 7f23c99fa640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23c415da50 con 0x7f23c40cd160 2023-10-18T10:46:03.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.254+0000 7f23c99fa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c40d3110 con 0x7f23c40d0760 2023-10-18T10:46:03.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.254+0000 7f23c2ffd640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f23c415c940 0x7f23c40d2a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55590/0 (socket says 172.21.15.89:55590) 2023-10-18T10:46:03.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.254+0000 7f23c2ffd640 1 -- 172.21.15.89:0/683798884 learned_addr learned my addr 172.21.15.89:0/683798884 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:03.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.254+0000 7f23c37fe640 1 --2- 172.21.15.89:0/683798884 >> v2:172.21.15.89:3300/0 conn(0x7f23c40cd160 0x7f23c40cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:03.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.254+0000 7f23c27fc640 1 --2- 172.21.15.89:0/683798884 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f23c40d0760 0x7f23c40d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:03.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.254+0000 7f23c37fe640 1 -- 172.21.15.89:0/683798884 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f23c40d0760 msgr2=0x7f23c40d0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:03.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.254+0000 7f23c37fe640 1 --2- 172.21.15.89:0/683798884 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f23c40d0760 0x7f23c40d0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.254+0000 7f23c37fe640 1 -- 172.21.15.89:0/683798884 >> v1:172.21.15.89:6789/0 conn(0x7f23c415c940 legacy=0x7f23c40d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:03.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.254+0000 7f23c37fe640 1 -- 172.21.15.89:0/683798884 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c415ce40 con 0x7f23c40cd160 2023-10-18T10:46:03.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.254+0000 7f23c37fe640 1 --2- 172.21.15.89:0/683798884 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23c40cd160 0x7f23c40cd5f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f23bc0097c0 tx=0x7f23bc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51fc2ef73eef94bd server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:03.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.254+0000 7f23c1ffb640 1 -- 172.21.15.89:0/683798884 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc0ab650 con 0x7f23c40cd160 2023-10-18T10:46:03.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.258+0000 7f23c1ffb640 1 -- 172.21.15.89:0/683798884 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23bc0ab7b0 con 0x7f23c40cd160 2023-10-18T10:46:03.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.258+0000 7f23c1ffb640 1 -- 172.21.15.89:0/683798884 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc0abb40 con 0x7f23c40cd160 2023-10-18T10:46:03.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.258+0000 7f23c99fa640 1 -- 172.21.15.89:0/683798884 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23c40cd160 msgr2=0x7f23c40cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:03.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.258+0000 7f23c99fa640 1 --2- 172.21.15.89:0/683798884 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23c40cd160 0x7f23c40cd5f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f23bc0097c0 tx=0x7f23bc01c450 comp rx=0 tx=0).stop 2023-10-18T10:46:03.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.258+0000 7f23c99fa640 1 -- 172.21.15.89:0/683798884 shutdown_connections 2023-10-18T10:46:03.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.258+0000 7f23c99fa640 1 --2- 172.21.15.89:0/683798884 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23c40cd160 0x7f23c40cd5f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.258+0000 7f23c99fa640 1 --2- 172.21.15.89:0/683798884 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f23c40d0760 0x7f23c40d0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.258+0000 7f23c99fa640 1 -- 172.21.15.89:0/683798884 >> 172.21.15.89:0/683798884 conn(0x7f23c4089fd0 msgr2=0x7f23c409b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:03.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.258+0000 7f23c99fa640 1 -- 172.21.15.89:0/683798884 shutdown_connections 2023-10-18T10:46:03.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.258+0000 7f23c99fa640 1 -- 172.21.15.89:0/683798884 wait complete. 2023-10-18T10:46:03.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.258+0000 7f23c99fa640 1 Processor -- start 2023-10-18T10:46:03.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.258+0000 7f23c99fa640 1 -- start start 2023-10-18T10:46:03.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.258+0000 7f23c99fa640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23c40cd160 0x7f23c40b7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:03.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.258+0000 7f23c99fa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f23c40d0760 0x7f23c40c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:03.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.258+0000 7f23c99fa640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23c4152010 0x7f23c40b5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:03.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.258+0000 7f23c99fa640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c41534b0 con 0x7f23c40cd160 2023-10-18T10:46:03.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.258+0000 7f23c99fa640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c41524b0 con 0x7f23c4152010 2023-10-18T10:46:03.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.258+0000 7f23c99fa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c4152790 con 0x7f23c40d0760 2023-10-18T10:46:03.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.258+0000 7f23c2ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23c40cd160 0x7f23c40b7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:03.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.258+0000 7f23c2ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23c40cd160 0x7f23c40b7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44978/0 (socket says 172.21.15.89:44978) 2023-10-18T10:46:03.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.258+0000 7f23c2ffd640 1 -- 172.21.15.89:0/1561960032 learned_addr learned my addr 172.21.15.89:0/1561960032 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:03.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.262+0000 7f23c2ffd640 1 -- 172.21.15.89:0/1561960032 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f23c40d0760 msgr2=0x7f23c40c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:03.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.262+0000 7f23c2ffd640 1 --2- 172.21.15.89:0/1561960032 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f23c40d0760 0x7f23c40c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.262+0000 7f23c2ffd640 1 -- 172.21.15.89:0/1561960032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23c4152010 msgr2=0x7f23c40b5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:03.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.262+0000 7f23c2ffd640 1 --2- 172.21.15.89:0/1561960032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23c4152010 0x7f23c40b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.262+0000 7f23c2ffd640 1 -- 172.21.15.89:0/1561960032 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23bc009420 con 0x7f23c40cd160 2023-10-18T10:46:03.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.262+0000 7f23c2ffd640 1 --2- 172.21.15.89:0/1561960032 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23c40cd160 0x7f23c40b7620 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f23b400cbc0 tx=0x7f23b400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:03.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.262+0000 7f23abfff640 1 -- 172.21.15.89:0/1561960032 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b400aca0 con 0x7f23c40cd160 2023-10-18T10:46:03.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.262+0000 7f23abfff640 1 -- 172.21.15.89:0/1561960032 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b400ae00 con 0x7f23c40cd160 2023-10-18T10:46:03.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.262+0000 7f23c99fa640 1 -- 172.21.15.89:0/1561960032 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c41536c0 con 0x7f23c40cd160 2023-10-18T10:46:03.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.262+0000 7f23abfff640 1 -- 172.21.15.89:0/1561960032 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b4012600 con 0x7f23c40cd160 2023-10-18T10:46:03.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.262+0000 7f23c99fa640 1 -- 172.21.15.89:0/1561960032 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c4159fb0 con 0x7f23c40cd160 2023-10-18T10:46:03.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.262+0000 7f23c99fa640 1 -- 172.21.15.89:0/1561960032 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23c404dbe0 con 0x7f23c40cd160 2023-10-18T10:46:03.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.262+0000 7f23abfff640 1 -- 172.21.15.89:0/1561960032 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f23b4007070 con 0x7f23c40cd160 2023-10-18T10:46:03.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.262+0000 7f23abfff640 1 --2- 172.21.15.89:0/1561960032 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f23b0065680 0x7f23b0067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:03.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.262+0000 7f23abfff640 1 -- 172.21.15.89:0/1561960032 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f23b4082fe0 con 0x7f23c40cd160 2023-10-18T10:46:03.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.262+0000 7f23c27fc640 1 --2- 172.21.15.89:0/1561960032 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f23b0065680 0x7f23b0067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:03.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.266+0000 7f23c27fc640 1 --2- 172.21.15.89:0/1561960032 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f23b0065680 0x7f23b0067b40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f23b8006aa0 tx=0x7f23b8006a30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:03.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.266+0000 7f23abfff640 1 -- 172.21.15.89:0/1561960032 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f23b4050820 con 0x7f23c40cd160 2023-10-18T10:46:03.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.450+0000 7f23c99fa640 1 -- 172.21.15.89:0/1561960032 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f23c4153b70 con 0x7f23c40cd160 2023-10-18T10:46:03.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.450+0000 7f23abfff640 1 -- 172.21.15.89:0/1561960032 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f23b4053e40 con 0x7f23c40cd160 2023-10-18T10:46:03.452 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-18T10:46:03.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.450+0000 7f23c99fa640 1 -- 172.21.15.89:0/1561960032 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f23b0065680 msgr2=0x7f23b0067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:03.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.450+0000 7f23c99fa640 1 --2- 172.21.15.89:0/1561960032 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f23b0065680 0x7f23b0067b40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f23b8006aa0 tx=0x7f23b8006a30 comp rx=0 tx=0).stop 2023-10-18T10:46:03.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.450+0000 7f23c99fa640 1 -- 172.21.15.89:0/1561960032 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23c40cd160 msgr2=0x7f23c40b7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:03.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.450+0000 7f23c99fa640 1 --2- 172.21.15.89:0/1561960032 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23c40cd160 0x7f23c40b7620 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f23b400cbc0 tx=0x7f23b400a060 comp rx=0 tx=0).stop 2023-10-18T10:46:03.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.450+0000 7f23c99fa640 1 -- 172.21.15.89:0/1561960032 shutdown_connections 2023-10-18T10:46:03.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.450+0000 7f23c99fa640 1 --2- 172.21.15.89:0/1561960032 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f23b0065680 0x7f23b0067b40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.450+0000 7f23c99fa640 1 --2- 172.21.15.89:0/1561960032 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f23c40d0760 0x7f23c40c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.450+0000 7f23c99fa640 1 --2- 172.21.15.89:0/1561960032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23c4152010 0x7f23c40b5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.450+0000 7f23c99fa640 1 --2- 172.21.15.89:0/1561960032 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23c40cd160 0x7f23c40b7620 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:03.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.450+0000 7f23c99fa640 1 -- 172.21.15.89:0/1561960032 >> 172.21.15.89:0/1561960032 conn(0x7f23c4089fd0 msgr2=0x7f23c40cf040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:03.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.450+0000 7f23c99fa640 1 -- 172.21.15.89:0/1561960032 shutdown_connections 2023-10-18T10:46:03.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:03.450+0000 7f23c99fa640 1 -- 172.21.15.89:0/1561960032 wait complete. 2023-10-18T10:46:03.472 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-18T10:46:03.472 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T10:46:03.473 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi089.front.sepia.ceph.com 2023-10-18T10:46:03.473 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-10-18T10:46:03.473 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-10-18T10:46:03.474 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-18T10:46:03.474 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-18T10:46:03.474 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T10:46:03.475 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-10-18T10:46:03.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:03.475 DEBUG:teuthology.orchestra.run.smithi089:> ip addr 2023-10-18T10:46:03.484 INFO:teuthology.orchestra.run.smithi089.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T10:46:03.485 INFO:teuthology.orchestra.run.smithi089.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T10:46:03.485 INFO:teuthology.orchestra.run.smithi089.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T10:46:03.485 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:03.485 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 ::1/128 scope host 2023-10-18T10:46:03.486 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:03.486 INFO:teuthology.orchestra.run.smithi089.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-18T10:46:03.486 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c6 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:46:03.486 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f0 2023-10-18T10:46:03.486 INFO:teuthology.orchestra.run.smithi089.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-18T10:46:03.487 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c7 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:46:03.487 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f1 2023-10-18T10:46:03.487 INFO:teuthology.orchestra.run.smithi089.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-18T10:46:03.487 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:00 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:46:03.487 INFO:teuthology.orchestra.run.smithi089.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T10:46:03.488 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:01 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:46:03.488 INFO:teuthology.orchestra.run.smithi089.stdout: inet 172.21.15.89/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-18T10:46:03.488 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft 42386sec preferred_lft 42386sec 2023-10-18T10:46:03.488 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec4:7aff:fe8f:d501/64 scope link 2023-10-18T10:46:03.488 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:03.488 INFO:teuthology.orchestra.run.smithi089.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-18T10:46:03.489 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 02:42:3f:fd:35:8d brd ff:ff:ff:ff:ff:ff 2023-10-18T10:46:03.489 INFO:teuthology.orchestra.run.smithi089.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-18T10:46:03.489 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:03.490 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-18T10:46:03.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:03.491 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-18T10:46:03.492 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-18T10:46:03.492 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link add name ceph-brx type bridge 2023-10-18T10:46:03.492 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip addr flush dev ceph-brx 2023-10-18T10:46:03.493 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set ceph-brx up 2023-10-18T10:46:03.493 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-18T10:46:03.494 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-18T10:46:03.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:03.605 DEBUG:teuthology.orchestra.run.smithi089:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T10:46:03.623 INFO:teuthology.orchestra.run.smithi089.stdout:1 2023-10-18T10:46:03.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:03.626 DEBUG:teuthology.orchestra.run.smithi089:> route 2023-10-18T10:46:03.713 INFO:teuthology.orchestra.run.smithi089.stdout:Kernel IP routing table 2023-10-18T10:46:03.713 INFO:teuthology.orchestra.run.smithi089.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T10:46:03.713 INFO:teuthology.orchestra.run.smithi089.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-18T10:46:03.714 INFO:teuthology.orchestra.run.smithi089.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T10:46:03.714 INFO:teuthology.orchestra.run.smithi089.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-18T10:46:03.714 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T10:46:03.714 INFO:teuthology.orchestra.run.smithi089.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-18T10:46:03.715 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-18T10:46:03.715 INFO:teuthology.orchestra.run.smithi089.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-18T10:46:03.715 INFO:teuthology.orchestra.run.smithi089.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T10:46:03.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:03.716 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-18T10:46:03.716 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-18T10:46:03.717 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T10:46:03.717 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T10:46:03.717 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T10:46:03.718 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-18T10:46:03.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:03.812 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-10-18T10:46:03.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:03.823 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list-id 2023-10-18T10:46:03.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:03.878 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-18T10:46:03.879 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-18T10:46:03.879 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-18T10:46:03.879 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-10-18T10:46:03.880 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-18T10:46:03.960 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-10-18T10:46:03.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:03.961 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-18T10:46:03.961 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-18T10:46:03.961 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-10-18T10:46:03.962 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-18T10:46:03.962 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-10-18T10:46:03.962 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-10-18T10:46:03.963 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-10-18T10:46:03.963 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-18T10:46:04.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:04.124 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-18T10:46:04.125 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-18T10:46:04.125 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set brx.0 up 2023-10-18T10:46:04.126 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set dev brx.0 master ceph-brx 2023-10-18T10:46:04.126 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-18T10:46:04.168 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-10-18T10:46:04.168 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:46:04.169 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-10-18T10:46:04.222 INFO:teuthology.orchestra.run.smithi089.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-10-18T10:46:04.222 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:46:04.223 DEBUG:teuthology.orchestra.run.smithi089:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-10-18T10:46:04.273 DEBUG:teuthology.orchestra.run.smithi089:> sudo modprobe fuse 2023-10-18T10:46:04.333 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-18T10:46:04.334 DEBUG:teuthology.orchestra.run.smithi089:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-18T10:46:04.396 INFO:teuthology.orchestra.run.smithi089.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-18T10:46:04.398 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-18T10:46:04.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:04.399 DEBUG:teuthology.orchestra.run.smithi089:> ls /sys/fs/fuse/connections 2023-10-18T10:46:04.449 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-18T10:46:04.450 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-10-18T10:46:04.496 DEBUG:teuthology.orchestra.run.smithi089:> sudo modprobe fuse 2023-10-18T10:46:04.517 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-18T10:46:04.518 DEBUG:teuthology.orchestra.run.smithi089:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-18T10:46:04.580 INFO:teuthology.orchestra.run.smithi089.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-18T10:46:04.581 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-18T10:46:04.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:04.582 DEBUG:teuthology.orchestra.run.smithi089:> ls /sys/fs/fuse/connections 2023-10-18T10:46:04.610 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi089.stderr:2023-10-18T10:46:04.606+0000 7f5115c45580 -1 init, newargv = 0x55e880f4b770 newargc=15 2023-10-18T10:46:04.610 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi089.stderr:ceph-fuse[44480]: starting ceph client 2023-10-18T10:46:04.620 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi089.stderr:ceph-fuse[44480]: starting fuse 2023-10-18T10:46:05.720 DEBUG:teuthology.orchestra.run.smithi089:> sudo modprobe fuse 2023-10-18T10:46:05.746 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-18T10:46:05.746 DEBUG:teuthology.orchestra.run.smithi089:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-18T10:46:05.809 INFO:teuthology.orchestra.run.smithi089.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-18T10:46:05.811 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-18T10:46:05.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:05.812 DEBUG:teuthology.orchestra.run.smithi089:> ls /sys/fs/fuse/connections 2023-10-18T10:46:05.866 INFO:teuthology.orchestra.run.smithi089.stdout:43 2023-10-18T10:46:05.866 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-10-18T10:46:05.867 DEBUG:teuthology.orchestra.run.smithi089:> sudo stdin-killer -- python3 -c ' 2023-10-18T10:46:05.867 DEBUG:teuthology.orchestra.run.smithi089:> import glob 2023-10-18T10:46:05.868 DEBUG:teuthology.orchestra.run.smithi089:> import re 2023-10-18T10:46:05.868 DEBUG:teuthology.orchestra.run.smithi089:> import os 2023-10-18T10:46:05.868 DEBUG:teuthology.orchestra.run.smithi089:> import subprocess 2023-10-18T10:46:05.869 DEBUG:teuthology.orchestra.run.smithi089:> 2023-10-18T10:46:05.869 DEBUG:teuthology.orchestra.run.smithi089:> def _find_admin_socket(client_name): 2023-10-18T10:46:05.869 DEBUG:teuthology.orchestra.run.smithi089:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-10-18T10:46:05.870 DEBUG:teuthology.orchestra.run.smithi089:> files = glob.glob(asok_path) 2023-10-18T10:46:05.870 DEBUG:teuthology.orchestra.run.smithi089:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-10-18T10:46:05.870 DEBUG:teuthology.orchestra.run.smithi089:> 2023-10-18T10:46:05.871 DEBUG:teuthology.orchestra.run.smithi089:> # Given a non-glob path, it better be there 2023-10-18T10:46:05.871 DEBUG:teuthology.orchestra.run.smithi089:> if "*" not in asok_path: 2023-10-18T10:46:05.871 DEBUG:teuthology.orchestra.run.smithi089:> assert(len(files) == 1) 2023-10-18T10:46:05.872 DEBUG:teuthology.orchestra.run.smithi089:> return files[0] 2023-10-18T10:46:05.872 DEBUG:teuthology.orchestra.run.smithi089:> 2023-10-18T10:46:05.872 DEBUG:teuthology.orchestra.run.smithi089:> for f in files: 2023-10-18T10:46:05.873 DEBUG:teuthology.orchestra.run.smithi089:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-18T10:46:05.873 DEBUG:teuthology.orchestra.run.smithi089:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-18T10:46:05.873 DEBUG:teuthology.orchestra.run.smithi089:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-18T10:46:05.874 DEBUG:teuthology.orchestra.run.smithi089:> contents = proc_f.read() 2023-10-18T10:46:05.874 DEBUG:teuthology.orchestra.run.smithi089:> if mountpoint in contents: 2023-10-18T10:46:05.875 DEBUG:teuthology.orchestra.run.smithi089:> return f 2023-10-18T10:46:05.875 DEBUG:teuthology.orchestra.run.smithi089:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-18T10:46:05.875 DEBUG:teuthology.orchestra.run.smithi089:> 2023-10-18T10:46:05.876 DEBUG:teuthology.orchestra.run.smithi089:> print(_find_admin_socket("client.admin")) 2023-10-18T10:46:05.876 DEBUG:teuthology.orchestra.run.smithi089:> ' 2023-10-18T10:46:05.994 INFO:teuthology.orchestra.run.smithi089.stdout:/var/run/ceph/ceph-client.admin.44480.asok 2023-10-18T10:46:05.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-18T10:46:06.005 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.44480.asok 2023-10-18T10:46:06.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:06.006 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.44480.asok status 2023-10-18T10:46:06.176 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-18T10:46:06.177 INFO:teuthology.orchestra.run.smithi089.stdout: "metadata": { 2023-10-18T10:46:06.177 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_sha1": "6f3e6b48891b9b50415c0822b6b870797fcda16d", 2023-10-18T10:46:06.177 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_version": "ceph version 18.0.0-5610-g6f3e6b48 (6f3e6b48891b9b50415c0822b6b870797fcda16d) reef (dev)", 2023-10-18T10:46:06.177 INFO:teuthology.orchestra.run.smithi089.stdout: "entity_id": "admin", 2023-10-18T10:46:06.178 INFO:teuthology.orchestra.run.smithi089.stdout: "hostname": "smithi089", 2023-10-18T10:46:06.178 INFO:teuthology.orchestra.run.smithi089.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-10-18T10:46:06.178 INFO:teuthology.orchestra.run.smithi089.stdout: "pid": "44480", 2023-10-18T10:46:06.178 INFO:teuthology.orchestra.run.smithi089.stdout: "root": "/" 2023-10-18T10:46:06.178 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-18T10:46:06.179 INFO:teuthology.orchestra.run.smithi089.stdout: "dentry_count": 0, 2023-10-18T10:46:06.179 INFO:teuthology.orchestra.run.smithi089.stdout: "dentry_pinned_count": 0, 2023-10-18T10:46:06.179 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 4358, 2023-10-18T10:46:06.179 INFO:teuthology.orchestra.run.smithi089.stdout: "inst": { 2023-10-18T10:46:06.180 INFO:teuthology.orchestra.run.smithi089.stdout: "name": { 2023-10-18T10:46:06.180 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "client", 2023-10-18T10:46:06.180 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 4358 2023-10-18T10:46:06.180 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-18T10:46:06.180 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": { 2023-10-18T10:46:06.180 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2023-10-18T10:46:06.181 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:0", 2023-10-18T10:46:06.181 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 1533837254 2023-10-18T10:46:06.181 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-18T10:46:06.182 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-18T10:46:06.182 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": { 2023-10-18T10:46:06.182 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2023-10-18T10:46:06.182 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:0", 2023-10-18T10:46:06.182 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 1533837254 2023-10-18T10:46:06.183 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-18T10:46:06.183 INFO:teuthology.orchestra.run.smithi089.stdout: "inst_str": "client.4358 172.21.15.89:0/1533837254", 2023-10-18T10:46:06.183 INFO:teuthology.orchestra.run.smithi089.stdout: "addr_str": "172.21.15.89:0/1533837254", 2023-10-18T10:46:06.183 INFO:teuthology.orchestra.run.smithi089.stdout: "inode_count": 1, 2023-10-18T10:46:06.183 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_epoch": 7, 2023-10-18T10:46:06.184 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_epoch": 27, 2023-10-18T10:46:06.184 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_epoch_barrier": 0, 2023-10-18T10:46:06.184 INFO:teuthology.orchestra.run.smithi089.stdout: "blocklisted": false, 2023-10-18T10:46:06.184 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs" 2023-10-18T10:46:06.184 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-18T10:46:06.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:06.191 DEBUG:teuthology.orchestra.run.smithi089:> stat --file-system '--printf=%T 2023-10-18T10:46:06.192 DEBUG:teuthology.orchestra.run.smithi089:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-10-18T10:46:06.202 INFO:teuthology.orchestra.run.smithi089.stdout:fuseblk 2023-10-18T10:46:06.202 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-10-18T10:46:06.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:06.203 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-10-18T10:46:06.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:06.271 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-10-18T10:46:06.346 INFO:teuthology.orchestra.run.smithi089.stdout:# file: . 2023-10-18T10:46:06.346 INFO:teuthology.orchestra.run.smithi089.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-10-18T10:46:06.347 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:06.348 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:46:06.348 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:46:06.397 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:46:06.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.574+0000 7f759bc34640 1 Processor -- start 2023-10-18T10:46:06.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.578+0000 7f759bc34640 1 -- start start 2023-10-18T10:46:06.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.578+0000 7f759bc34640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f759415c550 0x7f75940d2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:06.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.578+0000 7f759bc34640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f75940cfd20 0x7f75940b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:06.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.578+0000 7f759bc34640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75940d1e50 con 0x7f75940b1860 2023-10-18T10:46:06.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.578+0000 7f759bc34640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75940d2130 con 0x7f759415c550 2023-10-18T10:46:06.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.578+0000 7f759bc34640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75940d2410 con 0x7f75940cfd20 2023-10-18T10:46:06.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.578+0000 7f75991cf640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f75940b1860 0x7f75940cf610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55602/0 (socket says 172.21.15.89:55602) 2023-10-18T10:46:06.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.578+0000 7f75991cf640 1 -- 172.21.15.89:0/3913448692 learned_addr learned my addr 172.21.15.89:0/3913448692 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:06.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.578+0000 7f75999d0640 1 --2- 172.21.15.89:0/3913448692 >> v2:172.21.15.154:3300/0 conn(0x7f759415c550 0x7f75940d2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:06.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.578+0000 7f759a1d1640 1 --2- 172.21.15.89:0/3913448692 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f75940cfd20 0x7f75940b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:06.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.578+0000 7f75989ce640 1 -- 172.21.15.89:0/3913448692 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3074775066 0 0) 0x7f75940d1e50 con 0x7f75940b1860 2023-10-18T10:46:06.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.578+0000 7f75989ce640 1 -- 172.21.15.89:0/3913448692 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7580003530 con 0x7f75940b1860 2023-10-18T10:46:06.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.578+0000 7f75999d0640 1 -- 172.21.15.89:0/3913448692 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f75940cfd20 msgr2=0x7f75940b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:06.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.578+0000 7f75999d0640 1 --2- 172.21.15.89:0/3913448692 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f75940cfd20 0x7f75940b55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:06.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.578+0000 7f75999d0640 1 -- 172.21.15.89:0/3913448692 >> v1:172.21.15.89:6789/0 conn(0x7f75940b1860 legacy=0x7f75940cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:06.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.578+0000 7f75999d0640 1 -- 172.21.15.89:0/3913448692 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75940d26f0 con 0x7f759415c550 2023-10-18T10:46:06.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.582+0000 7f75999d0640 1 --2- 172.21.15.89:0/3913448692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f759415c550 0x7f75940d2ef0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f75900097c0 tx=0x7f759001ceb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3564c4327f87482a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:06.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.582+0000 7f75989ce640 1 -- 172.21.15.89:0/3913448692 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75900ac410 con 0x7f759415c550 2023-10-18T10:46:06.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.582+0000 7f75989ce640 1 -- 172.21.15.89:0/3913448692 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75900ac570 con 0x7f759415c550 2023-10-18T10:46:06.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.582+0000 7f75989ce640 1 -- 172.21.15.89:0/3913448692 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75900ac8e0 con 0x7f759415c550 2023-10-18T10:46:06.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.582+0000 7f759bc34640 1 -- 172.21.15.89:0/3913448692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f759415c550 msgr2=0x7f75940d2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:06.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.582+0000 7f759bc34640 1 --2- 172.21.15.89:0/3913448692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f759415c550 0x7f75940d2ef0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f75900097c0 tx=0x7f759001ceb0 comp rx=0 tx=0).stop 2023-10-18T10:46:06.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.582+0000 7f759bc34640 1 -- 172.21.15.89:0/3913448692 shutdown_connections 2023-10-18T10:46:06.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.582+0000 7f759bc34640 1 --2- 172.21.15.89:0/3913448692 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f75940cfd20 0x7f75940b55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:06.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.582+0000 7f759bc34640 1 --2- 172.21.15.89:0/3913448692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f759415c550 0x7f75940d2ef0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:06.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.582+0000 7f759bc34640 1 -- 172.21.15.89:0/3913448692 >> 172.21.15.89:0/3913448692 conn(0x7f7594069d00 msgr2=0x7f759406a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:06.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.582+0000 7f759bc34640 1 -- 172.21.15.89:0/3913448692 shutdown_connections 2023-10-18T10:46:06.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.582+0000 7f759bc34640 1 -- 172.21.15.89:0/3913448692 wait complete. 2023-10-18T10:46:06.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.582+0000 7f759bc34640 1 Processor -- start 2023-10-18T10:46:06.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.586+0000 7f759bc34640 1 -- start start 2023-10-18T10:46:06.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.586+0000 7f759bc34640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75940b1860 0x7f7594154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:06.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.586+0000 7f759bc34640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75940cfd20 0x7f75941529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:06.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.586+0000 7f759bc34640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f759415c550 0x7f7594166980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:06.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.586+0000 7f759bc34640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75941549d0 con 0x7f75940cfd20 2023-10-18T10:46:06.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.586+0000 7f759bc34640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7594152f20 con 0x7f75940b1860 2023-10-18T10:46:06.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.586+0000 7f759bc34640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7594166ec0 con 0x7f759415c550 2023-10-18T10:46:06.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.586+0000 7f75991cf640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75940cfd20 0x7f75941529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:06.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.586+0000 7f75991cf640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75940cfd20 0x7f75941529e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44980/0 (socket says 172.21.15.89:44980) 2023-10-18T10:46:06.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.586+0000 7f75991cf640 1 -- 172.21.15.89:0/1075315531 learned_addr learned my addr 172.21.15.89:0/1075315531 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:06.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.586+0000 7f75999d0640 1 --2- 172.21.15.89:0/1075315531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75940b1860 0x7f7594154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:06.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.586+0000 7f759a1d1640 1 --2- 172.21.15.89:0/1075315531 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f759415c550 0x7f7594166980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:06.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.586+0000 7f75991cf640 1 -- 172.21.15.89:0/1075315531 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f759415c550 msgr2=0x7f7594166980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:06.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.586+0000 7f75991cf640 1 --2- 172.21.15.89:0/1075315531 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f759415c550 0x7f7594166980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:06.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.586+0000 7f75991cf640 1 -- 172.21.15.89:0/1075315531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75940b1860 msgr2=0x7f7594154490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:06.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.586+0000 7f75991cf640 1 --2- 172.21.15.89:0/1075315531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75940b1860 0x7f7594154490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:06.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.586+0000 7f75991cf640 1 -- 172.21.15.89:0/1075315531 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7590009420 con 0x7f75940cfd20 2023-10-18T10:46:06.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.586+0000 7f75999d0640 1 --2- 172.21.15.89:0/1075315531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75940b1860 0x7f7594154490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:06.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.586+0000 7f75991cf640 1 --2- 172.21.15.89:0/1075315531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75940cfd20 0x7f75941529e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f758400c480 tx=0x7f758400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:06.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.586+0000 7f758a7fc640 1 -- 172.21.15.89:0/1075315531 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7584005cf0 con 0x7f75940cfd20 2023-10-18T10:46:06.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.586+0000 7f758a7fc640 1 -- 172.21.15.89:0/1075315531 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7584005e50 con 0x7f75940cfd20 2023-10-18T10:46:06.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.586+0000 7f758a7fc640 1 -- 172.21.15.89:0/1075315531 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7584007290 con 0x7f75940cfd20 2023-10-18T10:46:06.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.586+0000 7f759bc34640 1 -- 172.21.15.89:0/1075315531 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75940c99a0 con 0x7f75940cfd20 2023-10-18T10:46:06.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.586+0000 7f759bc34640 1 -- 172.21.15.89:0/1075315531 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7594153ed0 con 0x7f75940cfd20 2023-10-18T10:46:06.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.586+0000 7f759bc34640 1 -- 172.21.15.89:0/1075315531 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75940c9750 con 0x7f75940cfd20 2023-10-18T10:46:06.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.590+0000 7f758a7fc640 1 -- 172.21.15.89:0/1075315531 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f75840073f0 con 0x7f75940cfd20 2023-10-18T10:46:06.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.590+0000 7f758a7fc640 1 --2- 172.21.15.89:0/1075315531 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7578065680 0x7f7578067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:06.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.590+0000 7f758a7fc640 1 -- 172.21.15.89:0/1075315531 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7584082670 con 0x7f75940cfd20 2023-10-18T10:46:06.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.590+0000 7f75999d0640 1 --2- 172.21.15.89:0/1075315531 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7578065680 0x7f7578067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:06.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.590+0000 7f75999d0640 1 --2- 172.21.15.89:0/1075315531 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7578065680 0x7f7578067b40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f75900ae000 tx=0x7f7590004740 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:06.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.594+0000 7f758a7fc640 1 -- 172.21.15.89:0/1075315531 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f758404fed0 con 0x7f75940cfd20 2023-10-18T10:46:06.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.742+0000 7f759bc34640 1 -- 172.21.15.89:0/1075315531 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75940b3c90 con 0x7f75940cfd20 2023-10-18T10:46:06.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.742+0000 7f758a7fc640 1 -- 172.21.15.89:0/1075315531 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20096 (secure 0 0 0) 0x7f75840534f0 con 0x7f75940cfd20 2023-10-18T10:46:06.745 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:06.745 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":27,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:46:00.166702+0000","last_up_change":"2023-10-18T10:45:42.164688+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:45:53.309122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-18T10:45:54.659763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:45:57.796514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6824","nonce":773078364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6827","nonce":773078364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6831","nonce":773078364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6829","nonce":773078364}]},"public_addr":"172.21.15.154:6824/773078364","cluster_addr":"172.21.15.154:6827/773078364","heartbeat_back_addr":"172.21.15.154:6831/773078364","heartbeat_front_addr":"172.21.15.154:6829/773078364","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:46:06.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.746+0000 7f759bc34640 1 -- 172.21.15.89:0/1075315531 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7578065680 msgr2=0x7f7578067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:06.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.746+0000 7f759bc34640 1 --2- 172.21.15.89:0/1075315531 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7578065680 0x7f7578067b40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f75900ae000 tx=0x7f7590004740 comp rx=0 tx=0).stop 2023-10-18T10:46:06.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.746+0000 7f759bc34640 1 -- 172.21.15.89:0/1075315531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75940cfd20 msgr2=0x7f75941529e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:06.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.746+0000 7f759bc34640 1 --2- 172.21.15.89:0/1075315531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75940cfd20 0x7f75941529e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f758400c480 tx=0x7f758400c940 comp rx=0 tx=0).stop 2023-10-18T10:46:06.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.746+0000 7f759bc34640 1 -- 172.21.15.89:0/1075315531 shutdown_connections 2023-10-18T10:46:06.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.746+0000 7f759bc34640 1 --2- 172.21.15.89:0/1075315531 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7578065680 0x7f7578067b40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:06.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.746+0000 7f759bc34640 1 --2- 172.21.15.89:0/1075315531 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f759415c550 0x7f7594166980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:06.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.746+0000 7f759bc34640 1 --2- 172.21.15.89:0/1075315531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75940cfd20 0x7f75941529e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:06.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.746+0000 7f759bc34640 1 --2- 172.21.15.89:0/1075315531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75940b1860 0x7f7594154490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:06.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.746+0000 7f759bc34640 1 -- 172.21.15.89:0/1075315531 >> 172.21.15.89:0/1075315531 conn(0x7f7594069d00 msgr2=0x7f75940b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:06.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.746+0000 7f759bc34640 1 -- 172.21.15.89:0/1075315531 shutdown_connections 2023-10-18T10:46:06.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.746+0000 7f759bc34640 1 -- 172.21.15.89:0/1075315531 wait complete. 2023-10-18T10:46:06.765 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:45:43.114184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-18T10:45:48.730968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:45:53.309122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:45:54.659763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T10:45:57.796514+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T10:46:06.766 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:46:06.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.906+0000 7f61ac18f640 1 Processor -- start 2023-10-18T10:46:06.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.906+0000 7f61ac18f640 1 -- start start 2023-10-18T10:46:06.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.906+0000 7f61ac18f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f61a40c8960 0x7f61a406c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:06.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.906+0000 7f61ac18f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f61a40afd90 0x7f61a40b01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:06.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.906+0000 7f61ac18f640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61a40b0720 con 0x7f61a415e600 2023-10-18T10:46:06.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.906+0000 7f61ac18f640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61a406c6f0 con 0x7f61a40c8960 2023-10-18T10:46:06.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.906+0000 7f61ac18f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a4160f80 con 0x7f61a40afd90 2023-10-18T10:46:06.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.906+0000 7f61a9f2b640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f61a415e600 0x7f61a4161970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55616/0 (socket says 172.21.15.89:55616) 2023-10-18T10:46:06.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.906+0000 7f61a9f2b640 1 -- 172.21.15.89:0/675870998 learned_addr learned my addr 172.21.15.89:0/675870998 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:06.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.906+0000 7f61a972a640 1 --2- 172.21.15.89:0/675870998 >> v2:172.21.15.89:3300/0 conn(0x7f61a40c8960 0x7f61a406c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:06.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.906+0000 7f61aa72c640 1 --2- 172.21.15.89:0/675870998 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f61a40afd90 0x7f61a40b01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:06.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.906+0000 7f61a972a640 1 -- 172.21.15.89:0/675870998 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f61a40afd90 msgr2=0x7f61a40b01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:06.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.906+0000 7f61a972a640 1 --2- 172.21.15.89:0/675870998 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f61a40afd90 0x7f61a40b01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:06.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.906+0000 7f61a972a640 1 -- 172.21.15.89:0/675870998 >> v1:172.21.15.89:6789/0 conn(0x7f61a415e600 legacy=0x7f61a4161970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:06.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.906+0000 7f61a972a640 1 -- 172.21.15.89:0/675870998 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a41614e0 con 0x7f61a40c8960 2023-10-18T10:46:06.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.910+0000 7f61a972a640 1 --2- 172.21.15.89:0/675870998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61a40c8960 0x7f61a406c1b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f61940097f0 tx=0x7f619401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77209f61a0feb91f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:06.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.910+0000 7f61a8f29640 1 -- 172.21.15.89:0/675870998 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61940ab600 con 0x7f61a40c8960 2023-10-18T10:46:06.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.910+0000 7f61a8f29640 1 -- 172.21.15.89:0/675870998 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61940a7050 con 0x7f61a40c8960 2023-10-18T10:46:06.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.910+0000 7f61a8f29640 1 -- 172.21.15.89:0/675870998 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61940ab980 con 0x7f61a40c8960 2023-10-18T10:46:06.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.910+0000 7f61ac18f640 1 -- 172.21.15.89:0/675870998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61a40c8960 msgr2=0x7f61a406c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:06.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.910+0000 7f61ac18f640 1 --2- 172.21.15.89:0/675870998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61a40c8960 0x7f61a406c1b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f61940097f0 tx=0x7f619401c480 comp rx=0 tx=0).stop 2023-10-18T10:46:06.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.910+0000 7f61ac18f640 1 -- 172.21.15.89:0/675870998 shutdown_connections 2023-10-18T10:46:06.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.910+0000 7f61ac18f640 1 --2- 172.21.15.89:0/675870998 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f61a40afd90 0x7f61a40b01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:06.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.910+0000 7f61ac18f640 1 --2- 172.21.15.89:0/675870998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61a40c8960 0x7f61a406c1b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:06.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.910+0000 7f61ac18f640 1 -- 172.21.15.89:0/675870998 >> 172.21.15.89:0/675870998 conn(0x7f61a4069f00 msgr2=0x7f61a40cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:06.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.910+0000 7f61ac18f640 1 -- 172.21.15.89:0/675870998 shutdown_connections 2023-10-18T10:46:06.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.910+0000 7f61ac18f640 1 -- 172.21.15.89:0/675870998 wait complete. 2023-10-18T10:46:06.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.914+0000 7f61ac18f640 1 Processor -- start 2023-10-18T10:46:06.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.914+0000 7f61ac18f640 1 -- start start 2023-10-18T10:46:06.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.914+0000 7f61ac18f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61a40afd90 0x7f61a40b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:06.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.914+0000 7f61ac18f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61a40c8960 0x7f61a415a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:06.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.914+0000 7f61ac18f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f61a415e600 0x7f61a415fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:06.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.914+0000 7f61ac18f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a40c7fb0 con 0x7f61a40c8960 2023-10-18T10:46:06.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.914+0000 7f61ac18f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a415d770 con 0x7f61a40afd90 2023-10-18T10:46:06.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.914+0000 7f61ac18f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a40c7cd0 con 0x7f61a415e600 2023-10-18T10:46:06.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.914+0000 7f61a972a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61a40c8960 0x7f61a415a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:06.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.914+0000 7f61a972a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61a40c8960 0x7f61a415a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45002/0 (socket says 172.21.15.89:45002) 2023-10-18T10:46:06.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.914+0000 7f61a972a640 1 -- 172.21.15.89:0/3574201154 learned_addr learned my addr 172.21.15.89:0/3574201154 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:06.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.914+0000 7f61a9f2b640 1 --2- 172.21.15.89:0/3574201154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61a40afd90 0x7f61a40b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:06.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.914+0000 7f61aa72c640 1 --2- 172.21.15.89:0/3574201154 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f61a415e600 0x7f61a415fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:06.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.914+0000 7f61a972a640 1 -- 172.21.15.89:0/3574201154 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f61a415e600 msgr2=0x7f61a415fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:06.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.914+0000 7f61a972a640 1 --2- 172.21.15.89:0/3574201154 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f61a415e600 0x7f61a415fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:06.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.914+0000 7f61a972a640 1 -- 172.21.15.89:0/3574201154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61a40afd90 msgr2=0x7f61a40b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:06.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.914+0000 7f61a972a640 1 --2- 172.21.15.89:0/3574201154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61a40afd90 0x7f61a40b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:06.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.914+0000 7f61a972a640 1 -- 172.21.15.89:0/3574201154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6194009420 con 0x7f61a40c8960 2023-10-18T10:46:06.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.914+0000 7f61a972a640 1 --2- 172.21.15.89:0/3574201154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61a40c8960 0x7f61a415a6a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f619401a0b0 tx=0x7f6194019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:06.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.914+0000 7f619a7fc640 1 -- 172.21.15.89:0/3574201154 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61940b4840 con 0x7f61a40c8960 2023-10-18T10:46:06.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.914+0000 7f619a7fc640 1 -- 172.21.15.89:0/3574201154 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61940b49c0 con 0x7f61a40c8960 2023-10-18T10:46:06.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.914+0000 7f619a7fc640 1 -- 172.21.15.89:0/3574201154 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61940b4d70 con 0x7f61a40c8960 2023-10-18T10:46:06.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.914+0000 7f61ac18f640 1 -- 172.21.15.89:0/3574201154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61a40c1af0 con 0x7f61a40c8960 2023-10-18T10:46:06.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.914+0000 7f61ac18f640 1 -- 172.21.15.89:0/3574201154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a40c20b0 con 0x7f61a40c8960 2023-10-18T10:46:06.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.918+0000 7f61ac18f640 1 -- 172.21.15.89:0/3574201154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61a40c1950 con 0x7f61a40c8960 2023-10-18T10:46:06.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.918+0000 7f619a7fc640 1 -- 172.21.15.89:0/3574201154 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f61940b2040 con 0x7f61a40c8960 2023-10-18T10:46:06.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.918+0000 7f619a7fc640 1 --2- 172.21.15.89:0/3574201154 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6188065680 0x7f6188067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:06.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.918+0000 7f619a7fc640 1 -- 172.21.15.89:0/3574201154 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f619411d480 con 0x7f61a40c8960 2023-10-18T10:46:06.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.918+0000 7f61a9f2b640 1 --2- 172.21.15.89:0/3574201154 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6188065680 0x7f6188067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:06.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.918+0000 7f61a9f2b640 1 --2- 172.21.15.89:0/3574201154 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6188065680 0x7f6188067b40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f61a0005890 tx=0x7f61a0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:06.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:06.922+0000 7f619a7fc640 1 -- 172.21.15.89:0/3574201154 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f61a40c1950 con 0x7f61a40c8960 2023-10-18T10:46:07.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.070+0000 7f61ac18f640 1 -- 172.21.15.89:0/3574201154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f61a40c1950 con 0x7f61a40c8960 2023-10-18T10:46:07.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.070+0000 7f619a7fc640 1 -- 172.21.15.89:0/3574201154 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f61a40c1950 con 0x7f61a40c8960 2023-10-18T10:46:07.074 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-18T10:46:07.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.074+0000 7f61ac18f640 1 -- 172.21.15.89:0/3574201154 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6188065680 msgr2=0x7f6188067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:07.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.074+0000 7f61ac18f640 1 --2- 172.21.15.89:0/3574201154 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6188065680 0x7f6188067b40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f61a0005890 tx=0x7f61a0009040 comp rx=0 tx=0).stop 2023-10-18T10:46:07.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.074+0000 7f61ac18f640 1 -- 172.21.15.89:0/3574201154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61a40c8960 msgr2=0x7f61a415a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:07.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.074+0000 7f61ac18f640 1 --2- 172.21.15.89:0/3574201154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61a40c8960 0x7f61a415a6a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f619401a0b0 tx=0x7f6194019220 comp rx=0 tx=0).stop 2023-10-18T10:46:07.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.074+0000 7f61ac18f640 1 -- 172.21.15.89:0/3574201154 shutdown_connections 2023-10-18T10:46:07.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.074+0000 7f61ac18f640 1 --2- 172.21.15.89:0/3574201154 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6188065680 0x7f6188067b40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:07.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.074+0000 7f61ac18f640 1 --2- 172.21.15.89:0/3574201154 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f61a415e600 0x7f61a415fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:07.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.074+0000 7f61ac18f640 1 --2- 172.21.15.89:0/3574201154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61a40c8960 0x7f61a415a6a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:07.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.074+0000 7f61ac18f640 1 --2- 172.21.15.89:0/3574201154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61a40afd90 0x7f61a40b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:07.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.074+0000 7f61ac18f640 1 -- 172.21.15.89:0/3574201154 >> 172.21.15.89:0/3574201154 conn(0x7f61a4069f00 msgr2=0x7f61a40cc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:07.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.074+0000 7f61ac18f640 1 -- 172.21.15.89:0/3574201154 shutdown_connections 2023-10-18T10:46:07.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.074+0000 7f61ac18f640 1 -- 172.21.15.89:0/3574201154 wait complete. 2023-10-18T10:46:07.094 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-18T10:46:07.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.234+0000 7efd0f2bd640 1 Processor -- start 2023-10-18T10:46:07.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.234+0000 7efd0f2bd640 1 -- start start 2023-10-18T10:46:07.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.238+0000 7efd0f2bd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efd0806a010 0x7efd080d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:07.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.238+0000 7efd0f2bd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efd080b1950 0x7efd080b0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:07.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.238+0000 7efd0f2bd640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd080b1430 con 0x7efd0815ff30 2023-10-18T10:46:07.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.238+0000 7efd0f2bd640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd080d3690 con 0x7efd0806a010 2023-10-18T10:46:07.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.238+0000 7efd0f2bd640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd080d3080 con 0x7efd080b1950 2023-10-18T10:46:07.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.238+0000 7efd0c858640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efd0806a010 0x7efd080d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:07.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.238+0000 7efd0c858640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efd0806a010 0x7efd080d0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45010/0 (socket says 172.21.15.89:45010) 2023-10-18T10:46:07.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.238+0000 7efd0c858640 1 -- 172.21.15.89:0/2879720340 learned_addr learned my addr 172.21.15.89:0/2879720340 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:07.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.238+0000 7efd0d85a640 1 --2- 172.21.15.89:0/2879720340 >> v2:172.21.15.154:3300/0 conn(0x7efd080b1950 0x7efd080b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:07.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.238+0000 7efd0c858640 1 -- 172.21.15.89:0/2879720340 >> v2:172.21.15.154:3300/0 conn(0x7efd080b1950 msgr2=0x7efd080b0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:07.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.238+0000 7efd0c858640 1 --2- 172.21.15.89:0/2879720340 >> v2:172.21.15.154:3300/0 conn(0x7efd080b1950 0x7efd080b0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:07.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.238+0000 7efd0c858640 1 -- 172.21.15.89:0/2879720340 >> v1:172.21.15.89:6789/0 conn(0x7efd0815ff30 legacy=0x7efd08069900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:07.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.238+0000 7efd0c858640 1 -- 172.21.15.89:0/2879720340 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd080d3320 con 0x7efd0806a010 2023-10-18T10:46:07.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.238+0000 7efd0d85a640 1 --2- 172.21.15.89:0/2879720340 >> v2:172.21.15.154:3300/0 conn(0x7efd080b1950 0x7efd080b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:07.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.238+0000 7efd0c858640 1 --2- 172.21.15.89:0/2879720340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd0806a010 0x7efd080d0fd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efd0808cb90 tx=0x7efcf801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aaf3ac1e1bbb2f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:07.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.238+0000 7efcfffff640 1 -- 172.21.15.89:0/2879720340 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf80ab600 con 0x7efd0806a010 2023-10-18T10:46:07.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.238+0000 7efcfffff640 1 -- 172.21.15.89:0/2879720340 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcf80ab760 con 0x7efd0806a010 2023-10-18T10:46:07.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.238+0000 7efcfffff640 1 -- 172.21.15.89:0/2879720340 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf80abaf0 con 0x7efd0806a010 2023-10-18T10:46:07.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.242+0000 7efd0f2bd640 1 -- 172.21.15.89:0/2879720340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd0806a010 msgr2=0x7efd080d0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:07.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.242+0000 7efd0f2bd640 1 --2- 172.21.15.89:0/2879720340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd0806a010 0x7efd080d0fd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efd0808cb90 tx=0x7efcf801c3e0 comp rx=0 tx=0).stop 2023-10-18T10:46:07.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.242+0000 7efd0f2bd640 1 -- 172.21.15.89:0/2879720340 shutdown_connections 2023-10-18T10:46:07.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.242+0000 7efd0f2bd640 1 --2- 172.21.15.89:0/2879720340 >> v2:172.21.15.154:3300/0 conn(0x7efd080b1950 0x7efd080b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:07.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.242+0000 7efd0f2bd640 1 --2- 172.21.15.89:0/2879720340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd0806a010 0x7efd080d0fd0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:07.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.242+0000 7efd0f2bd640 1 -- 172.21.15.89:0/2879720340 >> 172.21.15.89:0/2879720340 conn(0x7efd0809b9d0 msgr2=0x7efd080d26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:07.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.242+0000 7efd0f2bd640 1 -- 172.21.15.89:0/2879720340 shutdown_connections 2023-10-18T10:46:07.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.242+0000 7efd0f2bd640 1 -- 172.21.15.89:0/2879720340 wait complete. 2023-10-18T10:46:07.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.242+0000 7efd0f2bd640 1 Processor -- start 2023-10-18T10:46:07.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.242+0000 7efd0f2bd640 1 -- start start 2023-10-18T10:46:07.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.242+0000 7efd0f2bd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd0806a010 0x7efd081546f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:07.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.242+0000 7efd0f2bd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efd080b1950 0x7efd081f8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:07.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.242+0000 7efd0f2bd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd0815ff30 0x7efd08157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:07.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.242+0000 7efd0f2bd640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd081f95f0 con 0x7efd0815ff30 2023-10-18T10:46:07.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.242+0000 7efd0f2bd640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd080cb430 con 0x7efd0806a010 2023-10-18T10:46:07.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.242+0000 7efd0f2bd640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd081f9030 con 0x7efd080b1950 2023-10-18T10:46:07.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.242+0000 7efd0d85a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd0815ff30 0x7efd08157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:07.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.242+0000 7efd0d85a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd0815ff30 0x7efd08157b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45014/0 (socket says 172.21.15.89:45014) 2023-10-18T10:46:07.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.242+0000 7efd0d85a640 1 -- 172.21.15.89:0/103193845 learned_addr learned my addr 172.21.15.89:0/103193845 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:07.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.242+0000 7efd0d059640 1 --2- 172.21.15.89:0/103193845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd0806a010 0x7efd081546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:07.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.242+0000 7efd0c858640 1 --2- 172.21.15.89:0/103193845 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efd080b1950 0x7efd081f8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:07.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.246+0000 7efd0d85a640 1 -- 172.21.15.89:0/103193845 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efd080b1950 msgr2=0x7efd081f8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:07.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.246+0000 7efd0d85a640 1 --2- 172.21.15.89:0/103193845 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efd080b1950 0x7efd081f8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:07.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.246+0000 7efd0d85a640 1 -- 172.21.15.89:0/103193845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd0806a010 msgr2=0x7efd081546f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:07.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.246+0000 7efd0d85a640 1 --2- 172.21.15.89:0/103193845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd0806a010 0x7efd081546f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:07.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.246+0000 7efd0d85a640 1 -- 172.21.15.89:0/103193845 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf80094d0 con 0x7efd0815ff30 2023-10-18T10:46:07.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.246+0000 7efd0d85a640 1 --2- 172.21.15.89:0/103193845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd0815ff30 0x7efd08157b00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efd0000cf20 tx=0x7efd0000a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:07.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.246+0000 7efcfdffb640 1 -- 172.21.15.89:0/103193845 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0000d840 con 0x7efd0815ff30 2023-10-18T10:46:07.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.246+0000 7efcfdffb640 1 -- 172.21.15.89:0/103193845 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd0000d9a0 con 0x7efd0815ff30 2023-10-18T10:46:07.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.246+0000 7efd0f2bd640 1 -- 172.21.15.89:0/103193845 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd081f9310 con 0x7efd0815ff30 2023-10-18T10:46:07.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.246+0000 7efcfdffb640 1 -- 172.21.15.89:0/103193845 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0000dd50 con 0x7efd0815ff30 2023-10-18T10:46:07.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.246+0000 7efd0f2bd640 1 -- 172.21.15.89:0/103193845 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd08154c30 con 0x7efd0815ff30 2023-10-18T10:46:07.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.246+0000 7efd0f2bd640 1 -- 172.21.15.89:0/103193845 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd080caeb0 con 0x7efd0815ff30 2023-10-18T10:46:07.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.246+0000 7efcfdffb640 1 -- 172.21.15.89:0/103193845 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7efd0001e3b0 con 0x7efd0815ff30 2023-10-18T10:46:07.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.246+0000 7efcfdffb640 1 --2- 172.21.15.89:0/103193845 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efcec065680 0x7efcec067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:07.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.246+0000 7efcfdffb640 1 -- 172.21.15.89:0/103193845 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7efd00083af0 con 0x7efd0815ff30 2023-10-18T10:46:07.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.246+0000 7efd0d059640 1 --2- 172.21.15.89:0/103193845 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efcec065680 0x7efcec067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:07.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.250+0000 7efd0d059640 1 --2- 172.21.15.89:0/103193845 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efcec065680 0x7efcec067b40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efd04007fd0 tx=0x7efd04009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:07.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.250+0000 7efcfdffb640 1 -- 172.21.15.89:0/103193845 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7efd080caeb0 con 0x7efd0815ff30 2023-10-18T10:46:07.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.402+0000 7efd0f2bd640 1 -- 172.21.15.89:0/103193845 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7efd080cb570 con 0x7efd0815ff30 2023-10-18T10:46:07.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.402+0000 7efcfdffb640 1 -- 172.21.15.89:0/103193845 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7efd0001a020 con 0x7efd0815ff30 2023-10-18T10:46:07.404 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-10-18T10:46:07.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.402+0000 7efd0f2bd640 1 -- 172.21.15.89:0/103193845 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efcec065680 msgr2=0x7efcec067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:07.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.402+0000 7efd0f2bd640 1 --2- 172.21.15.89:0/103193845 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efcec065680 0x7efcec067b40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efd04007fd0 tx=0x7efd04009040 comp rx=0 tx=0).stop 2023-10-18T10:46:07.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.402+0000 7efd0f2bd640 1 -- 172.21.15.89:0/103193845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd0815ff30 msgr2=0x7efd08157b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:07.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.402+0000 7efd0f2bd640 1 --2- 172.21.15.89:0/103193845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd0815ff30 0x7efd08157b00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efd0000cf20 tx=0x7efd0000a7d0 comp rx=0 tx=0).stop 2023-10-18T10:46:07.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.402+0000 7efd0f2bd640 1 -- 172.21.15.89:0/103193845 shutdown_connections 2023-10-18T10:46:07.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.402+0000 7efd0f2bd640 1 --2- 172.21.15.89:0/103193845 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efcec065680 0x7efcec067b40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:07.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.402+0000 7efd0f2bd640 1 --2- 172.21.15.89:0/103193845 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efd080b1950 0x7efd081f8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:07.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.402+0000 7efd0f2bd640 1 --2- 172.21.15.89:0/103193845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd0815ff30 0x7efd08157b00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:07.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.402+0000 7efd0f2bd640 1 --2- 172.21.15.89:0/103193845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd0806a010 0x7efd081546f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:07.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.402+0000 7efd0f2bd640 1 -- 172.21.15.89:0/103193845 >> 172.21.15.89:0/103193845 conn(0x7efd0809b9d0 msgr2=0x7efd0815f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:07.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.406+0000 7efd0f2bd640 1 -- 172.21.15.89:0/103193845 shutdown_connections 2023-10-18T10:46:07.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.406+0000 7efd0f2bd640 1 -- 172.21.15.89:0/103193845 wait complete. 2023-10-18T10:46:07.425 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:46:07.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.562+0000 7f31821fa640 1 Processor -- start 2023-10-18T10:46:07.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.562+0000 7f31821fa640 1 -- start start 2023-10-18T10:46:07.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.566+0000 7f31821fa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f317c0ca340 0x7f317c15e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:07.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.566+0000 7f31821fa640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f317c0c8960 0x7f317c0c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:07.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.566+0000 7f31821fa640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f317c15acb0 con 0x7f317c06ff00 2023-10-18T10:46:07.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.566+0000 7f31821fa640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f317c15ef30 con 0x7f317c0c8960 2023-10-18T10:46:07.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.566+0000 7f31821fa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f317c15aa10 con 0x7f317c0ca340 2023-10-18T10:46:07.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.566+0000 7f317bfff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f317c0c8960 0x7f317c0c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:07.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.566+0000 7f317bfff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f317c0c8960 0x7f317c0c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45018/0 (socket says 172.21.15.89:45018) 2023-10-18T10:46:07.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.566+0000 7f317bfff640 1 -- 172.21.15.89:0/3902757756 learned_addr learned my addr 172.21.15.89:0/3902757756 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:07.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.566+0000 7f3172ffd640 1 --2- 172.21.15.89:0/3902757756 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f317c0ca340 0x7f317c15e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:07.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.566+0000 7f317bfff640 1 -- 172.21.15.89:0/3902757756 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f317c0ca340 msgr2=0x7f317c15e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:07.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.566+0000 7f317bfff640 1 --2- 172.21.15.89:0/3902757756 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f317c0ca340 0x7f317c15e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:07.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.566+0000 7f317bfff640 1 -- 172.21.15.89:0/3902757756 >> v1:172.21.15.154:6789/0 conn(0x7f317c06ff00 legacy=0x7f317c15b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:07.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.566+0000 7f317bfff640 1 -- 172.21.15.89:0/3902757756 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f317c0702e0 con 0x7f317c0c8960 2023-10-18T10:46:07.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.566+0000 7f317bfff640 1 --2- 172.21.15.89:0/3902757756 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f317c0c8960 0x7f317c0c8db0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3174009870 tx=0x7f317401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06a6853f228282 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:07.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.566+0000 7f317affd640 1 -- 172.21.15.89:0/3902757756 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31740ac600 con 0x7f317c0c8960 2023-10-18T10:46:07.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.566+0000 7f317affd640 1 -- 172.21.15.89:0/3902757756 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31740a8050 con 0x7f317c0c8960 2023-10-18T10:46:07.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.566+0000 7f317affd640 1 -- 172.21.15.89:0/3902757756 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31740ac980 con 0x7f317c0c8960 2023-10-18T10:46:07.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.570+0000 7f31821fa640 1 -- 172.21.15.89:0/3902757756 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f317c0c8960 msgr2=0x7f317c0c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:07.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.570+0000 7f31821fa640 1 --2- 172.21.15.89:0/3902757756 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f317c0c8960 0x7f317c0c8db0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3174009870 tx=0x7f317401c4b0 comp rx=0 tx=0).stop 2023-10-18T10:46:07.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.570+0000 7f31821fa640 1 -- 172.21.15.89:0/3902757756 shutdown_connections 2023-10-18T10:46:07.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.570+0000 7f31821fa640 1 --2- 172.21.15.89:0/3902757756 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f317c0c8960 0x7f317c0c8db0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:07.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.570+0000 7f31821fa640 1 --2- 172.21.15.89:0/3902757756 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f317c0ca340 0x7f317c15e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:07.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.570+0000 7f31821fa640 1 -- 172.21.15.89:0/3902757756 >> 172.21.15.89:0/3902757756 conn(0x7f317c069f00 msgr2=0x7f317c0cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:07.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.570+0000 7f31821fa640 1 -- 172.21.15.89:0/3902757756 shutdown_connections 2023-10-18T10:46:07.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.570+0000 7f31821fa640 1 -- 172.21.15.89:0/3902757756 wait complete. 2023-10-18T10:46:07.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.570+0000 7f31821fa640 1 Processor -- start 2023-10-18T10:46:07.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.570+0000 7f31821fa640 1 -- start start 2023-10-18T10:46:07.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.570+0000 7f31821fa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f317c06ff00 0x7f317c14e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:07.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.570+0000 7f31821fa640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f317c0c8960 0x7f317c154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:07.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.570+0000 7f31821fa640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f317c0ca340 0x7f317c0b2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:07.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.570+0000 7f31821fa640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317c153d00 con 0x7f317c0c8960 2023-10-18T10:46:07.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.570+0000 7f31821fa640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317c0cd3b0 con 0x7f317c0ca340 2023-10-18T10:46:07.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.570+0000 7f31821fa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f317c14ece0 con 0x7f317c06ff00 2023-10-18T10:46:07.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.570+0000 7f3172ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f317c0c8960 0x7f317c154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:07.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.570+0000 7f3172ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f317c0c8960 0x7f317c154150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45022/0 (socket says 172.21.15.89:45022) 2023-10-18T10:46:07.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.570+0000 7f3172ffd640 1 -- 172.21.15.89:0/3912646167 learned_addr learned my addr 172.21.15.89:0/3912646167 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:07.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.574+0000 7f317bfff640 1 --2- 172.21.15.89:0/3912646167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f317c0ca340 0x7f317c0b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:07.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.574+0000 7f317b7fe640 1 --2- 172.21.15.89:0/3912646167 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f317c06ff00 0x7f317c14e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:07.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.574+0000 7f3172ffd640 1 -- 172.21.15.89:0/3912646167 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f317c06ff00 msgr2=0x7f317c14e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:07.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.574+0000 7f3172ffd640 1 --2- 172.21.15.89:0/3912646167 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f317c06ff00 0x7f317c14e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:07.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.574+0000 7f3172ffd640 1 -- 172.21.15.89:0/3912646167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f317c0ca340 msgr2=0x7f317c0b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:07.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.574+0000 7f3172ffd640 1 --2- 172.21.15.89:0/3912646167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f317c0ca340 0x7f317c0b2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:07.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.574+0000 7f3172ffd640 1 -- 172.21.15.89:0/3912646167 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31740094d0 con 0x7f317c0c8960 2023-10-18T10:46:07.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.574+0000 7f317bfff640 1 --2- 172.21.15.89:0/3912646167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f317c0ca340 0x7f317c0b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:07.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.574+0000 7f3172ffd640 1 --2- 172.21.15.89:0/3912646167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f317c0c8960 0x7f317c154150 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f316400e870 tx=0x7f316400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:07.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.574+0000 7f3178ff9640 1 -- 172.21.15.89:0/3912646167 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3164009510 con 0x7f317c0c8960 2023-10-18T10:46:07.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.574+0000 7f3178ff9640 1 -- 172.21.15.89:0/3912646167 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f316400eec0 con 0x7f317c0c8960 2023-10-18T10:46:07.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.574+0000 7f31821fa640 1 -- 172.21.15.89:0/3912646167 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f317c0cdea0 con 0x7f317c0c8960 2023-10-18T10:46:07.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.574+0000 7f3178ff9640 1 -- 172.21.15.89:0/3912646167 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316400cb40 con 0x7f317c0c8960 2023-10-18T10:46:07.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.574+0000 7f31821fa640 1 -- 172.21.15.89:0/3912646167 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f317c0cd8e0 con 0x7f317c0c8960 2023-10-18T10:46:07.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.574+0000 7f31821fa640 1 -- 172.21.15.89:0/3912646167 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f317c153e70 con 0x7f317c0c8960 2023-10-18T10:46:07.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.574+0000 7f3178ff9640 1 -- 172.21.15.89:0/3912646167 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f316400cca0 con 0x7f317c0c8960 2023-10-18T10:46:07.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.574+0000 7f3178ff9640 1 --2- 172.21.15.89:0/3912646167 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f315c0656b0 0x7f315c067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:07.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.574+0000 7f3178ff9640 1 -- 172.21.15.89:0/3912646167 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3164083150 con 0x7f317c0c8960 2023-10-18T10:46:07.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.578+0000 7f317b7fe640 1 --2- 172.21.15.89:0/3912646167 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f315c0656b0 0x7f315c067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:07.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.578+0000 7f317b7fe640 1 --2- 172.21.15.89:0/3912646167 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f315c0656b0 0x7f315c067b70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f316c0012f0 tx=0x7f316c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:07.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.578+0000 7f3178ff9640 1 -- 172.21.15.89:0/3912646167 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f317c153e70 con 0x7f317c0c8960 2023-10-18T10:46:07.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.726+0000 7f31821fa640 1 -- 172.21.15.89:0/3912646167 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f317c153e40 con 0x7f317c0c8960 2023-10-18T10:46:07.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.726+0000 7f3178ff9640 1 -- 172.21.15.89:0/3912646167 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3164054440 con 0x7f317c0c8960 2023-10-18T10:46:07.730 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-18T10:46:07.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.730+0000 7f31821fa640 1 -- 172.21.15.89:0/3912646167 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f315c0656b0 msgr2=0x7f315c067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:07.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.730+0000 7f31821fa640 1 --2- 172.21.15.89:0/3912646167 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f315c0656b0 0x7f315c067b70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f316c0012f0 tx=0x7f316c009040 comp rx=0 tx=0).stop 2023-10-18T10:46:07.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.730+0000 7f31821fa640 1 -- 172.21.15.89:0/3912646167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f317c0c8960 msgr2=0x7f317c154150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:07.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.730+0000 7f31821fa640 1 --2- 172.21.15.89:0/3912646167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f317c0c8960 0x7f317c154150 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f316400e870 tx=0x7f316400ec70 comp rx=0 tx=0).stop 2023-10-18T10:46:07.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.730+0000 7f31821fa640 1 -- 172.21.15.89:0/3912646167 shutdown_connections 2023-10-18T10:46:07.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.730+0000 7f31821fa640 1 --2- 172.21.15.89:0/3912646167 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f315c0656b0 0x7f315c067b70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:07.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.730+0000 7f31821fa640 1 --2- 172.21.15.89:0/3912646167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f317c0ca340 0x7f317c0b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:07.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.730+0000 7f31821fa640 1 --2- 172.21.15.89:0/3912646167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f317c0c8960 0x7f317c154150 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:07.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.730+0000 7f31821fa640 1 --2- 172.21.15.89:0/3912646167 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f317c06ff00 0x7f317c14e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:07.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.730+0000 7f31821fa640 1 -- 172.21.15.89:0/3912646167 >> 172.21.15.89:0/3912646167 conn(0x7f317c069f00 msgr2=0x7f317c0af460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:07.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.730+0000 7f31821fa640 1 -- 172.21.15.89:0/3912646167 shutdown_connections 2023-10-18T10:46:07.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.730+0000 7f31821fa640 1 -- 172.21.15.89:0/3912646167 wait complete. 2023-10-18T10:46:07.750 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:46:07.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.898+0000 7f1fbb7a3640 1 Processor -- start 2023-10-18T10:46:07.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.898+0000 7f1fbb7a3640 1 -- start start 2023-10-18T10:46:07.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.898+0000 7f1fbb7a3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1fb40c8960 0x7f1fb406c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:07.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.898+0000 7f1fbb7a3640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fb41614e0 con 0x7f1fb415e600 2023-10-18T10:46:07.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.898+0000 7f1fbb7a3640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fb40c8df0 con 0x7f1fb40afd90 2023-10-18T10:46:07.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.898+0000 7f1fbb7a3640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fb406c6f0 con 0x7f1fb40c8960 2023-10-18T10:46:07.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.898+0000 7f1fb953f640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1fb415e600 0x7f1fb4161970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55638/0 (socket says 172.21.15.89:55638) 2023-10-18T10:46:07.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.898+0000 7f1fb953f640 1 -- 172.21.15.89:0/3405448737 learned_addr learned my addr 172.21.15.89:0/3405448737 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:07.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.898+0000 7f1fb8d3e640 1 --2- 172.21.15.89:0/3405448737 >> v2:172.21.15.154:3300/0 conn(0x7f1fb40c8960 0x7f1fb406c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:07.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.898+0000 7f1fabfff640 1 -- 172.21.15.89:0/3405448737 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 424320626 0 0) 0x7f1fb41614e0 con 0x7f1fb415e600 2023-10-18T10:46:07.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.898+0000 7f1fabfff640 1 -- 172.21.15.89:0/3405448737 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fa0003530 con 0x7f1fb415e600 2023-10-18T10:46:07.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.898+0000 7f1fabfff640 1 -- 172.21.15.89:0/3405448737 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 542369473 0 0) 0x7f1fa0003530 con 0x7f1fb415e600 2023-10-18T10:46:07.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.898+0000 7f1fabfff640 1 -- 172.21.15.89:0/3405448737 >> v2:172.21.15.154:3300/0 conn(0x7f1fb40c8960 msgr2=0x7f1fb406c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:07.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.898+0000 7f1fabfff640 1 --2- 172.21.15.89:0/3405448737 >> v2:172.21.15.154:3300/0 conn(0x7f1fb40c8960 0x7f1fb406c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:07.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.898+0000 7f1fabfff640 1 -- 172.21.15.89:0/3405448737 >> v1:172.21.15.154:6789/0 conn(0x7f1fb40afd90 legacy=0x7f1fb40b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:07.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.898+0000 7f1fabfff640 1 -- 172.21.15.89:0/3405448737 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb415e9e0 con 0x7f1fb415e600 2023-10-18T10:46:07.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.898+0000 7f1fabfff640 1 -- 172.21.15.89:0/3405448737 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7f1fb0001680 con 0x7f1fb415e600 2023-10-18T10:46:07.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.898+0000 7f1fabfff640 1 -- 172.21.15.89:0/3405448737 >> v1:172.21.15.89:6789/0 conn(0x7f1fb415e600 legacy=0x7f1fb4161970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:07.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.898+0000 7f1fabfff640 1 --2- 172.21.15.89:0/3405448737 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fa0003980 0x7f1fa0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:07.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.898+0000 7f1fabfff640 1 --2- 172.21.15.89:0/3405448737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fa00047d0 0x7f1fa0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:07.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.898+0000 7f1fabfff640 1 --2- 172.21.15.89:0/3405448737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fa0005190 0x7f1fa000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:07.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.898+0000 7f1fabfff640 1 -- 172.21.15.89:0/3405448737 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb406c6f0 con 0x7f1fa0005190 2023-10-18T10:46:07.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.898+0000 7f1fabfff640 1 -- 172.21.15.89:0/3405448737 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa000c030 con 0x7f1fa00047d0 2023-10-18T10:46:07.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.898+0000 7f1fabfff640 1 -- 172.21.15.89:0/3405448737 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa000c1f0 con 0x7f1fa0003980 2023-10-18T10:46:07.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.898+0000 7f1fb9d40640 1 --2- 172.21.15.89:0/3405448737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fa0005190 0x7f1fa000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:07.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.898+0000 7f1fb8d3e640 1 --2- 172.21.15.89:0/3405448737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fa00047d0 0x7f1fa0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:07.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.898+0000 7f1fb953f640 1 --2- 172.21.15.89:0/3405448737 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fa0003980 0x7f1fa0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:07.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.898+0000 7f1fb9d40640 1 -- 172.21.15.89:0/3405448737 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fa0003980 msgr2=0x7f1fa0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:07.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.898+0000 7f1fb9d40640 1 --2- 172.21.15.89:0/3405448737 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fa0003980 0x7f1fa0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:07.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.898+0000 7f1fb9d40640 1 -- 172.21.15.89:0/3405448737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fa00047d0 msgr2=0x7f1fa0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:07.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.898+0000 7f1fb9d40640 1 --2- 172.21.15.89:0/3405448737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fa00047d0 0x7f1fa0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:07.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.898+0000 7f1fb9d40640 1 -- 172.21.15.89:0/3405448737 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa0003530 con 0x7f1fa0005190 2023-10-18T10:46:07.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.902+0000 7f1fb9d40640 1 -- 172.21.15.89:0/3405448737 reap_dead start 2023-10-18T10:46:07.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.902+0000 7f1fb9d40640 1 --2- 172.21.15.89:0/3405448737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fa0005190 0x7f1fa000b990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1fac001e70 tx=0x7f1fac01c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2b88b5f18adb191 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:07.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.902+0000 7f1fabfff640 1 -- 172.21.15.89:0/3405448737 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac003b80 con 0x7f1fa0005190 2023-10-18T10:46:07.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.902+0000 7f1fabfff640 1 -- 172.21.15.89:0/3405448737 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fac003d20 con 0x7f1fa0005190 2023-10-18T10:46:07.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.902+0000 7f1fabfff640 1 -- 172.21.15.89:0/3405448737 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac005260 con 0x7f1fa0005190 2023-10-18T10:46:07.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.902+0000 7f1fbb7a3640 1 -- 172.21.15.89:0/3405448737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fa0005190 msgr2=0x7f1fa000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:07.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.902+0000 7f1fbb7a3640 1 --2- 172.21.15.89:0/3405448737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fa0005190 0x7f1fa000b990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1fac001e70 tx=0x7f1fac01c760 comp rx=0 tx=0).stop 2023-10-18T10:46:07.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.902+0000 7f1fbb7a3640 1 -- 172.21.15.89:0/3405448737 shutdown_connections 2023-10-18T10:46:07.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.902+0000 7f1fbb7a3640 1 --2- 172.21.15.89:0/3405448737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fa0005190 0x7f1fa000b990 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:07.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.902+0000 7f1fbb7a3640 1 -- 172.21.15.89:0/3405448737 >> 172.21.15.89:0/3405448737 conn(0x7f1fb4069f00 msgr2=0x7f1fb40cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:07.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.902+0000 7f1fbb7a3640 1 -- 172.21.15.89:0/3405448737 shutdown_connections 2023-10-18T10:46:07.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.902+0000 7f1fbb7a3640 1 -- 172.21.15.89:0/3405448737 wait complete. 2023-10-18T10:46:07.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.906+0000 7f1fbb7a3640 1 Processor -- start 2023-10-18T10:46:07.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.906+0000 7f1fbb7a3640 1 -- start start 2023-10-18T10:46:07.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.906+0000 7f1fbb7a3640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fa0005190 0x7f1fb40c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:07.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.906+0000 7f1fbb7a3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fb41f6330 0x7f1fb41f67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:07.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.906+0000 7f1fbb7a3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fb40b12d0 0x7f1fb40b1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:07.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.906+0000 7f1fbb7a3640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb41f73e0 con 0x7f1fa0005190 2023-10-18T10:46:07.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.906+0000 7f1fbb7a3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb40c8220 con 0x7f1fb40b12d0 2023-10-18T10:46:07.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.906+0000 7f1fbb7a3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb40b1c90 con 0x7f1fb41f6330 2023-10-18T10:46:07.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.906+0000 7f1fb953f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fa0005190 0x7f1fb40c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:07.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.906+0000 7f1fb953f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fa0005190 0x7f1fb40c7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45042/0 (socket says 172.21.15.89:45042) 2023-10-18T10:46:07.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.906+0000 7f1fb953f640 1 -- 172.21.15.89:0/2350874827 learned_addr learned my addr 172.21.15.89:0/2350874827 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:07.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.906+0000 7f1fb8d3e640 1 --2- 172.21.15.89:0/2350874827 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fb41f6330 0x7f1fb41f67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:07.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.906+0000 7f1fb9d40640 1 --2- 172.21.15.89:0/2350874827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fb40b12d0 0x7f1fb40b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:07.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.906+0000 7f1fb953f640 1 -- 172.21.15.89:0/2350874827 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fb41f6330 msgr2=0x7f1fb41f67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:07.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.906+0000 7f1fb953f640 1 --2- 172.21.15.89:0/2350874827 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fb41f6330 0x7f1fb41f67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:07.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.906+0000 7f1fb953f640 1 -- 172.21.15.89:0/2350874827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fb40b12d0 msgr2=0x7f1fb40b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:07.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.906+0000 7f1fb953f640 1 --2- 172.21.15.89:0/2350874827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fb40b12d0 0x7f1fb40b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:07.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.906+0000 7f1fb953f640 1 -- 172.21.15.89:0/2350874827 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb41f6e20 con 0x7f1fa0005190 2023-10-18T10:46:07.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.906+0000 7f1fb9d40640 1 --2- 172.21.15.89:0/2350874827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fb40b12d0 0x7f1fb40b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:07.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.906+0000 7f1fb953f640 1 --2- 172.21.15.89:0/2350874827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fa0005190 0x7f1fb40c7ce0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0007290 tx=0x7f1fb0007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:07.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.906+0000 7f1fa9ffb640 1 -- 172.21.15.89:0/2350874827 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb000f9c0 con 0x7f1fa0005190 2023-10-18T10:46:07.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.906+0000 7f1fa9ffb640 1 -- 172.21.15.89:0/2350874827 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fb000fb40 con 0x7f1fa0005190 2023-10-18T10:46:07.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.906+0000 7f1fa9ffb640 1 -- 172.21.15.89:0/2350874827 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb00133f0 con 0x7f1fa0005190 2023-10-18T10:46:07.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.906+0000 7f1fbb7a3640 1 -- 172.21.15.89:0/2350874827 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fb41f7100 con 0x7f1fa0005190 2023-10-18T10:46:07.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.910+0000 7f1fbb7a3640 1 -- 172.21.15.89:0/2350874827 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb41f79c0 con 0x7f1fa0005190 2023-10-18T10:46:07.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.910+0000 7f1fbb7a3640 1 -- 172.21.15.89:0/2350874827 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fb41f7540 con 0x7f1fa0005190 2023-10-18T10:46:07.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.910+0000 7f1fa9ffb640 1 -- 172.21.15.89:0/2350874827 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f1fb0013550 con 0x7f1fa0005190 2023-10-18T10:46:07.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.910+0000 7f1fa9ffb640 1 --2- 172.21.15.89:0/2350874827 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1f980656b0 0x7f1f98067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:07.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.910+0000 7f1fa9ffb640 1 -- 172.21.15.89:0/2350874827 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1fb0083360 con 0x7f1fa0005190 2023-10-18T10:46:07.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.910+0000 7f1fb8d3e640 1 --2- 172.21.15.89:0/2350874827 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1f980656b0 0x7f1f98067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:07.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.910+0000 7f1fb8d3e640 1 --2- 172.21.15.89:0/2350874827 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1f980656b0 0x7f1f98067b70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1fb40cdb50 tx=0x7f1fa4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:07.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:07.914+0000 7f1fa9ffb640 1 -- 172.21.15.89:0/2350874827 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f1fb41f7540 con 0x7f1fa0005190 2023-10-18T10:46:08.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.066+0000 7f1fbb7a3640 1 -- 172.21.15.89:0/2350874827 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1fb41f7540 con 0x7f1fa0005190 2023-10-18T10:46:08.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.066+0000 7f1fa9ffb640 1 -- 172.21.15.89:0/2350874827 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1fb41f7540 con 0x7f1fa0005190 2023-10-18T10:46:08.069 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-18T10:46:08.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.070+0000 7f1fbb7a3640 1 -- 172.21.15.89:0/2350874827 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1f980656b0 msgr2=0x7f1f98067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:08.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.070+0000 7f1fbb7a3640 1 --2- 172.21.15.89:0/2350874827 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1f980656b0 0x7f1f98067b70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1fb40cdb50 tx=0x7f1fa4008040 comp rx=0 tx=0).stop 2023-10-18T10:46:08.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.070+0000 7f1fbb7a3640 1 -- 172.21.15.89:0/2350874827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fa0005190 msgr2=0x7f1fb40c7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:08.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.070+0000 7f1fbb7a3640 1 --2- 172.21.15.89:0/2350874827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fa0005190 0x7f1fb40c7ce0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0007290 tx=0x7f1fb0007750 comp rx=0 tx=0).stop 2023-10-18T10:46:08.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.070+0000 7f1fbb7a3640 1 -- 172.21.15.89:0/2350874827 shutdown_connections 2023-10-18T10:46:08.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.070+0000 7f1fbb7a3640 1 --2- 172.21.15.89:0/2350874827 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1f980656b0 0x7f1f98067b70 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:08.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.070+0000 7f1fbb7a3640 1 --2- 172.21.15.89:0/2350874827 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fb41f6330 0x7f1fb41f67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:08.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.070+0000 7f1fbb7a3640 1 --2- 172.21.15.89:0/2350874827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fb40b12d0 0x7f1fb40b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:08.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.070+0000 7f1fbb7a3640 1 --2- 172.21.15.89:0/2350874827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fa0005190 0x7f1fb40c7ce0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:08.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.070+0000 7f1fbb7a3640 1 -- 172.21.15.89:0/2350874827 >> 172.21.15.89:0/2350874827 conn(0x7f1fb4069f00 msgr2=0x7f1fb40cacf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:08.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.070+0000 7f1fbb7a3640 1 -- 172.21.15.89:0/2350874827 shutdown_connections 2023-10-18T10:46:08.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.070+0000 7f1fbb7a3640 1 -- 172.21.15.89:0/2350874827 wait complete. 2023-10-18T10:46:08.089 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T10:46:08.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.226+0000 7f7a83aca640 1 Processor -- start 2023-10-18T10:46:08.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.226+0000 7f7a83aca640 1 -- start start 2023-10-18T10:46:08.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.230+0000 7f7a83aca640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7a7c15ff30 0x7f7a7c0d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:08.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.230+0000 7f7a83aca640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7a7c06a010 0x7f7a7c0b0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:08.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.230+0000 7f7a83aca640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a7c0b1430 con 0x7f7a7c0b1950 2023-10-18T10:46:08.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.230+0000 7f7a83aca640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a7c0d3690 con 0x7f7a7c06a010 2023-10-18T10:46:08.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.230+0000 7f7a83aca640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a7c0d3080 con 0x7f7a7c15ff30 2023-10-18T10:46:08.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.230+0000 7f7a82067640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7a7c06a010 0x7f7a7c0b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:08.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.230+0000 7f7a82067640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7a7c06a010 0x7f7a7c0b0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45046/0 (socket says 172.21.15.89:45046) 2023-10-18T10:46:08.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.230+0000 7f7a82067640 1 -- 172.21.15.89:0/832693166 learned_addr learned my addr 172.21.15.89:0/832693166 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:08.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.230+0000 7f7a81866640 1 --2- 172.21.15.89:0/832693166 >> v2:172.21.15.154:3300/0 conn(0x7f7a7c15ff30 0x7f7a7c0d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:08.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.230+0000 7f7a82067640 1 -- 172.21.15.89:0/832693166 >> v2:172.21.15.154:3300/0 conn(0x7f7a7c15ff30 msgr2=0x7f7a7c0d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:08.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.230+0000 7f7a82067640 1 --2- 172.21.15.89:0/832693166 >> v2:172.21.15.154:3300/0 conn(0x7f7a7c15ff30 0x7f7a7c0d0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:08.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.230+0000 7f7a82067640 1 -- 172.21.15.89:0/832693166 >> v1:172.21.15.89:6789/0 conn(0x7f7a7c0b1950 legacy=0x7f7a7c069900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:08.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.230+0000 7f7a82067640 1 -- 172.21.15.89:0/832693166 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a7c0d3320 con 0x7f7a7c06a010 2023-10-18T10:46:08.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.234+0000 7f7a82067640 1 --2- 172.21.15.89:0/832693166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a7c06a010 0x7f7a7c0b0260 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c08a600 tx=0x7f7a7401c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27d0b7ef50ac885 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:08.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.234+0000 7f7a80864640 1 -- 172.21.15.89:0/832693166 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a740a5790 con 0x7f7a7c06a010 2023-10-18T10:46:08.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.234+0000 7f7a80864640 1 -- 172.21.15.89:0/832693166 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a740a58f0 con 0x7f7a7c06a010 2023-10-18T10:46:08.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.234+0000 7f7a80864640 1 -- 172.21.15.89:0/832693166 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a740a5c80 con 0x7f7a7c06a010 2023-10-18T10:46:08.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.234+0000 7f7a83aca640 1 -- 172.21.15.89:0/832693166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a7c06a010 msgr2=0x7f7a7c0b0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:08.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.234+0000 7f7a83aca640 1 --2- 172.21.15.89:0/832693166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a7c06a010 0x7f7a7c0b0260 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c08a600 tx=0x7f7a7401c8b0 comp rx=0 tx=0).stop 2023-10-18T10:46:08.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.234+0000 7f7a83aca640 1 -- 172.21.15.89:0/832693166 shutdown_connections 2023-10-18T10:46:08.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.234+0000 7f7a83aca640 1 --2- 172.21.15.89:0/832693166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a7c06a010 0x7f7a7c0b0260 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:08.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.234+0000 7f7a83aca640 1 --2- 172.21.15.89:0/832693166 >> v2:172.21.15.154:3300/0 conn(0x7f7a7c15ff30 0x7f7a7c0d0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:08.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.234+0000 7f7a83aca640 1 -- 172.21.15.89:0/832693166 >> 172.21.15.89:0/832693166 conn(0x7f7a7c09b9d0 msgr2=0x7f7a7c0d26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:08.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.234+0000 7f7a83aca640 1 -- 172.21.15.89:0/832693166 shutdown_connections 2023-10-18T10:46:08.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.234+0000 7f7a83aca640 1 -- 172.21.15.89:0/832693166 wait complete. 2023-10-18T10:46:08.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.234+0000 7f7a83aca640 1 Processor -- start 2023-10-18T10:46:08.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.234+0000 7f7a83aca640 1 -- start start 2023-10-18T10:46:08.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.238+0000 7f7a83aca640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a7c06a010 0x7f7a7c155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:08.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.238+0000 7f7a83aca640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a7c0b1950 0x7f7a7c0cc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:08.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.238+0000 7f7a83aca640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a7c15ff30 0x7f7a7c0b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:08.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.238+0000 7f7a83aca640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7c155e80 con 0x7f7a7c06a010 2023-10-18T10:46:08.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.238+0000 7f7a83aca640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7c0b8040 con 0x7f7a7c0b1950 2023-10-18T10:46:08.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.238+0000 7f7a83aca640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7c0cee80 con 0x7f7a7c15ff30 2023-10-18T10:46:08.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.238+0000 7f7a81866640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a7c06a010 0x7f7a7c155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:08.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.238+0000 7f7a81866640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a7c06a010 0x7f7a7c155940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45058/0 (socket says 172.21.15.89:45058) 2023-10-18T10:46:08.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.238+0000 7f7a81866640 1 -- 172.21.15.89:0/2543875608 learned_addr learned my addr 172.21.15.89:0/2543875608 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:08.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.238+0000 7f7a81065640 1 --2- 172.21.15.89:0/2543875608 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a7c0b1950 0x7f7a7c0cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:08.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.238+0000 7f7a82067640 1 --2- 172.21.15.89:0/2543875608 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a7c15ff30 0x7f7a7c0b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:08.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.238+0000 7f7a81866640 1 -- 172.21.15.89:0/2543875608 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a7c15ff30 msgr2=0x7f7a7c0b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:08.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.238+0000 7f7a81866640 1 --2- 172.21.15.89:0/2543875608 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a7c15ff30 0x7f7a7c0b25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:08.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.238+0000 7f7a81866640 1 -- 172.21.15.89:0/2543875608 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a7c0b1950 msgr2=0x7f7a7c0cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:08.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.238+0000 7f7a81866640 1 --2- 172.21.15.89:0/2543875608 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a7c0b1950 0x7f7a7c0cc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:08.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.238+0000 7f7a81866640 1 -- 172.21.15.89:0/2543875608 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a74009b30 con 0x7f7a7c06a010 2023-10-18T10:46:08.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.238+0000 7f7a82067640 1 --2- 172.21.15.89:0/2543875608 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a7c15ff30 0x7f7a7c0b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:08.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.238+0000 7f7a81866640 1 --2- 172.21.15.89:0/2543875608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a7c06a010 0x7f7a7c155940 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7a7800d7d0 tx=0x7f7a7800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:08.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.238+0000 7f7a727fc640 1 -- 172.21.15.89:0/2543875608 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7800b6c0 con 0x7f7a7c06a010 2023-10-18T10:46:08.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.238+0000 7f7a727fc640 1 -- 172.21.15.89:0/2543875608 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7800b820 con 0x7f7a7c06a010 2023-10-18T10:46:08.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.238+0000 7f7a83aca640 1 -- 172.21.15.89:0/2543875608 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a7c0ce270 con 0x7f7a7c06a010 2023-10-18T10:46:08.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.238+0000 7f7a727fc640 1 -- 172.21.15.89:0/2543875608 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7800bbd0 con 0x7f7a7c06a010 2023-10-18T10:46:08.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.238+0000 7f7a83aca640 1 -- 172.21.15.89:0/2543875608 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a7c0ce830 con 0x7f7a7c06a010 2023-10-18T10:46:08.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.238+0000 7f7a83aca640 1 -- 172.21.15.89:0/2543875608 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a7c04dbe0 con 0x7f7a7c06a010 2023-10-18T10:46:08.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.238+0000 7f7a727fc640 1 -- 172.21.15.89:0/2543875608 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f7a7800bd30 con 0x7f7a7c06a010 2023-10-18T10:46:08.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.238+0000 7f7a727fc640 1 --2- 172.21.15.89:0/2543875608 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7a60065680 0x7f7a60067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:08.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.242+0000 7f7a727fc640 1 -- 172.21.15.89:0/2543875608 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7a78083570 con 0x7f7a7c06a010 2023-10-18T10:46:08.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.242+0000 7f7a81065640 1 --2- 172.21.15.89:0/2543875608 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7a60065680 0x7f7a60067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:08.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.242+0000 7f7a81065640 1 --2- 172.21.15.89:0/2543875608 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7a60065680 0x7f7a60067b40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0b3140 tx=0x7f7a6c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:08.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.246+0000 7f7a727fc640 1 -- 172.21.15.89:0/2543875608 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f7a78054860 con 0x7f7a7c06a010 2023-10-18T10:46:08.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.390+0000 7f7a83aca640 1 -- 172.21.15.89:0/2543875608 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7a7c0cfe50 con 0x7f7a7c06a010 2023-10-18T10:46:08.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.390+0000 7f7a727fc640 1 -- 172.21.15.89:0/2543875608 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7a780543f0 con 0x7f7a7c06a010 2023-10-18T10:46:08.393 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-18T10:46:08.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.394+0000 7f7a83aca640 1 -- 172.21.15.89:0/2543875608 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7a60065680 msgr2=0x7f7a60067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:08.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.394+0000 7f7a83aca640 1 --2- 172.21.15.89:0/2543875608 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7a60065680 0x7f7a60067b40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0b3140 tx=0x7f7a6c009040 comp rx=0 tx=0).stop 2023-10-18T10:46:08.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.394+0000 7f7a83aca640 1 -- 172.21.15.89:0/2543875608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a7c06a010 msgr2=0x7f7a7c155940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:08.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.394+0000 7f7a83aca640 1 --2- 172.21.15.89:0/2543875608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a7c06a010 0x7f7a7c155940 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7a7800d7d0 tx=0x7f7a7800dc90 comp rx=0 tx=0).stop 2023-10-18T10:46:08.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.394+0000 7f7a83aca640 1 -- 172.21.15.89:0/2543875608 shutdown_connections 2023-10-18T10:46:08.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.394+0000 7f7a83aca640 1 --2- 172.21.15.89:0/2543875608 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7a60065680 0x7f7a60067b40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:08.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.394+0000 7f7a83aca640 1 --2- 172.21.15.89:0/2543875608 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a7c15ff30 0x7f7a7c0b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:08.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.394+0000 7f7a83aca640 1 --2- 172.21.15.89:0/2543875608 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a7c0b1950 0x7f7a7c0cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:08.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.394+0000 7f7a83aca640 1 --2- 172.21.15.89:0/2543875608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a7c06a010 0x7f7a7c155940 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:08.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.394+0000 7f7a83aca640 1 -- 172.21.15.89:0/2543875608 >> 172.21.15.89:0/2543875608 conn(0x7f7a7c09b9d0 msgr2=0x7f7a7c069900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:08.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.394+0000 7f7a83aca640 1 -- 172.21.15.89:0/2543875608 shutdown_connections 2023-10-18T10:46:08.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.394+0000 7f7a83aca640 1 -- 172.21.15.89:0/2543875608 wait complete. 2023-10-18T10:46:08.414 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T10:46:08.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.550+0000 7f1f90300640 1 Processor -- start 2023-10-18T10:46:08.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.550+0000 7f1f90300640 1 -- start start 2023-10-18T10:46:08.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.554+0000 7f1f90300640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1f8806a010 0x7f1f880d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:08.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.554+0000 7f1f90300640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f880b1430 con 0x7f1f880b1950 2023-10-18T10:46:08.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.554+0000 7f1f90300640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f880d3080 con 0x7f1f8815ff30 2023-10-18T10:46:08.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.554+0000 7f1f90300640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f880d3690 con 0x7f1f8806a010 2023-10-18T10:46:08.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.554+0000 7f1f8e89d640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1f880b1950 0x7f1f880b0260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55662/0 (socket says 172.21.15.89:55662) 2023-10-18T10:46:08.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.554+0000 7f1f8e89d640 1 -- 172.21.15.89:0/1331050135 learned_addr learned my addr 172.21.15.89:0/1331050135 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:08.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.554+0000 7f1f8d89b640 1 --2- 172.21.15.89:0/1331050135 >> v2:172.21.15.154:3300/0 conn(0x7f1f8806a010 0x7f1f880d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:08.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.554+0000 7f1f8d09a640 1 -- 172.21.15.89:0/1331050135 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1411069008 0 0) 0x7f1f880b1430 con 0x7f1f880b1950 2023-10-18T10:46:08.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.554+0000 7f1f8d09a640 1 -- 172.21.15.89:0/1331050135 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f74003530 con 0x7f1f880b1950 2023-10-18T10:46:08.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.554+0000 7f1f8d09a640 1 -- 172.21.15.89:0/1331050135 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1844901997 0 0) 0x7f1f74003530 con 0x7f1f880b1950 2023-10-18T10:46:08.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.554+0000 7f1f8d09a640 1 -- 172.21.15.89:0/1331050135 >> v2:172.21.15.154:3300/0 conn(0x7f1f8806a010 msgr2=0x7f1f880d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:08.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.554+0000 7f1f8d09a640 1 --2- 172.21.15.89:0/1331050135 >> v2:172.21.15.154:3300/0 conn(0x7f1f8806a010 0x7f1f880d0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:08.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.554+0000 7f1f8d09a640 1 -- 172.21.15.89:0/1331050135 >> v1:172.21.15.154:6789/0 conn(0x7f1f8815ff30 legacy=0x7f1f88069900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:08.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.554+0000 7f1f8d09a640 1 -- 172.21.15.89:0/1331050135 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f880d3320 con 0x7f1f880b1950 2023-10-18T10:46:08.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.554+0000 7f1f8d09a640 1 -- 172.21.15.89:0/1331050135 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7f1f80001680 con 0x7f1f880b1950 2023-10-18T10:46:08.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.554+0000 7f1f8d09a640 1 -- 172.21.15.89:0/1331050135 >> v1:172.21.15.89:6789/0 conn(0x7f1f880b1950 legacy=0x7f1f880b0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:08.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.554+0000 7f1f8d09a640 1 --2- 172.21.15.89:0/1331050135 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f74003980 0x7f1f74003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:08.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.554+0000 7f1f8d09a640 1 --2- 172.21.15.89:0/1331050135 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f740047d0 0x7f1f74004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:08.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.554+0000 7f1f8d09a640 1 --2- 172.21.15.89:0/1331050135 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f740052b0 0x7f1f7400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:08.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.554+0000 7f1f8d09a640 1 -- 172.21.15.89:0/1331050135 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f880d3690 con 0x7f1f740052b0 2023-10-18T10:46:08.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.554+0000 7f1f8d09a640 1 -- 172.21.15.89:0/1331050135 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7400c180 con 0x7f1f74003980 2023-10-18T10:46:08.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.554+0000 7f1f8d09a640 1 -- 172.21.15.89:0/1331050135 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7400c340 con 0x7f1f740047d0 2023-10-18T10:46:08.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.554+0000 7f1f8e89d640 1 --2- 172.21.15.89:0/1331050135 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f740052b0 0x7f1f7400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:08.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.554+0000 7f1f8e09c640 1 --2- 172.21.15.89:0/1331050135 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f74003980 0x7f1f74003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:08.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.554+0000 7f1f8d89b640 1 --2- 172.21.15.89:0/1331050135 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f740047d0 0x7f1f74004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:08.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.554+0000 7f1f8e89d640 1 -- 172.21.15.89:0/1331050135 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f740047d0 msgr2=0x7f1f74004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:08.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.554+0000 7f1f8e89d640 1 --2- 172.21.15.89:0/1331050135 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f740047d0 0x7f1f74004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:08.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.554+0000 7f1f8e89d640 1 -- 172.21.15.89:0/1331050135 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f74003980 msgr2=0x7f1f74003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:08.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.554+0000 7f1f8e89d640 1 --2- 172.21.15.89:0/1331050135 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f74003980 0x7f1f74003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:08.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.554+0000 7f1f8e89d640 1 -- 172.21.15.89:0/1331050135 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f74003530 con 0x7f1f740052b0 2023-10-18T10:46:08.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.558+0000 7f1f8e89d640 1 -- 172.21.15.89:0/1331050135 reap_dead start 2023-10-18T10:46:08.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.558+0000 7f1f8e89d640 1 --2- 172.21.15.89:0/1331050135 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f740052b0 0x7f1f7400b990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1f8000c190 tx=0x7f1f8001cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd849d7da4efd003 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:08.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.558+0000 7f1f8d09a640 1 -- 172.21.15.89:0/1331050135 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f800a5b10 con 0x7f1f740052b0 2023-10-18T10:46:08.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.558+0000 7f1f8d09a640 1 -- 172.21.15.89:0/1331050135 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f800a5c70 con 0x7f1f740052b0 2023-10-18T10:46:08.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.558+0000 7f1f8d09a640 1 -- 172.21.15.89:0/1331050135 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f800058e0 con 0x7f1f740052b0 2023-10-18T10:46:08.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.558+0000 7f1f90300640 1 -- 172.21.15.89:0/1331050135 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f740052b0 msgr2=0x7f1f7400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:08.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.558+0000 7f1f90300640 1 --2- 172.21.15.89:0/1331050135 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f740052b0 0x7f1f7400b990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1f8000c190 tx=0x7f1f8001cbb0 comp rx=0 tx=0).stop 2023-10-18T10:46:08.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.558+0000 7f1f90300640 1 -- 172.21.15.89:0/1331050135 shutdown_connections 2023-10-18T10:46:08.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.558+0000 7f1f90300640 1 --2- 172.21.15.89:0/1331050135 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f740052b0 0x7f1f7400b990 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:08.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.558+0000 7f1f90300640 1 -- 172.21.15.89:0/1331050135 >> 172.21.15.89:0/1331050135 conn(0x7f1f8809b9d0 msgr2=0x7f1f880d26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:08.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.558+0000 7f1f90300640 1 -- 172.21.15.89:0/1331050135 shutdown_connections 2023-10-18T10:46:08.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.558+0000 7f1f90300640 1 -- 172.21.15.89:0/1331050135 wait complete. 2023-10-18T10:46:08.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.562+0000 7f1f90300640 1 Processor -- start 2023-10-18T10:46:08.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.562+0000 7f1f90300640 1 -- start start 2023-10-18T10:46:08.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.562+0000 7f1f90300640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f740052b0 0x7f1f880b4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:08.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.562+0000 7f1f90300640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f880b4b60 0x7f1f8814b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:08.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.562+0000 7f1f90300640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f8814b840 0x7f1f880b71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:08.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.562+0000 7f1f90300640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8814bcf0 con 0x7f1f8814b840 2023-10-18T10:46:08.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.562+0000 7f1f90300640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8814bf40 con 0x7f1f740052b0 2023-10-18T10:46:08.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.562+0000 7f1f90300640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8814c220 con 0x7f1f880b4b60 2023-10-18T10:46:08.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.562+0000 7f1f8e89d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f8814b840 0x7f1f880b71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:08.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.562+0000 7f1f8e89d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f8814b840 0x7f1f880b71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45070/0 (socket says 172.21.15.89:45070) 2023-10-18T10:46:08.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.562+0000 7f1f8e89d640 1 -- 172.21.15.89:0/3823245976 learned_addr learned my addr 172.21.15.89:0/3823245976 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:08.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.562+0000 7f1f8d89b640 1 --2- 172.21.15.89:0/3823245976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f880b4b60 0x7f1f8814b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:08.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.562+0000 7f1f8e09c640 1 --2- 172.21.15.89:0/3823245976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f740052b0 0x7f1f880b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:08.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.562+0000 7f1f8e89d640 1 -- 172.21.15.89:0/3823245976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f880b4b60 msgr2=0x7f1f8814b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:08.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.562+0000 7f1f8e89d640 1 --2- 172.21.15.89:0/3823245976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f880b4b60 0x7f1f8814b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:08.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.562+0000 7f1f8e89d640 1 -- 172.21.15.89:0/3823245976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f740052b0 msgr2=0x7f1f880b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:08.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.562+0000 7f1f8e89d640 1 --2- 172.21.15.89:0/3823245976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f740052b0 0x7f1f880b4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:08.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.562+0000 7f1f8e89d640 1 -- 172.21.15.89:0/3823245976 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f8814c500 con 0x7f1f8814b840 2023-10-18T10:46:08.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.562+0000 7f1f8e89d640 1 --2- 172.21.15.89:0/3823245976 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f8814b840 0x7f1f880b71e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1f8001b2b0 tx=0x7f1f74005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:08.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.562+0000 7f1f7effd640 1 -- 172.21.15.89:0/3823245976 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f80000ee0 con 0x7f1f8814b840 2023-10-18T10:46:08.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.562+0000 7f1f7effd640 1 -- 172.21.15.89:0/3823245976 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f800a5910 con 0x7f1f8814b840 2023-10-18T10:46:08.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.562+0000 7f1f90300640 1 -- 172.21.15.89:0/3823245976 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f8814c7e0 con 0x7f1f8814b840 2023-10-18T10:46:08.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.562+0000 7f1f7effd640 1 -- 172.21.15.89:0/3823245976 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f800a5a80 con 0x7f1f8814b840 2023-10-18T10:46:08.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.562+0000 7f1f90300640 1 -- 172.21.15.89:0/3823245976 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f880b7720 con 0x7f1f8814b840 2023-10-18T10:46:08.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.562+0000 7f1f90300640 1 -- 172.21.15.89:0/3823245976 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f8814c360 con 0x7f1f8814b840 2023-10-18T10:46:08.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.566+0000 7f1f7effd640 1 -- 172.21.15.89:0/3823245976 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f1f800b4020 con 0x7f1f8814b840 2023-10-18T10:46:08.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.566+0000 7f1f7effd640 1 --2- 172.21.15.89:0/3823245976 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1f6c065680 0x7f1f6c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:08.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.566+0000 7f1f7effd640 1 -- 172.21.15.89:0/3823245976 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1f8011c890 con 0x7f1f8814b840 2023-10-18T10:46:08.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.566+0000 7f1f8e09c640 1 --2- 172.21.15.89:0/3823245976 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1f6c065680 0x7f1f6c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:08.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.566+0000 7f1f8e09c640 1 --2- 172.21.15.89:0/3823245976 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1f6c065680 0x7f1f6c067b40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1f84001280 tx=0x7f1f84009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:08.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.570+0000 7f1f7effd640 1 -- 172.21.15.89:0/3823245976 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f1f8814c360 con 0x7f1f8814b840 2023-10-18T10:46:08.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.730+0000 7f1f90300640 1 -- 172.21.15.89:0/3823245976 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1f8814c360 con 0x7f1f8814b840 2023-10-18T10:46:08.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.730+0000 7f1f7effd640 1 -- 172.21.15.89:0/3823245976 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1f8814c360 con 0x7f1f8814b840 2023-10-18T10:46:08.731 INFO:teuthology.orchestra.run.smithi089.stderr:listed 0 entries 2023-10-18T10:46:08.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.730+0000 7f1f90300640 1 -- 172.21.15.89:0/3823245976 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1f6c065680 msgr2=0x7f1f6c067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:08.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.730+0000 7f1f90300640 1 --2- 172.21.15.89:0/3823245976 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1f6c065680 0x7f1f6c067b40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1f84001280 tx=0x7f1f84009040 comp rx=0 tx=0).stop 2023-10-18T10:46:08.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.730+0000 7f1f90300640 1 -- 172.21.15.89:0/3823245976 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f8814b840 msgr2=0x7f1f880b71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:08.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.730+0000 7f1f90300640 1 --2- 172.21.15.89:0/3823245976 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f8814b840 0x7f1f880b71e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1f8001b2b0 tx=0x7f1f74005800 comp rx=0 tx=0).stop 2023-10-18T10:46:08.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.730+0000 7f1f90300640 1 -- 172.21.15.89:0/3823245976 shutdown_connections 2023-10-18T10:46:08.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.730+0000 7f1f90300640 1 --2- 172.21.15.89:0/3823245976 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1f6c065680 0x7f1f6c067b40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:08.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.730+0000 7f1f90300640 1 --2- 172.21.15.89:0/3823245976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f880b4b60 0x7f1f8814b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:08.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.730+0000 7f1f90300640 1 --2- 172.21.15.89:0/3823245976 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f8814b840 0x7f1f880b71e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:08.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.730+0000 7f1f90300640 1 --2- 172.21.15.89:0/3823245976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f740052b0 0x7f1f880b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:08.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.730+0000 7f1f90300640 1 -- 172.21.15.89:0/3823245976 >> 172.21.15.89:0/3823245976 conn(0x7f1f8809b9d0 msgr2=0x7f1f880d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:08.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.730+0000 7f1f90300640 1 -- 172.21.15.89:0/3823245976 shutdown_connections 2023-10-18T10:46:08.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.730+0000 7f1f90300640 1 -- 172.21.15.89:0/3823245976 wait complete. 2023-10-18T10:46:08.751 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:46:08.752 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:46:08.761 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:46:08.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.938+0000 7f36e9967640 1 Processor -- start 2023-10-18T10:46:08.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.942+0000 7f36e9967640 1 -- start start 2023-10-18T10:46:08.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.942+0000 7f36e9967640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36e415c940 0x7f36e40d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:08.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.942+0000 7f36e9967640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e415da50 con 0x7f36e40d2f40 2023-10-18T10:46:08.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.942+0000 7f36e9967640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e415ce40 con 0x7f36e40d0e70 2023-10-18T10:46:08.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.942+0000 7f36e9967640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e415d120 con 0x7f36e415c940 2023-10-18T10:46:08.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.942+0000 7f36e27fc640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f36e40d2f40 0x7f36e40d0760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55678/0 (socket says 172.21.15.89:55678) 2023-10-18T10:46:08.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.942+0000 7f36e27fc640 1 -- 172.21.15.89:0/2640440300 learned_addr learned my addr 172.21.15.89:0/2640440300 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:08.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.942+0000 7f36e2ffd640 1 --2- 172.21.15.89:0/2640440300 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36e415c940 0x7f36e40d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:08.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.942+0000 7f36e1ffb640 1 -- 172.21.15.89:0/2640440300 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 267005737 0 0) 0x7f36e415da50 con 0x7f36e40d2f40 2023-10-18T10:46:08.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.942+0000 7f36e1ffb640 1 -- 172.21.15.89:0/2640440300 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36d0003530 con 0x7f36e40d2f40 2023-10-18T10:46:08.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.942+0000 7f36e1ffb640 1 -- 172.21.15.89:0/2640440300 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1404801062 0 0) 0x7f36d0003530 con 0x7f36e40d2f40 2023-10-18T10:46:08.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.942+0000 7f36e1ffb640 1 -- 172.21.15.89:0/2640440300 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36e415c940 msgr2=0x7f36e40d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:08.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.942+0000 7f36e1ffb640 1 --2- 172.21.15.89:0/2640440300 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36e415c940 0x7f36e40d2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:08.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.942+0000 7f36e1ffb640 1 -- 172.21.15.89:0/2640440300 >> v1:172.21.15.154:6789/0 conn(0x7f36e40d0e70 legacy=0x7f36e40cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:08.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.942+0000 7f36e1ffb640 1 -- 172.21.15.89:0/2640440300 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e415d400 con 0x7f36e40d2f40 2023-10-18T10:46:08.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.942+0000 7f36e1ffb640 1 -- 172.21.15.89:0/2640440300 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7f36cc001680 con 0x7f36e40d2f40 2023-10-18T10:46:08.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.942+0000 7f36e1ffb640 1 -- 172.21.15.89:0/2640440300 >> v1:172.21.15.89:6789/0 conn(0x7f36e40d2f40 legacy=0x7f36e40d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:08.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.942+0000 7f36e1ffb640 1 --2- 172.21.15.89:0/2640440300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36d00039b0 0x7f36d0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:08.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.942+0000 7f36e1ffb640 1 --2- 172.21.15.89:0/2640440300 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36d0004360 0x7f36d00047f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:08.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.942+0000 7f36e1ffb640 1 --2- 172.21.15.89:0/2640440300 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36d0004f50 0x7f36d000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:08.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.942+0000 7f36e1ffb640 1 -- 172.21.15.89:0/2640440300 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e415d120 con 0x7f36d00039b0 2023-10-18T10:46:08.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.942+0000 7f36e1ffb640 1 -- 172.21.15.89:0/2640440300 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d0005670 con 0x7f36d0004f50 2023-10-18T10:46:08.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.942+0000 7f36e1ffb640 1 -- 172.21.15.89:0/2640440300 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36d000bf20 con 0x7f36d0004360 2023-10-18T10:46:08.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.942+0000 7f36e2ffd640 1 --2- 172.21.15.89:0/2640440300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36d00039b0 0x7f36d0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:08.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.942+0000 7f36e27fc640 1 --2- 172.21.15.89:0/2640440300 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36d0004360 0x7f36d00047f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:08.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.942+0000 7f36e37fe640 1 --2- 172.21.15.89:0/2640440300 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36d0004f50 0x7f36d000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:08.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.942+0000 7f36e2ffd640 1 -- 172.21.15.89:0/2640440300 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36d0004360 msgr2=0x7f36d00047f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:08.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.942+0000 7f36e2ffd640 1 --2- 172.21.15.89:0/2640440300 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36d0004360 0x7f36d00047f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:08.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.942+0000 7f36e2ffd640 1 -- 172.21.15.89:0/2640440300 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36d0004f50 msgr2=0x7f36d000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:08.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.942+0000 7f36e2ffd640 1 --2- 172.21.15.89:0/2640440300 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36d0004f50 0x7f36d000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:08.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.942+0000 7f36e2ffd640 1 -- 172.21.15.89:0/2640440300 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d0003530 con 0x7f36d00039b0 2023-10-18T10:46:08.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.946+0000 7f36e2ffd640 1 --2- 172.21.15.89:0/2640440300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36d00039b0 0x7f36d0003d90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f36d4009ae0 tx=0x7f36d401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae74617cfb89e545 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:08.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.946+0000 7f36e1ffb640 1 -- 172.21.15.89:0/2640440300 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d40a5640 con 0x7f36d00039b0 2023-10-18T10:46:08.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.946+0000 7f36e1ffb640 1 -- 172.21.15.89:0/2640440300 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d40a57a0 con 0x7f36d00039b0 2023-10-18T10:46:08.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.946+0000 7f36e1ffb640 1 -- 172.21.15.89:0/2640440300 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d40a5b50 con 0x7f36d00039b0 2023-10-18T10:46:08.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.946+0000 7f36e9967640 1 -- 172.21.15.89:0/2640440300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36d00039b0 msgr2=0x7f36d0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:08.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.946+0000 7f36e9967640 1 --2- 172.21.15.89:0/2640440300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36d00039b0 0x7f36d0003d90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f36d4009ae0 tx=0x7f36d401ca50 comp rx=0 tx=0).stop 2023-10-18T10:46:08.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.946+0000 7f36e37fe640 1 -- 172.21.15.89:0/2640440300 reap_dead start 2023-10-18T10:46:08.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.946+0000 7f36e9967640 1 -- 172.21.15.89:0/2640440300 shutdown_connections 2023-10-18T10:46:08.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.946+0000 7f36e9967640 1 -- 172.21.15.89:0/2640440300 >> 172.21.15.89:0/2640440300 conn(0x7f36e4089fd0 msgr2=0x7f36e409b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:08.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.946+0000 7f36e9967640 1 -- 172.21.15.89:0/2640440300 shutdown_connections 2023-10-18T10:46:08.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.946+0000 7f36e9967640 1 -- 172.21.15.89:0/2640440300 wait complete. 2023-10-18T10:46:08.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.950+0000 7f36e9967640 1 Processor -- start 2023-10-18T10:46:08.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.950+0000 7f36e9967640 1 -- start start 2023-10-18T10:46:08.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.950+0000 7f36e9967640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36d00039b0 0x7f36e4159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:08.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.950+0000 7f36e9967640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36e40cb730 0x7f36e40cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:08.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.950+0000 7f36e9967640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36e41f1e90 0x7f36e41f22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:08.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.950+0000 7f36e9967640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e40cc0f0 con 0x7f36e40cb730 2023-10-18T10:46:08.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.950+0000 7f36e9967640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e40b5240 con 0x7f36e41f1e90 2023-10-18T10:46:08.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.950+0000 7f36e9967640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e41f2bc0 con 0x7f36d00039b0 2023-10-18T10:46:08.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.950+0000 7f36e27fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36e40cb730 0x7f36e40cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:08.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.950+0000 7f36e27fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36e40cb730 0x7f36e40cbbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45094/0 (socket says 172.21.15.89:45094) 2023-10-18T10:46:08.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.950+0000 7f36e27fc640 1 -- 172.21.15.89:0/2902182904 learned_addr learned my addr 172.21.15.89:0/2902182904 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:08.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.950+0000 7f36e2ffd640 1 --2- 172.21.15.89:0/2902182904 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36d00039b0 0x7f36e4159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:08.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.950+0000 7f36e37fe640 1 --2- 172.21.15.89:0/2902182904 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36e41f1e90 0x7f36e41f22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:08.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.950+0000 7f36e27fc640 1 -- 172.21.15.89:0/2902182904 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36d00039b0 msgr2=0x7f36e4159a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:08.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.950+0000 7f36e27fc640 1 --2- 172.21.15.89:0/2902182904 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36d00039b0 0x7f36e4159a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:08.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.950+0000 7f36e27fc640 1 -- 172.21.15.89:0/2902182904 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36e41f1e90 msgr2=0x7f36e41f22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:08.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.950+0000 7f36e27fc640 1 --2- 172.21.15.89:0/2902182904 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36e41f1e90 0x7f36e41f22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:08.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.950+0000 7f36e27fc640 1 -- 172.21.15.89:0/2902182904 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e40b5500 con 0x7f36e40cb730 2023-10-18T10:46:08.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.950+0000 7f36e27fc640 1 --2- 172.21.15.89:0/2902182904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36e40cb730 0x7f36e40cbbb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f36cc00e7f0 tx=0x7f36cc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:08.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.950+0000 7f36dbfff640 1 -- 172.21.15.89:0/2902182904 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36cc00c870 con 0x7f36e40cb730 2023-10-18T10:46:08.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.950+0000 7f36dbfff640 1 -- 172.21.15.89:0/2902182904 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36cc00c9d0 con 0x7f36e40cb730 2023-10-18T10:46:08.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.950+0000 7f36e9967640 1 -- 172.21.15.89:0/2902182904 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e40b57e0 con 0x7f36e40cb730 2023-10-18T10:46:08.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.950+0000 7f36dbfff640 1 -- 172.21.15.89:0/2902182904 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36cc00cd40 con 0x7f36e40cb730 2023-10-18T10:46:08.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.950+0000 7f36e9967640 1 -- 172.21.15.89:0/2902182904 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e40b5fe0 con 0x7f36e40cb730 2023-10-18T10:46:08.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.954+0000 7f36e9967640 1 -- 172.21.15.89:0/2902182904 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36e404dbe0 con 0x7f36e40cb730 2023-10-18T10:46:08.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.954+0000 7f36dbfff640 1 -- 172.21.15.89:0/2902182904 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f36cc01d460 con 0x7f36e40cb730 2023-10-18T10:46:08.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.954+0000 7f36dbfff640 1 --2- 172.21.15.89:0/2902182904 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f36c4065680 0x7f36c4067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:08.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.954+0000 7f36dbfff640 1 -- 172.21.15.89:0/2902182904 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f36cc082ea0 con 0x7f36e40cb730 2023-10-18T10:46:08.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.954+0000 7f36e2ffd640 1 --2- 172.21.15.89:0/2902182904 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f36c4065680 0x7f36c4067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:08.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.954+0000 7f36e2ffd640 1 --2- 172.21.15.89:0/2902182904 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f36c4065680 0x7f36c4067b40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f36d40af000 tx=0x7f36d40046d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:08.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:08.958+0000 7f36dbfff640 1 -- 172.21.15.89:0/2902182904 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f36cc050790 con 0x7f36e40cb730 2023-10-18T10:46:09.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.106+0000 7f36e9967640 1 -- 172.21.15.89:0/2902182904 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f36e40d3330 con 0x7f36e40cb730 2023-10-18T10:46:09.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.110+0000 7f36dbfff640 1 -- 172.21.15.89:0/2902182904 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20096 (secure 0 0 0) 0x7f36cc053db0 con 0x7f36e40cb730 2023-10-18T10:46:09.111 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:09.111 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":27,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:46:00.166702+0000","last_up_change":"2023-10-18T10:45:42.164688+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:45:53.309122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-18T10:45:54.659763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:45:57.796514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6824","nonce":773078364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6827","nonce":773078364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6831","nonce":773078364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6829","nonce":773078364}]},"public_addr":"172.21.15.154:6824/773078364","cluster_addr":"172.21.15.154:6827/773078364","heartbeat_back_addr":"172.21.15.154:6831/773078364","heartbeat_front_addr":"172.21.15.154:6829/773078364","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:46:09.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.110+0000 7f36e9967640 1 -- 172.21.15.89:0/2902182904 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f36c4065680 msgr2=0x7f36c4067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:09.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.110+0000 7f36e9967640 1 --2- 172.21.15.89:0/2902182904 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f36c4065680 0x7f36c4067b40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f36d40af000 tx=0x7f36d40046d0 comp rx=0 tx=0).stop 2023-10-18T10:46:09.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.110+0000 7f36e9967640 1 -- 172.21.15.89:0/2902182904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36e40cb730 msgr2=0x7f36e40cbbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:09.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.110+0000 7f36e9967640 1 --2- 172.21.15.89:0/2902182904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36e40cb730 0x7f36e40cbbb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f36cc00e7f0 tx=0x7f36cc00ecb0 comp rx=0 tx=0).stop 2023-10-18T10:46:09.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.110+0000 7f36e9967640 1 -- 172.21.15.89:0/2902182904 shutdown_connections 2023-10-18T10:46:09.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.110+0000 7f36e9967640 1 --2- 172.21.15.89:0/2902182904 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f36c4065680 0x7f36c4067b40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:09.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.110+0000 7f36e9967640 1 --2- 172.21.15.89:0/2902182904 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36e41f1e90 0x7f36e41f22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:09.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.110+0000 7f36e9967640 1 --2- 172.21.15.89:0/2902182904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36e40cb730 0x7f36e40cbbb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:09.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.110+0000 7f36e9967640 1 --2- 172.21.15.89:0/2902182904 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36d00039b0 0x7f36e4159a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:09.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.110+0000 7f36e9967640 1 -- 172.21.15.89:0/2902182904 >> 172.21.15.89:0/2902182904 conn(0x7f36e4089fd0 msgr2=0x7f36e415edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:09.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.110+0000 7f36e9967640 1 -- 172.21.15.89:0/2902182904 shutdown_connections 2023-10-18T10:46:09.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.110+0000 7f36e9967640 1 -- 172.21.15.89:0/2902182904 wait complete. 2023-10-18T10:46:09.132 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:45:43.114184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-18T10:45:48.730968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:45:53.309122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:45:54.659763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T10:45:57.796514+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T10:46:09.132 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:46:09.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.270+0000 7f5fcd66a640 1 Processor -- start 2023-10-18T10:46:09.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.270+0000 7f5fcd66a640 1 -- start start 2023-10-18T10:46:09.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.270+0000 7f5fcd66a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5fc80d0760 0x7f5fc80d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:09.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.270+0000 7f5fcd66a640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fc80d1100 con 0x7f5fc80cd160 2023-10-18T10:46:09.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.270+0000 7f5fcd66a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fc815da50 con 0x7f5fc815c940 2023-10-18T10:46:09.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.270+0000 7f5fcd66a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc80d3110 con 0x7f5fc80d0760 2023-10-18T10:46:09.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.274+0000 7f5fc77fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5fc80cd160 0x7f5fc80cd5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55686/0 (socket says 172.21.15.89:55686) 2023-10-18T10:46:09.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.274+0000 7f5fc77fe640 1 -- 172.21.15.89:0/3389651802 learned_addr learned my addr 172.21.15.89:0/3389651802 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:09.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.274+0000 7f5fbe7fc640 1 --2- 172.21.15.89:0/3389651802 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5fc80d0760 0x7f5fc80d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:09.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.274+0000 7f5fc67fc640 1 -- 172.21.15.89:0/3389651802 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2132066837 0 0) 0x7f5fc80d1100 con 0x7f5fc80cd160 2023-10-18T10:46:09.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.274+0000 7f5fc67fc640 1 -- 172.21.15.89:0/3389651802 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fb4003530 con 0x7f5fc80cd160 2023-10-18T10:46:09.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.274+0000 7f5fbe7fc640 1 -- 172.21.15.89:0/3389651802 >> v1:172.21.15.154:6789/0 conn(0x7f5fc815c940 legacy=0x7f5fc80d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:09.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.274+0000 7f5fc67fc640 1 -- 172.21.15.89:0/3389651802 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1430153427 0 0) 0x7f5fc815da50 con 0x7f5fc815c940 2023-10-18T10:46:09.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.274+0000 7f5fbe7fc640 1 -- 172.21.15.89:0/3389651802 >> v1:172.21.15.89:6789/0 conn(0x7f5fc80cd160 legacy=0x7f5fc80cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:09.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.274+0000 7f5fbe7fc640 1 -- 172.21.15.89:0/3389651802 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc815ce40 con 0x7f5fc80d0760 2023-10-18T10:46:09.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.278+0000 7f5fbe7fc640 1 --2- 172.21.15.89:0/3389651802 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5fc80d0760 0x7f5fc80d0bc0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5fb80097f0 tx=0x7f5fb801c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a38b7da8a7a24127 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.278+0000 7f5fc67fc640 1 -- 172.21.15.89:0/3389651802 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb80aa5f0 con 0x7f5fc80d0760 2023-10-18T10:46:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.278+0000 7f5fc67fc640 1 -- 172.21.15.89:0/3389651802 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb80aa750 con 0x7f5fc80d0760 2023-10-18T10:46:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.278+0000 7f5fc67fc640 1 -- 172.21.15.89:0/3389651802 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb80aaae0 con 0x7f5fc80d0760 2023-10-18T10:46:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.278+0000 7f5fcd66a640 1 -- 172.21.15.89:0/3389651802 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5fc80d0760 msgr2=0x7f5fc80d0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.278+0000 7f5fcd66a640 1 --2- 172.21.15.89:0/3389651802 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5fc80d0760 0x7f5fc80d0bc0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5fb80097f0 tx=0x7f5fb801c490 comp rx=0 tx=0).stop 2023-10-18T10:46:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.278+0000 7f5fcd66a640 1 -- 172.21.15.89:0/3389651802 shutdown_connections 2023-10-18T10:46:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.278+0000 7f5fcd66a640 1 --2- 172.21.15.89:0/3389651802 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5fc80d0760 0x7f5fc80d0bc0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.278+0000 7f5fcd66a640 1 -- 172.21.15.89:0/3389651802 >> 172.21.15.89:0/3389651802 conn(0x7f5fc8089fd0 msgr2=0x7f5fc809b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.278+0000 7f5fcd66a640 1 -- 172.21.15.89:0/3389651802 shutdown_connections 2023-10-18T10:46:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.278+0000 7f5fcd66a640 1 -- 172.21.15.89:0/3389651802 wait complete. 2023-10-18T10:46:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.278+0000 7f5fcd66a640 1 Processor -- start 2023-10-18T10:46:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.278+0000 7f5fcd66a640 1 -- start start 2023-10-18T10:46:09.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.282+0000 7f5fcd66a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fc80cd160 0x7f5fc80cc040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:09.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.282+0000 7f5fcd66a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5fc80d0760 0x7f5fc8152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:09.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.282+0000 7f5fcd66a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fc81f8e50 0x7f5fc815f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:09.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.282+0000 7f5fcd66a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc80cc580 con 0x7f5fc81f8e50 2023-10-18T10:46:09.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.282+0000 7f5fcd66a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc8152690 con 0x7f5fc80cd160 2023-10-18T10:46:09.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.282+0000 7f5fcd66a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc81fcc30 con 0x7f5fc80d0760 2023-10-18T10:46:09.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.282+0000 7f5fc77fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fc81f8e50 0x7f5fc815f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:09.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.282+0000 7f5fc77fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fc81f8e50 0x7f5fc815f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45102/0 (socket says 172.21.15.89:45102) 2023-10-18T10:46:09.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.282+0000 7f5fc77fe640 1 -- 172.21.15.89:0/3460348970 learned_addr learned my addr 172.21.15.89:0/3460348970 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:09.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.282+0000 7f5fbe7fc640 1 --2- 172.21.15.89:0/3460348970 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5fc80d0760 0x7f5fc8152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:09.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.282+0000 7f5fc6ffd640 1 --2- 172.21.15.89:0/3460348970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fc80cd160 0x7f5fc80cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:09.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.282+0000 7f5fc77fe640 1 -- 172.21.15.89:0/3460348970 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5fc80d0760 msgr2=0x7f5fc8152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:09.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.282+0000 7f5fc77fe640 1 --2- 172.21.15.89:0/3460348970 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5fc80d0760 0x7f5fc8152150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:09.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.282+0000 7f5fc77fe640 1 -- 172.21.15.89:0/3460348970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fc80cd160 msgr2=0x7f5fc80cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:09.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.282+0000 7f5fc77fe640 1 --2- 172.21.15.89:0/3460348970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fc80cd160 0x7f5fc80cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:09.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.282+0000 7f5fc77fe640 1 -- 172.21.15.89:0/3460348970 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb8009420 con 0x7f5fc81f8e50 2023-10-18T10:46:09.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.282+0000 7f5fbe7fc640 1 --2- 172.21.15.89:0/3460348970 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5fc80d0760 0x7f5fc8152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:09.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.282+0000 7f5fc77fe640 1 --2- 172.21.15.89:0/3460348970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fc81f8e50 0x7f5fc815f8f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5fc000cb40 tx=0x7f5fc000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:09.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.282+0000 7f5fbffff640 1 -- 172.21.15.89:0/3460348970 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc000aca0 con 0x7f5fc81f8e50 2023-10-18T10:46:09.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.282+0000 7f5fbffff640 1 -- 172.21.15.89:0/3460348970 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc000ae00 con 0x7f5fc81f8e50 2023-10-18T10:46:09.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.282+0000 7f5fbffff640 1 -- 172.21.15.89:0/3460348970 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0012620 con 0x7f5fc81f8e50 2023-10-18T10:46:09.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.282+0000 7f5fcd66a640 1 -- 172.21.15.89:0/3460348970 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fc81fcf10 con 0x7f5fc81f8e50 2023-10-18T10:46:09.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.282+0000 7f5fcd66a640 1 -- 172.21.15.89:0/3460348970 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fc81fd4d0 con 0x7f5fc81f8e50 2023-10-18T10:46:09.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.282+0000 7f5fcd66a640 1 -- 172.21.15.89:0/3460348970 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fc804dbe0 con 0x7f5fc81f8e50 2023-10-18T10:46:09.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.282+0000 7f5fbffff640 1 -- 172.21.15.89:0/3460348970 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f5fc0007070 con 0x7f5fc81f8e50 2023-10-18T10:46:09.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.286+0000 7f5fbffff640 1 --2- 172.21.15.89:0/3460348970 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5fa80656b0 0x7f5fa8067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:09.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.286+0000 7f5fbffff640 1 -- 172.21.15.89:0/3460348970 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5fc0083180 con 0x7f5fc81f8e50 2023-10-18T10:46:09.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.286+0000 7f5fc6ffd640 1 --2- 172.21.15.89:0/3460348970 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5fa80656b0 0x7f5fa8067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:09.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.286+0000 7f5fc6ffd640 1 --2- 172.21.15.89:0/3460348970 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5fa80656b0 0x7f5fa8067b70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0000f70 tx=0x7f5fb0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:09.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.290+0000 7f5fbffff640 1 -- 172.21.15.89:0/3460348970 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f5fc00508b0 con 0x7f5fc81f8e50 2023-10-18T10:46:09.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.434+0000 7f5fcd66a640 1 -- 172.21.15.89:0/3460348970 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5fc806fd00 con 0x7f5fc81f8e50 2023-10-18T10:46:09.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.434+0000 7f5fbffff640 1 -- 172.21.15.89:0/3460348970 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5fc0053ed0 con 0x7f5fc81f8e50 2023-10-18T10:46:09.437 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-18T10:46:09.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.438+0000 7f5fcd66a640 1 -- 172.21.15.89:0/3460348970 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5fa80656b0 msgr2=0x7f5fa8067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:09.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.438+0000 7f5fcd66a640 1 --2- 172.21.15.89:0/3460348970 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5fa80656b0 0x7f5fa8067b70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0000f70 tx=0x7f5fb0009040 comp rx=0 tx=0).stop 2023-10-18T10:46:09.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.438+0000 7f5fcd66a640 1 -- 172.21.15.89:0/3460348970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fc81f8e50 msgr2=0x7f5fc815f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:09.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.438+0000 7f5fcd66a640 1 --2- 172.21.15.89:0/3460348970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fc81f8e50 0x7f5fc815f8f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5fc000cb40 tx=0x7f5fc000a060 comp rx=0 tx=0).stop 2023-10-18T10:46:09.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.438+0000 7f5fcd66a640 1 -- 172.21.15.89:0/3460348970 shutdown_connections 2023-10-18T10:46:09.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.438+0000 7f5fcd66a640 1 --2- 172.21.15.89:0/3460348970 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5fa80656b0 0x7f5fa8067b70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:09.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.438+0000 7f5fcd66a640 1 --2- 172.21.15.89:0/3460348970 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5fc80d0760 0x7f5fc8152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:09.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.438+0000 7f5fcd66a640 1 --2- 172.21.15.89:0/3460348970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fc81f8e50 0x7f5fc815f8f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:09.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.438+0000 7f5fcd66a640 1 --2- 172.21.15.89:0/3460348970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fc80cd160 0x7f5fc80cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:09.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.438+0000 7f5fcd66a640 1 -- 172.21.15.89:0/3460348970 >> 172.21.15.89:0/3460348970 conn(0x7f5fc8089fd0 msgr2=0x7f5fc80b3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:09.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.438+0000 7f5fcd66a640 1 -- 172.21.15.89:0/3460348970 shutdown_connections 2023-10-18T10:46:09.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.438+0000 7f5fcd66a640 1 -- 172.21.15.89:0/3460348970 wait complete. 2023-10-18T10:46:09.458 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-18T10:46:09.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.598+0000 7fd1364ca640 1 Processor -- start 2023-10-18T10:46:09.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.598+0000 7fd1364ca640 1 -- start start 2023-10-18T10:46:09.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.598+0000 7fd1364ca640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd130158d20 0x7fd13015d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:09.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.598+0000 7fd1364ca640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd1300b01f0 0x7fd13006bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:09.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.598+0000 7fd1364ca640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd13006fe10 con 0x7fd1300afd90 2023-10-18T10:46:09.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.598+0000 7fd1364ca640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd13015d650 con 0x7fd1300b01f0 2023-10-18T10:46:09.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.598+0000 7fd1364ca640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1300c9220 con 0x7fd130158d20 2023-10-18T10:46:09.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.598+0000 7fd12f7fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd1300afd90 0x7fd13006b7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55696/0 (socket says 172.21.15.89:55696) 2023-10-18T10:46:09.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.598+0000 7fd12f7fe640 1 -- 172.21.15.89:0/908288095 learned_addr learned my addr 172.21.15.89:0/908288095 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:09.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.598+0000 7fd12effd640 1 -- 172.21.15.89:0/908288095 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3218559728 0 0) 0x7fd13006fe10 con 0x7fd1300afd90 2023-10-18T10:46:09.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.598+0000 7fd12ffff640 1 --2- 172.21.15.89:0/908288095 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd130158d20 0x7fd13015d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:09.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.598+0000 7fd134a67640 1 --2- 172.21.15.89:0/908288095 >> v2:172.21.15.154:3300/0 conn(0x7fd1300b01f0 0x7fd13006bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:09.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.598+0000 7fd12effd640 1 -- 172.21.15.89:0/908288095 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd118003530 con 0x7fd1300afd90 2023-10-18T10:46:09.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.598+0000 7fd12effd640 1 -- 172.21.15.89:0/908288095 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3343309884 0 0) 0x7fd118003530 con 0x7fd1300afd90 2023-10-18T10:46:09.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.598+0000 7fd12effd640 1 -- 172.21.15.89:0/908288095 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd130158d20 msgr2=0x7fd13015d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:09.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.598+0000 7fd12effd640 1 --2- 172.21.15.89:0/908288095 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd130158d20 0x7fd13015d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:09.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.598+0000 7fd12effd640 1 -- 172.21.15.89:0/908288095 >> v2:172.21.15.154:3300/0 conn(0x7fd1300b01f0 msgr2=0x7fd13006bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:09.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.598+0000 7fd12effd640 1 --2- 172.21.15.89:0/908288095 >> v2:172.21.15.154:3300/0 conn(0x7fd1300b01f0 0x7fd13006bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:09.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.598+0000 7fd12effd640 1 -- 172.21.15.89:0/908288095 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1300c9500 con 0x7fd1300afd90 2023-10-18T10:46:09.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.598+0000 7fd12effd640 1 -- 172.21.15.89:0/908288095 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7fd124001680 con 0x7fd1300afd90 2023-10-18T10:46:09.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.598+0000 7fd12effd640 1 -- 172.21.15.89:0/908288095 >> v1:172.21.15.89:6789/0 conn(0x7fd1300afd90 legacy=0x7fd13006b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:09.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.598+0000 7fd12effd640 1 --2- 172.21.15.89:0/908288095 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd118003980 0x7fd118003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:09.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.598+0000 7fd12effd640 1 --2- 172.21.15.89:0/908288095 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd130158d20 0x7fd1180047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:09.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.598+0000 7fd12effd640 1 --2- 172.21.15.89:0/908288095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd118004d10 0x7fd118005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:09.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.598+0000 7fd12effd640 1 -- 172.21.15.89:0/908288095 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13015d650 con 0x7fd118003980 2023-10-18T10:46:09.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.598+0000 7fd12effd640 1 -- 172.21.15.89:0/908288095 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1300c9220 con 0x7fd118004d10 2023-10-18T10:46:09.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.598+0000 7fd12effd640 1 -- 172.21.15.89:0/908288095 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd11800dba0 con 0x7fd130158d20 2023-10-18T10:46:09.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.598+0000 7fd12ffff640 1 --2- 172.21.15.89:0/908288095 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd118003980 0x7fd118003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:09.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.598+0000 7fd134a67640 1 --2- 172.21.15.89:0/908288095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd118004d10 0x7fd118005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:09.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.598+0000 7fd12f7fe640 1 --2- 172.21.15.89:0/908288095 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd130158d20 0x7fd1180047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:09.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.602+0000 7fd12ffff640 1 -- 172.21.15.89:0/908288095 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd130158d20 msgr2=0x7fd1180047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:09.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.602+0000 7fd12ffff640 1 --2- 172.21.15.89:0/908288095 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd130158d20 0x7fd1180047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:09.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.602+0000 7fd12ffff640 1 -- 172.21.15.89:0/908288095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd118004d10 msgr2=0x7fd118005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:09.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.602+0000 7fd12ffff640 1 --2- 172.21.15.89:0/908288095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd118004d10 0x7fd118005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:09.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.602+0000 7fd12ffff640 1 -- 172.21.15.89:0/908288095 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd118003530 con 0x7fd118003980 2023-10-18T10:46:09.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.602+0000 7fd12ffff640 1 --2- 172.21.15.89:0/908288095 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd118003980 0x7fd118003da0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd12000aaf0 tx=0x7fd12001c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e59a5e895d032c40 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:09.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.602+0000 7fd12effd640 1 -- 172.21.15.89:0/908288095 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1200059a0 con 0x7fd118003980 2023-10-18T10:46:09.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.602+0000 7fd12effd640 1 -- 172.21.15.89:0/908288095 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd120005b00 con 0x7fd118003980 2023-10-18T10:46:09.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.602+0000 7fd12effd640 1 -- 172.21.15.89:0/908288095 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd120005e90 con 0x7fd118003980 2023-10-18T10:46:09.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.602+0000 7fd1364ca640 1 -- 172.21.15.89:0/908288095 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd118003980 msgr2=0x7fd118003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:09.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.602+0000 7fd1364ca640 1 --2- 172.21.15.89:0/908288095 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd118003980 0x7fd118003da0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd12000aaf0 tx=0x7fd12001c9b0 comp rx=0 tx=0).stop 2023-10-18T10:46:09.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.602+0000 7fd134a67640 1 -- 172.21.15.89:0/908288095 reap_dead start 2023-10-18T10:46:09.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.602+0000 7fd1364ca640 1 -- 172.21.15.89:0/908288095 shutdown_connections 2023-10-18T10:46:09.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.602+0000 7fd1364ca640 1 -- 172.21.15.89:0/908288095 >> 172.21.15.89:0/908288095 conn(0x7fd130069f00 msgr2=0x7fd1300cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:09.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.602+0000 7fd1364ca640 1 -- 172.21.15.89:0/908288095 shutdown_connections 2023-10-18T10:46:09.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.602+0000 7fd1364ca640 1 -- 172.21.15.89:0/908288095 wait complete. 2023-10-18T10:46:09.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.606+0000 7fd1364ca640 1 Processor -- start 2023-10-18T10:46:09.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.606+0000 7fd1364ca640 1 -- start start 2023-10-18T10:46:09.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.606+0000 7fd1364ca640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd118003980 0x7fd13015a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:09.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.606+0000 7fd1364ca640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd130154f70 0x7fd13006b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:09.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.606+0000 7fd1364ca640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1301553c0 0x7fd13006bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:09.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.606+0000 7fd1364ca640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1300b3db0 con 0x7fd1301553c0 2023-10-18T10:46:09.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.606+0000 7fd1364ca640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1301593b0 con 0x7fd130154f70 2023-10-18T10:46:09.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.606+0000 7fd1364ca640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd130159690 con 0x7fd118003980 2023-10-18T10:46:09.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.606+0000 7fd134a67640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1301553c0 0x7fd13006bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:09.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.606+0000 7fd134a67640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1301553c0 0x7fd13006bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45120/0 (socket says 172.21.15.89:45120) 2023-10-18T10:46:09.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.606+0000 7fd134a67640 1 -- 172.21.15.89:0/2436869993 learned_addr learned my addr 172.21.15.89:0/2436869993 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:09.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.606+0000 7fd12f7fe640 1 --2- 172.21.15.89:0/2436869993 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd130154f70 0x7fd13006b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:09.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.606+0000 7fd12ffff640 1 --2- 172.21.15.89:0/2436869993 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd118003980 0x7fd13015a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:09.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.606+0000 7fd134a67640 1 -- 172.21.15.89:0/2436869993 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd118003980 msgr2=0x7fd13015a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:09.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.606+0000 7fd134a67640 1 --2- 172.21.15.89:0/2436869993 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd118003980 0x7fd13015a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:09.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.606+0000 7fd134a67640 1 -- 172.21.15.89:0/2436869993 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd130154f70 msgr2=0x7fd13006b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:09.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.606+0000 7fd134a67640 1 --2- 172.21.15.89:0/2436869993 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd130154f70 0x7fd13006b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:09.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.606+0000 7fd134a67640 1 -- 172.21.15.89:0/2436869993 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd130159970 con 0x7fd1301553c0 2023-10-18T10:46:09.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.606+0000 7fd12f7fe640 1 --2- 172.21.15.89:0/2436869993 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd130154f70 0x7fd13006b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:09.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.606+0000 7fd134a67640 1 --2- 172.21.15.89:0/2436869993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1301553c0 0x7fd13006bd10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd13006c8a0 tx=0x7fd12800f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:09.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.606+0000 7fd12cff9640 1 -- 172.21.15.89:0/2436869993 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12800e040 con 0x7fd1301553c0 2023-10-18T10:46:09.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.610+0000 7fd12cff9640 1 -- 172.21.15.89:0/2436869993 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd12800f040 con 0x7fd1301553c0 2023-10-18T10:46:09.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.610+0000 7fd1364ca640 1 -- 172.21.15.89:0/2436869993 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd130159c50 con 0x7fd1301553c0 2023-10-18T10:46:09.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.610+0000 7fd12cff9640 1 -- 172.21.15.89:0/2436869993 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd128015430 con 0x7fd1301553c0 2023-10-18T10:46:09.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.610+0000 7fd1364ca640 1 -- 172.21.15.89:0/2436869993 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1300b3810 con 0x7fd1301553c0 2023-10-18T10:46:09.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.610+0000 7fd1364ca640 1 -- 172.21.15.89:0/2436869993 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd13004dbe0 con 0x7fd1301553c0 2023-10-18T10:46:09.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.610+0000 7fd12cff9640 1 -- 172.21.15.89:0/2436869993 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fd128015590 con 0x7fd1301553c0 2023-10-18T10:46:09.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.610+0000 7fd12cff9640 1 --2- 172.21.15.89:0/2436869993 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd11c065680 0x7fd11c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:09.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.610+0000 7fd12cff9640 1 -- 172.21.15.89:0/2436869993 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd128083ac0 con 0x7fd1301553c0 2023-10-18T10:46:09.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.610+0000 7fd12ffff640 1 --2- 172.21.15.89:0/2436869993 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd11c065680 0x7fd11c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:09.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.610+0000 7fd12ffff640 1 --2- 172.21.15.89:0/2436869993 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd11c065680 0x7fd11c067b40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd1200ae000 tx=0x7fd120004740 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:09.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.614+0000 7fd12cff9640 1 -- 172.21.15.89:0/2436869993 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fd1280511f0 con 0x7fd1301553c0 2023-10-18T10:46:09.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.766+0000 7fd1364ca640 1 -- 172.21.15.89:0/2436869993 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd13015b0d0 con 0x7fd1301553c0 2023-10-18T10:46:09.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.766+0000 7fd12cff9640 1 -- 172.21.15.89:0/2436869993 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd128054810 con 0x7fd1301553c0 2023-10-18T10:46:09.767 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-10-18T10:46:09.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.766+0000 7fd1364ca640 1 -- 172.21.15.89:0/2436869993 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd11c065680 msgr2=0x7fd11c067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:09.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.766+0000 7fd1364ca640 1 --2- 172.21.15.89:0/2436869993 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd11c065680 0x7fd11c067b40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd1200ae000 tx=0x7fd120004740 comp rx=0 tx=0).stop 2023-10-18T10:46:09.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.766+0000 7fd1364ca640 1 -- 172.21.15.89:0/2436869993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1301553c0 msgr2=0x7fd13006bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:09.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.766+0000 7fd1364ca640 1 --2- 172.21.15.89:0/2436869993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1301553c0 0x7fd13006bd10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd13006c8a0 tx=0x7fd12800f1b0 comp rx=0 tx=0).stop 2023-10-18T10:46:09.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.766+0000 7fd1364ca640 1 -- 172.21.15.89:0/2436869993 shutdown_connections 2023-10-18T10:46:09.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.766+0000 7fd1364ca640 1 --2- 172.21.15.89:0/2436869993 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd11c065680 0x7fd11c067b40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:09.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.766+0000 7fd1364ca640 1 --2- 172.21.15.89:0/2436869993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1301553c0 0x7fd13006bd10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:09.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.766+0000 7fd1364ca640 1 --2- 172.21.15.89:0/2436869993 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd130154f70 0x7fd13006b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:09.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.766+0000 7fd1364ca640 1 --2- 172.21.15.89:0/2436869993 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd118003980 0x7fd13015a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:09.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.766+0000 7fd1364ca640 1 -- 172.21.15.89:0/2436869993 >> 172.21.15.89:0/2436869993 conn(0x7fd130069f00 msgr2=0x7fd1300cbe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:09.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.766+0000 7fd1364ca640 1 -- 172.21.15.89:0/2436869993 shutdown_connections 2023-10-18T10:46:09.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.766+0000 7fd1364ca640 1 -- 172.21.15.89:0/2436869993 wait complete. 2023-10-18T10:46:09.787 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:46:09.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.926+0000 7efe18fd2640 1 Processor -- start 2023-10-18T10:46:09.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.926+0000 7efe18fd2640 1 -- start start 2023-10-18T10:46:09.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.930+0000 7efe18fd2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efe1415c5d0 0x7efe140d1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:09.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.930+0000 7efe18fd2640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe140cf580 0x7efe140d1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:09.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.930+0000 7efe18fd2640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe140d0520 con 0x7efe140cd860 2023-10-18T10:46:09.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.930+0000 7efe18fd2640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe140d15c0 con 0x7efe1415c5d0 2023-10-18T10:46:09.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.930+0000 7efe18fd2640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe140d0800 con 0x7efe140cf580 2023-10-18T10:46:09.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.930+0000 7efe12ffd640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7efe140cd860 0x7efe140cee70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55700/0 (socket says 172.21.15.89:55700) 2023-10-18T10:46:09.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.930+0000 7efe12ffd640 1 -- 172.21.15.89:0/2720737656 learned_addr learned my addr 172.21.15.89:0/2720737656 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:09.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.930+0000 7efe137fe640 1 --2- 172.21.15.89:0/2720737656 >> v2:172.21.15.154:3300/0 conn(0x7efe1415c5d0 0x7efe140d1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:09.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.930+0000 7efe13fff640 1 --2- 172.21.15.89:0/2720737656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe140cf580 0x7efe140d1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:09.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.930+0000 7efe127fc640 1 -- 172.21.15.89:0/2720737656 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2508553253 0 0) 0x7efe140d0520 con 0x7efe140cd860 2023-10-18T10:46:09.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.930+0000 7efe127fc640 1 -- 172.21.15.89:0/2720737656 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdfc003530 con 0x7efe140cd860 2023-10-18T10:46:09.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.930+0000 7efe137fe640 1 -- 172.21.15.89:0/2720737656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe140cf580 msgr2=0x7efe140d1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:09.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.930+0000 7efe137fe640 1 --2- 172.21.15.89:0/2720737656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe140cf580 0x7efe140d1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:09.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.930+0000 7efe137fe640 1 -- 172.21.15.89:0/2720737656 >> v1:172.21.15.89:6789/0 conn(0x7efe140cd860 legacy=0x7efe140cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:09.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.930+0000 7efe127fc640 1 -- 172.21.15.89:0/2720737656 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1062285557 0 0) 0x7efdfc003530 con 0x7efe140cd860 2023-10-18T10:46:09.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.930+0000 7efe137fe640 1 -- 172.21.15.89:0/2720737656 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe140d0ae0 con 0x7efe1415c5d0 2023-10-18T10:46:09.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.930+0000 7efe13fff640 1 --2- 172.21.15.89:0/2720737656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe140cf580 0x7efe140d1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:46:09.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.934+0000 7efe137fe640 1 --2- 172.21.15.89:0/2720737656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe1415c5d0 0x7efe140d1930 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efe040097c0 tx=0x7efe0401eb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3ed032e4b267cb1 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:09.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.934+0000 7efe127fc640 1 -- 172.21.15.89:0/2720737656 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe08001680 con 0x7efe1415c5d0 2023-10-18T10:46:09.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.934+0000 7efe127fc640 1 -- 172.21.15.89:0/2720737656 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe040a85d0 con 0x7efe1415c5d0 2023-10-18T10:46:09.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.934+0000 7efe127fc640 1 -- 172.21.15.89:0/2720737656 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe040a8960 con 0x7efe1415c5d0 2023-10-18T10:46:09.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.934+0000 7efe18fd2640 1 -- 172.21.15.89:0/2720737656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe1415c5d0 msgr2=0x7efe140d1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:09.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.934+0000 7efe18fd2640 1 --2- 172.21.15.89:0/2720737656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe1415c5d0 0x7efe140d1930 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efe040097c0 tx=0x7efe0401eb90 comp rx=0 tx=0).stop 2023-10-18T10:46:09.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.934+0000 7efe18fd2640 1 -- 172.21.15.89:0/2720737656 shutdown_connections 2023-10-18T10:46:09.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.934+0000 7efe18fd2640 1 --2- 172.21.15.89:0/2720737656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe140cf580 0x7efe140d1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:09.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.934+0000 7efe18fd2640 1 --2- 172.21.15.89:0/2720737656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe1415c5d0 0x7efe140d1930 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:09.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.934+0000 7efe18fd2640 1 -- 172.21.15.89:0/2720737656 >> 172.21.15.89:0/2720737656 conn(0x7efe140b3f50 msgr2=0x7efe140b4360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:09.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.934+0000 7efe18fd2640 1 -- 172.21.15.89:0/2720737656 shutdown_connections 2023-10-18T10:46:09.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.934+0000 7efe18fd2640 1 -- 172.21.15.89:0/2720737656 wait complete. 2023-10-18T10:46:09.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.934+0000 7efe18fd2640 1 Processor -- start 2023-10-18T10:46:09.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.934+0000 7efe18fd2640 1 -- start start 2023-10-18T10:46:09.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.938+0000 7efe18fd2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe140cd860 0x7efe1414f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:09.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.938+0000 7efe18fd2640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe140cf580 0x7efe140c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:09.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.938+0000 7efe18fd2640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe1415c5d0 0x7efe140b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:09.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.938+0000 7efe18fd2640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1414fdc0 con 0x7efe140cf580 2023-10-18T10:46:09.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.938+0000 7efe18fd2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe140c92d0 con 0x7efe140cd860 2023-10-18T10:46:09.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.938+0000 7efe18fd2640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe1415daf0 con 0x7efe1415c5d0 2023-10-18T10:46:09.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.938+0000 7efe12ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe140cf580 0x7efe140c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:09.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.938+0000 7efe12ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe140cf580 0x7efe140c85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45126/0 (socket says 172.21.15.89:45126) 2023-10-18T10:46:09.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.938+0000 7efe12ffd640 1 -- 172.21.15.89:0/1414824156 learned_addr learned my addr 172.21.15.89:0/1414824156 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:09.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.938+0000 7efe137fe640 1 --2- 172.21.15.89:0/1414824156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe140cd860 0x7efe1414f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:09.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.938+0000 7efe13fff640 1 --2- 172.21.15.89:0/1414824156 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe1415c5d0 0x7efe140b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:09.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.938+0000 7efe12ffd640 1 -- 172.21.15.89:0/1414824156 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe1415c5d0 msgr2=0x7efe140b1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:09.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.938+0000 7efe12ffd640 1 --2- 172.21.15.89:0/1414824156 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe1415c5d0 0x7efe140b1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:09.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.938+0000 7efe12ffd640 1 -- 172.21.15.89:0/1414824156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe140cd860 msgr2=0x7efe1414f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:09.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.938+0000 7efe12ffd640 1 --2- 172.21.15.89:0/1414824156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe140cd860 0x7efe1414f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:09.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.938+0000 7efe12ffd640 1 -- 172.21.15.89:0/1414824156 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe0c009a70 con 0x7efe140cf580 2023-10-18T10:46:09.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.938+0000 7efe13fff640 1 --2- 172.21.15.89:0/1414824156 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe1415c5d0 0x7efe140b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:09.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.938+0000 7efe12ffd640 1 --2- 172.21.15.89:0/1414824156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe140cf580 0x7efe140c85e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efe0800c760 tx=0x7efe0800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:09.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.938+0000 7efdfbfff640 1 -- 172.21.15.89:0/1414824156 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe08005ae0 con 0x7efe140cf580 2023-10-18T10:46:09.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.938+0000 7efdfbfff640 1 -- 172.21.15.89:0/1414824156 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe08005c40 con 0x7efe140cf580 2023-10-18T10:46:09.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.938+0000 7efe18fd2640 1 -- 172.21.15.89:0/1414824156 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe04009420 con 0x7efe140cf580 2023-10-18T10:46:09.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.938+0000 7efdfbfff640 1 -- 172.21.15.89:0/1414824156 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe08007070 con 0x7efe140cf580 2023-10-18T10:46:09.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.938+0000 7efe18fd2640 1 -- 172.21.15.89:0/1414824156 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe1415dd90 con 0x7efe140cf580 2023-10-18T10:46:09.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.938+0000 7efe18fd2640 1 -- 172.21.15.89:0/1414824156 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe14073b50 con 0x7efe140cf580 2023-10-18T10:46:09.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.938+0000 7efdfbfff640 1 -- 172.21.15.89:0/1414824156 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7efe080071d0 con 0x7efe140cf580 2023-10-18T10:46:09.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.942+0000 7efdfbfff640 1 --2- 172.21.15.89:0/1414824156 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efe00065680 0x7efe00067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:09.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.942+0000 7efdfbfff640 1 -- 172.21.15.89:0/1414824156 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7efe08083000 con 0x7efe140cf580 2023-10-18T10:46:09.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.942+0000 7efe137fe640 1 --2- 172.21.15.89:0/1414824156 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efe00065680 0x7efe00067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:09.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.942+0000 7efe137fe640 1 --2- 172.21.15.89:0/1414824156 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efe00065680 0x7efe00067b40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efe040b2000 tx=0x7efe04008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:09.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:09.946+0000 7efdfbfff640 1 -- 172.21.15.89:0/1414824156 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7efe08054260 con 0x7efe140cf580 2023-10-18T10:46:10.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.094+0000 7efe18fd2640 1 -- 172.21.15.89:0/1414824156 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efe140b13a0 con 0x7efe140cf580 2023-10-18T10:46:10.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.094+0000 7efdfbfff640 1 -- 172.21.15.89:0/1414824156 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7efe08053df0 con 0x7efe140cf580 2023-10-18T10:46:10.096 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-18T10:46:10.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.094+0000 7efe18fd2640 1 -- 172.21.15.89:0/1414824156 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efe00065680 msgr2=0x7efe00067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:10.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.094+0000 7efe18fd2640 1 --2- 172.21.15.89:0/1414824156 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efe00065680 0x7efe00067b40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efe040b2000 tx=0x7efe04008040 comp rx=0 tx=0).stop 2023-10-18T10:46:10.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.094+0000 7efe18fd2640 1 -- 172.21.15.89:0/1414824156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe140cf580 msgr2=0x7efe140c85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:10.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.094+0000 7efe18fd2640 1 --2- 172.21.15.89:0/1414824156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe140cf580 0x7efe140c85e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efe0800c760 tx=0x7efe0800cc20 comp rx=0 tx=0).stop 2023-10-18T10:46:10.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.094+0000 7efe18fd2640 1 -- 172.21.15.89:0/1414824156 shutdown_connections 2023-10-18T10:46:10.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.094+0000 7efe18fd2640 1 --2- 172.21.15.89:0/1414824156 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efe00065680 0x7efe00067b40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:10.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.094+0000 7efe18fd2640 1 --2- 172.21.15.89:0/1414824156 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe1415c5d0 0x7efe140b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:10.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.094+0000 7efe18fd2640 1 --2- 172.21.15.89:0/1414824156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe140cf580 0x7efe140c85e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:10.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.094+0000 7efe18fd2640 1 --2- 172.21.15.89:0/1414824156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe140cd860 0x7efe1414f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:10.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.094+0000 7efe18fd2640 1 -- 172.21.15.89:0/1414824156 >> 172.21.15.89:0/1414824156 conn(0x7efe140b3f50 msgr2=0x7efe1415b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:10.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.098+0000 7efe18fd2640 1 -- 172.21.15.89:0/1414824156 shutdown_connections 2023-10-18T10:46:10.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.098+0000 7efe18fd2640 1 -- 172.21.15.89:0/1414824156 wait complete. 2023-10-18T10:46:10.116 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:46:10.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.254+0000 7f0343d37640 1 Processor -- start 2023-10-18T10:46:10.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.254+0000 7f0343d37640 1 -- start start 2023-10-18T10:46:10.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.254+0000 7f0343d37640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f033c0b1860 0x7f033c0cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:10.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.258+0000 7f0343d37640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f033c0d1e50 con 0x7f033c0b1cc0 2023-10-18T10:46:10.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.258+0000 7f0343d37640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f033c0d2130 con 0x7f033c15c550 2023-10-18T10:46:10.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.258+0000 7f0343d37640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f033c0d2410 con 0x7f033c0b1860 2023-10-18T10:46:10.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.258+0000 7f03422d4640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f033c0b1cc0 0x7f033c0b55f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55704/0 (socket says 172.21.15.89:55704) 2023-10-18T10:46:10.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.258+0000 7f03422d4640 1 -- 172.21.15.89:0/3388410189 learned_addr learned my addr 172.21.15.89:0/3388410189 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:10.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.258+0000 7f03412d2640 1 --2- 172.21.15.89:0/3388410189 >> v2:172.21.15.154:3300/0 conn(0x7f033c0b1860 0x7f033c0cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:10.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.258+0000 7f0340ad1640 1 -- 172.21.15.89:0/3388410189 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2659961769 0 0) 0x7f033c0d1e50 con 0x7f033c0b1cc0 2023-10-18T10:46:10.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.258+0000 7f0340ad1640 1 -- 172.21.15.89:0/3388410189 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0328003530 con 0x7f033c0b1cc0 2023-10-18T10:46:10.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.258+0000 7f03412d2640 1 -- 172.21.15.89:0/3388410189 >> v1:172.21.15.154:6789/0 conn(0x7f033c15c550 legacy=0x7f033c0d2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:10.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.258+0000 7f0340ad1640 1 -- 172.21.15.89:0/3388410189 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2394756568 0 0) 0x7f033c0d2130 con 0x7f033c15c550 2023-10-18T10:46:10.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.258+0000 7f03412d2640 1 -- 172.21.15.89:0/3388410189 >> v1:172.21.15.89:6789/0 conn(0x7f033c0b1cc0 legacy=0x7f033c0b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:10.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.258+0000 7f03412d2640 1 -- 172.21.15.89:0/3388410189 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033c0d26f0 con 0x7f033c0b1860 2023-10-18T10:46:10.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.258+0000 7f03412d2640 1 --2- 172.21.15.89:0/3388410189 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f033c0b1860 0x7f033c0cf610 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f032c0097c0 tx=0x7f032c01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fc423a0ecf00e8d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:10.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.262+0000 7f0340ad1640 1 -- 172.21.15.89:0/3388410189 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c0ad660 con 0x7f033c0b1860 2023-10-18T10:46:10.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.262+0000 7f0340ad1640 1 -- 172.21.15.89:0/3388410189 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032c0ad7c0 con 0x7f033c0b1860 2023-10-18T10:46:10.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.262+0000 7f0340ad1640 1 -- 172.21.15.89:0/3388410189 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c0adb50 con 0x7f033c0b1860 2023-10-18T10:46:10.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.262+0000 7f0343d37640 1 -- 172.21.15.89:0/3388410189 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f033c0b1860 msgr2=0x7f033c0cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:10.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.262+0000 7f0343d37640 1 --2- 172.21.15.89:0/3388410189 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f033c0b1860 0x7f033c0cf610 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f032c0097c0 tx=0x7f032c01c450 comp rx=0 tx=0).stop 2023-10-18T10:46:10.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.262+0000 7f0343d37640 1 -- 172.21.15.89:0/3388410189 shutdown_connections 2023-10-18T10:46:10.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.262+0000 7f0343d37640 1 --2- 172.21.15.89:0/3388410189 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f033c0b1860 0x7f033c0cf610 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:10.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.262+0000 7f0343d37640 1 -- 172.21.15.89:0/3388410189 >> 172.21.15.89:0/3388410189 conn(0x7f033c069d00 msgr2=0x7f033c06a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:10.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.262+0000 7f0343d37640 1 -- 172.21.15.89:0/3388410189 shutdown_connections 2023-10-18T10:46:10.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.262+0000 7f0343d37640 1 -- 172.21.15.89:0/3388410189 wait complete. 2023-10-18T10:46:10.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.262+0000 7f0343d37640 1 Processor -- start 2023-10-18T10:46:10.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.262+0000 7f0343d37640 1 -- start start 2023-10-18T10:46:10.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.262+0000 7f0343d37640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f033c0b1860 0x7f033c151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:10.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.262+0000 7f0343d37640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f033c0b1cc0 0x7f033c14ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:10.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.262+0000 7f0343d37640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f033c15c550 0x7f033c14f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:10.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.262+0000 7f0343d37640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033c14e940 con 0x7f033c0b1860 2023-10-18T10:46:10.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.262+0000 7f0343d37640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033c152a80 con 0x7f033c15c550 2023-10-18T10:46:10.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.262+0000 7f0343d37640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033c152d30 con 0x7f033c0b1cc0 2023-10-18T10:46:10.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.262+0000 7f0341ad3640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f033c0b1860 0x7f033c151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:10.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.262+0000 7f0341ad3640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f033c0b1860 0x7f033c151110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45140/0 (socket says 172.21.15.89:45140) 2023-10-18T10:46:10.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.262+0000 7f0341ad3640 1 -- 172.21.15.89:0/165501823 learned_addr learned my addr 172.21.15.89:0/165501823 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:10.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.262+0000 7f03422d4640 1 --2- 172.21.15.89:0/165501823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f033c15c550 0x7f033c14f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:10.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.262+0000 7f03412d2640 1 --2- 172.21.15.89:0/165501823 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f033c0b1cc0 0x7f033c14ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:10.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.266+0000 7f0341ad3640 1 -- 172.21.15.89:0/165501823 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f033c0b1cc0 msgr2=0x7f033c14ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:10.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.266+0000 7f0341ad3640 1 --2- 172.21.15.89:0/165501823 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f033c0b1cc0 0x7f033c14ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:10.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.266+0000 7f0341ad3640 1 -- 172.21.15.89:0/165501823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f033c15c550 msgr2=0x7f033c14f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:10.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.266+0000 7f0341ad3640 1 --2- 172.21.15.89:0/165501823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f033c15c550 0x7f033c14f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:10.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.266+0000 7f0341ad3640 1 -- 172.21.15.89:0/165501823 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032c009420 con 0x7f033c0b1860 2023-10-18T10:46:10.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.266+0000 7f0341ad3640 1 --2- 172.21.15.89:0/165501823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f033c0b1860 0x7f033c151110 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0338001ec0 tx=0x7f033800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:10.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.266+0000 7f03327fc640 1 -- 172.21.15.89:0/165501823 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03380054e0 con 0x7f033c0b1860 2023-10-18T10:46:10.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.266+0000 7f03327fc640 1 -- 172.21.15.89:0/165501823 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0338003600 con 0x7f033c0b1860 2023-10-18T10:46:10.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.266+0000 7f0343d37640 1 -- 172.21.15.89:0/165501823 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f033c153010 con 0x7f033c0b1860 2023-10-18T10:46:10.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.266+0000 7f03327fc640 1 -- 172.21.15.89:0/165501823 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03380039b0 con 0x7f033c0b1860 2023-10-18T10:46:10.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.266+0000 7f0343d37640 1 -- 172.21.15.89:0/165501823 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f033c1535d0 con 0x7f033c0b1860 2023-10-18T10:46:10.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.266+0000 7f0343d37640 1 -- 172.21.15.89:0/165501823 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f033c04dbe0 con 0x7f033c0b1860 2023-10-18T10:46:10.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.266+0000 7f03327fc640 1 -- 172.21.15.89:0/165501823 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f0338003b10 con 0x7f033c0b1860 2023-10-18T10:46:10.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.266+0000 7f03327fc640 1 --2- 172.21.15.89:0/165501823 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0320065680 0x7f0320067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:10.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.266+0000 7f03327fc640 1 -- 172.21.15.89:0/165501823 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0338082650 con 0x7f033c0b1860 2023-10-18T10:46:10.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.266+0000 7f03412d2640 1 --2- 172.21.15.89:0/165501823 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0320065680 0x7f0320067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:10.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.270+0000 7f03412d2640 1 --2- 172.21.15.89:0/165501823 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0320065680 0x7f0320067b40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f032c0af000 tx=0x7f032c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:10.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.270+0000 7f03327fc640 1 -- 172.21.15.89:0/165501823 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f033804fed0 con 0x7f033c0b1860 2023-10-18T10:46:10.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.418+0000 7f0343d37640 1 -- 172.21.15.89:0/165501823 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f033c0d0da0 con 0x7f033c0b1860 2023-10-18T10:46:10.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.418+0000 7f03327fc640 1 -- 172.21.15.89:0/165501823 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f03380534f0 con 0x7f033c0b1860 2023-10-18T10:46:10.422 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-18T10:46:10.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.422+0000 7f0343d37640 1 -- 172.21.15.89:0/165501823 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0320065680 msgr2=0x7f0320067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:10.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.422+0000 7f0343d37640 1 --2- 172.21.15.89:0/165501823 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0320065680 0x7f0320067b40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f032c0af000 tx=0x7f032c008040 comp rx=0 tx=0).stop 2023-10-18T10:46:10.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.422+0000 7f0343d37640 1 -- 172.21.15.89:0/165501823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f033c0b1860 msgr2=0x7f033c151110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:10.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.422+0000 7f0343d37640 1 --2- 172.21.15.89:0/165501823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f033c0b1860 0x7f033c151110 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0338001ec0 tx=0x7f033800c8e0 comp rx=0 tx=0).stop 2023-10-18T10:46:10.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.422+0000 7f0343d37640 1 -- 172.21.15.89:0/165501823 shutdown_connections 2023-10-18T10:46:10.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.422+0000 7f0343d37640 1 --2- 172.21.15.89:0/165501823 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0320065680 0x7f0320067b40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:10.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.422+0000 7f0343d37640 1 --2- 172.21.15.89:0/165501823 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f033c0b1cc0 0x7f033c14ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:10.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.422+0000 7f0343d37640 1 --2- 172.21.15.89:0/165501823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f033c15c550 0x7f033c14f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:10.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.422+0000 7f0343d37640 1 --2- 172.21.15.89:0/165501823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f033c0b1860 0x7f033c151110 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:10.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.422+0000 7f0343d37640 1 -- 172.21.15.89:0/165501823 >> 172.21.15.89:0/165501823 conn(0x7f033c069d00 msgr2=0x7f033c0b5270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:10.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.422+0000 7f0343d37640 1 -- 172.21.15.89:0/165501823 shutdown_connections 2023-10-18T10:46:10.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.422+0000 7f0343d37640 1 -- 172.21.15.89:0/165501823 wait complete. 2023-10-18T10:46:10.442 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T10:46:10.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.582+0000 7f17b4cbd640 1 Processor -- start 2023-10-18T10:46:10.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.582+0000 7f17b4cbd640 1 -- start start 2023-10-18T10:46:10.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.582+0000 7f17b4cbd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f17b006ff00 0x7f17b015e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:10.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.582+0000 7f17b4cbd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f17b00ca340 0x7f17b015b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:10.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.582+0000 7f17b4cbd640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17b015acb0 con 0x7f17b015b680 2023-10-18T10:46:10.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.582+0000 7f17b4cbd640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17b015ef30 con 0x7f17b00ca340 2023-10-18T10:46:10.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.582+0000 7f17b4cbd640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17b015aa10 con 0x7f17b006ff00 2023-10-18T10:46:10.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.582+0000 7f17add9b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f17b00ca340 0x7f17b015b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:10.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.582+0000 7f17add9b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f17b00ca340 0x7f17b015b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45154/0 (socket says 172.21.15.89:45154) 2023-10-18T10:46:10.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.582+0000 7f17add9b640 1 -- 172.21.15.89:0/96201636 learned_addr learned my addr 172.21.15.89:0/96201636 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:10.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.582+0000 7f17ae59c640 1 --2- 172.21.15.89:0/96201636 >> v2:172.21.15.154:3300/0 conn(0x7f17b006ff00 0x7f17b015e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:10.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.582+0000 7f17add9b640 1 -- 172.21.15.89:0/96201636 >> v2:172.21.15.154:3300/0 conn(0x7f17b006ff00 msgr2=0x7f17b015e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:10.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.582+0000 7f17add9b640 1 --2- 172.21.15.89:0/96201636 >> v2:172.21.15.154:3300/0 conn(0x7f17b006ff00 0x7f17b015e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:10.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.582+0000 7f17add9b640 1 -- 172.21.15.89:0/96201636 >> v1:172.21.15.154:6789/0 conn(0x7f17b015b680 legacy=0x7f17b00c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:10.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.582+0000 7f17add9b640 1 -- 172.21.15.89:0/96201636 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b00702e0 con 0x7f17b00ca340 2023-10-18T10:46:10.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.586+0000 7f17add9b640 1 --2- 172.21.15.89:0/96201636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17b00ca340 0x7f17b015b140 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f17b006a830 tx=0x7f17a401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac208e0dea1cf7cc server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:10.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.586+0000 7f17ad59a640 1 -- 172.21.15.89:0/96201636 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a40ab640 con 0x7f17b00ca340 2023-10-18T10:46:10.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.586+0000 7f17ad59a640 1 -- 172.21.15.89:0/96201636 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a40a7070 con 0x7f17b00ca340 2023-10-18T10:46:10.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.586+0000 7f17ad59a640 1 -- 172.21.15.89:0/96201636 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a40ab9c0 con 0x7f17b00ca340 2023-10-18T10:46:10.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.586+0000 7f17b4cbd640 1 -- 172.21.15.89:0/96201636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17b00ca340 msgr2=0x7f17b015b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:10.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.586+0000 7f17b4cbd640 1 --2- 172.21.15.89:0/96201636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17b00ca340 0x7f17b015b140 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f17b006a830 tx=0x7f17a401c440 comp rx=0 tx=0).stop 2023-10-18T10:46:10.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.586+0000 7f17b4cbd640 1 -- 172.21.15.89:0/96201636 shutdown_connections 2023-10-18T10:46:10.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.586+0000 7f17b4cbd640 1 --2- 172.21.15.89:0/96201636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17b00ca340 0x7f17b015b140 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:10.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.586+0000 7f17b4cbd640 1 --2- 172.21.15.89:0/96201636 >> v2:172.21.15.154:3300/0 conn(0x7f17b006ff00 0x7f17b015e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:10.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.586+0000 7f17b4cbd640 1 -- 172.21.15.89:0/96201636 >> 172.21.15.89:0/96201636 conn(0x7f17b0069f00 msgr2=0x7f17b00cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:10.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.586+0000 7f17b4cbd640 1 -- 172.21.15.89:0/96201636 shutdown_connections 2023-10-18T10:46:10.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.586+0000 7f17b4cbd640 1 -- 172.21.15.89:0/96201636 wait complete. 2023-10-18T10:46:10.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.590+0000 7f17b4cbd640 1 Processor -- start 2023-10-18T10:46:10.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.590+0000 7f17b4cbd640 1 -- start start 2023-10-18T10:46:10.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.590+0000 7f17b4cbd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17b006ff00 0x7f17b0159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:10.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.590+0000 7f17b4cbd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17b00ca340 0x7f17b0153430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:10.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.590+0000 7f17b4cbd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17b00c1890 0x7f17b0153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:10.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.590+0000 7f17b4cbd640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b00cd830 con 0x7f17b00c1890 2023-10-18T10:46:10.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.590+0000 7f17b4cbd640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b00b2a90 con 0x7f17b00ca340 2023-10-18T10:46:10.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.590+0000 7f17b4cbd640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b00cdaa0 con 0x7f17b006ff00 2023-10-18T10:46:10.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.590+0000 7f17aed9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17b00c1890 0x7f17b0153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:10.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.590+0000 7f17ae59c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17b006ff00 0x7f17b0159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:10.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.590+0000 7f17aed9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17b00c1890 0x7f17b0153970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45162/0 (socket says 172.21.15.89:45162) 2023-10-18T10:46:10.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.590+0000 7f17aed9d640 1 -- 172.21.15.89:0/3456194732 learned_addr learned my addr 172.21.15.89:0/3456194732 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:10.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.590+0000 7f17add9b640 1 --2- 172.21.15.89:0/3456194732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17b00ca340 0x7f17b0153430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:10.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.590+0000 7f17aed9d640 1 -- 172.21.15.89:0/3456194732 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17b006ff00 msgr2=0x7f17b0159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:10.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.590+0000 7f17aed9d640 1 --2- 172.21.15.89:0/3456194732 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17b006ff00 0x7f17b0159690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:10.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.590+0000 7f17aed9d640 1 -- 172.21.15.89:0/3456194732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17b00ca340 msgr2=0x7f17b0153430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:10.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.590+0000 7f17aed9d640 1 --2- 172.21.15.89:0/3456194732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17b00ca340 0x7f17b0153430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:10.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.590+0000 7f17aed9d640 1 -- 172.21.15.89:0/3456194732 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a40094d0 con 0x7f17b00c1890 2023-10-18T10:46:10.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.590+0000 7f17aed9d640 1 --2- 172.21.15.89:0/3456194732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17b00c1890 0x7f17b0153970 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f17a800c3c0 tx=0x7f17a800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:10.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.590+0000 7f179effd640 1 -- 172.21.15.89:0/3456194732 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a80052d0 con 0x7f17b00c1890 2023-10-18T10:46:10.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.590+0000 7f179effd640 1 -- 172.21.15.89:0/3456194732 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a8005430 con 0x7f17b00c1890 2023-10-18T10:46:10.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.590+0000 7f17b4cbd640 1 -- 172.21.15.89:0/3456194732 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b00cdd80 con 0x7f17b00c1890 2023-10-18T10:46:10.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.590+0000 7f179effd640 1 -- 172.21.15.89:0/3456194732 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a8003840 con 0x7f17b00c1890 2023-10-18T10:46:10.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.590+0000 7f17b4cbd640 1 -- 172.21.15.89:0/3456194732 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b0150760 con 0x7f17b00c1890 2023-10-18T10:46:10.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.594+0000 7f17b4cbd640 1 -- 172.21.15.89:0/3456194732 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17b004dbe0 con 0x7f17b00c1890 2023-10-18T10:46:10.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.594+0000 7f179effd640 1 -- 172.21.15.89:0/3456194732 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f17a80039a0 con 0x7f17b00c1890 2023-10-18T10:46:10.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.594+0000 7f179effd640 1 --2- 172.21.15.89:0/3456194732 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1790065680 0x7f1790067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:10.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.594+0000 7f179effd640 1 -- 172.21.15.89:0/3456194732 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f17a8082530 con 0x7f17b00c1890 2023-10-18T10:46:10.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.594+0000 7f17ae59c640 1 --2- 172.21.15.89:0/3456194732 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1790065680 0x7f1790067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:10.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.594+0000 7f17ae59c640 1 --2- 172.21.15.89:0/3456194732 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1790065680 0x7f1790067b40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f17a0003b90 tx=0x7f17a00098d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:10.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.598+0000 7f179effd640 1 -- 172.21.15.89:0/3456194732 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f17a804fd10 con 0x7f17b00c1890 2023-10-18T10:46:10.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.746+0000 7f17b4cbd640 1 -- 172.21.15.89:0/3456194732 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f17b00cb820 con 0x7f17b00c1890 2023-10-18T10:46:10.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.746+0000 7f179effd640 1 -- 172.21.15.89:0/3456194732 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f17a8053330 con 0x7f17b00c1890 2023-10-18T10:46:10.748 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-18T10:46:10.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.746+0000 7f17b4cbd640 1 -- 172.21.15.89:0/3456194732 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1790065680 msgr2=0x7f1790067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:10.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.746+0000 7f17b4cbd640 1 --2- 172.21.15.89:0/3456194732 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1790065680 0x7f1790067b40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f17a0003b90 tx=0x7f17a00098d0 comp rx=0 tx=0).stop 2023-10-18T10:46:10.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.746+0000 7f17b4cbd640 1 -- 172.21.15.89:0/3456194732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17b00c1890 msgr2=0x7f17b0153970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:10.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.746+0000 7f17b4cbd640 1 --2- 172.21.15.89:0/3456194732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17b00c1890 0x7f17b0153970 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f17a800c3c0 tx=0x7f17a800c880 comp rx=0 tx=0).stop 2023-10-18T10:46:10.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.746+0000 7f17b4cbd640 1 -- 172.21.15.89:0/3456194732 shutdown_connections 2023-10-18T10:46:10.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.746+0000 7f17b4cbd640 1 --2- 172.21.15.89:0/3456194732 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1790065680 0x7f1790067b40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:10.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.746+0000 7f17b4cbd640 1 --2- 172.21.15.89:0/3456194732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17b00c1890 0x7f17b0153970 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:10.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.746+0000 7f17b4cbd640 1 --2- 172.21.15.89:0/3456194732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17b00ca340 0x7f17b0153430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:10.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.746+0000 7f17b4cbd640 1 --2- 172.21.15.89:0/3456194732 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17b006ff00 0x7f17b0159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:10.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.746+0000 7f17b4cbd640 1 -- 172.21.15.89:0/3456194732 >> 172.21.15.89:0/3456194732 conn(0x7f17b0069f00 msgr2=0x7f17b015c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:10.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.746+0000 7f17b4cbd640 1 -- 172.21.15.89:0/3456194732 shutdown_connections 2023-10-18T10:46:10.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.746+0000 7f17b4cbd640 1 -- 172.21.15.89:0/3456194732 wait complete. 2023-10-18T10:46:10.769 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T10:46:10.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.906+0000 7f8d44f3a640 1 Processor -- start 2023-10-18T10:46:10.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.906+0000 7f8d44f3a640 1 -- start start 2023-10-18T10:46:10.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.910+0000 7f8d44f3a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8d4015c550 0x7f8d400d2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:10.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.910+0000 7f8d44f3a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d400cfd20 0x7f8d400b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:10.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.910+0000 7f8d44f3a640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d400d1e50 con 0x7f8d400b1860 2023-10-18T10:46:10.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.910+0000 7f8d44f3a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d400d2130 con 0x7f8d4015c550 2023-10-18T10:46:10.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.910+0000 7f8d44f3a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d400d2410 con 0x7f8d400cfd20 2023-10-18T10:46:10.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.910+0000 7f8d3dd9b640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f8d400b1860 0x7f8d400cf610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:55716/0 (socket says 172.21.15.89:55716) 2023-10-18T10:46:10.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.910+0000 7f8d3dd9b640 1 -- 172.21.15.89:0/741879763 learned_addr learned my addr 172.21.15.89:0/741879763 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:10.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.910+0000 7f8d3e59c640 1 --2- 172.21.15.89:0/741879763 >> v2:172.21.15.154:3300/0 conn(0x7f8d4015c550 0x7f8d400d2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:10.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.910+0000 7f8d3ed9d640 1 --2- 172.21.15.89:0/741879763 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d400cfd20 0x7f8d400b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:10.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.910+0000 7f8d3d59a640 1 -- 172.21.15.89:0/741879763 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2359360724 0 0) 0x7f8d400d1e50 con 0x7f8d400b1860 2023-10-18T10:46:10.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.910+0000 7f8d3d59a640 1 -- 172.21.15.89:0/741879763 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d28003530 con 0x7f8d400b1860 2023-10-18T10:46:10.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.910+0000 7f8d3e59c640 1 -- 172.21.15.89:0/741879763 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d400cfd20 msgr2=0x7f8d400b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:10.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.910+0000 7f8d3e59c640 1 --2- 172.21.15.89:0/741879763 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d400cfd20 0x7f8d400b55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:10.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.910+0000 7f8d3d59a640 1 -- 172.21.15.89:0/741879763 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 252522430 0 0) 0x7f8d28003530 con 0x7f8d400b1860 2023-10-18T10:46:10.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.910+0000 7f8d3e59c640 1 -- 172.21.15.89:0/741879763 >> v1:172.21.15.89:6789/0 conn(0x7f8d400b1860 legacy=0x7f8d400cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:10.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.910+0000 7f8d3e59c640 1 -- 172.21.15.89:0/741879763 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d400d26f0 con 0x7f8d4015c550 2023-10-18T10:46:10.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.914+0000 7f8d3e59c640 1 --2- 172.21.15.89:0/741879763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d4015c550 0x7f8d400d2ef0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8d300097c0 tx=0x7f8d3001cdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaa83659e8728042 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:10.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.914+0000 7f8d3d59a640 1 -- 172.21.15.89:0/741879763 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d300a4ea0 con 0x7f8d4015c550 2023-10-18T10:46:10.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.914+0000 7f8d3d59a640 1 -- 172.21.15.89:0/741879763 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d300a7050 con 0x7f8d4015c550 2023-10-18T10:46:10.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.914+0000 7f8d3d59a640 1 -- 172.21.15.89:0/741879763 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d34001680 con 0x7f8d4015c550 2023-10-18T10:46:10.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.914+0000 7f8d44f3a640 1 -- 172.21.15.89:0/741879763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d4015c550 msgr2=0x7f8d400d2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:10.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.914+0000 7f8d44f3a640 1 --2- 172.21.15.89:0/741879763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d4015c550 0x7f8d400d2ef0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8d300097c0 tx=0x7f8d3001cdd0 comp rx=0 tx=0).stop 2023-10-18T10:46:10.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.914+0000 7f8d44f3a640 1 -- 172.21.15.89:0/741879763 shutdown_connections 2023-10-18T10:46:10.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.914+0000 7f8d44f3a640 1 --2- 172.21.15.89:0/741879763 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d400cfd20 0x7f8d400b55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:10.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.914+0000 7f8d44f3a640 1 --2- 172.21.15.89:0/741879763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d4015c550 0x7f8d400d2ef0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:10.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.914+0000 7f8d44f3a640 1 -- 172.21.15.89:0/741879763 >> 172.21.15.89:0/741879763 conn(0x7f8d40069d00 msgr2=0x7f8d4006a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:10.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.914+0000 7f8d44f3a640 1 -- 172.21.15.89:0/741879763 shutdown_connections 2023-10-18T10:46:10.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.914+0000 7f8d44f3a640 1 -- 172.21.15.89:0/741879763 wait complete. 2023-10-18T10:46:10.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.914+0000 7f8d44f3a640 1 Processor -- start 2023-10-18T10:46:10.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.918+0000 7f8d44f3a640 1 -- start start 2023-10-18T10:46:10.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.918+0000 7f8d44f3a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d400b1860 0x7f8d40150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:10.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.918+0000 7f8d44f3a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d400cfd20 0x7f8d4014cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:10.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.918+0000 7f8d44f3a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d4015c550 0x7f8d4014c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:10.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.918+0000 7f8d44f3a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d400c6730 con 0x7f8d4015c550 2023-10-18T10:46:10.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.918+0000 7f8d44f3a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d400c6a10 con 0x7f8d400cfd20 2023-10-18T10:46:10.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.918+0000 7f8d44f3a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d400c6cf0 con 0x7f8d400b1860 2023-10-18T10:46:10.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.918+0000 7f8d3ed9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d4015c550 0x7f8d4014c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:10.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.918+0000 7f8d3ed9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d4015c550 0x7f8d4014c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45168/0 (socket says 172.21.15.89:45168) 2023-10-18T10:46:10.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.918+0000 7f8d3ed9d640 1 -- 172.21.15.89:0/3282902724 learned_addr learned my addr 172.21.15.89:0/3282902724 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:10.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.918+0000 7f8d3dd9b640 1 --2- 172.21.15.89:0/3282902724 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d400cfd20 0x7f8d4014cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:10.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.918+0000 7f8d3e59c640 1 --2- 172.21.15.89:0/3282902724 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d400b1860 0x7f8d40150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:10.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.918+0000 7f8d3ed9d640 1 -- 172.21.15.89:0/3282902724 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d400b1860 msgr2=0x7f8d40150bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:10.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.918+0000 7f8d3ed9d640 1 --2- 172.21.15.89:0/3282902724 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d400b1860 0x7f8d40150bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:10.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.918+0000 7f8d3ed9d640 1 -- 172.21.15.89:0/3282902724 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d400cfd20 msgr2=0x7f8d4014cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:10.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.918+0000 7f8d3ed9d640 1 --2- 172.21.15.89:0/3282902724 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d400cfd20 0x7f8d4014cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:10.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.918+0000 7f8d3ed9d640 1 -- 172.21.15.89:0/3282902724 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d30009420 con 0x7f8d4015c550 2023-10-18T10:46:10.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.918+0000 7f8d3ed9d640 1 --2- 172.21.15.89:0/3282902724 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d4015c550 0x7f8d4014c1f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8d3800ef90 tx=0x7f8d3800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:10.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.918+0000 7f8d2effd640 1 -- 172.21.15.89:0/3282902724 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d38009ca0 con 0x7f8d4015c550 2023-10-18T10:46:10.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.918+0000 7f8d2effd640 1 -- 172.21.15.89:0/3282902724 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3800f040 con 0x7f8d4015c550 2023-10-18T10:46:10.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.918+0000 7f8d44f3a640 1 -- 172.21.15.89:0/3282902724 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d400c6fd0 con 0x7f8d4015c550 2023-10-18T10:46:10.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.918+0000 7f8d2effd640 1 -- 172.21.15.89:0/3282902724 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d380146c0 con 0x7f8d4015c550 2023-10-18T10:46:10.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.918+0000 7f8d44f3a640 1 -- 172.21.15.89:0/3282902724 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d4014d010 con 0x7f8d4015c550 2023-10-18T10:46:10.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.918+0000 7f8d44f3a640 1 -- 172.21.15.89:0/3282902724 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d4004dbe0 con 0x7f8d4015c550 2023-10-18T10:46:10.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.922+0000 7f8d2effd640 1 -- 172.21.15.89:0/3282902724 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f8d38014820 con 0x7f8d4015c550 2023-10-18T10:46:10.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.922+0000 7f8d2effd640 1 --2- 172.21.15.89:0/3282902724 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8d20065680 0x7f8d20067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:10.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.922+0000 7f8d2effd640 1 -- 172.21.15.89:0/3282902724 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8d38083ec0 con 0x7f8d4015c550 2023-10-18T10:46:10.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.922+0000 7f8d3e59c640 1 --2- 172.21.15.89:0/3282902724 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8d20065680 0x7f8d20067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:10.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.922+0000 7f8d3e59c640 1 --2- 172.21.15.89:0/3282902724 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8d20065680 0x7f8d20067b40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8d300ad000 tx=0x7f8d30004740 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:10.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:10.926+0000 7f8d2effd640 1 -- 172.21.15.89:0/3282902724 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f8d380516a0 con 0x7f8d4015c550 2023-10-18T10:46:11.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.086+0000 7f8d44f3a640 1 -- 172.21.15.89:0/3282902724 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8d400c7480 con 0x7f8d4015c550 2023-10-18T10:46:11.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.086+0000 7f8d2effd640 1 -- 172.21.15.89:0/3282902724 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8d38018020 con 0x7f8d4015c550 2023-10-18T10:46:11.089 INFO:teuthology.orchestra.run.smithi089.stderr:listed 0 entries 2023-10-18T10:46:11.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.090+0000 7f8d44f3a640 1 -- 172.21.15.89:0/3282902724 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8d20065680 msgr2=0x7f8d20067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:11.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.090+0000 7f8d44f3a640 1 --2- 172.21.15.89:0/3282902724 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8d20065680 0x7f8d20067b40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8d300ad000 tx=0x7f8d30004740 comp rx=0 tx=0).stop 2023-10-18T10:46:11.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.090+0000 7f8d44f3a640 1 -- 172.21.15.89:0/3282902724 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d4015c550 msgr2=0x7f8d4014c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:11.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.090+0000 7f8d44f3a640 1 --2- 172.21.15.89:0/3282902724 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d4015c550 0x7f8d4014c1f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8d3800ef90 tx=0x7f8d3800c490 comp rx=0 tx=0).stop 2023-10-18T10:46:11.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.090+0000 7f8d44f3a640 1 -- 172.21.15.89:0/3282902724 shutdown_connections 2023-10-18T10:46:11.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.090+0000 7f8d44f3a640 1 --2- 172.21.15.89:0/3282902724 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8d20065680 0x7f8d20067b40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:11.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.090+0000 7f8d44f3a640 1 --2- 172.21.15.89:0/3282902724 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d4015c550 0x7f8d4014c1f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:11.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.090+0000 7f8d44f3a640 1 --2- 172.21.15.89:0/3282902724 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d400cfd20 0x7f8d4014cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:11.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.090+0000 7f8d44f3a640 1 --2- 172.21.15.89:0/3282902724 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d400b1860 0x7f8d40150bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:11.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.090+0000 7f8d44f3a640 1 -- 172.21.15.89:0/3282902724 >> 172.21.15.89:0/3282902724 conn(0x7f8d40069d00 msgr2=0x7f8d400d0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:11.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.090+0000 7f8d44f3a640 1 -- 172.21.15.89:0/3282902724 shutdown_connections 2023-10-18T10:46:11.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.090+0000 7f8d44f3a640 1 -- 172.21.15.89:0/3282902724 wait complete. 2023-10-18T10:46:11.109 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi089.front.sepia.ceph.com... 2023-10-18T10:46:11.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:11.111 DEBUG:teuthology.orchestra.run.smithi089:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-10-18T10:46:11.157 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi089.stderr:ceph-fuse[44480]: fuse finished with error 0 and tester_r 0 2023-10-18T10:46:11.160 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:46:11.197 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi089.front.sepia.ceph.com 2023-10-18T10:46:11.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:11.197 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-10-18T10:46:11.255 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:46:11.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.434+0000 7fdea9396640 1 Processor -- start 2023-10-18T10:46:11.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.438+0000 7fdea9396640 1 -- start start 2023-10-18T10:46:11.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.438+0000 7fdea9396640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdea415c940 0x7fdea40d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:11.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.438+0000 7fdea9396640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdea40d2f40 0x7fdea40d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:11.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.438+0000 7fdea9396640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdea40d0ca0 0x7fdea40cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:11.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.438+0000 7fdea9396640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea40d1130 con 0x7fdea40d0ca0 2023-10-18T10:46:11.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.438+0000 7fdea9396640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea415da50 con 0x7fdea415c940 2023-10-18T10:46:11.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.438+0000 7fdea9396640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea415ce40 con 0x7fdea40d2f40 2023-10-18T10:46:11.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.438+0000 7fdea37fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdea40d0ca0 0x7fdea40cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:11.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.438+0000 7fdea37fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdea40d0ca0 0x7fdea40cf060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45178/0 (socket says 172.21.15.89:45178) 2023-10-18T10:46:11.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.438+0000 7fdea37fe640 1 -- 172.21.15.89:0/1137071362 learned_addr learned my addr 172.21.15.89:0/1137071362 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:11.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.438+0000 7fdea2ffd640 1 --2- 172.21.15.89:0/1137071362 >> v2:172.21.15.154:3300/0 conn(0x7fdea415c940 0x7fdea40d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:11.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.438+0000 7fdea27fc640 1 --2- 172.21.15.89:0/1137071362 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdea40d2f40 0x7fdea40d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:11.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.438+0000 7fdea37fe640 1 -- 172.21.15.89:0/1137071362 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdea40d2f40 msgr2=0x7fdea40d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:11.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.438+0000 7fdea37fe640 1 --2- 172.21.15.89:0/1137071362 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdea40d2f40 0x7fdea40d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:11.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.438+0000 7fdea37fe640 1 -- 172.21.15.89:0/1137071362 >> v2:172.21.15.154:3300/0 conn(0x7fdea415c940 msgr2=0x7fdea40d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:11.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.438+0000 7fdea37fe640 1 --2- 172.21.15.89:0/1137071362 >> v2:172.21.15.154:3300/0 conn(0x7fdea415c940 0x7fdea40d2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:11.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.438+0000 7fdea37fe640 1 -- 172.21.15.89:0/1137071362 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea415d120 con 0x7fdea40d0ca0 2023-10-18T10:46:11.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.442+0000 7fdea37fe640 1 --2- 172.21.15.89:0/1137071362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdea40d0ca0 0x7fdea40cf060 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdea40725b0 tx=0x7fde9c01c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c59be4cb65275edd server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:11.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.442+0000 7fdea1ffb640 1 -- 172.21.15.89:0/1137071362 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9c0ab600 con 0x7fdea40d0ca0 2023-10-18T10:46:11.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.442+0000 7fdea1ffb640 1 -- 172.21.15.89:0/1137071362 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9c0a7050 con 0x7fdea40d0ca0 2023-10-18T10:46:11.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.442+0000 7fdea1ffb640 1 -- 172.21.15.89:0/1137071362 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9c0ab980 con 0x7fdea40d0ca0 2023-10-18T10:46:11.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.442+0000 7fdea9396640 1 -- 172.21.15.89:0/1137071362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdea40d0ca0 msgr2=0x7fdea40cf060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:11.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.442+0000 7fdea9396640 1 --2- 172.21.15.89:0/1137071362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdea40d0ca0 0x7fdea40cf060 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdea40725b0 tx=0x7fde9c01c210 comp rx=0 tx=0).stop 2023-10-18T10:46:11.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.442+0000 7fdea9396640 1 -- 172.21.15.89:0/1137071362 shutdown_connections 2023-10-18T10:46:11.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.442+0000 7fdea9396640 1 --2- 172.21.15.89:0/1137071362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdea40d0ca0 0x7fdea40cf060 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:11.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.442+0000 7fdea9396640 1 --2- 172.21.15.89:0/1137071362 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdea40d2f40 0x7fdea40d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:11.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.442+0000 7fdea9396640 1 --2- 172.21.15.89:0/1137071362 >> v2:172.21.15.154:3300/0 conn(0x7fdea415c940 0x7fdea40d2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:11.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.442+0000 7fdea9396640 1 -- 172.21.15.89:0/1137071362 >> 172.21.15.89:0/1137071362 conn(0x7fdea4089fd0 msgr2=0x7fdea409b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:11.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.442+0000 7fdea9396640 1 -- 172.21.15.89:0/1137071362 shutdown_connections 2023-10-18T10:46:11.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.442+0000 7fdea9396640 1 -- 172.21.15.89:0/1137071362 wait complete. 2023-10-18T10:46:11.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.442+0000 7fdea9396640 1 Processor -- start 2023-10-18T10:46:11.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.442+0000 7fdea9396640 1 -- start start 2023-10-18T10:46:11.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.446+0000 7fdea9396640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdea40d0ca0 0x7fdea4155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:11.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.446+0000 7fdea9396640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdea4156010 0x7fdea40c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:11.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.446+0000 7fdea9396640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdea415f0e0 0x7fdea415f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:11.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.446+0000 7fdea9396640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea415fab0 con 0x7fdea415f0e0 2023-10-18T10:46:11.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.446+0000 7fdea9396640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea40c9720 con 0x7fdea4156010 2023-10-18T10:46:11.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.446+0000 7fdea9396640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea40b5c30 con 0x7fdea40d0ca0 2023-10-18T10:46:11.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.446+0000 7fdea37fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdea415f0e0 0x7fdea415f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:11.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.446+0000 7fdea37fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdea415f0e0 0x7fdea415f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45186/0 (socket says 172.21.15.89:45186) 2023-10-18T10:46:11.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.446+0000 7fdea37fe640 1 -- 172.21.15.89:0/90854380 learned_addr learned my addr 172.21.15.89:0/90854380 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:11.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.446+0000 7fdea2ffd640 1 --2- 172.21.15.89:0/90854380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdea40d0ca0 0x7fdea4155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:11.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.446+0000 7fdea27fc640 1 --2- 172.21.15.89:0/90854380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdea4156010 0x7fdea40c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:11.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.446+0000 7fdea37fe640 1 -- 172.21.15.89:0/90854380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdea40d0ca0 msgr2=0x7fdea4155720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:11.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.446+0000 7fdea37fe640 1 --2- 172.21.15.89:0/90854380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdea40d0ca0 0x7fdea4155720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:11.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.446+0000 7fdea37fe640 1 -- 172.21.15.89:0/90854380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdea4156010 msgr2=0x7fdea40c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:11.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.446+0000 7fdea37fe640 1 --2- 172.21.15.89:0/90854380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdea4156010 0x7fdea40c6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:11.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.446+0000 7fdea37fe640 1 -- 172.21.15.89:0/90854380 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde9c0094d0 con 0x7fdea415f0e0 2023-10-18T10:46:11.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.446+0000 7fdea37fe640 1 --2- 172.21.15.89:0/90854380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdea415f0e0 0x7fdea415f570 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fde9c0ae040 tx=0x7fde9c01aa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:11.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.446+0000 7fde8b7fe640 1 -- 172.21.15.89:0/90854380 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9c0b6420 con 0x7fdea415f0e0 2023-10-18T10:46:11.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.446+0000 7fde8b7fe640 1 -- 172.21.15.89:0/90854380 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9c0b6580 con 0x7fdea415f0e0 2023-10-18T10:46:11.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.446+0000 7fde8b7fe640 1 -- 172.21.15.89:0/90854380 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9c0b68f0 con 0x7fdea415f0e0 2023-10-18T10:46:11.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.446+0000 7fdea9396640 1 -- 172.21.15.89:0/90854380 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdea40b5ed0 con 0x7fdea415f0e0 2023-10-18T10:46:11.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.446+0000 7fdea9396640 1 -- 172.21.15.89:0/90854380 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdea40b6490 con 0x7fdea415f0e0 2023-10-18T10:46:11.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.446+0000 7fdea9396640 1 -- 172.21.15.89:0/90854380 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdea404dbe0 con 0x7fdea415f0e0 2023-10-18T10:46:11.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.446+0000 7fde8b7fe640 1 -- 172.21.15.89:0/90854380 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fde9c0b2040 con 0x7fdea415f0e0 2023-10-18T10:46:11.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.450+0000 7fde8b7fe640 1 --2- 172.21.15.89:0/90854380 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fde900656b0 0x7fde90067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:11.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.450+0000 7fde8b7fe640 1 -- 172.21.15.89:0/90854380 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fde9c11d0b0 con 0x7fdea415f0e0 2023-10-18T10:46:11.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.450+0000 7fdea2ffd640 1 --2- 172.21.15.89:0/90854380 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fde900656b0 0x7fde90067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:11.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.450+0000 7fdea2ffd640 1 --2- 172.21.15.89:0/90854380 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fde900656b0 0x7fde90067b70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdea406a290 tx=0x7fde9400b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:11.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.454+0000 7fde8b7fe640 1 -- 172.21.15.89:0/90854380 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fde9c0ea890 con 0x7fdea415f0e0 2023-10-18T10:46:11.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.602+0000 7fdea9396640 1 -- 172.21.15.89:0/90854380 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdea40b85d0 con 0x7fdea415f0e0 2023-10-18T10:46:11.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.602+0000 7fde8b7fe640 1 -- 172.21.15.89:0/90854380 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20096 (secure 0 0 0) 0x7fde9c0edeb0 con 0x7fdea415f0e0 2023-10-18T10:46:11.605 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:11.606 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":27,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:46:00.166702+0000","last_up_change":"2023-10-18T10:45:42.164688+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:45:53.309122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-18T10:45:54.659763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:45:57.796514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6824","nonce":773078364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6827","nonce":773078364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6831","nonce":773078364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6829","nonce":773078364}]},"public_addr":"172.21.15.154:6824/773078364","cluster_addr":"172.21.15.154:6827/773078364","heartbeat_back_addr":"172.21.15.154:6831/773078364","heartbeat_front_addr":"172.21.15.154:6829/773078364","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:46:11.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.606+0000 7fdea9396640 1 -- 172.21.15.89:0/90854380 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fde900656b0 msgr2=0x7fde90067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:11.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.606+0000 7fdea9396640 1 --2- 172.21.15.89:0/90854380 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fde900656b0 0x7fde90067b70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdea406a290 tx=0x7fde9400b040 comp rx=0 tx=0).stop 2023-10-18T10:46:11.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.606+0000 7fdea9396640 1 -- 172.21.15.89:0/90854380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdea415f0e0 msgr2=0x7fdea415f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:11.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.606+0000 7fdea9396640 1 --2- 172.21.15.89:0/90854380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdea415f0e0 0x7fdea415f570 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fde9c0ae040 tx=0x7fde9c01aa50 comp rx=0 tx=0).stop 2023-10-18T10:46:11.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.606+0000 7fdea9396640 1 -- 172.21.15.89:0/90854380 shutdown_connections 2023-10-18T10:46:11.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.606+0000 7fdea9396640 1 --2- 172.21.15.89:0/90854380 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fde900656b0 0x7fde90067b70 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:11.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.606+0000 7fdea9396640 1 --2- 172.21.15.89:0/90854380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdea415f0e0 0x7fdea415f570 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:11.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.606+0000 7fdea9396640 1 --2- 172.21.15.89:0/90854380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdea4156010 0x7fdea40c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:11.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.606+0000 7fdea9396640 1 --2- 172.21.15.89:0/90854380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdea40d0ca0 0x7fdea4155720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:11.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.606+0000 7fdea9396640 1 -- 172.21.15.89:0/90854380 >> 172.21.15.89:0/90854380 conn(0x7fdea4089fd0 msgr2=0x7fdea415dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:11.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.606+0000 7fdea9396640 1 -- 172.21.15.89:0/90854380 shutdown_connections 2023-10-18T10:46:11.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.606+0000 7fdea9396640 1 -- 172.21.15.89:0/90854380 wait complete. 2023-10-18T10:46:11.627 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:46:11.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.762+0000 7f940344f640 1 Processor -- start 2023-10-18T10:46:11.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.762+0000 7f940344f640 1 -- start start 2023-10-18T10:46:11.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.766+0000 7f940344f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f93fc15c2f0 0x7f93fc0b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:11.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.766+0000 7f940344f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93fc0d0270 0x7f93fc0d06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:11.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.766+0000 7f940344f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f93fc15c6d0 0x7f93fc15cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:11.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.766+0000 7f940344f640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93fc15d1b0 con 0x7f93fc15c6d0 2023-10-18T10:46:11.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.766+0000 7f940344f640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93fc15d490 con 0x7f93fc15c2f0 2023-10-18T10:46:11.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.766+0000 7f940344f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc15d770 con 0x7f93fc0d0270 2023-10-18T10:46:11.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.766+0000 7f94019ec640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f93fc15c6d0 0x7f93fc15cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:11.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.766+0000 7f94019ec640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f93fc15c6d0 0x7f93fc15cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45192/0 (socket says 172.21.15.89:45192) 2023-10-18T10:46:11.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.766+0000 7f94019ec640 1 -- 172.21.15.89:0/480222379 learned_addr learned my addr 172.21.15.89:0/480222379 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:11.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.766+0000 7f94011eb640 1 --2- 172.21.15.89:0/480222379 >> v2:172.21.15.154:3300/0 conn(0x7f93fc15c2f0 0x7f93fc0b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:11.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.766+0000 7f94009ea640 1 --2- 172.21.15.89:0/480222379 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93fc0d0270 0x7f93fc0d06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:11.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.766+0000 7f94019ec640 1 -- 172.21.15.89:0/480222379 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93fc0d0270 msgr2=0x7f93fc0d06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:11.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.766+0000 7f94019ec640 1 --2- 172.21.15.89:0/480222379 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93fc0d0270 0x7f93fc0d06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:11.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.766+0000 7f94019ec640 1 -- 172.21.15.89:0/480222379 >> v2:172.21.15.154:3300/0 conn(0x7f93fc15c2f0 msgr2=0x7f93fc0b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:11.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.766+0000 7f94019ec640 1 --2- 172.21.15.89:0/480222379 >> v2:172.21.15.154:3300/0 conn(0x7f93fc15c2f0 0x7f93fc0b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:11.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.766+0000 7f94019ec640 1 -- 172.21.15.89:0/480222379 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93fc15da50 con 0x7f93fc15c6d0 2023-10-18T10:46:11.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.770+0000 7f94019ec640 1 --2- 172.21.15.89:0/480222379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93fc15c6d0 0x7f93fc15cb20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f93fc0b4250 tx=0x7f93f401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d83f29f0d4d2721 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:11.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.770+0000 7f93f3fff640 1 -- 172.21.15.89:0/480222379 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f40a5840 con 0x7f93fc15c6d0 2023-10-18T10:46:11.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.770+0000 7f93f3fff640 1 -- 172.21.15.89:0/480222379 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f40a59a0 con 0x7f93fc15c6d0 2023-10-18T10:46:11.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.770+0000 7f93f3fff640 1 -- 172.21.15.89:0/480222379 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f40a5d30 con 0x7f93fc15c6d0 2023-10-18T10:46:11.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.770+0000 7f940344f640 1 -- 172.21.15.89:0/480222379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93fc15c6d0 msgr2=0x7f93fc15cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:11.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.770+0000 7f940344f640 1 --2- 172.21.15.89:0/480222379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93fc15c6d0 0x7f93fc15cb20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f93fc0b4250 tx=0x7f93f401c9a0 comp rx=0 tx=0).stop 2023-10-18T10:46:11.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.770+0000 7f940344f640 1 -- 172.21.15.89:0/480222379 shutdown_connections 2023-10-18T10:46:11.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.770+0000 7f940344f640 1 --2- 172.21.15.89:0/480222379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93fc15c6d0 0x7f93fc15cb20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:11.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.770+0000 7f940344f640 1 --2- 172.21.15.89:0/480222379 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93fc0d0270 0x7f93fc0d06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:11.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.770+0000 7f940344f640 1 --2- 172.21.15.89:0/480222379 >> v2:172.21.15.154:3300/0 conn(0x7f93fc15c2f0 0x7f93fc0b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:11.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.770+0000 7f940344f640 1 -- 172.21.15.89:0/480222379 >> 172.21.15.89:0/480222379 conn(0x7f93fc069ad0 msgr2=0x7f93fc069eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:11.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.770+0000 7f940344f640 1 -- 172.21.15.89:0/480222379 shutdown_connections 2023-10-18T10:46:11.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.770+0000 7f940344f640 1 -- 172.21.15.89:0/480222379 wait complete. 2023-10-18T10:46:11.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.770+0000 7f940344f640 1 Processor -- start 2023-10-18T10:46:11.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.770+0000 7f940344f640 1 -- start start 2023-10-18T10:46:11.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.774+0000 7f940344f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93fc0d0270 0x7f93fc0b7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:11.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.774+0000 7f940344f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93fc15c2f0 0x7f93fc0b2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:11.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.774+0000 7f940344f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93fc15c6d0 0x7f93fc0b2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:11.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.774+0000 7f940344f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc0d3000 con 0x7f93fc0d0270 2023-10-18T10:46:11.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.774+0000 7f940344f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc0c7d00 con 0x7f93fc15c2f0 2023-10-18T10:46:11.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.774+0000 7f940344f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc0c7fe0 con 0x7f93fc15c6d0 2023-10-18T10:46:11.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.774+0000 7f94011eb640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93fc0d0270 0x7f93fc0b7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:11.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.774+0000 7f94011eb640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93fc0d0270 0x7f93fc0b7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45200/0 (socket says 172.21.15.89:45200) 2023-10-18T10:46:11.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.774+0000 7f94011eb640 1 -- 172.21.15.89:0/4028525628 learned_addr learned my addr 172.21.15.89:0/4028525628 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:11.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.774+0000 7f94009ea640 1 --2- 172.21.15.89:0/4028525628 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93fc15c2f0 0x7f93fc0b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:11.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.774+0000 7f94019ec640 1 --2- 172.21.15.89:0/4028525628 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93fc15c6d0 0x7f93fc0b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:11.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.774+0000 7f94011eb640 1 -- 172.21.15.89:0/4028525628 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93fc15c6d0 msgr2=0x7f93fc0b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:11.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.774+0000 7f94011eb640 1 --2- 172.21.15.89:0/4028525628 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93fc15c6d0 0x7f93fc0b2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:11.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.774+0000 7f94011eb640 1 -- 172.21.15.89:0/4028525628 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93fc15c2f0 msgr2=0x7f93fc0b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:11.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.774+0000 7f94011eb640 1 --2- 172.21.15.89:0/4028525628 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93fc15c2f0 0x7f93fc0b2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:11.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.774+0000 7f94011eb640 1 -- 172.21.15.89:0/4028525628 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f4009b30 con 0x7f93fc0d0270 2023-10-18T10:46:11.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.774+0000 7f94011eb640 1 --2- 172.21.15.89:0/4028525628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93fc0d0270 0x7f93fc0b7590 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f93f800e930 tx=0x7f93f800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:11.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.774+0000 7f93f1ffb640 1 -- 172.21.15.89:0/4028525628 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f8009690 con 0x7f93fc0d0270 2023-10-18T10:46:11.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.774+0000 7f93f1ffb640 1 -- 172.21.15.89:0/4028525628 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f800ca00 con 0x7f93fc0d0270 2023-10-18T10:46:11.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.774+0000 7f93f1ffb640 1 -- 172.21.15.89:0/4028525628 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f800cd70 con 0x7f93fc0d0270 2023-10-18T10:46:11.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.774+0000 7f940344f640 1 -- 172.21.15.89:0/4028525628 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93fc0c82c0 con 0x7f93fc0d0270 2023-10-18T10:46:11.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.774+0000 7f940344f640 1 -- 172.21.15.89:0/4028525628 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93fc0c85a0 con 0x7f93fc0d0270 2023-10-18T10:46:11.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.774+0000 7f940344f640 1 -- 172.21.15.89:0/4028525628 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93fc066ca0 con 0x7f93fc0d0270 2023-10-18T10:46:11.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.778+0000 7f93f1ffb640 1 -- 172.21.15.89:0/4028525628 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f93f8013470 con 0x7f93fc0d0270 2023-10-18T10:46:11.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.778+0000 7f93f1ffb640 1 --2- 172.21.15.89:0/4028525628 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f93e00656b0 0x7f93e0067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:11.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.778+0000 7f93f1ffb640 1 -- 172.21.15.89:0/4028525628 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f93f80832a0 con 0x7f93fc0d0270 2023-10-18T10:46:11.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.778+0000 7f94009ea640 1 --2- 172.21.15.89:0/4028525628 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f93e00656b0 0x7f93e0067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:11.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.778+0000 7f94009ea640 1 --2- 172.21.15.89:0/4028525628 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f93e00656b0 0x7f93e0067b70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f93fc0c7920 tx=0x7f93ec009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:11.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.782+0000 7f93f1ffb640 1 -- 172.21.15.89:0/4028525628 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f93f8050b40 con 0x7f93fc0d0270 2023-10-18T10:46:11.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.934+0000 7f940344f640 1 -- 172.21.15.89:0/4028525628 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93fc0ce6b0 con 0x7f93fc0d0270 2023-10-18T10:46:11.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.934+0000 7f93f1ffb640 1 -- 172.21.15.89:0/4028525628 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20096 (secure 0 0 0) 0x7f93f8054160 con 0x7f93fc0d0270 2023-10-18T10:46:11.935 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:11.936 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":27,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:46:00.166702+0000","last_up_change":"2023-10-18T10:45:42.164688+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:45:53.309122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-18T10:45:54.659763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:45:57.796514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6824","nonce":773078364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6827","nonce":773078364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6831","nonce":773078364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6829","nonce":773078364}]},"public_addr":"172.21.15.154:6824/773078364","cluster_addr":"172.21.15.154:6827/773078364","heartbeat_back_addr":"172.21.15.154:6831/773078364","heartbeat_front_addr":"172.21.15.154:6829/773078364","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:46:11.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.934+0000 7f940344f640 1 -- 172.21.15.89:0/4028525628 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f93e00656b0 msgr2=0x7f93e0067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:11.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.934+0000 7f940344f640 1 --2- 172.21.15.89:0/4028525628 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f93e00656b0 0x7f93e0067b70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f93fc0c7920 tx=0x7f93ec009190 comp rx=0 tx=0).stop 2023-10-18T10:46:11.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.934+0000 7f940344f640 1 -- 172.21.15.89:0/4028525628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93fc0d0270 msgr2=0x7f93fc0b7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:11.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.934+0000 7f940344f640 1 --2- 172.21.15.89:0/4028525628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93fc0d0270 0x7f93fc0b7590 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f93f800e930 tx=0x7f93f800edf0 comp rx=0 tx=0).stop 2023-10-18T10:46:11.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.934+0000 7f940344f640 1 -- 172.21.15.89:0/4028525628 shutdown_connections 2023-10-18T10:46:11.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.934+0000 7f940344f640 1 --2- 172.21.15.89:0/4028525628 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f93e00656b0 0x7f93e0067b70 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:11.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.934+0000 7f940344f640 1 --2- 172.21.15.89:0/4028525628 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f93fc15c6d0 0x7f93fc0b2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:11.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.934+0000 7f940344f640 1 --2- 172.21.15.89:0/4028525628 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93fc15c2f0 0x7f93fc0b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:11.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.934+0000 7f940344f640 1 --2- 172.21.15.89:0/4028525628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93fc0d0270 0x7f93fc0b7590 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:11.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.934+0000 7f940344f640 1 -- 172.21.15.89:0/4028525628 >> 172.21.15.89:0/4028525628 conn(0x7f93fc069ad0 msgr2=0x7f93fc15e780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:11.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.934+0000 7f940344f640 1 -- 172.21.15.89:0/4028525628 shutdown_connections 2023-10-18T10:46:11.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:11.934+0000 7f940344f640 1 -- 172.21.15.89:0/4028525628 wait complete. 2023-10-18T10:46:11.956 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-18T10:46:12.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.094+0000 7f7233100640 1 Processor -- start 2023-10-18T10:46:12.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.094+0000 7f7233100640 1 -- start start 2023-10-18T10:46:12.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.094+0000 7f7233100640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f722c159490 0x7f722c15d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:12.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.094+0000 7f7233100640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f722c06bee0 0x7f722c0c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:12.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.094+0000 7f7233100640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f722c15ddf0 con 0x7f722c070600 2023-10-18T10:46:12.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.094+0000 7f7233100640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f722c070b90 con 0x7f722c06bee0 2023-10-18T10:46:12.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.094+0000 7f7233100640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f722c070e70 con 0x7f722c159490 2023-10-18T10:46:12.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.098+0000 7f723169d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f722c06bee0 0x7f722c0c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:12.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.098+0000 7f7230e9c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f722c159490 0x7f722c15d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:12.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.098+0000 7f723169d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f722c06bee0 0x7f722c0c8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:49256/0 (socket says 172.21.15.89:49256) 2023-10-18T10:46:12.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.098+0000 7f723169d640 1 -- 172.21.15.89:0/975074527 learned_addr learned my addr 172.21.15.89:0/975074527 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:12.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.098+0000 7f722bfff640 1 --1- 172.21.15.89:0/975074527 >> v1:172.21.15.154:6789/0 conn(0x7f722c070600 0x7f722c06b7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.89:48846/0 (socket says 172.21.15.89:48846) 2023-10-18T10:46:12.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.098+0000 7f722b7fe640 1 -- 172.21.15.89:0/975074527 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3982668549 0 0) 0x7f722c15ddf0 con 0x7f722c070600 2023-10-18T10:46:12.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.098+0000 7f722b7fe640 1 -- 172.21.15.89:0/975074527 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7214003530 con 0x7f722c070600 2023-10-18T10:46:12.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.098+0000 7f723169d640 1 -- 172.21.15.89:0/975074527 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f722c159490 msgr2=0x7f722c15d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:12.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.098+0000 7f723169d640 1 --2- 172.21.15.89:0/975074527 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f722c159490 0x7f722c15d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:12.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.098+0000 7f723169d640 1 -- 172.21.15.89:0/975074527 >> v1:172.21.15.154:6789/0 conn(0x7f722c070600 legacy=0x7f722c06b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:12.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.098+0000 7f723169d640 1 -- 172.21.15.89:0/975074527 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f722c071150 con 0x7f722c06bee0 2023-10-18T10:46:12.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.098+0000 7f7230e9c640 1 --2- 172.21.15.89:0/975074527 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f722c159490 0x7f722c15d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:46:12.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.098+0000 7f723169d640 1 --2- 172.21.15.89:0/975074527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f722c06bee0 0x7f722c0c8960 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7224009870 tx=0x7f722401c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3288a9fb0a7d4ab0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:12.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.102+0000 7f722b7fe640 1 -- 172.21.15.89:0/975074527 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72240ab680 con 0x7f722c06bee0 2023-10-18T10:46:12.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.102+0000 7f722b7fe640 1 -- 172.21.15.89:0/975074527 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72240ab7e0 con 0x7f722c06bee0 2023-10-18T10:46:12.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.102+0000 7f722b7fe640 1 -- 172.21.15.89:0/975074527 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72240abb70 con 0x7f722c06bee0 2023-10-18T10:46:12.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.102+0000 7f7233100640 1 -- 172.21.15.89:0/975074527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f722c06bee0 msgr2=0x7f722c0c8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:12.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.102+0000 7f7233100640 1 --2- 172.21.15.89:0/975074527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f722c06bee0 0x7f722c0c8960 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7224009870 tx=0x7f722401c480 comp rx=0 tx=0).stop 2023-10-18T10:46:12.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.102+0000 7f7233100640 1 -- 172.21.15.89:0/975074527 shutdown_connections 2023-10-18T10:46:12.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.102+0000 7f7233100640 1 --2- 172.21.15.89:0/975074527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f722c06bee0 0x7f722c0c8960 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:12.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.102+0000 7f7233100640 1 --2- 172.21.15.89:0/975074527 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f722c159490 0x7f722c15d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:12.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.102+0000 7f7233100640 1 -- 172.21.15.89:0/975074527 >> 172.21.15.89:0/975074527 conn(0x7f722c069f00 msgr2=0x7f722c0cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:12.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.102+0000 7f7233100640 1 -- 172.21.15.89:0/975074527 shutdown_connections 2023-10-18T10:46:12.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.102+0000 7f7233100640 1 -- 172.21.15.89:0/975074527 wait complete. 2023-10-18T10:46:12.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.102+0000 7f7233100640 1 Processor -- start 2023-10-18T10:46:12.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.102+0000 7f7233100640 1 -- start start 2023-10-18T10:46:12.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.102+0000 7f7233100640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f722c06bee0 0x7f722c153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:12.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.102+0000 7f7233100640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f722c070600 0x7f722c0c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:12.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.102+0000 7f7233100640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f722c159490 0x7f722c0c3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:12.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.102+0000 7f7233100640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722c0c5cf0 con 0x7f722c06bee0 2023-10-18T10:46:12.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.102+0000 7f7233100640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722c0c53e0 con 0x7f722c070600 2023-10-18T10:46:12.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.102+0000 7f7233100640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f722c0c7d60 con 0x7f722c159490 2023-10-18T10:46:12.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.102+0000 7f7230e9c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f722c06bee0 0x7f722c153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:12.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.102+0000 7f7230e9c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f722c06bee0 0x7f722c153860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45214/0 (socket says 172.21.15.89:45214) 2023-10-18T10:46:12.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.102+0000 7f7230e9c640 1 -- 172.21.15.89:0/821196253 learned_addr learned my addr 172.21.15.89:0/821196253 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:12.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.102+0000 7f722bfff640 1 --2- 172.21.15.89:0/821196253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f722c070600 0x7f722c0c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:12.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.106+0000 7f723169d640 1 --2- 172.21.15.89:0/821196253 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f722c159490 0x7f722c0c3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:12.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.106+0000 7f7230e9c640 1 -- 172.21.15.89:0/821196253 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f722c159490 msgr2=0x7f722c0c3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:12.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.106+0000 7f7230e9c640 1 --2- 172.21.15.89:0/821196253 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f722c159490 0x7f722c0c3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:12.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.106+0000 7f7230e9c640 1 -- 172.21.15.89:0/821196253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f722c070600 msgr2=0x7f722c0c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:12.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.106+0000 7f7230e9c640 1 --2- 172.21.15.89:0/821196253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f722c070600 0x7f722c0c7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:12.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.106+0000 7f7230e9c640 1 -- 172.21.15.89:0/821196253 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721c009a70 con 0x7f722c06bee0 2023-10-18T10:46:12.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.106+0000 7f7230e9c640 1 --2- 172.21.15.89:0/821196253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f722c06bee0 0x7f722c153860 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f721c00efc0 tx=0x7f721c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:12.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.106+0000 7f72297fa640 1 -- 172.21.15.89:0/821196253 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c014680 con 0x7f722c06bee0 2023-10-18T10:46:12.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.106+0000 7f72297fa640 1 -- 172.21.15.89:0/821196253 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721c00f040 con 0x7f722c06bee0 2023-10-18T10:46:12.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.106+0000 7f72297fa640 1 -- 172.21.15.89:0/821196253 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c014a00 con 0x7f722c06bee0 2023-10-18T10:46:12.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.106+0000 7f7233100640 1 -- 172.21.15.89:0/821196253 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72240094d0 con 0x7f722c06bee0 2023-10-18T10:46:12.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.106+0000 7f7233100640 1 -- 172.21.15.89:0/821196253 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f722c0c5980 con 0x7f722c06bee0 2023-10-18T10:46:12.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.106+0000 7f7233100640 1 -- 172.21.15.89:0/821196253 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f722c04dbe0 con 0x7f722c06bee0 2023-10-18T10:46:12.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.106+0000 7f72297fa640 1 -- 172.21.15.89:0/821196253 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f721c014b60 con 0x7f722c06bee0 2023-10-18T10:46:12.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.106+0000 7f72297fa640 1 --2- 172.21.15.89:0/821196253 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7218065680 0x7f7218067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:12.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.106+0000 7f72297fa640 1 -- 172.21.15.89:0/821196253 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f721c083ed0 con 0x7f722c06bee0 2023-10-18T10:46:12.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.110+0000 7f722bfff640 1 --2- 172.21.15.89:0/821196253 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7218065680 0x7f7218067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:12.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.110+0000 7f722bfff640 1 --2- 172.21.15.89:0/821196253 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7218065680 0x7f7218067b40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f722c0c1bd0 tx=0x7f7220007d40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:12.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.110+0000 7f72297fa640 1 -- 172.21.15.89:0/821196253 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f721c0517e0 con 0x7f722c06bee0 2023-10-18T10:46:12.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.290+0000 7f7233100640 1 -- 172.21.15.89:0/821196253 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f722c0cbb60 con 0x7f722c06bee0 2023-10-18T10:46:12.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.830+0000 7f72297fa640 1 -- 172.21.15.89:0/821196253 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7f721c054e00 con 0x7f722c06bee0 2023-10-18T10:46:12.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.834+0000 7f7233100640 1 -- 172.21.15.89:0/821196253 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7218065680 msgr2=0x7f7218067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:12.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.834+0000 7f7233100640 1 --2- 172.21.15.89:0/821196253 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7218065680 0x7f7218067b40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f722c0c1bd0 tx=0x7f7220007d40 comp rx=0 tx=0).stop 2023-10-18T10:46:12.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.834+0000 7f7233100640 1 -- 172.21.15.89:0/821196253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f722c06bee0 msgr2=0x7f722c153860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:12.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.834+0000 7f7233100640 1 --2- 172.21.15.89:0/821196253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f722c06bee0 0x7f722c153860 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f721c00efc0 tx=0x7f721c00c490 comp rx=0 tx=0).stop 2023-10-18T10:46:12.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.834+0000 7f7233100640 1 -- 172.21.15.89:0/821196253 shutdown_connections 2023-10-18T10:46:12.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.834+0000 7f7233100640 1 --2- 172.21.15.89:0/821196253 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7218065680 0x7f7218067b40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:12.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.834+0000 7f7233100640 1 --2- 172.21.15.89:0/821196253 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f722c159490 0x7f722c0c3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:12.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.834+0000 7f7233100640 1 --2- 172.21.15.89:0/821196253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f722c070600 0x7f722c0c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:12.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.834+0000 7f7233100640 1 --2- 172.21.15.89:0/821196253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f722c06bee0 0x7f722c153860 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:12.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.834+0000 7f7233100640 1 -- 172.21.15.89:0/821196253 >> 172.21.15.89:0/821196253 conn(0x7f722c069f00 msgr2=0x7f722c0af530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:12.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.834+0000 7f7233100640 1 -- 172.21.15.89:0/821196253 shutdown_connections 2023-10-18T10:46:12.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.834+0000 7f7233100640 1 -- 172.21.15.89:0/821196253 wait complete. 2023-10-18T10:46:12.857 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2023-10-18T10:46:12.858 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-18T10:46:13.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.998+0000 7f6d5a8ca640 1 Processor -- start 2023-10-18T10:46:13.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:12.998+0000 7f6d5a8ca640 1 -- start start 2023-10-18T10:46:13.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.002+0000 7f6d5a8ca640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6d5415c5d0 0x7f6d540d1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:13.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.002+0000 7f6d5a8ca640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6d540cd860 0x7f6d540d1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:13.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.002+0000 7f6d5a8ca640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6d540cee70 0x7f6d540cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:13.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.002+0000 7f6d5a8ca640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d540d15c0 con 0x7f6d540cee70 2023-10-18T10:46:13.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.002+0000 7f6d5a8ca640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d540cf820 con 0x7f6d540cd860 2023-10-18T10:46:13.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.002+0000 7f6d5a8ca640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d540d0520 con 0x7f6d5415c5d0 2023-10-18T10:46:13.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.002+0000 7f6d5a0c9640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6d540cee70 0x7f6d540cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:13.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.002+0000 7f6d5a0c9640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6d540cee70 0x7f6d540cf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44982/0 (socket says 172.21.15.89:44982) 2023-10-18T10:46:13.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.002+0000 7f6d5a0c9640 1 -- 172.21.15.89:0/423313953 learned_addr learned my addr 172.21.15.89:0/423313953 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:13.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.002+0000 7f6d590c7640 1 --2- 172.21.15.89:0/423313953 >> v2:172.21.15.154:3300/0 conn(0x7f6d540cd860 0x7f6d540d1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:13.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.002+0000 7f6d598c8640 1 --2- 172.21.15.89:0/423313953 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6d5415c5d0 0x7f6d540d1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:13.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.002+0000 7f6d5a0c9640 1 -- 172.21.15.89:0/423313953 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6d5415c5d0 msgr2=0x7f6d540d1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:13.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.002+0000 7f6d5a0c9640 1 --2- 172.21.15.89:0/423313953 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6d5415c5d0 0x7f6d540d1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:13.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.002+0000 7f6d5a0c9640 1 -- 172.21.15.89:0/423313953 >> v2:172.21.15.154:3300/0 conn(0x7f6d540cd860 msgr2=0x7f6d540d1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:13.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.002+0000 7f6d5a0c9640 1 --2- 172.21.15.89:0/423313953 >> v2:172.21.15.154:3300/0 conn(0x7f6d540cd860 0x7f6d540d1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:13.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.002+0000 7f6d5a0c9640 1 -- 172.21.15.89:0/423313953 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d540d0800 con 0x7f6d540cee70 2023-10-18T10:46:13.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.002+0000 7f6d598c8640 1 --2- 172.21.15.89:0/423313953 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6d5415c5d0 0x7f6d540d1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:13.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.006+0000 7f6d5a0c9640 1 --2- 172.21.15.89:0/423313953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d540cee70 0x7f6d540cf2e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6d5406a9d0 tx=0x7f6d4c01f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b9a35548400734 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:13.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.006+0000 7f6d588c6640 1 -- 172.21.15.89:0/423313953 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c0af400 con 0x7f6d540cee70 2023-10-18T10:46:13.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.006+0000 7f6d588c6640 1 -- 172.21.15.89:0/423313953 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d4c0ab050 con 0x7f6d540cee70 2023-10-18T10:46:13.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.006+0000 7f6d588c6640 1 -- 172.21.15.89:0/423313953 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c0af780 con 0x7f6d540cee70 2023-10-18T10:46:13.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.006+0000 7f6d5a8ca640 1 -- 172.21.15.89:0/423313953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d540cee70 msgr2=0x7f6d540cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:13.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.006+0000 7f6d5a8ca640 1 --2- 172.21.15.89:0/423313953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d540cee70 0x7f6d540cf2e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6d5406a9d0 tx=0x7f6d4c01f1c0 comp rx=0 tx=0).stop 2023-10-18T10:46:13.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.006+0000 7f6d5a8ca640 1 -- 172.21.15.89:0/423313953 shutdown_connections 2023-10-18T10:46:13.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.006+0000 7f6d5a8ca640 1 --2- 172.21.15.89:0/423313953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d540cee70 0x7f6d540cf2e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:13.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.006+0000 7f6d5a8ca640 1 --2- 172.21.15.89:0/423313953 >> v2:172.21.15.154:3300/0 conn(0x7f6d540cd860 0x7f6d540d1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:13.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.006+0000 7f6d5a8ca640 1 --2- 172.21.15.89:0/423313953 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6d5415c5d0 0x7f6d540d1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:13.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.006+0000 7f6d5a8ca640 1 -- 172.21.15.89:0/423313953 >> 172.21.15.89:0/423313953 conn(0x7f6d540b3f50 msgr2=0x7f6d540b4360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:13.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.006+0000 7f6d5a8ca640 1 -- 172.21.15.89:0/423313953 shutdown_connections 2023-10-18T10:46:13.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.006+0000 7f6d5a8ca640 1 -- 172.21.15.89:0/423313953 wait complete. 2023-10-18T10:46:13.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.006+0000 7f6d5a8ca640 1 Processor -- start 2023-10-18T10:46:13.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.006+0000 7f6d5a8ca640 1 -- start start 2023-10-18T10:46:13.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.010+0000 7f6d5a8ca640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d540cd860 0x7f6d541f25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:13.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.010+0000 7f6d5a8ca640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6d540cee70 0x7f6d540cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:13.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.010+0000 7f6d5a8ca640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d5415c5d0 0x7f6d540cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:13.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.010+0000 7f6d5a8ca640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d54159440 con 0x7f6d5415c5d0 2023-10-18T10:46:13.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.010+0000 7f6d5a8ca640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d540cc970 con 0x7f6d540cd860 2023-10-18T10:46:13.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.010+0000 7f6d5a8ca640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d540c96d0 con 0x7f6d540cee70 2023-10-18T10:46:13.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.010+0000 7f6d5a0c9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d5415c5d0 0x7f6d540cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:13.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.010+0000 7f6d5a0c9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d5415c5d0 0x7f6d540cc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44994/0 (socket says 172.21.15.89:44994) 2023-10-18T10:46:13.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.010+0000 7f6d5a0c9640 1 -- 172.21.15.89:0/2818757448 learned_addr learned my addr 172.21.15.89:0/2818757448 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:13.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.010+0000 7f6d590c7640 1 --2- 172.21.15.89:0/2818757448 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6d540cee70 0x7f6d540cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:13.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.010+0000 7f6d598c8640 1 --2- 172.21.15.89:0/2818757448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d540cd860 0x7f6d541f25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:13.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.010+0000 7f6d5a0c9640 1 -- 172.21.15.89:0/2818757448 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6d540cee70 msgr2=0x7f6d540cbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:13.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.010+0000 7f6d5a0c9640 1 --2- 172.21.15.89:0/2818757448 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6d540cee70 0x7f6d540cbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:13.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.010+0000 7f6d5a0c9640 1 -- 172.21.15.89:0/2818757448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d540cd860 msgr2=0x7f6d541f25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:13.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.010+0000 7f6d5a0c9640 1 --2- 172.21.15.89:0/2818757448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d540cd860 0x7f6d541f25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:13.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.010+0000 7f6d5a0c9640 1 -- 172.21.15.89:0/2818757448 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d4c009b30 con 0x7f6d5415c5d0 2023-10-18T10:46:13.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.010+0000 7f6d5a0c9640 1 --2- 172.21.15.89:0/2818757448 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d5415c5d0 0x7f6d540cc430 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c01d870 tx=0x7f6d4c01bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:13.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.010+0000 7f6d4a7fc640 1 -- 172.21.15.89:0/2818757448 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c0c0070 con 0x7f6d5415c5d0 2023-10-18T10:46:13.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.010+0000 7f6d4a7fc640 1 -- 172.21.15.89:0/2818757448 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d4c009510 con 0x7f6d5415c5d0 2023-10-18T10:46:13.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.010+0000 7f6d4a7fc640 1 -- 172.21.15.89:0/2818757448 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c0afcf0 con 0x7f6d5415c5d0 2023-10-18T10:46:13.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.010+0000 7f6d5a8ca640 1 -- 172.21.15.89:0/2818757448 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d540c99b0 con 0x7f6d5415c5d0 2023-10-18T10:46:13.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.010+0000 7f6d5a8ca640 1 -- 172.21.15.89:0/2818757448 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d540c9f70 con 0x7f6d5415c5d0 2023-10-18T10:46:13.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.010+0000 7f6d5a8ca640 1 -- 172.21.15.89:0/2818757448 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d54073b50 con 0x7f6d5415c5d0 2023-10-18T10:46:13.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.010+0000 7f6d4a7fc640 1 -- 172.21.15.89:0/2818757448 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f6d4c0b6040 con 0x7f6d5415c5d0 2023-10-18T10:46:13.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.010+0000 7f6d4a7fc640 1 --2- 172.21.15.89:0/2818757448 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6d38065680 0x7f6d38067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:13.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.010+0000 7f6d4a7fc640 1 -- 172.21.15.89:0/2818757448 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6d4c120a90 con 0x7f6d5415c5d0 2023-10-18T10:46:13.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.014+0000 7f6d598c8640 1 --2- 172.21.15.89:0/2818757448 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6d38065680 0x7f6d38067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:13.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.014+0000 7f6d598c8640 1 --2- 172.21.15.89:0/2818757448 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6d38065680 0x7f6d38067b40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6d540b5f70 tx=0x7f6d50008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:13.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.014+0000 7f6d4a7fc640 1 -- 172.21.15.89:0/2818757448 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f6d4c0f1da0 con 0x7f6d5415c5d0 2023-10-18T10:46:13.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.198+0000 7f6d5a8ca640 1 -- 172.21.15.89:0/2818757448 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6d540cf5e0 con 0x7f6d5415c5d0 2023-10-18T10:46:13.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.834+0000 7f6d4a7fc640 1 -- 172.21.15.89:0/2818757448 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6d4c0f1930 con 0x7f6d5415c5d0 2023-10-18T10:46:13.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.838+0000 7f6d5a8ca640 1 -- 172.21.15.89:0/2818757448 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6d38065680 msgr2=0x7f6d38067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:13.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.838+0000 7f6d5a8ca640 1 --2- 172.21.15.89:0/2818757448 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6d38065680 0x7f6d38067b40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6d540b5f70 tx=0x7f6d50008040 comp rx=0 tx=0).stop 2023-10-18T10:46:13.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.838+0000 7f6d5a8ca640 1 -- 172.21.15.89:0/2818757448 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d5415c5d0 msgr2=0x7f6d540cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:13.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.838+0000 7f6d5a8ca640 1 --2- 172.21.15.89:0/2818757448 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d5415c5d0 0x7f6d540cc430 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c01d870 tx=0x7f6d4c01bf60 comp rx=0 tx=0).stop 2023-10-18T10:46:13.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.838+0000 7f6d5a8ca640 1 -- 172.21.15.89:0/2818757448 shutdown_connections 2023-10-18T10:46:13.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.838+0000 7f6d5a8ca640 1 --2- 172.21.15.89:0/2818757448 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6d38065680 0x7f6d38067b40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:13.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.838+0000 7f6d5a8ca640 1 --2- 172.21.15.89:0/2818757448 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6d540cee70 0x7f6d540cbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:13.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.838+0000 7f6d5a8ca640 1 --2- 172.21.15.89:0/2818757448 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d5415c5d0 0x7f6d540cc430 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:13.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.838+0000 7f6d5a8ca640 1 --2- 172.21.15.89:0/2818757448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d540cd860 0x7f6d541f25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:13.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.838+0000 7f6d5a8ca640 1 -- 172.21.15.89:0/2818757448 >> 172.21.15.89:0/2818757448 conn(0x7f6d540b3f50 msgr2=0x7f6d540d1720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:13.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.838+0000 7f6d5a8ca640 1 -- 172.21.15.89:0/2818757448 shutdown_connections 2023-10-18T10:46:13.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:13.838+0000 7f6d5a8ca640 1 -- 172.21.15.89:0/2818757448 wait complete. 2023-10-18T10:46:13.861 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-18T10:46:13.862 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:46:14.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.002+0000 7fdf6e9d5640 1 Processor -- start 2023-10-18T10:46:14.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.002+0000 7fdf6e9d5640 1 -- start start 2023-10-18T10:46:14.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.002+0000 7fdf6e9d5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdf6815c7f0 0x7fdf6806b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:14.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.002+0000 7fdf6e9d5640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf68161160 con 0x7fdf680ad5b0 2023-10-18T10:46:14.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.002+0000 7fdf6e9d5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf681605c0 con 0x7fdf680afd90 2023-10-18T10:46:14.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.002+0000 7fdf6e9d5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf6806c000 con 0x7fdf6815c7f0 2023-10-18T10:46:14.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.006+0000 7fdf6cf72640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fdf680ad5b0 0x7fdf68160a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35480/0 (socket says 172.21.15.89:35480) 2023-10-18T10:46:14.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.006+0000 7fdf6cf72640 1 -- 172.21.15.89:0/1751185278 learned_addr learned my addr 172.21.15.89:0/1751185278 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:14.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.006+0000 7fdf677fe640 1 --2- 172.21.15.89:0/1751185278 >> v2:172.21.15.154:3300/0 conn(0x7fdf6815c7f0 0x7fdf6806b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:14.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.006+0000 7fdf66ffd640 1 -- 172.21.15.89:0/1751185278 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2203811065 0 0) 0x7fdf600014a0 con 0x7fdf680ad5b0 2023-10-18T10:46:14.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.006+0000 7fdf66ffd640 1 -- 172.21.15.89:0/1751185278 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf50003530 con 0x7fdf680ad5b0 2023-10-18T10:46:14.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.006+0000 7fdf677fe640 1 -- 172.21.15.89:0/1751185278 >> v1:172.21.15.154:6789/0 conn(0x7fdf680afd90 legacy=0x7fdf680b0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:14.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.006+0000 7fdf677fe640 1 -- 172.21.15.89:0/1751185278 >> v1:172.21.15.89:6789/0 conn(0x7fdf680ad5b0 legacy=0x7fdf68160a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:14.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.006+0000 7fdf677fe640 1 -- 172.21.15.89:0/1751185278 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf6806c2e0 con 0x7fdf6815c7f0 2023-10-18T10:46:14.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.006+0000 7fdf677fe640 1 --2- 172.21.15.89:0/1751185278 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf6815c7f0 0x7fdf6806b7d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c0097c0 tx=0x7fdf5c01c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=502c5eb06149095b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:14.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.010+0000 7fdf66ffd640 1 -- 172.21.15.89:0/1751185278 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c0ab600 con 0x7fdf6815c7f0 2023-10-18T10:46:14.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.010+0000 7fdf66ffd640 1 -- 172.21.15.89:0/1751185278 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5c0a7050 con 0x7fdf6815c7f0 2023-10-18T10:46:14.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.010+0000 7fdf66ffd640 1 -- 172.21.15.89:0/1751185278 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c0ab960 con 0x7fdf6815c7f0 2023-10-18T10:46:14.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.010+0000 7fdf6e9d5640 1 -- 172.21.15.89:0/1751185278 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf6815c7f0 msgr2=0x7fdf6806b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:14.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.010+0000 7fdf6e9d5640 1 --2- 172.21.15.89:0/1751185278 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf6815c7f0 0x7fdf6806b7d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c0097c0 tx=0x7fdf5c01c3f0 comp rx=0 tx=0).stop 2023-10-18T10:46:14.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.010+0000 7fdf6e9d5640 1 -- 172.21.15.89:0/1751185278 shutdown_connections 2023-10-18T10:46:14.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.010+0000 7fdf6e9d5640 1 --2- 172.21.15.89:0/1751185278 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf6815c7f0 0x7fdf6806b7d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:14.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.010+0000 7fdf6e9d5640 1 -- 172.21.15.89:0/1751185278 >> 172.21.15.89:0/1751185278 conn(0x7fdf68069f00 msgr2=0x7fdf680cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:14.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.010+0000 7fdf6e9d5640 1 -- 172.21.15.89:0/1751185278 shutdown_connections 2023-10-18T10:46:14.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.010+0000 7fdf6e9d5640 1 -- 172.21.15.89:0/1751185278 wait complete. 2023-10-18T10:46:14.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.010+0000 7fdf6e9d5640 1 Processor -- start 2023-10-18T10:46:14.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.010+0000 7fdf6e9d5640 1 -- start start 2023-10-18T10:46:14.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.010+0000 7fdf6e9d5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf680ad5b0 0x7fdf680712f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:14.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.010+0000 7fdf6e9d5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf680afd90 0x7fdf680c48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:14.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.010+0000 7fdf6e9d5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf6815c7f0 0x7fdf680ac030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:14.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.010+0000 7fdf6e9d5640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf680b0f50 con 0x7fdf6815c7f0 2023-10-18T10:46:14.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.010+0000 7fdf6e9d5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6814aa60 con 0x7fdf680afd90 2023-10-18T10:46:14.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.010+0000 7fdf6e9d5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf68071900 con 0x7fdf680ad5b0 2023-10-18T10:46:14.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.014+0000 7fdf6cf72640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf6815c7f0 0x7fdf680ac030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:14.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.014+0000 7fdf6cf72640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf6815c7f0 0x7fdf680ac030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45012/0 (socket says 172.21.15.89:45012) 2023-10-18T10:46:14.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.014+0000 7fdf6cf72640 1 -- 172.21.15.89:0/1348435746 learned_addr learned my addr 172.21.15.89:0/1348435746 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:14.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.014+0000 7fdf677fe640 1 --2- 172.21.15.89:0/1348435746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf680afd90 0x7fdf680c48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:14.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.014+0000 7fdf67fff640 1 --2- 172.21.15.89:0/1348435746 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf680ad5b0 0x7fdf680712f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:14.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.014+0000 7fdf6cf72640 1 -- 172.21.15.89:0/1348435746 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf680ad5b0 msgr2=0x7fdf680712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:14.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.014+0000 7fdf6cf72640 1 --2- 172.21.15.89:0/1348435746 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf680ad5b0 0x7fdf680712f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:14.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.014+0000 7fdf6cf72640 1 -- 172.21.15.89:0/1348435746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf680afd90 msgr2=0x7fdf680c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:14.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.014+0000 7fdf6cf72640 1 --2- 172.21.15.89:0/1348435746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf680afd90 0x7fdf680c48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:14.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.014+0000 7fdf6cf72640 1 -- 172.21.15.89:0/1348435746 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5c009420 con 0x7fdf6815c7f0 2023-10-18T10:46:14.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.014+0000 7fdf6cf72640 1 --2- 172.21.15.89:0/1348435746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf6815c7f0 0x7fdf680ac030 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000caa0 tx=0x7fdf6000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:14.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.014+0000 7fdf64ff9640 1 -- 172.21.15.89:0/1348435746 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6000aae0 con 0x7fdf6815c7f0 2023-10-18T10:46:14.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.014+0000 7fdf64ff9640 1 -- 172.21.15.89:0/1348435746 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf6000ac40 con 0x7fdf6815c7f0 2023-10-18T10:46:14.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.014+0000 7fdf6e9d5640 1 -- 172.21.15.89:0/1348435746 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf6806ffb0 con 0x7fdf6815c7f0 2023-10-18T10:46:14.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.014+0000 7fdf64ff9640 1 -- 172.21.15.89:0/1348435746 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf60012400 con 0x7fdf6815c7f0 2023-10-18T10:46:14.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.014+0000 7fdf6e9d5640 1 -- 172.21.15.89:0/1348435746 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf68070570 con 0x7fdf6815c7f0 2023-10-18T10:46:14.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.014+0000 7fdf6e9d5640 1 -- 172.21.15.89:0/1348435746 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf6804dbe0 con 0x7fdf6815c7f0 2023-10-18T10:46:14.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.014+0000 7fdf64ff9640 1 -- 172.21.15.89:0/1348435746 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fdf60007070 con 0x7fdf6815c7f0 2023-10-18T10:46:14.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.018+0000 7fdf64ff9640 1 --2- 172.21.15.89:0/1348435746 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdf54065680 0x7fdf54067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:14.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.018+0000 7fdf64ff9640 1 -- 172.21.15.89:0/1348435746 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdf60082c60 con 0x7fdf6815c7f0 2023-10-18T10:46:14.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.018+0000 7fdf67fff640 1 --2- 172.21.15.89:0/1348435746 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdf54065680 0x7fdf54067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:14.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.018+0000 7fdf67fff640 1 --2- 172.21.15.89:0/1348435746 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdf54065680 0x7fdf54067b40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdf58007fd0 tx=0x7fdf58009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:14.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.018+0000 7fdf64ff9640 1 -- 172.21.15.89:0/1348435746 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fdf600504d0 con 0x7fdf6815c7f0 2023-10-18T10:46:14.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.198+0000 7fdf6e9d5640 1 -- 172.21.15.89:0/1348435746 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf68161160 con 0x7fdf6815c7f0 2023-10-18T10:46:14.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.202+0000 7fdf64ff9640 1 -- 172.21.15.89:0/1348435746 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4874 (secure 0 0 0) 0x7fdf68161160 con 0x7fdf6815c7f0 2023-10-18T10:46:14.202 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:14.202 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/1247973498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1247973498},{"type":"v1","addr":"172.21.15.154:6836","nonce":1247973498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4284,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/516057316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":516057316},{"type":"v1","addr":"172.21.15.154:6837","nonce":516057316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4299,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6834/3957051407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3957051407},{"type":"v1","addr":"172.21.15.89:6834","nonce":3957051407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:45:55.655016+0000","modified":"2023-10-18T10:46:13.833215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4312":{"gid":4312,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6835/2906113039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6833","nonce":2906113039},{"type":"v1","addr":"172.21.15.89:6835","nonce":2906113039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T10:46:14.203 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 9 2023-10-18T10:46:14.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.202+0000 7fdf6e9d5640 1 -- 172.21.15.89:0/1348435746 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdf54065680 msgr2=0x7fdf54067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:14.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.202+0000 7fdf6e9d5640 1 --2- 172.21.15.89:0/1348435746 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdf54065680 0x7fdf54067b40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdf58007fd0 tx=0x7fdf58009040 comp rx=0 tx=0).stop 2023-10-18T10:46:14.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.202+0000 7fdf6e9d5640 1 -- 172.21.15.89:0/1348435746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf6815c7f0 msgr2=0x7fdf680ac030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:14.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.202+0000 7fdf6e9d5640 1 --2- 172.21.15.89:0/1348435746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf6815c7f0 0x7fdf680ac030 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000caa0 tx=0x7fdf6000cf60 comp rx=0 tx=0).stop 2023-10-18T10:46:14.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.202+0000 7fdf6e9d5640 1 -- 172.21.15.89:0/1348435746 shutdown_connections 2023-10-18T10:46:14.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.202+0000 7fdf6e9d5640 1 --2- 172.21.15.89:0/1348435746 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdf54065680 0x7fdf54067b40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:14.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.202+0000 7fdf6e9d5640 1 --2- 172.21.15.89:0/1348435746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf6815c7f0 0x7fdf680ac030 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:14.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.202+0000 7fdf6e9d5640 1 --2- 172.21.15.89:0/1348435746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf680afd90 0x7fdf680c48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:14.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.202+0000 7fdf6e9d5640 1 --2- 172.21.15.89:0/1348435746 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf680ad5b0 0x7fdf680712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:14.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.202+0000 7fdf6e9d5640 1 -- 172.21.15.89:0/1348435746 >> 172.21.15.89:0/1348435746 conn(0x7fdf68069f00 msgr2=0x7fdf68157680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:14.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.202+0000 7fdf6e9d5640 1 -- 172.21.15.89:0/1348435746 shutdown_connections 2023-10-18T10:46:14.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.202+0000 7fdf6e9d5640 1 -- 172.21.15.89:0/1348435746 wait complete. 2023-10-18T10:46:14.223 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:46:14.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef63093640 1 Processor -- start 2023-10-18T10:46:14.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef63093640 1 -- start start 2023-10-18T10:46:14.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef63093640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fef5c15c940 0x7fef5c0d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:14.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef63093640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fef5c0d0e70 0x7fef5c0cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:14.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef63093640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef5c15da50 con 0x7fef5c0d2f40 2023-10-18T10:46:14.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef63093640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef5c15ce40 con 0x7fef5c15c940 2023-10-18T10:46:14.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef63093640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef5c15d120 con 0x7fef5c0d0e70 2023-10-18T10:46:14.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef5bfff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fef5c0d2f40 0x7fef5c0d0760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35490/0 (socket says 172.21.15.89:35490) 2023-10-18T10:46:14.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef5bfff640 1 -- 172.21.15.89:0/330757439 learned_addr learned my addr 172.21.15.89:0/330757439 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:14.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef60e2f640 1 --2- 172.21.15.89:0/330757439 >> v2:172.21.15.154:3300/0 conn(0x7fef5c15c940 0x7fef5c0d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:14.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef61630640 1 --2- 172.21.15.89:0/330757439 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fef5c0d0e70 0x7fef5c0cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:14.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef5b7fe640 1 -- 172.21.15.89:0/330757439 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3011038403 0 0) 0x7fef5c15da50 con 0x7fef5c0d2f40 2023-10-18T10:46:14.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef5b7fe640 1 -- 172.21.15.89:0/330757439 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef4c003530 con 0x7fef5c0d2f40 2023-10-18T10:46:14.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef5b7fe640 1 -- 172.21.15.89:0/330757439 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2077125472 0 0) 0x7fef4c003530 con 0x7fef5c0d2f40 2023-10-18T10:46:14.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef5b7fe640 1 -- 172.21.15.89:0/330757439 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fef5c0d0e70 msgr2=0x7fef5c0cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:14.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef5b7fe640 1 --2- 172.21.15.89:0/330757439 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fef5c0d0e70 0x7fef5c0cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:14.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef5b7fe640 1 -- 172.21.15.89:0/330757439 >> v2:172.21.15.154:3300/0 conn(0x7fef5c15c940 msgr2=0x7fef5c0d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:14.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef5b7fe640 1 --2- 172.21.15.89:0/330757439 >> v2:172.21.15.154:3300/0 conn(0x7fef5c15c940 0x7fef5c0d2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:14.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef5b7fe640 1 -- 172.21.15.89:0/330757439 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef5c15d400 con 0x7fef5c0d2f40 2023-10-18T10:46:14.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef5b7fe640 1 -- 172.21.15.89:0/330757439 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7fef48001680 con 0x7fef5c0d2f40 2023-10-18T10:46:14.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef5b7fe640 1 -- 172.21.15.89:0/330757439 >> v1:172.21.15.89:6789/0 conn(0x7fef5c0d2f40 legacy=0x7fef5c0d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:14.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef5b7fe640 1 --2- 172.21.15.89:0/330757439 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef4c003980 0x7fef4c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:14.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef5b7fe640 1 --2- 172.21.15.89:0/330757439 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fef4c0047d0 0x7fef4c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:14.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef5b7fe640 1 --2- 172.21.15.89:0/330757439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef4c005190 0x7fef4c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:14.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef5b7fe640 1 -- 172.21.15.89:0/330757439 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5c15ce40 con 0x7fef4c005190 2023-10-18T10:46:14.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef5b7fe640 1 -- 172.21.15.89:0/330757439 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5c15d120 con 0x7fef4c003980 2023-10-18T10:46:14.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef5b7fe640 1 -- 172.21.15.89:0/330757439 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c00bed0 con 0x7fef4c0047d0 2023-10-18T10:46:14.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef61630640 1 --2- 172.21.15.89:0/330757439 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fef5c0d0e70 0x7fef5c0cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:46:14.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef60e2f640 1 --2- 172.21.15.89:0/330757439 >> v2:172.21.15.154:3300/0 conn(0x7fef5c15c940 0x7fef5c0d2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:46:14.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef61630640 1 --2- 172.21.15.89:0/330757439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef4c005190 0x7fef4c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:14.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef5bfff640 1 --2- 172.21.15.89:0/330757439 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fef4c0047d0 0x7fef4c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:14.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef60e2f640 1 --2- 172.21.15.89:0/330757439 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef4c003980 0x7fef4c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:14.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef61630640 1 -- 172.21.15.89:0/330757439 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fef4c0047d0 msgr2=0x7fef4c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:14.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef61630640 1 --2- 172.21.15.89:0/330757439 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fef4c0047d0 0x7fef4c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:14.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef61630640 1 -- 172.21.15.89:0/330757439 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef4c003980 msgr2=0x7fef4c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:14.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef61630640 1 --2- 172.21.15.89:0/330757439 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef4c003980 0x7fef4c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:14.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef61630640 1 -- 172.21.15.89:0/330757439 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef4c003530 con 0x7fef4c005190 2023-10-18T10:46:14.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.362+0000 7fef5bfff640 1 --2- 172.21.15.89:0/330757439 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fef4c0047d0 0x7fef4c004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:14.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.366+0000 7fef61630640 1 --2- 172.21.15.89:0/330757439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef4c005190 0x7fef4c00b990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fef5c0b85a0 tx=0x7fef5401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0165efed916b486 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:14.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.366+0000 7fef5b7fe640 1 -- 172.21.15.89:0/330757439 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef540a5cd0 con 0x7fef4c005190 2023-10-18T10:46:14.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.366+0000 7fef5b7fe640 1 -- 172.21.15.89:0/330757439 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5400f040 con 0x7fef4c005190 2023-10-18T10:46:14.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.366+0000 7fef5b7fe640 1 -- 172.21.15.89:0/330757439 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef54003bc0 con 0x7fef4c005190 2023-10-18T10:46:14.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.366+0000 7fef63093640 1 -- 172.21.15.89:0/330757439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef4c005190 msgr2=0x7fef4c00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:14.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.366+0000 7fef63093640 1 --2- 172.21.15.89:0/330757439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef4c005190 0x7fef4c00b990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fef5c0b85a0 tx=0x7fef5401cc80 comp rx=0 tx=0).stop 2023-10-18T10:46:14.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.366+0000 7fef61630640 1 -- 172.21.15.89:0/330757439 reap_dead start 2023-10-18T10:46:14.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.366+0000 7fef63093640 1 -- 172.21.15.89:0/330757439 shutdown_connections 2023-10-18T10:46:14.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.366+0000 7fef63093640 1 -- 172.21.15.89:0/330757439 >> 172.21.15.89:0/330757439 conn(0x7fef5c089fd0 msgr2=0x7fef5c09b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:14.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.366+0000 7fef63093640 1 -- 172.21.15.89:0/330757439 shutdown_connections 2023-10-18T10:46:14.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.366+0000 7fef63093640 1 -- 172.21.15.89:0/330757439 wait complete. 2023-10-18T10:46:14.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.370+0000 7fef63093640 1 Processor -- start 2023-10-18T10:46:14.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.370+0000 7fef63093640 1 -- start start 2023-10-18T10:46:14.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.370+0000 7fef63093640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef4c005190 0x7fef5c0c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:14.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.370+0000 7fef63093640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef5c148890 0x7fef5c14bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:14.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.370+0000 7fef63093640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fef5c0cf040 0x7fef5c0cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:14.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.370+0000 7fef63093640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5c0cfa00 con 0x7fef5c148890 2023-10-18T10:46:14.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.370+0000 7fef63093640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5c0c8520 con 0x7fef4c005190 2023-10-18T10:46:14.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.370+0000 7fef63093640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef5c0c8800 con 0x7fef5c0cf040 2023-10-18T10:46:14.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.370+0000 7fef5bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef5c148890 0x7fef5c14bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:14.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.370+0000 7fef5bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef5c148890 0x7fef5c14bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45032/0 (socket says 172.21.15.89:45032) 2023-10-18T10:46:14.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.370+0000 7fef5bfff640 1 -- 172.21.15.89:0/3239250863 learned_addr learned my addr 172.21.15.89:0/3239250863 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:14.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.370+0000 7fef60e2f640 1 --2- 172.21.15.89:0/3239250863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef4c005190 0x7fef5c0c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:14.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.370+0000 7fef61630640 1 --2- 172.21.15.89:0/3239250863 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fef5c0cf040 0x7fef5c0cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:14.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.370+0000 7fef5bfff640 1 -- 172.21.15.89:0/3239250863 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fef5c0cf040 msgr2=0x7fef5c0cf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:14.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.370+0000 7fef5bfff640 1 --2- 172.21.15.89:0/3239250863 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fef5c0cf040 0x7fef5c0cf4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:14.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.370+0000 7fef5bfff640 1 -- 172.21.15.89:0/3239250863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef4c005190 msgr2=0x7fef5c0c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:14.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.370+0000 7fef5bfff640 1 --2- 172.21.15.89:0/3239250863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef4c005190 0x7fef5c0c7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:14.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.370+0000 7fef5bfff640 1 -- 172.21.15.89:0/3239250863 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef5c0c8ae0 con 0x7fef5c148890 2023-10-18T10:46:14.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.370+0000 7fef60e2f640 1 --2- 172.21.15.89:0/3239250863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef4c005190 0x7fef5c0c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:14.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.370+0000 7fef5bfff640 1 --2- 172.21.15.89:0/3239250863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef5c148890 0x7fef5c14bb30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fef480017b0 tx=0x7fef4800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:14.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.370+0000 7fef597fa640 1 -- 172.21.15.89:0/3239250863 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4800c8d0 con 0x7fef5c148890 2023-10-18T10:46:14.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.370+0000 7fef597fa640 1 -- 172.21.15.89:0/3239250863 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef4800ca30 con 0x7fef5c148890 2023-10-18T10:46:14.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.370+0000 7fef597fa640 1 -- 172.21.15.89:0/3239250863 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4800cda0 con 0x7fef5c148890 2023-10-18T10:46:14.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.370+0000 7fef63093640 1 -- 172.21.15.89:0/3239250863 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef5c0c8dc0 con 0x7fef5c148890 2023-10-18T10:46:14.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.370+0000 7fef63093640 1 -- 172.21.15.89:0/3239250863 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef5c14c070 con 0x7fef5c148890 2023-10-18T10:46:14.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.374+0000 7fef63093640 1 -- 172.21.15.89:0/3239250863 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef5c0d0ce0 con 0x7fef5c148890 2023-10-18T10:46:14.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.374+0000 7fef597fa640 1 -- 172.21.15.89:0/3239250863 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fef4801d460 con 0x7fef5c148890 2023-10-18T10:46:14.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.374+0000 7fef597fa640 1 --2- 172.21.15.89:0/3239250863 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fef400656b0 0x7fef40067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:14.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.374+0000 7fef597fa640 1 -- 172.21.15.89:0/3239250863 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fef480830a0 con 0x7fef5c148890 2023-10-18T10:46:14.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.374+0000 7fef60e2f640 1 --2- 172.21.15.89:0/3239250863 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fef400656b0 0x7fef40067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:14.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.374+0000 7fef60e2f640 1 --2- 172.21.15.89:0/3239250863 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fef400656b0 0x7fef40067b70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fef50009c80 tx=0x7fef50008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:14.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.378+0000 7fef597fa640 1 -- 172.21.15.89:0/3239250863 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fef48050870 con 0x7fef5c148890 2023-10-18T10:46:14.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.526+0000 7fef63093640 1 -- 172.21.15.89:0/3239250863 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef5c0c9570 con 0x7fef5c148890 2023-10-18T10:46:14.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.530+0000 7fef597fa640 1 -- 172.21.15.89:0/3239250863 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20096 (secure 0 0 0) 0x7fef48053e90 con 0x7fef5c148890 2023-10-18T10:46:14.530 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:14.531 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":27,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:46:00.166702+0000","last_up_change":"2023-10-18T10:45:42.164688+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:45:53.309122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-18T10:45:54.659763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:45:57.796514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6824","nonce":773078364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6827","nonce":773078364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6831","nonce":773078364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6829","nonce":773078364}]},"public_addr":"172.21.15.154:6824/773078364","cluster_addr":"172.21.15.154:6827/773078364","heartbeat_back_addr":"172.21.15.154:6831/773078364","heartbeat_front_addr":"172.21.15.154:6829/773078364","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:46:14.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.530+0000 7fef63093640 1 -- 172.21.15.89:0/3239250863 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fef400656b0 msgr2=0x7fef40067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:14.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.530+0000 7fef63093640 1 --2- 172.21.15.89:0/3239250863 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fef400656b0 0x7fef40067b70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fef50009c80 tx=0x7fef50008040 comp rx=0 tx=0).stop 2023-10-18T10:46:14.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.530+0000 7fef63093640 1 -- 172.21.15.89:0/3239250863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef5c148890 msgr2=0x7fef5c14bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:14.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.530+0000 7fef63093640 1 --2- 172.21.15.89:0/3239250863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef5c148890 0x7fef5c14bb30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fef480017b0 tx=0x7fef4800ec80 comp rx=0 tx=0).stop 2023-10-18T10:46:14.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.530+0000 7fef63093640 1 -- 172.21.15.89:0/3239250863 shutdown_connections 2023-10-18T10:46:14.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.530+0000 7fef63093640 1 --2- 172.21.15.89:0/3239250863 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fef400656b0 0x7fef40067b70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:14.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.530+0000 7fef63093640 1 --2- 172.21.15.89:0/3239250863 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fef5c0cf040 0x7fef5c0cf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:14.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.530+0000 7fef63093640 1 --2- 172.21.15.89:0/3239250863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef5c148890 0x7fef5c14bb30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:14.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.530+0000 7fef63093640 1 --2- 172.21.15.89:0/3239250863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef4c005190 0x7fef5c0c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:14.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.530+0000 7fef63093640 1 -- 172.21.15.89:0/3239250863 >> 172.21.15.89:0/3239250863 conn(0x7fef5c089fd0 msgr2=0x7fef5c15ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:14.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.530+0000 7fef63093640 1 -- 172.21.15.89:0/3239250863 shutdown_connections 2023-10-18T10:46:14.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.530+0000 7fef63093640 1 -- 172.21.15.89:0/3239250863 wait complete. 2023-10-18T10:46:14.551 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-18T10:46:14.552 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:46:14.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.686+0000 7fcfa9093640 1 Processor -- start 2023-10-18T10:46:14.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.686+0000 7fcfa9093640 1 -- start start 2023-10-18T10:46:14.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.686+0000 7fcfa9093640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcfa40cd4f0 0x7fcfa4143470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:14.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.686+0000 7fcfa9093640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcfa4145b70 0x7fcfa4160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:14.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.686+0000 7fcfa9093640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfa40cd8d0 con 0x7fcfa40ce070 2023-10-18T10:46:14.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.686+0000 7fcfa9093640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfa4069900 con 0x7fcfa4145b70 2023-10-18T10:46:14.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.686+0000 7fcfa9093640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfa415cbf0 con 0x7fcfa40cd4f0 2023-10-18T10:46:14.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.686+0000 7fcf9a59c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcfa4145b70 0x7fcfa4160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:14.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.690+0000 7fcf9a59c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcfa4145b70 0x7fcfa4160cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45044/0 (socket says 172.21.15.89:45044) 2023-10-18T10:46:14.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.690+0000 7fcf9a59c640 1 -- 172.21.15.89:0/754396451 learned_addr learned my addr 172.21.15.89:0/754396451 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:14.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.690+0000 7fcfa2d9d640 1 --2- 172.21.15.89:0/754396451 >> v2:172.21.15.154:3300/0 conn(0x7fcfa40cd4f0 0x7fcfa4143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:14.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.690+0000 7fcf9a59c640 1 -- 172.21.15.89:0/754396451 >> v2:172.21.15.154:3300/0 conn(0x7fcfa40cd4f0 msgr2=0x7fcfa4143470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:14.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.690+0000 7fcf9a59c640 1 --2- 172.21.15.89:0/754396451 >> v2:172.21.15.154:3300/0 conn(0x7fcfa40cd4f0 0x7fcfa4143470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:14.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.690+0000 7fcf9a59c640 1 -- 172.21.15.89:0/754396451 >> v1:172.21.15.89:6789/0 conn(0x7fcfa40ce070 legacy=0x7fcfa406abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:14.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.690+0000 7fcf9a59c640 1 -- 172.21.15.89:0/754396451 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa4161230 con 0x7fcfa4145b70 2023-10-18T10:46:14.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.690+0000 7fcf9a59c640 1 --2- 172.21.15.89:0/754396451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfa4145b70 0x7fcfa4160cf0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcfa40716d0 tx=0x7fcf8c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f48ec443bfb754a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:14.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.690+0000 7fcfa259c640 1 -- 172.21.15.89:0/754396451 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c0aa430 con 0x7fcfa4145b70 2023-10-18T10:46:14.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.690+0000 7fcfa259c640 1 -- 172.21.15.89:0/754396451 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf8c0aa590 con 0x7fcfa4145b70 2023-10-18T10:46:14.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.690+0000 7fcfa259c640 1 -- 172.21.15.89:0/754396451 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c0aa920 con 0x7fcfa4145b70 2023-10-18T10:46:14.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.694+0000 7fcfa9093640 1 -- 172.21.15.89:0/754396451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfa4145b70 msgr2=0x7fcfa4160cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:14.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.694+0000 7fcfa9093640 1 --2- 172.21.15.89:0/754396451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfa4145b70 0x7fcfa4160cf0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcfa40716d0 tx=0x7fcf8c01c4d0 comp rx=0 tx=0).stop 2023-10-18T10:46:14.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.694+0000 7fcfa9093640 1 -- 172.21.15.89:0/754396451 shutdown_connections 2023-10-18T10:46:14.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.694+0000 7fcfa9093640 1 --2- 172.21.15.89:0/754396451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfa4145b70 0x7fcfa4160cf0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:14.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.694+0000 7fcfa9093640 1 --2- 172.21.15.89:0/754396451 >> v2:172.21.15.154:3300/0 conn(0x7fcfa40cd4f0 0x7fcfa4143470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:14.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.694+0000 7fcfa9093640 1 -- 172.21.15.89:0/754396451 >> 172.21.15.89:0/754396451 conn(0x7fcfa406dad0 msgr2=0x7fcfa406dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:14.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.694+0000 7fcfa9093640 1 -- 172.21.15.89:0/754396451 shutdown_connections 2023-10-18T10:46:14.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.694+0000 7fcfa9093640 1 -- 172.21.15.89:0/754396451 wait complete. 2023-10-18T10:46:14.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.694+0000 7fcfa9093640 1 Processor -- start 2023-10-18T10:46:14.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.694+0000 7fcfa9093640 1 -- start start 2023-10-18T10:46:14.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.694+0000 7fcfa9093640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfa40cd4f0 0x7fcfa40c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:14.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.694+0000 7fcfa9093640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcfa40ce070 0x7fcfa40b3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:14.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.694+0000 7fcfa9093640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfa4145b70 0x7fcfa40b3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:14.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.694+0000 7fcfa9093640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa40c2020 con 0x7fcfa40cd4f0 2023-10-18T10:46:14.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.694+0000 7fcfa9093640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa40c16f0 con 0x7fcfa4145b70 2023-10-18T10:46:14.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.694+0000 7fcfa9093640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa40c19d0 con 0x7fcfa40ce070 2023-10-18T10:46:14.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.694+0000 7fcfa2d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfa40cd4f0 0x7fcfa40c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:14.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.694+0000 7fcfa2d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfa40cd4f0 0x7fcfa40c9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45052/0 (socket says 172.21.15.89:45052) 2023-10-18T10:46:14.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.694+0000 7fcfa2d9d640 1 -- 172.21.15.89:0/1380011618 learned_addr learned my addr 172.21.15.89:0/1380011618 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:14.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.694+0000 7fcf9a59c640 1 --2- 172.21.15.89:0/1380011618 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcfa40ce070 0x7fcfa40b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:14.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.694+0000 7fcfa359e640 1 --2- 172.21.15.89:0/1380011618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfa4145b70 0x7fcfa40b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:14.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.694+0000 7fcfa2d9d640 1 -- 172.21.15.89:0/1380011618 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcfa40ce070 msgr2=0x7fcfa40b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:14.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.694+0000 7fcfa2d9d640 1 --2- 172.21.15.89:0/1380011618 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcfa40ce070 0x7fcfa40b3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:14.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.694+0000 7fcfa2d9d640 1 -- 172.21.15.89:0/1380011618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfa4145b70 msgr2=0x7fcfa40b3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:14.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.694+0000 7fcfa2d9d640 1 --2- 172.21.15.89:0/1380011618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfa4145b70 0x7fcfa40b3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:14.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.694+0000 7fcfa2d9d640 1 -- 172.21.15.89:0/1380011618 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf8c0094d0 con 0x7fcfa40cd4f0 2023-10-18T10:46:14.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.694+0000 7fcf9a59c640 1 --2- 172.21.15.89:0/1380011618 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcfa40ce070 0x7fcfa40b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:14.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.694+0000 7fcfa359e640 1 --2- 172.21.15.89:0/1380011618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfa4145b70 0x7fcfa40b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:14.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.698+0000 7fcfa2d9d640 1 --2- 172.21.15.89:0/1380011618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfa40cd4f0 0x7fcfa40c9ac0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400e870 tx=0x7fcf9400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:14.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.698+0000 7fcf9bfff640 1 -- 172.21.15.89:0/1380011618 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf94009510 con 0x7fcfa40cd4f0 2023-10-18T10:46:14.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.698+0000 7fcf9bfff640 1 -- 172.21.15.89:0/1380011618 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf9400eec0 con 0x7fcfa40cd4f0 2023-10-18T10:46:14.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.698+0000 7fcfa9093640 1 -- 172.21.15.89:0/1380011618 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfa40c1cb0 con 0x7fcfa40cd4f0 2023-10-18T10:46:14.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.698+0000 7fcf9bfff640 1 -- 172.21.15.89:0/1380011618 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9400cb40 con 0x7fcfa40cd4f0 2023-10-18T10:46:14.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.698+0000 7fcfa9093640 1 -- 172.21.15.89:0/1380011618 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa4154980 con 0x7fcfa40cd4f0 2023-10-18T10:46:14.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.698+0000 7fcfa9093640 1 -- 172.21.15.89:0/1380011618 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfa40d1410 con 0x7fcfa40cd4f0 2023-10-18T10:46:14.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.698+0000 7fcf9bfff640 1 -- 172.21.15.89:0/1380011618 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fcf9400cca0 con 0x7fcfa40cd4f0 2023-10-18T10:46:14.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.698+0000 7fcf9bfff640 1 --2- 172.21.15.89:0/1380011618 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fcf840656b0 0x7fcf84067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:14.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.698+0000 7fcf9bfff640 1 -- 172.21.15.89:0/1380011618 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fcf94083280 con 0x7fcfa40cd4f0 2023-10-18T10:46:14.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.702+0000 7fcf9a59c640 1 --2- 172.21.15.89:0/1380011618 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fcf840656b0 0x7fcf84067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:14.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.702+0000 7fcf9a59c640 1 --2- 172.21.15.89:0/1380011618 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fcf840656b0 0x7fcf84067b70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c0ae000 tx=0x7fcf8c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:14.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.702+0000 7fcf9bfff640 1 -- 172.21.15.89:0/1380011618 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fcf940509b0 con 0x7fcfa40cd4f0 2023-10-18T10:46:14.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.850+0000 7fcfa9093640 1 -- 172.21.15.89:0/1380011618 --> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcfa40c1860 con 0x7fcf840656b0 2023-10-18T10:46:14.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.862+0000 7fcf9bfff640 1 -- 172.21.15.89:0/1380011618 <== mgr.4099 v2:172.21.15.154:6832/1086608968 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367424 (secure 0 0 0) 0x7fcfa40c1860 con 0x7fcf840656b0 2023-10-18T10:46:14.870 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:14.872 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-18T10:46:14.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.866+0000 7fcfa9093640 1 -- 172.21.15.89:0/1380011618 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fcf840656b0 msgr2=0x7fcf84067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:14.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.866+0000 7fcfa9093640 1 --2- 172.21.15.89:0/1380011618 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fcf840656b0 0x7fcf84067b70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c0ae000 tx=0x7fcf8c008040 comp rx=0 tx=0).stop 2023-10-18T10:46:14.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.866+0000 7fcfa9093640 1 -- 172.21.15.89:0/1380011618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfa40cd4f0 msgr2=0x7fcfa40c9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:14.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.866+0000 7fcfa9093640 1 --2- 172.21.15.89:0/1380011618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfa40cd4f0 0x7fcfa40c9ac0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400e870 tx=0x7fcf9400ec70 comp rx=0 tx=0).stop 2023-10-18T10:46:14.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.866+0000 7fcfa9093640 1 -- 172.21.15.89:0/1380011618 shutdown_connections 2023-10-18T10:46:14.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.866+0000 7fcfa9093640 1 --2- 172.21.15.89:0/1380011618 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fcf840656b0 0x7fcf84067b70 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:14.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.866+0000 7fcfa9093640 1 --2- 172.21.15.89:0/1380011618 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcfa40ce070 0x7fcfa40b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:14.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.866+0000 7fcfa9093640 1 --2- 172.21.15.89:0/1380011618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfa4145b70 0x7fcfa40b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:14.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.866+0000 7fcfa9093640 1 --2- 172.21.15.89:0/1380011618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfa40cd4f0 0x7fcfa40c9ac0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:14.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.866+0000 7fcfa9093640 1 -- 172.21.15.89:0/1380011618 >> 172.21.15.89:0/1380011618 conn(0x7fcfa406dad0 msgr2=0x7fcfa40cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:14.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.870+0000 7fcfa9093640 1 -- 172.21.15.89:0/1380011618 shutdown_connections 2023-10-18T10:46:14.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:14.870+0000 7fcfa9093640 1 -- 172.21.15.89:0/1380011618 wait complete. 2023-10-18T10:46:14.889 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2023-10-18T10:46:13.056875+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219660,"kb_used_data":3496,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754755060,"statfs":{"total":773094113280,"available":772869181440,"internally_reserved":0,"allocated":3579904,"data_stored":2149222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2229,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003137"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084532+0000","last_change":"2023-10-18T10:45:59.809013+0000","last_active":"2023-10-18T10:46:01.084532+0000","last_peered":"2023-10-18T10:46:01.084532+0000","last_clean":"2023-10-18T10:46:01.084532+0000","last_became_active":"2023-10-18T10:45:59.808636+0000","last_became_peered":"2023-10-18T10:45:59.808636+0000","last_unstale":"2023-10-18T10:46:01.084532+0000","last_undegraded":"2023-10-18T10:46:01.084532+0000","last_fullsized":"2023-10-18T10:46:01.084532+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:30:08.797534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085907+0000","last_change":"2023-10-18T10:45:55.297617+0000","last_active":"2023-10-18T10:46:01.085907+0000","last_peered":"2023-10-18T10:46:01.085907+0000","last_clean":"2023-10-18T10:46:01.085907+0000","last_became_active":"2023-10-18T10:45:55.296996+0000","last_became_peered":"2023-10-18T10:45:55.296996+0000","last_unstale":"2023-10-18T10:46:01.085907+0000","last_undegraded":"2023-10-18T10:46:01.085907+0000","last_fullsized":"2023-10-18T10:46:01.085907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:04:27.847910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768182+0000","last_change":"2023-10-18T10:45:55.770859+0000","last_active":"2023-10-18T10:46:00.768182+0000","last_peered":"2023-10-18T10:46:00.768182+0000","last_clean":"2023-10-18T10:46:00.768182+0000","last_became_active":"2023-10-18T10:45:55.769578+0000","last_became_peered":"2023-10-18T10:45:55.769578+0000","last_unstale":"2023-10-18T10:46:00.768182+0000","last_undegraded":"2023-10-18T10:46:00.768182+0000","last_fullsized":"2023-10-18T10:46:00.768182+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:08:17.812888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.767991+0000","last_change":"2023-10-18T10:45:59.790837+0000","last_active":"2023-10-18T10:46:00.767991+0000","last_peered":"2023-10-18T10:46:00.767991+0000","last_clean":"2023-10-18T10:46:00.767991+0000","last_became_active":"2023-10-18T10:45:59.790488+0000","last_became_peered":"2023-10-18T10:45:59.790488+0000","last_unstale":"2023-10-18T10:46:00.767991+0000","last_undegraded":"2023-10-18T10:46:00.767991+0000","last_fullsized":"2023-10-18T10:46:00.767991+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:43:23.609561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902666+0000","last_change":"2023-10-18T10:45:55.295114+0000","last_active":"2023-10-18T10:46:00.902666+0000","last_peered":"2023-10-18T10:46:00.902666+0000","last_clean":"2023-10-18T10:46:00.902666+0000","last_became_active":"2023-10-18T10:45:55.294672+0000","last_became_peered":"2023-10-18T10:45:55.294672+0000","last_unstale":"2023-10-18T10:46:00.902666+0000","last_undegraded":"2023-10-18T10:46:00.902666+0000","last_fullsized":"2023-10-18T10:46:00.902666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:06:43.162381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902162+0000","last_change":"2023-10-18T10:45:55.756885+0000","last_active":"2023-10-18T10:46:00.902162+0000","last_peered":"2023-10-18T10:46:00.902162+0000","last_clean":"2023-10-18T10:46:00.902162+0000","last_became_active":"2023-10-18T10:45:55.756184+0000","last_became_peered":"2023-10-18T10:45:55.756184+0000","last_unstale":"2023-10-18T10:46:00.902162+0000","last_undegraded":"2023-10-18T10:46:00.902162+0000","last_fullsized":"2023-10-18T10:46:00.902162+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:20:46.641913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085021+0000","last_change":"2023-10-18T10:45:59.796520+0000","last_active":"2023-10-18T10:46:01.085021+0000","last_peered":"2023-10-18T10:46:01.085021+0000","last_clean":"2023-10-18T10:46:01.085021+0000","last_became_active":"2023-10-18T10:45:59.796056+0000","last_became_peered":"2023-10-18T10:45:59.796056+0000","last_unstale":"2023-10-18T10:46:01.085021+0000","last_undegraded":"2023-10-18T10:46:01.085021+0000","last_fullsized":"2023-10-18T10:46:01.085021+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:55:12.299657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085752+0000","last_change":"2023-10-18T10:45:55.302985+0000","last_active":"2023-10-18T10:46:01.085752+0000","last_peered":"2023-10-18T10:46:01.085752+0000","last_clean":"2023-10-18T10:46:01.085752+0000","last_became_active":"2023-10-18T10:45:55.302657+0000","last_became_peered":"2023-10-18T10:45:55.302657+0000","last_unstale":"2023-10-18T10:46:01.085752+0000","last_undegraded":"2023-10-18T10:46:01.085752+0000","last_fullsized":"2023-10-18T10:46:01.085752+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:55:03.412092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768360+0000","last_change":"2023-10-18T10:45:55.776143+0000","last_active":"2023-10-18T10:46:00.768360+0000","last_peered":"2023-10-18T10:46:00.768360+0000","last_clean":"2023-10-18T10:46:00.768360+0000","last_became_active":"2023-10-18T10:45:55.770969+0000","last_became_peered":"2023-10-18T10:45:55.770969+0000","last_unstale":"2023-10-18T10:46:00.768360+0000","last_undegraded":"2023-10-18T10:46:00.768360+0000","last_fullsized":"2023-10-18T10:46:00.768360+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:53:59.036438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085289+0000","last_change":"2023-10-18T10:45:59.810699+0000","last_active":"2023-10-18T10:46:01.085289+0000","last_peered":"2023-10-18T10:46:01.085289+0000","last_clean":"2023-10-18T10:46:01.085289+0000","last_became_active":"2023-10-18T10:45:59.810219+0000","last_became_peered":"2023-10-18T10:45:59.810219+0000","last_unstale":"2023-10-18T10:46:01.085289+0000","last_undegraded":"2023-10-18T10:46:01.085289+0000","last_fullsized":"2023-10-18T10:46:01.085289+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:06:30.723321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085485+0000","last_change":"2023-10-18T10:45:55.300287+0000","last_active":"2023-10-18T10:46:01.085485+0000","last_peered":"2023-10-18T10:46:01.085485+0000","last_clean":"2023-10-18T10:46:01.085485+0000","last_became_active":"2023-10-18T10:45:55.300014+0000","last_became_peered":"2023-10-18T10:45:55.300014+0000","last_unstale":"2023-10-18T10:46:01.085485+0000","last_undegraded":"2023-10-18T10:46:01.085485+0000","last_fullsized":"2023-10-18T10:46:01.085485+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:40:14.773073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768476+0000","last_change":"2023-10-18T10:45:55.772481+0000","last_active":"2023-10-18T10:46:00.768476+0000","last_peered":"2023-10-18T10:46:00.768476+0000","last_clean":"2023-10-18T10:46:00.768476+0000","last_became_active":"2023-10-18T10:45:55.770365+0000","last_became_peered":"2023-10-18T10:45:55.770365+0000","last_unstale":"2023-10-18T10:46:00.768476+0000","last_undegraded":"2023-10-18T10:46:00.768476+0000","last_fullsized":"2023-10-18T10:46:00.768476+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:21:41.323647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085226+0000","last_change":"2023-10-18T10:45:59.796654+0000","last_active":"2023-10-18T10:46:01.085226+0000","last_peered":"2023-10-18T10:46:01.085226+0000","last_clean":"2023-10-18T10:46:01.085226+0000","last_became_active":"2023-10-18T10:45:59.796115+0000","last_became_peered":"2023-10-18T10:45:59.796115+0000","last_unstale":"2023-10-18T10:46:01.085226+0000","last_undegraded":"2023-10-18T10:46:01.085226+0000","last_fullsized":"2023-10-18T10:46:01.085226+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:28:08.869152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086358+0000","last_change":"2023-10-18T10:45:55.302642+0000","last_active":"2023-10-18T10:46:01.086358+0000","last_peered":"2023-10-18T10:46:01.086358+0000","last_clean":"2023-10-18T10:46:01.086358+0000","last_became_active":"2023-10-18T10:45:55.302307+0000","last_became_peered":"2023-10-18T10:45:55.302307+0000","last_unstale":"2023-10-18T10:46:01.086358+0000","last_undegraded":"2023-10-18T10:46:01.086358+0000","last_fullsized":"2023-10-18T10:46:01.086358+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:40:52.185453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.767207+0000","last_change":"2023-10-18T10:45:55.765319+0000","last_active":"2023-10-18T10:46:00.767207+0000","last_peered":"2023-10-18T10:46:00.767207+0000","last_clean":"2023-10-18T10:46:00.767207+0000","last_became_active":"2023-10-18T10:45:55.764958+0000","last_became_peered":"2023-10-18T10:45:55.764958+0000","last_unstale":"2023-10-18T10:46:00.767207+0000","last_undegraded":"2023-10-18T10:46:00.767207+0000","last_fullsized":"2023-10-18T10:46:00.767207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:30:49.094396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768337+0000","last_change":"2023-10-18T10:45:59.791333+0000","last_active":"2023-10-18T10:46:00.768337+0000","last_peered":"2023-10-18T10:46:00.768337+0000","last_clean":"2023-10-18T10:46:00.768337+0000","last_became_active":"2023-10-18T10:45:59.790903+0000","last_became_peered":"2023-10-18T10:45:59.790903+0000","last_unstale":"2023-10-18T10:46:00.768337+0000","last_undegraded":"2023-10-18T10:46:00.768337+0000","last_fullsized":"2023-10-18T10:46:00.768337+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:52:13.857250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768597+0000","last_change":"2023-10-18T10:45:55.302330+0000","last_active":"2023-10-18T10:46:00.768597+0000","last_peered":"2023-10-18T10:46:00.768597+0000","last_clean":"2023-10-18T10:46:00.768597+0000","last_became_active":"2023-10-18T10:45:55.301709+0000","last_became_peered":"2023-10-18T10:45:55.301709+0000","last_unstale":"2023-10-18T10:46:00.768597+0000","last_undegraded":"2023-10-18T10:46:00.768597+0000","last_fullsized":"2023-10-18T10:46:00.768597+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:40:49.128175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902373+0000","last_change":"2023-10-18T10:45:55.767859+0000","last_active":"2023-10-18T10:46:00.902373+0000","last_peered":"2023-10-18T10:46:00.902373+0000","last_clean":"2023-10-18T10:46:00.902373+0000","last_became_active":"2023-10-18T10:45:55.767505+0000","last_became_peered":"2023-10-18T10:45:55.767505+0000","last_unstale":"2023-10-18T10:46:00.902373+0000","last_undegraded":"2023-10-18T10:46:00.902373+0000","last_fullsized":"2023-10-18T10:46:00.902373+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:12:23.654089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.770164+0000","last_change":"2023-10-18T10:45:59.794892+0000","last_active":"2023-10-18T10:46:00.770164+0000","last_peered":"2023-10-18T10:46:00.770164+0000","last_clean":"2023-10-18T10:46:00.770164+0000","last_became_active":"2023-10-18T10:45:59.794477+0000","last_became_peered":"2023-10-18T10:45:59.794477+0000","last_unstale":"2023-10-18T10:46:00.770164+0000","last_undegraded":"2023-10-18T10:46:00.770164+0000","last_fullsized":"2023-10-18T10:46:00.770164+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:04:07.040979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084932+0000","last_change":"2023-10-18T10:45:55.290491+0000","last_active":"2023-10-18T10:46:01.084932+0000","last_peered":"2023-10-18T10:46:01.084932+0000","last_clean":"2023-10-18T10:46:01.084932+0000","last_became_active":"2023-10-18T10:45:55.290134+0000","last_became_peered":"2023-10-18T10:45:55.290134+0000","last_unstale":"2023-10-18T10:46:01.084932+0000","last_undegraded":"2023-10-18T10:46:01.084932+0000","last_fullsized":"2023-10-18T10:46:01.084932+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:39.977541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086354+0000","last_change":"2023-10-18T10:45:55.765415+0000","last_active":"2023-10-18T10:46:01.086354+0000","last_peered":"2023-10-18T10:46:01.086354+0000","last_clean":"2023-10-18T10:46:01.086354+0000","last_became_active":"2023-10-18T10:45:55.764573+0000","last_became_peered":"2023-10-18T10:45:55.764573+0000","last_unstale":"2023-10-18T10:46:01.086354+0000","last_undegraded":"2023-10-18T10:46:01.086354+0000","last_fullsized":"2023-10-18T10:46:01.086354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:33:49.751994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086225+0000","last_change":"2023-10-18T10:45:59.801842+0000","last_active":"2023-10-18T10:46:01.086225+0000","last_peered":"2023-10-18T10:46:01.086225+0000","last_clean":"2023-10-18T10:46:01.086225+0000","last_became_active":"2023-10-18T10:45:59.801367+0000","last_became_peered":"2023-10-18T10:45:59.801367+0000","last_unstale":"2023-10-18T10:46:01.086225+0000","last_undegraded":"2023-10-18T10:46:01.086225+0000","last_fullsized":"2023-10-18T10:46:01.086225+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:56:46.630947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902154+0000","last_change":"2023-10-18T10:45:55.298611+0000","last_active":"2023-10-18T10:46:00.902154+0000","last_peered":"2023-10-18T10:46:00.902154+0000","last_clean":"2023-10-18T10:46:00.902154+0000","last_became_active":"2023-10-18T10:45:55.297980+0000","last_became_peered":"2023-10-18T10:45:55.297980+0000","last_unstale":"2023-10-18T10:46:00.902154+0000","last_undegraded":"2023-10-18T10:46:00.902154+0000","last_fullsized":"2023-10-18T10:46:00.902154+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:05:56.466612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.770285+0000","last_change":"2023-10-18T10:45:55.775435+0000","last_active":"2023-10-18T10:46:00.770285+0000","last_peered":"2023-10-18T10:46:00.770285+0000","last_clean":"2023-10-18T10:46:00.770285+0000","last_became_active":"2023-10-18T10:45:55.774841+0000","last_became_peered":"2023-10-18T10:45:55.774841+0000","last_unstale":"2023-10-18T10:46:00.770285+0000","last_undegraded":"2023-10-18T10:46:00.770285+0000","last_fullsized":"2023-10-18T10:46:00.770285+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:27:03.226674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.901875+0000","last_change":"2023-10-18T10:45:55.768221+0000","last_active":"2023-10-18T10:46:00.901875+0000","last_peered":"2023-10-18T10:46:00.901875+0000","last_clean":"2023-10-18T10:46:00.901875+0000","last_became_active":"2023-10-18T10:45:55.767975+0000","last_became_peered":"2023-10-18T10:45:55.767975+0000","last_unstale":"2023-10-18T10:46:00.901875+0000","last_undegraded":"2023-10-18T10:46:00.901875+0000","last_fullsized":"2023-10-18T10:46:00.901875+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:14:00.849685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086003+0000","last_change":"2023-10-18T10:45:55.291798+0000","last_active":"2023-10-18T10:46:01.086003+0000","last_peered":"2023-10-18T10:46:01.086003+0000","last_clean":"2023-10-18T10:46:01.086003+0000","last_became_active":"2023-10-18T10:45:55.291380+0000","last_became_peered":"2023-10-18T10:45:55.291380+0000","last_unstale":"2023-10-18T10:46:01.086003+0000","last_undegraded":"2023-10-18T10:46:01.086003+0000","last_fullsized":"2023-10-18T10:46:01.086003+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:49:45.673807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.769758+0000","last_change":"2023-10-18T10:45:59.794170+0000","last_active":"2023-10-18T10:46:00.769758+0000","last_peered":"2023-10-18T10:46:00.769758+0000","last_clean":"2023-10-18T10:46:00.769758+0000","last_became_active":"2023-10-18T10:45:59.793675+0000","last_became_peered":"2023-10-18T10:45:59.793675+0000","last_unstale":"2023-10-18T10:46:00.769758+0000","last_undegraded":"2023-10-18T10:46:00.769758+0000","last_fullsized":"2023-10-18T10:46:00.769758+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:08:00.737560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.901966+0000","last_change":"2023-10-18T10:45:55.765758+0000","last_active":"2023-10-18T10:46:00.901966+0000","last_peered":"2023-10-18T10:46:00.901966+0000","last_clean":"2023-10-18T10:46:00.901966+0000","last_became_active":"2023-10-18T10:45:55.765436+0000","last_became_peered":"2023-10-18T10:45:55.765436+0000","last_unstale":"2023-10-18T10:46:00.901966+0000","last_undegraded":"2023-10-18T10:46:00.901966+0000","last_fullsized":"2023-10-18T10:46:00.901966+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:40:53.192771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086132+0000","last_change":"2023-10-18T10:45:55.295720+0000","last_active":"2023-10-18T10:46:01.086132+0000","last_peered":"2023-10-18T10:46:01.086132+0000","last_clean":"2023-10-18T10:46:01.086132+0000","last_became_active":"2023-10-18T10:45:55.295401+0000","last_became_peered":"2023-10-18T10:45:55.295401+0000","last_unstale":"2023-10-18T10:46:01.086132+0000","last_undegraded":"2023-10-18T10:46:01.086132+0000","last_fullsized":"2023-10-18T10:46:01.086132+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:49:45.673807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084670+0000","last_change":"2023-10-18T10:45:59.798312+0000","last_active":"2023-10-18T10:46:01.084670+0000","last_peered":"2023-10-18T10:46:01.084670+0000","last_clean":"2023-10-18T10:46:01.084670+0000","last_became_active":"2023-10-18T10:45:59.797944+0000","last_became_peered":"2023-10-18T10:45:59.797944+0000","last_unstale":"2023-10-18T10:46:01.084670+0000","last_undegraded":"2023-10-18T10:46:01.084670+0000","last_fullsized":"2023-10-18T10:46:01.084670+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:51:40.857910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085288+0000","last_change":"2023-10-18T10:45:55.307908+0000","last_active":"2023-10-18T10:46:01.085288+0000","last_peered":"2023-10-18T10:46:01.085288+0000","last_clean":"2023-10-18T10:46:01.085288+0000","last_became_active":"2023-10-18T10:45:55.307593+0000","last_became_peered":"2023-10-18T10:45:55.307593+0000","last_unstale":"2023-10-18T10:46:01.085288+0000","last_undegraded":"2023-10-18T10:46:01.085288+0000","last_fullsized":"2023-10-18T10:46:01.085288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:20:45.634595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084888+0000","last_change":"2023-10-18T10:45:55.778356+0000","last_active":"2023-10-18T10:46:01.084888+0000","last_peered":"2023-10-18T10:46:01.084888+0000","last_clean":"2023-10-18T10:46:01.084888+0000","last_became_active":"2023-10-18T10:45:55.776465+0000","last_became_peered":"2023-10-18T10:45:55.776465+0000","last_unstale":"2023-10-18T10:46:01.084888+0000","last_undegraded":"2023-10-18T10:46:01.084888+0000","last_fullsized":"2023-10-18T10:46:01.084888+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:10:59.545961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902026+0000","last_change":"2023-10-18T10:45:59.788972+0000","last_active":"2023-10-18T10:46:00.902026+0000","last_peered":"2023-10-18T10:46:00.902026+0000","last_clean":"2023-10-18T10:46:00.902026+0000","last_became_active":"2023-10-18T10:45:59.788613+0000","last_became_peered":"2023-10-18T10:45:59.788613+0000","last_unstale":"2023-10-18T10:46:00.902026+0000","last_undegraded":"2023-10-18T10:46:00.902026+0000","last_fullsized":"2023-10-18T10:46:00.902026+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:33:04.407310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"27'9","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:06.342753+0000","last_change":"2023-10-18T10:45:55.298794+0000","last_active":"2023-10-18T10:46:06.342753+0000","last_peered":"2023-10-18T10:46:06.342753+0000","last_clean":"2023-10-18T10:46:06.342753+0000","last_became_active":"2023-10-18T10:45:55.297181+0000","last_became_peered":"2023-10-18T10:45:55.297181+0000","last_unstale":"2023-10-18T10:46:06.342753+0000","last_undegraded":"2023-10-18T10:46:06.342753+0000","last_fullsized":"2023-10-18T10:46:06.342753+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:16:43.562003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768593+0000","last_change":"2023-10-18T10:45:55.777329+0000","last_active":"2023-10-18T10:46:00.768593+0000","last_peered":"2023-10-18T10:46:00.768593+0000","last_clean":"2023-10-18T10:46:00.768593+0000","last_became_active":"2023-10-18T10:45:55.777047+0000","last_became_peered":"2023-10-18T10:45:55.777047+0000","last_unstale":"2023-10-18T10:46:00.768593+0000","last_undegraded":"2023-10-18T10:46:00.768593+0000","last_fullsized":"2023-10-18T10:46:00.768593+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:09:20.566242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084662+0000","last_change":"2023-10-18T10:45:59.804498+0000","last_active":"2023-10-18T10:46:01.084662+0000","last_peered":"2023-10-18T10:46:01.084662+0000","last_clean":"2023-10-18T10:46:01.084662+0000","last_became_active":"2023-10-18T10:45:59.804083+0000","last_became_peered":"2023-10-18T10:45:59.804083+0000","last_unstale":"2023-10-18T10:46:01.084662+0000","last_undegraded":"2023-10-18T10:46:01.084662+0000","last_fullsized":"2023-10-18T10:46:01.084662+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:04:58.822427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.769276+0000","last_change":"2023-10-18T10:45:55.774541+0000","last_active":"2023-10-18T10:46:00.769276+0000","last_peered":"2023-10-18T10:46:00.769276+0000","last_clean":"2023-10-18T10:46:00.769276+0000","last_became_active":"2023-10-18T10:45:55.774258+0000","last_became_peered":"2023-10-18T10:45:55.774258+0000","last_unstale":"2023-10-18T10:46:00.769276+0000","last_undegraded":"2023-10-18T10:46:00.769276+0000","last_fullsized":"2023-10-18T10:46:00.769276+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:11:42.497290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.770380+0000","last_change":"2023-10-18T10:45:50.252021+0000","last_active":"2023-10-18T10:46:00.770380+0000","last_peered":"2023-10-18T10:46:00.770380+0000","last_clean":"2023-10-18T10:46:00.770380+0000","last_became_active":"2023-10-18T10:45:50.251427+0000","last_became_peered":"2023-10-18T10:45:50.251427+0000","last_unstale":"2023-10-18T10:46:00.770380+0000","last_undegraded":"2023-10-18T10:46:00.770380+0000","last_fullsized":"2023-10-18T10:46:00.770380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:43.575003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"26'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086531+0000","last_change":"2023-10-18T10:45:55.296927+0000","last_active":"2023-10-18T10:46:01.086531+0000","last_peered":"2023-10-18T10:46:01.086531+0000","last_clean":"2023-10-18T10:46:01.086531+0000","last_became_active":"2023-10-18T10:45:55.296614+0000","last_became_peered":"2023-10-18T10:45:55.296614+0000","last_unstale":"2023-10-18T10:46:01.086531+0000","last_undegraded":"2023-10-18T10:46:01.086531+0000","last_fullsized":"2023-10-18T10:46:01.086531+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:05:56.466612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902246+0000","last_change":"2023-10-18T10:45:59.786937+0000","last_active":"2023-10-18T10:46:00.902246+0000","last_peered":"2023-10-18T10:46:00.902246+0000","last_clean":"2023-10-18T10:46:00.902246+0000","last_became_active":"2023-10-18T10:45:59.786436+0000","last_became_peered":"2023-10-18T10:45:59.786436+0000","last_unstale":"2023-10-18T10:46:00.902246+0000","last_undegraded":"2023-10-18T10:46:00.902246+0000","last_fullsized":"2023-10-18T10:46:00.902246+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:37:40.801018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902006+0000","last_change":"2023-10-18T10:45:55.766947+0000","last_active":"2023-10-18T10:46:00.902006+0000","last_peered":"2023-10-18T10:46:00.902006+0000","last_clean":"2023-10-18T10:46:00.902006+0000","last_became_active":"2023-10-18T10:45:55.765951+0000","last_became_peered":"2023-10-18T10:45:55.765951+0000","last_unstale":"2023-10-18T10:46:00.902006+0000","last_undegraded":"2023-10-18T10:46:00.902006+0000","last_fullsized":"2023-10-18T10:46:00.902006+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:06:27.237867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085276+0000","last_change":"2023-10-18T10:45:50.252876+0000","last_active":"2023-10-18T10:46:01.085276+0000","last_peered":"2023-10-18T10:46:01.085276+0000","last_clean":"2023-10-18T10:46:01.085276+0000","last_became_active":"2023-10-18T10:45:50.252559+0000","last_became_peered":"2023-10-18T10:45:50.252559+0000","last_unstale":"2023-10-18T10:46:01.085276+0000","last_undegraded":"2023-10-18T10:46:01.085276+0000","last_fullsized":"2023-10-18T10:46:01.085276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:55:15.913186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902858+0000","last_change":"2023-10-18T10:45:55.300421+0000","last_active":"2023-10-18T10:46:00.902858+0000","last_peered":"2023-10-18T10:46:00.902858+0000","last_clean":"2023-10-18T10:46:00.902858+0000","last_became_active":"2023-10-18T10:45:55.300170+0000","last_became_peered":"2023-10-18T10:45:55.300170+0000","last_unstale":"2023-10-18T10:46:00.902858+0000","last_undegraded":"2023-10-18T10:46:00.902858+0000","last_fullsized":"2023-10-18T10:46:00.902858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:52:09.364478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768261+0000","last_change":"2023-10-18T10:45:59.799805+0000","last_active":"2023-10-18T10:46:00.768261+0000","last_peered":"2023-10-18T10:46:00.768261+0000","last_clean":"2023-10-18T10:46:00.768261+0000","last_became_active":"2023-10-18T10:45:59.799112+0000","last_became_peered":"2023-10-18T10:45:59.799112+0000","last_unstale":"2023-10-18T10:46:00.768261+0000","last_undegraded":"2023-10-18T10:46:00.768261+0000","last_fullsized":"2023-10-18T10:46:00.768261+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:49:10.356115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086095+0000","last_change":"2023-10-18T10:45:55.774299+0000","last_active":"2023-10-18T10:46:01.086095+0000","last_peered":"2023-10-18T10:46:01.086095+0000","last_clean":"2023-10-18T10:46:01.086095+0000","last_became_active":"2023-10-18T10:45:55.773409+0000","last_became_peered":"2023-10-18T10:45:55.773409+0000","last_unstale":"2023-10-18T10:46:01.086095+0000","last_undegraded":"2023-10-18T10:46:01.086095+0000","last_fullsized":"2023-10-18T10:46:01.086095+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:37:37.315564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085581+0000","last_change":"2023-10-18T10:45:50.585224+0000","last_active":"2023-10-18T10:46:01.085581+0000","last_peered":"2023-10-18T10:46:01.085581+0000","last_clean":"2023-10-18T10:46:01.085581+0000","last_became_active":"2023-10-18T10:45:50.584872+0000","last_became_peered":"2023-10-18T10:45:50.584872+0000","last_unstale":"2023-10-18T10:46:01.085581+0000","last_undegraded":"2023-10-18T10:46:01.085581+0000","last_fullsized":"2023-10-18T10:46:01.085581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:07:41.932960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085445+0000","last_change":"2023-10-18T10:45:55.298601+0000","last_active":"2023-10-18T10:46:01.085445+0000","last_peered":"2023-10-18T10:46:01.085445+0000","last_clean":"2023-10-18T10:46:01.085445+0000","last_became_active":"2023-10-18T10:45:55.295209+0000","last_became_peered":"2023-10-18T10:45:55.295209+0000","last_unstale":"2023-10-18T10:46:01.085445+0000","last_undegraded":"2023-10-18T10:46:01.085445+0000","last_fullsized":"2023-10-18T10:46:01.085445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:55:07.806885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.769905+0000","last_change":"2023-10-18T10:45:59.787239+0000","last_active":"2023-10-18T10:46:00.769905+0000","last_peered":"2023-10-18T10:46:00.769905+0000","last_clean":"2023-10-18T10:46:00.769905+0000","last_became_active":"2023-10-18T10:45:59.786770+0000","last_became_peered":"2023-10-18T10:45:59.786770+0000","last_unstale":"2023-10-18T10:46:00.769905+0000","last_undegraded":"2023-10-18T10:46:00.769905+0000","last_fullsized":"2023-10-18T10:46:00.769905+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:09:26.681314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084957+0000","last_change":"2023-10-18T10:45:55.770838+0000","last_active":"2023-10-18T10:46:01.084957+0000","last_peered":"2023-10-18T10:46:01.084957+0000","last_clean":"2023-10-18T10:46:01.084957+0000","last_became_active":"2023-10-18T10:45:55.770308+0000","last_became_peered":"2023-10-18T10:45:55.770308+0000","last_unstale":"2023-10-18T10:46:01.084957+0000","last_undegraded":"2023-10-18T10:46:01.084957+0000","last_fullsized":"2023-10-18T10:46:01.084957+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:42:54.758946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085651+0000","last_change":"2023-10-18T10:45:50.584946+0000","last_active":"2023-10-18T10:46:01.085651+0000","last_peered":"2023-10-18T10:46:01.085651+0000","last_clean":"2023-10-18T10:46:01.085651+0000","last_became_active":"2023-10-18T10:45:50.584638+0000","last_became_peered":"2023-10-18T10:45:50.584638+0000","last_unstale":"2023-10-18T10:46:01.085651+0000","last_undegraded":"2023-10-18T10:46:01.085651+0000","last_fullsized":"2023-10-18T10:46:01.085651+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:40:09.742761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085828+0000","last_change":"2023-10-18T10:45:55.299483+0000","last_active":"2023-10-18T10:46:01.085828+0000","last_peered":"2023-10-18T10:46:01.085828+0000","last_clean":"2023-10-18T10:46:01.085828+0000","last_became_active":"2023-10-18T10:45:55.299195+0000","last_became_peered":"2023-10-18T10:45:55.299195+0000","last_unstale":"2023-10-18T10:46:01.085828+0000","last_undegraded":"2023-10-18T10:46:01.085828+0000","last_fullsized":"2023-10-18T10:46:01.085828+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:25:17.957984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085926+0000","last_change":"2023-10-18T10:45:59.801291+0000","last_active":"2023-10-18T10:46:01.085926+0000","last_peered":"2023-10-18T10:46:01.085926+0000","last_clean":"2023-10-18T10:46:01.085926+0000","last_became_active":"2023-10-18T10:45:59.800819+0000","last_became_peered":"2023-10-18T10:45:59.800819+0000","last_unstale":"2023-10-18T10:46:01.085926+0000","last_undegraded":"2023-10-18T10:46:01.085926+0000","last_fullsized":"2023-10-18T10:46:01.085926+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:06:30.412609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086211+0000","last_change":"2023-10-18T10:45:55.773950+0000","last_active":"2023-10-18T10:46:01.086211+0000","last_peered":"2023-10-18T10:46:01.086211+0000","last_clean":"2023-10-18T10:46:01.086211+0000","last_became_active":"2023-10-18T10:45:55.769581+0000","last_became_peered":"2023-10-18T10:45:55.769581+0000","last_unstale":"2023-10-18T10:46:01.086211+0000","last_undegraded":"2023-10-18T10:46:01.086211+0000","last_fullsized":"2023-10-18T10:46:01.086211+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:31:36.488372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768705+0000","last_change":"2023-10-18T10:45:55.297879+0000","last_active":"2023-10-18T10:46:00.768705+0000","last_peered":"2023-10-18T10:46:00.768705+0000","last_clean":"2023-10-18T10:46:00.768705+0000","last_became_active":"2023-10-18T10:45:55.297097+0000","last_became_peered":"2023-10-18T10:45:55.297097+0000","last_unstale":"2023-10-18T10:46:00.768705+0000","last_undegraded":"2023-10-18T10:46:00.768705+0000","last_fullsized":"2023-10-18T10:46:00.768705+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:27:53.149351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086078+0000","last_change":"2023-10-18T10:45:50.251505+0000","last_active":"2023-10-18T10:46:01.086078+0000","last_peered":"2023-10-18T10:46:01.086078+0000","last_clean":"2023-10-18T10:46:01.086078+0000","last_became_active":"2023-10-18T10:45:50.251111+0000","last_became_peered":"2023-10-18T10:45:50.251111+0000","last_unstale":"2023-10-18T10:46:01.086078+0000","last_undegraded":"2023-10-18T10:46:01.086078+0000","last_fullsized":"2023-10-18T10:46:01.086078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:04:22.817598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085546+0000","last_change":"2023-10-18T10:45:59.800297+0000","last_active":"2023-10-18T10:46:01.085546+0000","last_peered":"2023-10-18T10:46:01.085546+0000","last_clean":"2023-10-18T10:46:01.085546+0000","last_became_active":"2023-10-18T10:45:59.799434+0000","last_became_peered":"2023-10-18T10:45:59.799434+0000","last_unstale":"2023-10-18T10:46:01.085546+0000","last_undegraded":"2023-10-18T10:46:01.085546+0000","last_fullsized":"2023-10-18T10:46:01.085546+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:55:07.904865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086122+0000","last_change":"2023-10-18T10:45:55.774137+0000","last_active":"2023-10-18T10:46:01.086122+0000","last_peered":"2023-10-18T10:46:01.086122+0000","last_clean":"2023-10-18T10:46:01.086122+0000","last_became_active":"2023-10-18T10:45:55.773832+0000","last_became_peered":"2023-10-18T10:45:55.773832+0000","last_unstale":"2023-10-18T10:46:01.086122+0000","last_undegraded":"2023-10-18T10:46:01.086122+0000","last_fullsized":"2023-10-18T10:46:01.086122+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:31:21.466919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084731+0000","last_change":"2023-10-18T10:45:55.294225+0000","last_active":"2023-10-18T10:46:01.084731+0000","last_peered":"2023-10-18T10:46:01.084731+0000","last_clean":"2023-10-18T10:46:01.084731+0000","last_became_active":"2023-10-18T10:45:55.292869+0000","last_became_peered":"2023-10-18T10:45:55.292869+0000","last_unstale":"2023-10-18T10:46:01.084731+0000","last_undegraded":"2023-10-18T10:46:01.084731+0000","last_fullsized":"2023-10-18T10:46:01.084731+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:18:31.743377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.766604+0000","last_change":"2023-10-18T10:45:50.259019+0000","last_active":"2023-10-18T10:46:00.766604+0000","last_peered":"2023-10-18T10:46:00.766604+0000","last_clean":"2023-10-18T10:46:00.766604+0000","last_became_active":"2023-10-18T10:45:50.258607+0000","last_became_peered":"2023-10-18T10:45:50.258607+0000","last_unstale":"2023-10-18T10:46:00.766604+0000","last_undegraded":"2023-10-18T10:46:00.766604+0000","last_fullsized":"2023-10-18T10:46:00.766604+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:14:34.875058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.767128+0000","last_change":"2023-10-18T10:45:59.803747+0000","last_active":"2023-10-18T10:46:00.767128+0000","last_peered":"2023-10-18T10:46:00.767128+0000","last_clean":"2023-10-18T10:46:00.767128+0000","last_became_active":"2023-10-18T10:45:59.795250+0000","last_became_peered":"2023-10-18T10:45:59.795250+0000","last_unstale":"2023-10-18T10:46:00.767128+0000","last_undegraded":"2023-10-18T10:46:00.767128+0000","last_fullsized":"2023-10-18T10:46:00.767128+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:25.942126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086446+0000","last_change":"2023-10-18T10:45:55.763100+0000","last_active":"2023-10-18T10:46:01.086446+0000","last_peered":"2023-10-18T10:46:01.086446+0000","last_clean":"2023-10-18T10:46:01.086446+0000","last_became_active":"2023-10-18T10:45:55.762545+0000","last_became_peered":"2023-10-18T10:45:55.762545+0000","last_unstale":"2023-10-18T10:46:01.086446+0000","last_undegraded":"2023-10-18T10:46:01.086446+0000","last_fullsized":"2023-10-18T10:46:01.086446+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:33:00.921856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085280+0000","last_change":"2023-10-18T10:45:55.305049+0000","last_active":"2023-10-18T10:46:01.085280+0000","last_peered":"2023-10-18T10:46:01.085280+0000","last_clean":"2023-10-18T10:46:01.085280+0000","last_became_active":"2023-10-18T10:45:55.304746+0000","last_became_peered":"2023-10-18T10:45:55.304746+0000","last_unstale":"2023-10-18T10:46:01.085280+0000","last_undegraded":"2023-10-18T10:46:01.085280+0000","last_fullsized":"2023-10-18T10:46:01.085280+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:37:42.159482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084766+0000","last_change":"2023-10-18T10:45:50.251375+0000","last_active":"2023-10-18T10:46:01.084766+0000","last_peered":"2023-10-18T10:46:01.084766+0000","last_clean":"2023-10-18T10:46:01.084766+0000","last_became_active":"2023-10-18T10:45:50.251055+0000","last_became_peered":"2023-10-18T10:45:50.251055+0000","last_unstale":"2023-10-18T10:46:01.084766+0000","last_undegraded":"2023-10-18T10:46:01.084766+0000","last_fullsized":"2023-10-18T10:46:01.084766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:49:40.643495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085615+0000","last_change":"2023-10-18T10:45:59.791676+0000","last_active":"2023-10-18T10:46:01.085615+0000","last_peered":"2023-10-18T10:46:01.085615+0000","last_clean":"2023-10-18T10:46:01.085615+0000","last_became_active":"2023-10-18T10:45:59.791341+0000","last_became_peered":"2023-10-18T10:45:59.791341+0000","last_unstale":"2023-10-18T10:46:01.085615+0000","last_undegraded":"2023-10-18T10:46:01.085615+0000","last_fullsized":"2023-10-18T10:46:01.085615+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:15:44.664969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.770245+0000","last_change":"2023-10-18T10:45:55.777240+0000","last_active":"2023-10-18T10:46:00.770245+0000","last_peered":"2023-10-18T10:46:00.770245+0000","last_clean":"2023-10-18T10:46:00.770245+0000","last_became_active":"2023-10-18T10:45:55.775286+0000","last_became_peered":"2023-10-18T10:45:55.775286+0000","last_unstale":"2023-10-18T10:46:00.770245+0000","last_undegraded":"2023-10-18T10:46:00.770245+0000","last_fullsized":"2023-10-18T10:46:00.770245+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:52.500160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902463+0000","last_change":"2023-10-18T10:45:55.296440+0000","last_active":"2023-10-18T10:46:00.902463+0000","last_peered":"2023-10-18T10:46:00.902463+0000","last_clean":"2023-10-18T10:46:00.902463+0000","last_became_active":"2023-10-18T10:45:55.296124+0000","last_became_peered":"2023-10-18T10:45:55.296124+0000","last_unstale":"2023-10-18T10:46:00.902463+0000","last_undegraded":"2023-10-18T10:46:00.902463+0000","last_fullsized":"2023-10-18T10:46:00.902463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:43:19.116789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085874+0000","last_change":"2023-10-18T10:45:50.251310+0000","last_active":"2023-10-18T10:46:01.085874+0000","last_peered":"2023-10-18T10:46:01.085874+0000","last_clean":"2023-10-18T10:46:01.085874+0000","last_became_active":"2023-10-18T10:45:50.250823+0000","last_became_peered":"2023-10-18T10:45:50.250823+0000","last_unstale":"2023-10-18T10:46:01.085874+0000","last_undegraded":"2023-10-18T10:46:01.085874+0000","last_fullsized":"2023-10-18T10:46:01.085874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:33:28.469766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":183,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084881+0000","last_change":"2023-10-18T10:45:44.563531+0000","last_active":"2023-10-18T10:46:01.084881+0000","last_peered":"2023-10-18T10:46:01.084881+0000","last_clean":"2023-10-18T10:46:01.084881+0000","last_became_active":"2023-10-18T10:45:44.562504+0000","last_became_peered":"2023-10-18T10:45:44.562504+0000","last_unstale":"2023-10-18T10:46:01.084881+0000","last_undegraded":"2023-10-18T10:46:01.084881+0000","last_fullsized":"2023-10-18T10:46:01.084881+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:43.181461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:43.181461+0000","last_clean_scrub_stamp":"2023-10-18T10:45:43.181461+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:37:25.232224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084962+0000","last_change":"2023-10-18T10:45:59.807765+0000","last_active":"2023-10-18T10:46:01.084962+0000","last_peered":"2023-10-18T10:46:01.084962+0000","last_clean":"2023-10-18T10:46:01.084962+0000","last_became_active":"2023-10-18T10:45:59.807369+0000","last_became_peered":"2023-10-18T10:45:59.807369+0000","last_unstale":"2023-10-18T10:46:01.084962+0000","last_undegraded":"2023-10-18T10:46:01.084962+0000","last_fullsized":"2023-10-18T10:46:01.084962+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:14:04.335139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086027+0000","last_change":"2023-10-18T10:45:55.763363+0000","last_active":"2023-10-18T10:46:01.086027+0000","last_peered":"2023-10-18T10:46:01.086027+0000","last_clean":"2023-10-18T10:46:01.086027+0000","last_became_active":"2023-10-18T10:45:55.763155+0000","last_became_peered":"2023-10-18T10:45:55.763155+0000","last_unstale":"2023-10-18T10:46:01.086027+0000","last_undegraded":"2023-10-18T10:46:01.086027+0000","last_fullsized":"2023-10-18T10:46:01.086027+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:15:41.179515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902288+0000","last_change":"2023-10-18T10:45:55.304658+0000","last_active":"2023-10-18T10:46:00.902288+0000","last_peered":"2023-10-18T10:46:00.902288+0000","last_clean":"2023-10-18T10:46:00.902288+0000","last_became_active":"2023-10-18T10:45:55.303900+0000","last_became_peered":"2023-10-18T10:45:55.303900+0000","last_unstale":"2023-10-18T10:46:00.902288+0000","last_undegraded":"2023-10-18T10:46:00.902288+0000","last_fullsized":"2023-10-18T10:46:00.902288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:34:05.567387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768906+0000","last_change":"2023-10-18T10:45:59.792438+0000","last_active":"2023-10-18T10:46:00.768906+0000","last_peered":"2023-10-18T10:46:00.768906+0000","last_clean":"2023-10-18T10:46:00.768906+0000","last_became_active":"2023-10-18T10:45:59.791975+0000","last_became_peered":"2023-10-18T10:45:59.791975+0000","last_unstale":"2023-10-18T10:46:00.768906+0000","last_undegraded":"2023-10-18T10:46:00.768906+0000","last_fullsized":"2023-10-18T10:46:00.768906+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:19:39.692866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768991+0000","last_change":"2023-10-18T10:45:55.773190+0000","last_active":"2023-10-18T10:46:00.768991+0000","last_peered":"2023-10-18T10:46:00.768991+0000","last_clean":"2023-10-18T10:46:00.768991+0000","last_became_active":"2023-10-18T10:45:55.769619+0000","last_became_peered":"2023-10-18T10:45:55.769619+0000","last_unstale":"2023-10-18T10:46:00.768991+0000","last_undegraded":"2023-10-18T10:46:00.768991+0000","last_fullsized":"2023-10-18T10:46:00.768991+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:12:47.136700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.769371+0000","last_change":"2023-10-18T10:45:55.299320+0000","last_active":"2023-10-18T10:46:00.769371+0000","last_peered":"2023-10-18T10:46:00.769371+0000","last_clean":"2023-10-18T10:46:00.769371+0000","last_became_active":"2023-10-18T10:45:55.299008+0000","last_became_peered":"2023-10-18T10:45:55.299008+0000","last_unstale":"2023-10-18T10:46:00.769371+0000","last_undegraded":"2023-10-18T10:46:00.769371+0000","last_fullsized":"2023-10-18T10:46:00.769371+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:51.786437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085465+0000","last_change":"2023-10-18T10:45:59.801383+0000","last_active":"2023-10-18T10:46:01.085465+0000","last_peered":"2023-10-18T10:46:01.085465+0000","last_clean":"2023-10-18T10:46:01.085465+0000","last_became_active":"2023-10-18T10:45:59.801052+0000","last_became_peered":"2023-10-18T10:45:59.801052+0000","last_unstale":"2023-10-18T10:46:01.085465+0000","last_undegraded":"2023-10-18T10:46:01.085465+0000","last_fullsized":"2023-10-18T10:46:01.085465+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:57:31.735612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768713+0000","last_change":"2023-10-18T10:45:55.762399+0000","last_active":"2023-10-18T10:46:00.768713+0000","last_peered":"2023-10-18T10:46:00.768713+0000","last_clean":"2023-10-18T10:46:00.768713+0000","last_became_active":"2023-10-18T10:45:55.762074+0000","last_became_peered":"2023-10-18T10:45:55.762074+0000","last_unstale":"2023-10-18T10:46:00.768713+0000","last_undegraded":"2023-10-18T10:46:00.768713+0000","last_fullsized":"2023-10-18T10:46:00.768713+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:44:41.551817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.770066+0000","last_change":"2023-10-18T10:45:55.301204+0000","last_active":"2023-10-18T10:46:00.770066+0000","last_peered":"2023-10-18T10:46:00.770066+0000","last_clean":"2023-10-18T10:46:00.770066+0000","last_became_active":"2023-10-18T10:45:55.300895+0000","last_became_peered":"2023-10-18T10:45:55.300895+0000","last_unstale":"2023-10-18T10:46:00.770066+0000","last_undegraded":"2023-10-18T10:46:00.770066+0000","last_fullsized":"2023-10-18T10:46:00.770066+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:33:30.055966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084972+0000","last_change":"2023-10-18T10:45:59.794810+0000","last_active":"2023-10-18T10:46:01.084972+0000","last_peered":"2023-10-18T10:46:01.084972+0000","last_clean":"2023-10-18T10:46:01.084972+0000","last_became_active":"2023-10-18T10:45:59.794392+0000","last_became_peered":"2023-10-18T10:45:59.794392+0000","last_unstale":"2023-10-18T10:46:01.084972+0000","last_undegraded":"2023-10-18T10:46:01.084972+0000","last_fullsized":"2023-10-18T10:46:01.084972+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:05:09.364081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768556+0000","last_change":"2023-10-18T10:45:55.775591+0000","last_active":"2023-10-18T10:46:00.768556+0000","last_peered":"2023-10-18T10:46:00.768556+0000","last_clean":"2023-10-18T10:46:00.768556+0000","last_became_active":"2023-10-18T10:45:55.770394+0000","last_became_peered":"2023-10-18T10:45:55.770394+0000","last_unstale":"2023-10-18T10:46:00.768556+0000","last_undegraded":"2023-10-18T10:46:00.768556+0000","last_fullsized":"2023-10-18T10:46:00.768556+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:03:08.191047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902332+0000","last_change":"2023-10-18T10:45:55.299012+0000","last_active":"2023-10-18T10:46:00.902332+0000","last_peered":"2023-10-18T10:46:00.902332+0000","last_clean":"2023-10-18T10:46:00.902332+0000","last_became_active":"2023-10-18T10:45:55.296941+0000","last_became_peered":"2023-10-18T10:45:55.296941+0000","last_unstale":"2023-10-18T10:46:00.902332+0000","last_undegraded":"2023-10-18T10:46:00.902332+0000","last_fullsized":"2023-10-18T10:46:00.902332+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:07:46.963272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.766976+0000","last_change":"2023-10-18T10:45:59.797124+0000","last_active":"2023-10-18T10:46:00.766976+0000","last_peered":"2023-10-18T10:46:00.766976+0000","last_clean":"2023-10-18T10:46:00.766976+0000","last_became_active":"2023-10-18T10:45:59.796721+0000","last_became_peered":"2023-10-18T10:45:59.796721+0000","last_unstale":"2023-10-18T10:46:00.766976+0000","last_undegraded":"2023-10-18T10:46:00.766976+0000","last_fullsized":"2023-10-18T10:46:00.766976+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:11:32.963010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085998+0000","last_change":"2023-10-18T10:45:55.767007+0000","last_active":"2023-10-18T10:46:01.085998+0000","last_peered":"2023-10-18T10:46:01.085998+0000","last_clean":"2023-10-18T10:46:01.085998+0000","last_became_active":"2023-10-18T10:45:55.766727+0000","last_became_peered":"2023-10-18T10:45:55.766727+0000","last_unstale":"2023-10-18T10:46:01.085998+0000","last_undegraded":"2023-10-18T10:46:01.085998+0000","last_fullsized":"2023-10-18T10:46:01.085998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:51:37.372456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085606+0000","last_change":"2023-10-18T10:45:55.293762+0000","last_active":"2023-10-18T10:46:01.085606+0000","last_peered":"2023-10-18T10:46:01.085606+0000","last_clean":"2023-10-18T10:46:01.085606+0000","last_became_active":"2023-10-18T10:45:55.293349+0000","last_became_peered":"2023-10-18T10:45:55.293349+0000","last_unstale":"2023-10-18T10:46:01.085606+0000","last_undegraded":"2023-10-18T10:46:01.085606+0000","last_fullsized":"2023-10-18T10:46:01.085606+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:31:20.459601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.766635+0000","last_change":"2023-10-18T10:45:59.790292+0000","last_active":"2023-10-18T10:46:00.766635+0000","last_peered":"2023-10-18T10:46:00.766635+0000","last_clean":"2023-10-18T10:46:00.766635+0000","last_became_active":"2023-10-18T10:45:59.789901+0000","last_became_peered":"2023-10-18T10:45:59.789901+0000","last_unstale":"2023-10-18T10:46:00.766635+0000","last_undegraded":"2023-10-18T10:46:00.766635+0000","last_fullsized":"2023-10-18T10:46:00.766635+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:26:52.984157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.769853+0000","last_change":"2023-10-18T10:45:55.768064+0000","last_active":"2023-10-18T10:46:00.769853+0000","last_peered":"2023-10-18T10:46:00.769853+0000","last_clean":"2023-10-18T10:46:00.769853+0000","last_became_active":"2023-10-18T10:45:55.767773+0000","last_became_peered":"2023-10-18T10:45:55.767773+0000","last_unstale":"2023-10-18T10:46:00.769853+0000","last_undegraded":"2023-10-18T10:46:00.769853+0000","last_fullsized":"2023-10-18T10:46:00.769853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:30:00.014556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085414+0000","last_change":"2023-10-18T10:45:55.303880+0000","last_active":"2023-10-18T10:46:01.085414+0000","last_peered":"2023-10-18T10:46:01.085414+0000","last_clean":"2023-10-18T10:46:01.085414+0000","last_became_active":"2023-10-18T10:45:55.303577+0000","last_became_peered":"2023-10-18T10:45:55.303577+0000","last_unstale":"2023-10-18T10:46:01.085414+0000","last_undegraded":"2023-10-18T10:46:01.085414+0000","last_fullsized":"2023-10-18T10:46:01.085414+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:02:55.960777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.766839+0000","last_change":"2023-10-18T10:45:59.789006+0000","last_active":"2023-10-18T10:46:00.766839+0000","last_peered":"2023-10-18T10:46:00.766839+0000","last_clean":"2023-10-18T10:46:00.766839+0000","last_became_active":"2023-10-18T10:45:59.788544+0000","last_became_peered":"2023-10-18T10:45:59.788544+0000","last_unstale":"2023-10-18T10:46:00.766839+0000","last_undegraded":"2023-10-18T10:46:00.766839+0000","last_fullsized":"2023-10-18T10:46:00.766839+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:32:31.878208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.769128+0000","last_change":"2023-10-18T10:45:55.775207+0000","last_active":"2023-10-18T10:46:00.769128+0000","last_peered":"2023-10-18T10:46:00.769128+0000","last_clean":"2023-10-18T10:46:00.769128+0000","last_became_active":"2023-10-18T10:45:55.774909+0000","last_became_peered":"2023-10-18T10:45:55.774909+0000","last_unstale":"2023-10-18T10:46:00.769128+0000","last_undegraded":"2023-10-18T10:46:00.769128+0000","last_fullsized":"2023-10-18T10:46:00.769128+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:07:30.694604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085978+0000","last_change":"2023-10-18T10:45:55.302100+0000","last_active":"2023-10-18T10:46:01.085978+0000","last_peered":"2023-10-18T10:46:01.085978+0000","last_clean":"2023-10-18T10:46:01.085978+0000","last_became_active":"2023-10-18T10:45:55.301652+0000","last_became_peered":"2023-10-18T10:45:55.301652+0000","last_unstale":"2023-10-18T10:46:01.085978+0000","last_undegraded":"2023-10-18T10:46:01.085978+0000","last_fullsized":"2023-10-18T10:46:01.085978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:12:22.646771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.766536+0000","last_change":"2023-10-18T10:45:59.794997+0000","last_active":"2023-10-18T10:46:00.766536+0000","last_peered":"2023-10-18T10:46:00.766536+0000","last_clean":"2023-10-18T10:46:00.766536+0000","last_became_active":"2023-10-18T10:45:59.794587+0000","last_became_peered":"2023-10-18T10:45:59.794587+0000","last_unstale":"2023-10-18T10:46:00.766536+0000","last_undegraded":"2023-10-18T10:46:00.766536+0000","last_fullsized":"2023-10-18T10:46:00.766536+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:22:23.799739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084896+0000","last_change":"2023-10-18T10:45:55.772538+0000","last_active":"2023-10-18T10:46:01.084896+0000","last_peered":"2023-10-18T10:46:01.084896+0000","last_clean":"2023-10-18T10:46:01.084896+0000","last_became_active":"2023-10-18T10:45:55.772207+0000","last_became_peered":"2023-10-18T10:45:55.772207+0000","last_unstale":"2023-10-18T10:46:01.084896+0000","last_undegraded":"2023-10-18T10:46:01.084896+0000","last_fullsized":"2023-10-18T10:46:01.084896+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:55:21.950816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086120+0000","last_change":"2023-10-18T10:45:55.305858+0000","last_active":"2023-10-18T10:46:01.086120+0000","last_peered":"2023-10-18T10:46:01.086120+0000","last_clean":"2023-10-18T10:46:01.086120+0000","last_became_active":"2023-10-18T10:45:55.305591+0000","last_became_peered":"2023-10-18T10:45:55.305591+0000","last_unstale":"2023-10-18T10:46:01.086120+0000","last_undegraded":"2023-10-18T10:46:01.086120+0000","last_fullsized":"2023-10-18T10:46:01.086120+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:43:19.116789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084960+0000","last_change":"2023-10-18T10:45:59.799125+0000","last_active":"2023-10-18T10:46:01.084960+0000","last_peered":"2023-10-18T10:46:01.084960+0000","last_clean":"2023-10-18T10:46:01.084960+0000","last_became_active":"2023-10-18T10:45:59.798734+0000","last_became_peered":"2023-10-18T10:45:59.798734+0000","last_unstale":"2023-10-18T10:46:01.084960+0000","last_undegraded":"2023-10-18T10:46:01.084960+0000","last_fullsized":"2023-10-18T10:46:01.084960+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:56:46.630947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085751+0000","last_change":"2023-10-18T10:45:59.794340+0000","last_active":"2023-10-18T10:46:01.085751+0000","last_peered":"2023-10-18T10:46:01.085751+0000","last_clean":"2023-10-18T10:46:01.085751+0000","last_became_active":"2023-10-18T10:45:59.793965+0000","last_became_peered":"2023-10-18T10:45:59.793965+0000","last_unstale":"2023-10-18T10:46:01.085751+0000","last_undegraded":"2023-10-18T10:46:01.085751+0000","last_fullsized":"2023-10-18T10:46:01.085751+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:29.702107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.767968+0000","last_change":"2023-10-18T10:45:55.292258+0000","last_active":"2023-10-18T10:46:00.767968+0000","last_peered":"2023-10-18T10:46:00.767968+0000","last_clean":"2023-10-18T10:46:00.767968+0000","last_became_active":"2023-10-18T10:45:55.291007+0000","last_became_peered":"2023-10-18T10:45:55.291007+0000","last_unstale":"2023-10-18T10:46:00.767968+0000","last_undegraded":"2023-10-18T10:46:00.767968+0000","last_fullsized":"2023-10-18T10:46:00.767968+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:06:37.792273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902848+0000","last_change":"2023-10-18T10:45:55.765261+0000","last_active":"2023-10-18T10:46:00.902848+0000","last_peered":"2023-10-18T10:46:00.902848+0000","last_clean":"2023-10-18T10:46:00.902848+0000","last_became_active":"2023-10-18T10:45:55.765025+0000","last_became_peered":"2023-10-18T10:45:55.765025+0000","last_unstale":"2023-10-18T10:46:00.902848+0000","last_undegraded":"2023-10-18T10:46:00.902848+0000","last_fullsized":"2023-10-18T10:46:00.902848+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:49:06.870660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902759+0000","last_change":"2023-10-18T10:45:59.795015+0000","last_active":"2023-10-18T10:46:00.902759+0000","last_peered":"2023-10-18T10:46:00.902759+0000","last_clean":"2023-10-18T10:46:00.902759+0000","last_became_active":"2023-10-18T10:45:59.794646+0000","last_became_peered":"2023-10-18T10:45:59.794646+0000","last_unstale":"2023-10-18T10:46:00.902759+0000","last_undegraded":"2023-10-18T10:46:00.902759+0000","last_fullsized":"2023-10-18T10:46:00.902759+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:08:21.298342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085153+0000","last_change":"2023-10-18T10:45:55.305369+0000","last_active":"2023-10-18T10:46:01.085153+0000","last_peered":"2023-10-18T10:46:01.085153+0000","last_clean":"2023-10-18T10:46:01.085153+0000","last_became_active":"2023-10-18T10:45:55.304821+0000","last_became_peered":"2023-10-18T10:45:55.304821+0000","last_unstale":"2023-10-18T10:46:01.085153+0000","last_undegraded":"2023-10-18T10:46:01.085153+0000","last_fullsized":"2023-10-18T10:46:01.085153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:32:59.914538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086514+0000","last_change":"2023-10-18T10:45:55.763301+0000","last_active":"2023-10-18T10:46:01.086514+0000","last_peered":"2023-10-18T10:46:01.086514+0000","last_clean":"2023-10-18T10:46:01.086514+0000","last_became_active":"2023-10-18T10:45:55.762910+0000","last_became_peered":"2023-10-18T10:45:55.762910+0000","last_unstale":"2023-10-18T10:46:01.086514+0000","last_undegraded":"2023-10-18T10:46:01.086514+0000","last_fullsized":"2023-10-18T10:46:01.086514+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:06:27.237867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085615+0000","last_change":"2023-10-18T10:45:55.302395+0000","last_active":"2023-10-18T10:46:01.085615+0000","last_peered":"2023-10-18T10:46:01.085615+0000","last_clean":"2023-10-18T10:46:01.085615+0000","last_became_active":"2023-10-18T10:45:55.302090+0000","last_became_peered":"2023-10-18T10:45:55.302090+0000","last_unstale":"2023-10-18T10:46:01.085615+0000","last_undegraded":"2023-10-18T10:46:01.085615+0000","last_fullsized":"2023-10-18T10:46:01.085615+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:25:17.957984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085682+0000","last_change":"2023-10-18T10:45:55.776022+0000","last_active":"2023-10-18T10:46:01.085682+0000","last_peered":"2023-10-18T10:46:01.085682+0000","last_clean":"2023-10-18T10:46:01.085682+0000","last_became_active":"2023-10-18T10:45:55.775711+0000","last_became_peered":"2023-10-18T10:45:55.775711+0000","last_unstale":"2023-10-18T10:46:01.085682+0000","last_undegraded":"2023-10-18T10:46:01.085682+0000","last_fullsized":"2023-10-18T10:46:01.085682+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:31:36.488372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902232+0000","last_change":"2023-10-18T10:45:59.798164+0000","last_active":"2023-10-18T10:46:00.902232+0000","last_peered":"2023-10-18T10:46:00.902232+0000","last_clean":"2023-10-18T10:46:00.902232+0000","last_became_active":"2023-10-18T10:45:59.797723+0000","last_became_peered":"2023-10-18T10:45:59.797723+0000","last_unstale":"2023-10-18T10:46:00.902232+0000","last_undegraded":"2023-10-18T10:46:00.902232+0000","last_fullsized":"2023-10-18T10:46:00.902232+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:03:00.453549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085393+0000","last_change":"2023-10-18T10:45:55.306329+0000","last_active":"2023-10-18T10:46:01.085393+0000","last_peered":"2023-10-18T10:46:01.085393+0000","last_clean":"2023-10-18T10:46:01.085393+0000","last_became_active":"2023-10-18T10:45:55.306033+0000","last_became_peered":"2023-10-18T10:45:55.306033+0000","last_unstale":"2023-10-18T10:46:01.085393+0000","last_undegraded":"2023-10-18T10:46:01.085393+0000","last_fullsized":"2023-10-18T10:46:01.085393+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:25:36.844395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902485+0000","last_change":"2023-10-18T10:45:55.768454+0000","last_active":"2023-10-18T10:46:00.902485+0000","last_peered":"2023-10-18T10:46:00.902485+0000","last_clean":"2023-10-18T10:46:00.902485+0000","last_became_active":"2023-10-18T10:45:55.767547+0000","last_became_peered":"2023-10-18T10:45:55.767547+0000","last_unstale":"2023-10-18T10:46:00.902485+0000","last_undegraded":"2023-10-18T10:46:00.902485+0000","last_fullsized":"2023-10-18T10:46:00.902485+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:19:20.955967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085524+0000","last_change":"2023-10-18T10:45:59.790629+0000","last_active":"2023-10-18T10:46:01.085524+0000","last_peered":"2023-10-18T10:46:01.085524+0000","last_clean":"2023-10-18T10:46:01.085524+0000","last_became_active":"2023-10-18T10:45:59.790163+0000","last_became_peered":"2023-10-18T10:45:59.790163+0000","last_unstale":"2023-10-18T10:46:01.085524+0000","last_undegraded":"2023-10-18T10:46:01.085524+0000","last_fullsized":"2023-10-18T10:46:01.085524+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:10:56.439036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.767840+0000","last_change":"2023-10-18T10:45:55.759248+0000","last_active":"2023-10-18T10:46:00.767840+0000","last_peered":"2023-10-18T10:46:00.767840+0000","last_clean":"2023-10-18T10:46:00.767840+0000","last_became_active":"2023-10-18T10:45:55.758926+0000","last_became_peered":"2023-10-18T10:45:55.758926+0000","last_unstale":"2023-10-18T10:46:00.767840+0000","last_undegraded":"2023-10-18T10:46:00.767840+0000","last_fullsized":"2023-10-18T10:46:00.767840+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:37:00.951309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.901903+0000","last_change":"2023-10-18T10:45:55.777594+0000","last_active":"2023-10-18T10:46:00.901903+0000","last_peered":"2023-10-18T10:46:00.901903+0000","last_clean":"2023-10-18T10:46:00.901903+0000","last_became_active":"2023-10-18T10:45:55.777126+0000","last_became_peered":"2023-10-18T10:45:55.777126+0000","last_unstale":"2023-10-18T10:46:00.901903+0000","last_undegraded":"2023-10-18T10:46:00.901903+0000","last_fullsized":"2023-10-18T10:46:00.901903+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:33:49.751994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768469+0000","last_change":"2023-10-18T10:45:55.770662+0000","last_active":"2023-10-18T10:46:00.768469+0000","last_peered":"2023-10-18T10:46:00.768469+0000","last_clean":"2023-10-18T10:46:00.768469+0000","last_became_active":"2023-10-18T10:45:55.769816+0000","last_became_peered":"2023-10-18T10:45:55.769816+0000","last_unstale":"2023-10-18T10:46:00.768469+0000","last_undegraded":"2023-10-18T10:46:00.768469+0000","last_fullsized":"2023-10-18T10:46:00.768469+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:40:54.772659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902183+0000","last_change":"2023-10-18T10:45:55.764434+0000","last_active":"2023-10-18T10:46:00.902183+0000","last_peered":"2023-10-18T10:46:00.902183+0000","last_clean":"2023-10-18T10:46:00.902183+0000","last_became_active":"2023-10-18T10:45:55.764164+0000","last_became_peered":"2023-10-18T10:45:55.764164+0000","last_unstale":"2023-10-18T10:46:00.902183+0000","last_undegraded":"2023-10-18T10:46:00.902183+0000","last_fullsized":"2023-10-18T10:46:00.902183+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:25:37.851713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085549+0000","last_change":"2023-10-18T10:45:55.773793+0000","last_active":"2023-10-18T10:46:01.085549+0000","last_peered":"2023-10-18T10:46:01.085549+0000","last_clean":"2023-10-18T10:46:01.085549+0000","last_became_active":"2023-10-18T10:45:55.769530+0000","last_became_peered":"2023-10-18T10:45:55.769530+0000","last_unstale":"2023-10-18T10:46:01.085549+0000","last_undegraded":"2023-10-18T10:46:01.085549+0000","last_fullsized":"2023-10-18T10:46:01.085549+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:57:28.250158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085697+0000","last_change":"2023-10-18T10:45:55.780769+0000","last_active":"2023-10-18T10:46:01.085697+0000","last_peered":"2023-10-18T10:46:01.085697+0000","last_clean":"2023-10-18T10:46:01.085697+0000","last_became_active":"2023-10-18T10:45:55.780434+0000","last_became_peered":"2023-10-18T10:45:55.780434+0000","last_unstale":"2023-10-18T10:46:01.085697+0000","last_undegraded":"2023-10-18T10:46:01.085697+0000","last_fullsized":"2023-10-18T10:46:01.085697+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:37:43.166800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085181+0000","last_change":"2023-10-18T10:45:55.770696+0000","last_active":"2023-10-18T10:46:01.085181+0000","last_peered":"2023-10-18T10:46:01.085181+0000","last_clean":"2023-10-18T10:46:01.085181+0000","last_became_active":"2023-10-18T10:45:55.769723+0000","last_became_peered":"2023-10-18T10:45:55.769723+0000","last_unstale":"2023-10-18T10:46:01.085181+0000","last_undegraded":"2023-10-18T10:46:01.085181+0000","last_fullsized":"2023-10-18T10:46:01.085181+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:59:15.197613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085919+0000","last_change":"2023-10-18T10:45:55.766491+0000","last_active":"2023-10-18T10:46:01.085919+0000","last_peered":"2023-10-18T10:46:01.085919+0000","last_clean":"2023-10-18T10:46:01.085919+0000","last_became_active":"2023-10-18T10:45:55.766215+0000","last_became_peered":"2023-10-18T10:45:55.766215+0000","last_unstale":"2023-10-18T10:46:01.085919+0000","last_undegraded":"2023-10-18T10:46:01.085919+0000","last_fullsized":"2023-10-18T10:46:01.085919+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:10:52.953582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085494+0000","last_change":"2023-10-18T10:45:55.767712+0000","last_active":"2023-10-18T10:46:01.085494+0000","last_peered":"2023-10-18T10:46:01.085494+0000","last_clean":"2023-10-18T10:46:01.085494+0000","last_became_active":"2023-10-18T10:45:55.765004+0000","last_became_peered":"2023-10-18T10:45:55.765004+0000","last_unstale":"2023-10-18T10:46:01.085494+0000","last_undegraded":"2023-10-18T10:46:01.085494+0000","last_fullsized":"2023-10-18T10:46:01.085494+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:34:06.574705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.901824+0000","last_change":"2023-10-18T10:45:55.757810+0000","last_active":"2023-10-18T10:46:00.901824+0000","last_peered":"2023-10-18T10:46:00.901824+0000","last_clean":"2023-10-18T10:46:00.901824+0000","last_became_active":"2023-10-18T10:45:55.756492+0000","last_became_peered":"2023-10-18T10:45:55.756492+0000","last_unstale":"2023-10-18T10:46:00.901824+0000","last_undegraded":"2023-10-18T10:46:00.901824+0000","last_fullsized":"2023-10-18T10:46:00.901824+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:56:43.145493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085491+0000","last_change":"2023-10-18T10:45:55.767803+0000","last_active":"2023-10-18T10:46:01.085491+0000","last_peered":"2023-10-18T10:46:01.085491+0000","last_clean":"2023-10-18T10:46:01.085491+0000","last_became_active":"2023-10-18T10:45:55.767525+0000","last_became_peered":"2023-10-18T10:45:55.767525+0000","last_unstale":"2023-10-18T10:46:01.085491+0000","last_undegraded":"2023-10-18T10:46:01.085491+0000","last_fullsized":"2023-10-18T10:46:01.085491+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:26.216653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085266+0000","last_change":"2023-10-18T10:45:55.769366+0000","last_active":"2023-10-18T10:46:01.085266+0000","last_peered":"2023-10-18T10:46:01.085266+0000","last_clean":"2023-10-18T10:46:01.085266+0000","last_became_active":"2023-10-18T10:45:55.769083+0000","last_became_peered":"2023-10-18T10:45:55.769083+0000","last_unstale":"2023-10-18T10:46:01.085266+0000","last_undegraded":"2023-10-18T10:46:01.085266+0000","last_fullsized":"2023-10-18T10:46:01.085266+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:05:57.473930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768585+0000","last_change":"2023-10-18T10:45:55.770711+0000","last_active":"2023-10-18T10:46:00.768585+0000","last_peered":"2023-10-18T10:46:00.768585+0000","last_clean":"2023-10-18T10:46:00.768585+0000","last_became_active":"2023-10-18T10:45:55.770147+0000","last_became_peered":"2023-10-18T10:45:55.770147+0000","last_unstale":"2023-10-18T10:46:00.768585+0000","last_undegraded":"2023-10-18T10:46:00.768585+0000","last_fullsized":"2023-10-18T10:46:00.768585+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:20:18.473140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.767102+0000","last_change":"2023-10-18T10:45:55.758011+0000","last_active":"2023-10-18T10:46:00.767102+0000","last_peered":"2023-10-18T10:46:00.767102+0000","last_clean":"2023-10-18T10:46:00.767102+0000","last_became_active":"2023-10-18T10:45:55.757668+0000","last_became_peered":"2023-10-18T10:45:55.757668+0000","last_unstale":"2023-10-18T10:46:00.767102+0000","last_undegraded":"2023-10-18T10:46:00.767102+0000","last_fullsized":"2023-10-18T10:46:00.767102+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:41:48.881411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.769446+0000","last_change":"2023-10-18T10:45:55.763084+0000","last_active":"2023-10-18T10:46:00.769446+0000","last_peered":"2023-10-18T10:46:00.769446+0000","last_clean":"2023-10-18T10:46:00.769446+0000","last_became_active":"2023-10-18T10:45:55.762776+0000","last_became_peered":"2023-10-18T10:45:55.762776+0000","last_unstale":"2023-10-18T10:46:00.769446+0000","last_undegraded":"2023-10-18T10:46:00.769446+0000","last_fullsized":"2023-10-18T10:46:00.769446+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:28:02.667670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085807+0000","last_change":"2023-10-18T10:45:55.762254+0000","last_active":"2023-10-18T10:46:01.085807+0000","last_peered":"2023-10-18T10:46:01.085807+0000","last_clean":"2023-10-18T10:46:01.085807+0000","last_became_active":"2023-10-18T10:45:55.761870+0000","last_became_peered":"2023-10-18T10:45:55.761870+0000","last_unstale":"2023-10-18T10:46:01.085807+0000","last_undegraded":"2023-10-18T10:46:01.085807+0000","last_fullsized":"2023-10-18T10:46:01.085807+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:43:01.654769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768118+0000","last_change":"2023-10-18T10:45:55.765743+0000","last_active":"2023-10-18T10:46:00.768118+0000","last_peered":"2023-10-18T10:46:00.768118+0000","last_clean":"2023-10-18T10:46:00.768118+0000","last_became_active":"2023-10-18T10:45:55.765470+0000","last_became_peered":"2023-10-18T10:45:55.765470+0000","last_unstale":"2023-10-18T10:46:00.768118+0000","last_undegraded":"2023-10-18T10:46:00.768118+0000","last_fullsized":"2023-10-18T10:46:00.768118+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:19:20.955967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085124+0000","last_change":"2023-10-18T10:45:55.781643+0000","last_active":"2023-10-18T10:46:01.085124+0000","last_peered":"2023-10-18T10:46:01.085124+0000","last_clean":"2023-10-18T10:46:01.085124+0000","last_became_active":"2023-10-18T10:45:55.781252+0000","last_became_peered":"2023-10-18T10:45:55.781252+0000","last_unstale":"2023-10-18T10:46:01.085124+0000","last_undegraded":"2023-10-18T10:46:01.085124+0000","last_fullsized":"2023-10-18T10:46:01.085124+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:40:53.192771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085958+0000","last_change":"2023-10-18T10:45:55.774683+0000","last_active":"2023-10-18T10:46:01.085958+0000","last_peered":"2023-10-18T10:46:01.085958+0000","last_clean":"2023-10-18T10:46:01.085958+0000","last_became_active":"2023-10-18T10:45:55.773664+0000","last_became_peered":"2023-10-18T10:45:55.773664+0000","last_unstale":"2023-10-18T10:46:01.085958+0000","last_undegraded":"2023-10-18T10:46:01.085958+0000","last_fullsized":"2023-10-18T10:46:01.085958+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:06:26.927155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.766819+0000","last_change":"2023-10-18T10:45:55.755077+0000","last_active":"2023-10-18T10:46:00.766819+0000","last_peered":"2023-10-18T10:46:00.766819+0000","last_clean":"2023-10-18T10:46:00.766819+0000","last_became_active":"2023-10-18T10:45:55.754723+0000","last_became_peered":"2023-10-18T10:45:55.754723+0000","last_unstale":"2023-10-18T10:46:00.766819+0000","last_undegraded":"2023-10-18T10:46:00.766819+0000","last_fullsized":"2023-10-18T10:46:00.766819+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:17:50.477786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.767162+0000","last_change":"2023-10-18T10:45:55.771484+0000","last_active":"2023-10-18T10:46:00.767162+0000","last_peered":"2023-10-18T10:46:00.767162+0000","last_clean":"2023-10-18T10:46:00.767162+0000","last_became_active":"2023-10-18T10:45:55.771108+0000","last_became_peered":"2023-10-18T10:45:55.771108+0000","last_unstale":"2023-10-18T10:46:00.767162+0000","last_undegraded":"2023-10-18T10:46:00.767162+0000","last_fullsized":"2023-10-18T10:46:00.767162+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:18.131261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768708+0000","last_change":"2023-10-18T10:45:55.778320+0000","last_active":"2023-10-18T10:46:00.768708+0000","last_peered":"2023-10-18T10:46:00.768708+0000","last_clean":"2023-10-18T10:46:00.768708+0000","last_became_active":"2023-10-18T10:45:55.777767+0000","last_became_peered":"2023-10-18T10:45:55.777767+0000","last_unstale":"2023-10-18T10:46:00.768708+0000","last_undegraded":"2023-10-18T10:46:00.768708+0000","last_fullsized":"2023-10-18T10:46:00.768708+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:05:56.688630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086126+0000","last_change":"2023-10-18T10:45:55.771747+0000","last_active":"2023-10-18T10:46:01.086126+0000","last_peered":"2023-10-18T10:46:01.086126+0000","last_clean":"2023-10-18T10:46:01.086126+0000","last_became_active":"2023-10-18T10:45:55.771297+0000","last_became_peered":"2023-10-18T10:45:55.771297+0000","last_unstale":"2023-10-18T10:46:01.086126+0000","last_undegraded":"2023-10-18T10:46:01.086126+0000","last_fullsized":"2023-10-18T10:46:01.086126+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:51:37.372456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085611+0000","last_change":"2023-10-18T10:45:55.756546+0000","last_active":"2023-10-18T10:46:01.085611+0000","last_peered":"2023-10-18T10:46:01.085611+0000","last_clean":"2023-10-18T10:46:01.085611+0000","last_became_active":"2023-10-18T10:45:55.756277+0000","last_became_peered":"2023-10-18T10:45:55.756277+0000","last_unstale":"2023-10-18T10:46:01.085611+0000","last_undegraded":"2023-10-18T10:46:01.085611+0000","last_fullsized":"2023-10-18T10:46:01.085611+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:33:34.507396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084785+0000","last_change":"2023-10-18T10:45:55.778442+0000","last_active":"2023-10-18T10:46:01.084785+0000","last_peered":"2023-10-18T10:46:01.084785+0000","last_clean":"2023-10-18T10:46:01.084785+0000","last_became_active":"2023-10-18T10:45:55.778198+0000","last_became_peered":"2023-10-18T10:45:55.778198+0000","last_unstale":"2023-10-18T10:46:01.084785+0000","last_undegraded":"2023-10-18T10:46:01.084785+0000","last_fullsized":"2023-10-18T10:46:01.084785+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:28:05.383698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085042+0000","last_change":"2023-10-18T10:45:55.779285+0000","last_active":"2023-10-18T10:46:01.085042+0000","last_peered":"2023-10-18T10:46:01.085042+0000","last_clean":"2023-10-18T10:46:01.085042+0000","last_became_active":"2023-10-18T10:45:55.778975+0000","last_became_peered":"2023-10-18T10:45:55.778975+0000","last_unstale":"2023-10-18T10:46:01.085042+0000","last_undegraded":"2023-10-18T10:46:01.085042+0000","last_fullsized":"2023-10-18T10:46:01.085042+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:33:49.751994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768170+0000","last_change":"2023-10-18T10:45:55.770892+0000","last_active":"2023-10-18T10:46:00.768170+0000","last_peered":"2023-10-18T10:46:00.768170+0000","last_clean":"2023-10-18T10:46:00.768170+0000","last_became_active":"2023-10-18T10:45:55.770181+0000","last_became_peered":"2023-10-18T10:45:55.770181+0000","last_unstale":"2023-10-18T10:46:00.768170+0000","last_undegraded":"2023-10-18T10:46:00.768170+0000","last_fullsized":"2023-10-18T10:46:00.768170+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:06:44.169699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086251+0000","last_change":"2023-10-18T10:45:55.759114+0000","last_active":"2023-10-18T10:46:01.086251+0000","last_peered":"2023-10-18T10:46:01.086251+0000","last_clean":"2023-10-18T10:46:01.086251+0000","last_became_active":"2023-10-18T10:45:55.758825+0000","last_became_peered":"2023-10-18T10:45:55.758825+0000","last_unstale":"2023-10-18T10:46:01.086251+0000","last_undegraded":"2023-10-18T10:46:01.086251+0000","last_fullsized":"2023-10-18T10:46:01.086251+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:14:00.849685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.770488+0000","last_change":"2023-10-18T10:45:55.763407+0000","last_active":"2023-10-18T10:46:00.770488+0000","last_peered":"2023-10-18T10:46:00.770488+0000","last_clean":"2023-10-18T10:46:00.770488+0000","last_became_active":"2023-10-18T10:45:55.763083+0000","last_became_peered":"2023-10-18T10:45:55.763083+0000","last_unstale":"2023-10-18T10:46:00.770488+0000","last_undegraded":"2023-10-18T10:46:00.770488+0000","last_fullsized":"2023-10-18T10:46:00.770488+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:30.982488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768596+0000","last_change":"2023-10-18T10:45:55.770469+0000","last_active":"2023-10-18T10:46:00.768596+0000","last_peered":"2023-10-18T10:46:00.768596+0000","last_clean":"2023-10-18T10:46:00.768596+0000","last_became_active":"2023-10-18T10:45:55.769998+0000","last_became_peered":"2023-10-18T10:45:55.769998+0000","last_unstale":"2023-10-18T10:46:00.768596+0000","last_undegraded":"2023-10-18T10:46:00.768596+0000","last_fullsized":"2023-10-18T10:46:00.768596+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:00:14.580995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085929+0000","last_change":"2023-10-18T10:45:55.781899+0000","last_active":"2023-10-18T10:46:01.085929+0000","last_peered":"2023-10-18T10:46:01.085929+0000","last_clean":"2023-10-18T10:46:01.085929+0000","last_became_active":"2023-10-18T10:45:55.781624+0000","last_became_peered":"2023-10-18T10:45:55.781624+0000","last_unstale":"2023-10-18T10:46:01.085929+0000","last_undegraded":"2023-10-18T10:46:01.085929+0000","last_fullsized":"2023-10-18T10:46:01.085929+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:43:01.654769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.767009+0000","last_change":"2023-10-18T10:45:55.757924+0000","last_active":"2023-10-18T10:46:00.767009+0000","last_peered":"2023-10-18T10:46:00.767009+0000","last_clean":"2023-10-18T10:46:00.767009+0000","last_became_active":"2023-10-18T10:45:55.757644+0000","last_became_peered":"2023-10-18T10:45:55.757644+0000","last_unstale":"2023-10-18T10:46:00.767009+0000","last_undegraded":"2023-10-18T10:46:00.767009+0000","last_fullsized":"2023-10-18T10:46:00.767009+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:08:54.769872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343932,"statfs":{"total":96636764160,"available":96608186368,"internally_reserved":0,"allocated":913408,"data_stored":714326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":120553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":311296,"data_stored":123798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":119991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":888832,"data_stored":710851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:46:14.896 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:46:15.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.066+0000 7f1f4cd59640 1 Processor -- start 2023-10-18T10:46:15.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.066+0000 7f1f4cd59640 1 -- start start 2023-10-18T10:46:15.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.066+0000 7f1f4cd59640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f4815c940 0x7f1f480d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:15.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.066+0000 7f1f4cd59640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f4815da50 con 0x7f1f480d2f40 2023-10-18T10:46:15.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.066+0000 7f1f4cd59640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f4815ce40 con 0x7f1f480d0e70 2023-10-18T10:46:15.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.066+0000 7f1f4cd59640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4815d120 con 0x7f1f4815c940 2023-10-18T10:46:15.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.066+0000 7f1f45d9b640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1f480d2f40 0x7f1f480d0760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35510/0 (socket says 172.21.15.89:35510) 2023-10-18T10:46:15.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.066+0000 7f1f45d9b640 1 -- 172.21.15.89:0/2660363655 learned_addr learned my addr 172.21.15.89:0/2660363655 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:15.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.066+0000 7f1f4559a640 1 -- 172.21.15.89:0/2660363655 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 178838647 0 0) 0x7f1f4815da50 con 0x7f1f480d2f40 2023-10-18T10:46:15.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.066+0000 7f1f4659c640 1 --2- 172.21.15.89:0/2660363655 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f4815c940 0x7f1f480d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:15.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.066+0000 7f1f4559a640 1 -- 172.21.15.89:0/2660363655 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f30003530 con 0x7f1f480d2f40 2023-10-18T10:46:15.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.066+0000 7f1f4559a640 1 -- 172.21.15.89:0/2660363655 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3086263810 0 0) 0x7f1f30003530 con 0x7f1f480d2f40 2023-10-18T10:46:15.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.066+0000 7f1f4559a640 1 -- 172.21.15.89:0/2660363655 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f4815c940 msgr2=0x7f1f480d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.066+0000 7f1f4559a640 1 --2- 172.21.15.89:0/2660363655 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f4815c940 0x7f1f480d2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.066+0000 7f1f4559a640 1 -- 172.21.15.89:0/2660363655 >> v1:172.21.15.154:6789/0 conn(0x7f1f480d0e70 legacy=0x7f1f480cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.066+0000 7f1f4559a640 1 -- 172.21.15.89:0/2660363655 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4815d400 con 0x7f1f480d2f40 2023-10-18T10:46:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.066+0000 7f1f4559a640 1 -- 172.21.15.89:0/2660363655 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7f1f3c001680 con 0x7f1f480d2f40 2023-10-18T10:46:15.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.066+0000 7f1f4559a640 1 -- 172.21.15.89:0/2660363655 >> v1:172.21.15.89:6789/0 conn(0x7f1f480d2f40 legacy=0x7f1f480d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:15.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.066+0000 7f1f4559a640 1 --2- 172.21.15.89:0/2660363655 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f300039b0 0x7f1f30003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:15.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.066+0000 7f1f4559a640 1 --2- 172.21.15.89:0/2660363655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f30004310 0x7f1f300047f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:15.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.066+0000 7f1f4559a640 1 --2- 172.21.15.89:0/2660363655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f30004ff0 0x7f1f3000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:15.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.066+0000 7f1f4559a640 1 -- 172.21.15.89:0/2660363655 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4815d120 con 0x7f1f30004ff0 2023-10-18T10:46:15.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.066+0000 7f1f4559a640 1 -- 172.21.15.89:0/2660363655 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3000bed0 con 0x7f1f30004310 2023-10-18T10:46:15.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.066+0000 7f1f4559a640 1 -- 172.21.15.89:0/2660363655 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3000c040 con 0x7f1f300039b0 2023-10-18T10:46:15.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.066+0000 7f1f46d9d640 1 --2- 172.21.15.89:0/2660363655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f30004ff0 0x7f1f3000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:15.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.066+0000 7f1f4659c640 1 --2- 172.21.15.89:0/2660363655 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f300039b0 0x7f1f30003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:15.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.066+0000 7f1f45d9b640 1 --2- 172.21.15.89:0/2660363655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f30004310 0x7f1f300047f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:15.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.066+0000 7f1f46d9d640 1 -- 172.21.15.89:0/2660363655 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f300039b0 msgr2=0x7f1f30003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:15.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.066+0000 7f1f46d9d640 1 --2- 172.21.15.89:0/2660363655 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f300039b0 0x7f1f30003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:15.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.066+0000 7f1f46d9d640 1 -- 172.21.15.89:0/2660363655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f30004310 msgr2=0x7f1f300047f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:15.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.066+0000 7f1f46d9d640 1 --2- 172.21.15.89:0/2660363655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f30004310 0x7f1f300047f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:15.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.066+0000 7f1f46d9d640 1 -- 172.21.15.89:0/2660363655 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f30003530 con 0x7f1f30004ff0 2023-10-18T10:46:15.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.070+0000 7f1f46d9d640 1 --2- 172.21.15.89:0/2660363655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f30004ff0 0x7f1f3000b990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1f40001e40 tx=0x7f1f4001c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=933a7bee7c554101 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:15.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.070+0000 7f1f4559a640 1 -- 172.21.15.89:0/2660363655 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f40003b80 con 0x7f1f30004ff0 2023-10-18T10:46:15.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.070+0000 7f1f4559a640 1 -- 172.21.15.89:0/2660363655 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f40003ce0 con 0x7f1f30004ff0 2023-10-18T10:46:15.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.070+0000 7f1f4559a640 1 -- 172.21.15.89:0/2660363655 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f400052c0 con 0x7f1f30004ff0 2023-10-18T10:46:15.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.070+0000 7f1f4cd59640 1 -- 172.21.15.89:0/2660363655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f30004ff0 msgr2=0x7f1f3000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:15.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.070+0000 7f1f4cd59640 1 --2- 172.21.15.89:0/2660363655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f30004ff0 0x7f1f3000b990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1f40001e40 tx=0x7f1f4001c740 comp rx=0 tx=0).stop 2023-10-18T10:46:15.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.070+0000 7f1f46d9d640 1 -- 172.21.15.89:0/2660363655 reap_dead start 2023-10-18T10:46:15.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.070+0000 7f1f4cd59640 1 -- 172.21.15.89:0/2660363655 shutdown_connections 2023-10-18T10:46:15.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.070+0000 7f1f4cd59640 1 -- 172.21.15.89:0/2660363655 >> 172.21.15.89:0/2660363655 conn(0x7f1f48089fd0 msgr2=0x7f1f4809b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:15.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.070+0000 7f1f4cd59640 1 -- 172.21.15.89:0/2660363655 shutdown_connections 2023-10-18T10:46:15.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.070+0000 7f1f4cd59640 1 -- 172.21.15.89:0/2660363655 wait complete. 2023-10-18T10:46:15.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.074+0000 7f1f4cd59640 1 Processor -- start 2023-10-18T10:46:15.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.074+0000 7f1f4cd59640 1 -- start start 2023-10-18T10:46:15.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.074+0000 7f1f4cd59640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f30004ff0 0x7f1f480b7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:15.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.074+0000 7f1f4cd59640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f48151bb0 0x7f1f480c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:15.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.074+0000 7f1f4cd59640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f480c8aa0 0x7f1f480c7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:15.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.074+0000 7f1f4cd59640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f480c8f20 con 0x7f1f480c8aa0 2023-10-18T10:46:15.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.074+0000 7f1f4cd59640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f480b6a40 con 0x7f1f30004ff0 2023-10-18T10:46:15.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.074+0000 7f1f4cd59640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f480b72e0 con 0x7f1f48151bb0 2023-10-18T10:46:15.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.074+0000 7f1f46d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f480c8aa0 0x7f1f480c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:15.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.074+0000 7f1f46d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f480c8aa0 0x7f1f480c7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45072/0 (socket says 172.21.15.89:45072) 2023-10-18T10:46:15.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.074+0000 7f1f46d9d640 1 -- 172.21.15.89:0/1132475352 learned_addr learned my addr 172.21.15.89:0/1132475352 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:15.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.074+0000 7f1f45d9b640 1 --2- 172.21.15.89:0/1132475352 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f48151bb0 0x7f1f480c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:15.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.074+0000 7f1f4659c640 1 --2- 172.21.15.89:0/1132475352 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f30004ff0 0x7f1f480b7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:15.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.074+0000 7f1f46d9d640 1 -- 172.21.15.89:0/1132475352 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f48151bb0 msgr2=0x7f1f480c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:15.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.074+0000 7f1f46d9d640 1 --2- 172.21.15.89:0/1132475352 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f48151bb0 0x7f1f480c8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:15.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.074+0000 7f1f46d9d640 1 -- 172.21.15.89:0/1132475352 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f30004ff0 msgr2=0x7f1f480b7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:15.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.074+0000 7f1f46d9d640 1 --2- 172.21.15.89:0/1132475352 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f30004ff0 0x7f1f480b7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:15.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.074+0000 7f1f46d9d640 1 -- 172.21.15.89:0/1132475352 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f480c7f90 con 0x7f1f480c8aa0 2023-10-18T10:46:15.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.074+0000 7f1f46d9d640 1 --2- 172.21.15.89:0/1132475352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f480c8aa0 0x7f1f480c7240 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1f4001cc20 tx=0x7f1f30005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:15.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.074+0000 7f1f36ffd640 1 -- 172.21.15.89:0/1132475352 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f40006d70 con 0x7f1f480c8aa0 2023-10-18T10:46:15.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.074+0000 7f1f36ffd640 1 -- 172.21.15.89:0/1132475352 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f40005030 con 0x7f1f480c8aa0 2023-10-18T10:46:15.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.074+0000 7f1f4cd59640 1 -- 172.21.15.89:0/1132475352 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f480b6c90 con 0x7f1f480c8aa0 2023-10-18T10:46:15.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.074+0000 7f1f36ffd640 1 -- 172.21.15.89:0/1132475352 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f400053a0 con 0x7f1f480c8aa0 2023-10-18T10:46:15.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.074+0000 7f1f4cd59640 1 -- 172.21.15.89:0/1132475352 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f480c7780 con 0x7f1f480c8aa0 2023-10-18T10:46:15.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.078+0000 7f1f4cd59640 1 -- 172.21.15.89:0/1132475352 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f480b7450 con 0x7f1f480c8aa0 2023-10-18T10:46:15.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.078+0000 7f1f36ffd640 1 -- 172.21.15.89:0/1132475352 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f1f40005500 con 0x7f1f480c8aa0 2023-10-18T10:46:15.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.078+0000 7f1f36ffd640 1 --2- 172.21.15.89:0/1132475352 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1f280656b0 0x7f1f28067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:15.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.078+0000 7f1f36ffd640 1 -- 172.21.15.89:0/1132475352 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1f4011ca10 con 0x7f1f480c8aa0 2023-10-18T10:46:15.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.078+0000 7f1f4659c640 1 --2- 172.21.15.89:0/1132475352 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1f280656b0 0x7f1f28067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:15.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.078+0000 7f1f4659c640 1 --2- 172.21.15.89:0/1132475352 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1f280656b0 0x7f1f28067b70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1f38009a30 tx=0x7f1f38008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:15.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.082+0000 7f1f36ffd640 1 -- 172.21.15.89:0/1132475352 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f1f400ea140 con 0x7f1f480c8aa0 2023-10-18T10:46:15.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.226+0000 7f1f4cd59640 1 -- 172.21.15.89:0/1132475352 --> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1f480c8f20 con 0x7f1f280656b0 2023-10-18T10:46:15.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.238+0000 7f1f36ffd640 1 -- 172.21.15.89:0/1132475352 <== mgr.4099 v2:172.21.15.154:6832/1086608968 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367424 (secure 0 0 0) 0x7f1f480c8f20 con 0x7f1f280656b0 2023-10-18T10:46:15.243 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:15.246 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-18T10:46:15.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.242+0000 7f1f4cd59640 1 -- 172.21.15.89:0/1132475352 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1f280656b0 msgr2=0x7f1f28067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:15.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.242+0000 7f1f4cd59640 1 --2- 172.21.15.89:0/1132475352 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1f280656b0 0x7f1f28067b70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1f38009a30 tx=0x7f1f38008040 comp rx=0 tx=0).stop 2023-10-18T10:46:15.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.242+0000 7f1f4cd59640 1 -- 172.21.15.89:0/1132475352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f480c8aa0 msgr2=0x7f1f480c7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:15.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.242+0000 7f1f4cd59640 1 --2- 172.21.15.89:0/1132475352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f480c8aa0 0x7f1f480c7240 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1f4001cc20 tx=0x7f1f30005800 comp rx=0 tx=0).stop 2023-10-18T10:46:15.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.242+0000 7f1f4cd59640 1 -- 172.21.15.89:0/1132475352 shutdown_connections 2023-10-18T10:46:15.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.242+0000 7f1f4cd59640 1 --2- 172.21.15.89:0/1132475352 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1f280656b0 0x7f1f28067b70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:15.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.242+0000 7f1f4cd59640 1 --2- 172.21.15.89:0/1132475352 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f48151bb0 0x7f1f480c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:15.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.242+0000 7f1f4cd59640 1 --2- 172.21.15.89:0/1132475352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f480c8aa0 0x7f1f480c7240 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:15.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.242+0000 7f1f4cd59640 1 --2- 172.21.15.89:0/1132475352 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f30004ff0 0x7f1f480b7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:15.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.242+0000 7f1f4cd59640 1 -- 172.21.15.89:0/1132475352 >> 172.21.15.89:0/1132475352 conn(0x7f1f48089fd0 msgr2=0x7f1f4815df50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:15.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.242+0000 7f1f4cd59640 1 -- 172.21.15.89:0/1132475352 shutdown_connections 2023-10-18T10:46:15.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.242+0000 7f1f4cd59640 1 -- 172.21.15.89:0/1132475352 wait complete. 2023-10-18T10:46:15.263 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2023-10-18T10:46:15.057592+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219660,"kb_used_data":3496,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754755060,"statfs":{"total":773094113280,"available":772869181440,"internally_reserved":0,"allocated":3579904,"data_stored":2149222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2229,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003521"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084532+0000","last_change":"2023-10-18T10:45:59.809013+0000","last_active":"2023-10-18T10:46:01.084532+0000","last_peered":"2023-10-18T10:46:01.084532+0000","last_clean":"2023-10-18T10:46:01.084532+0000","last_became_active":"2023-10-18T10:45:59.808636+0000","last_became_peered":"2023-10-18T10:45:59.808636+0000","last_unstale":"2023-10-18T10:46:01.084532+0000","last_undegraded":"2023-10-18T10:46:01.084532+0000","last_fullsized":"2023-10-18T10:46:01.084532+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:30:08.797534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085907+0000","last_change":"2023-10-18T10:45:55.297617+0000","last_active":"2023-10-18T10:46:01.085907+0000","last_peered":"2023-10-18T10:46:01.085907+0000","last_clean":"2023-10-18T10:46:01.085907+0000","last_became_active":"2023-10-18T10:45:55.296996+0000","last_became_peered":"2023-10-18T10:45:55.296996+0000","last_unstale":"2023-10-18T10:46:01.085907+0000","last_undegraded":"2023-10-18T10:46:01.085907+0000","last_fullsized":"2023-10-18T10:46:01.085907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:04:27.847910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768182+0000","last_change":"2023-10-18T10:45:55.770859+0000","last_active":"2023-10-18T10:46:00.768182+0000","last_peered":"2023-10-18T10:46:00.768182+0000","last_clean":"2023-10-18T10:46:00.768182+0000","last_became_active":"2023-10-18T10:45:55.769578+0000","last_became_peered":"2023-10-18T10:45:55.769578+0000","last_unstale":"2023-10-18T10:46:00.768182+0000","last_undegraded":"2023-10-18T10:46:00.768182+0000","last_fullsized":"2023-10-18T10:46:00.768182+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:08:17.812888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.767991+0000","last_change":"2023-10-18T10:45:59.790837+0000","last_active":"2023-10-18T10:46:00.767991+0000","last_peered":"2023-10-18T10:46:00.767991+0000","last_clean":"2023-10-18T10:46:00.767991+0000","last_became_active":"2023-10-18T10:45:59.790488+0000","last_became_peered":"2023-10-18T10:45:59.790488+0000","last_unstale":"2023-10-18T10:46:00.767991+0000","last_undegraded":"2023-10-18T10:46:00.767991+0000","last_fullsized":"2023-10-18T10:46:00.767991+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:43:23.609561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902666+0000","last_change":"2023-10-18T10:45:55.295114+0000","last_active":"2023-10-18T10:46:00.902666+0000","last_peered":"2023-10-18T10:46:00.902666+0000","last_clean":"2023-10-18T10:46:00.902666+0000","last_became_active":"2023-10-18T10:45:55.294672+0000","last_became_peered":"2023-10-18T10:45:55.294672+0000","last_unstale":"2023-10-18T10:46:00.902666+0000","last_undegraded":"2023-10-18T10:46:00.902666+0000","last_fullsized":"2023-10-18T10:46:00.902666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:06:43.162381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902162+0000","last_change":"2023-10-18T10:45:55.756885+0000","last_active":"2023-10-18T10:46:00.902162+0000","last_peered":"2023-10-18T10:46:00.902162+0000","last_clean":"2023-10-18T10:46:00.902162+0000","last_became_active":"2023-10-18T10:45:55.756184+0000","last_became_peered":"2023-10-18T10:45:55.756184+0000","last_unstale":"2023-10-18T10:46:00.902162+0000","last_undegraded":"2023-10-18T10:46:00.902162+0000","last_fullsized":"2023-10-18T10:46:00.902162+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:20:46.641913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085021+0000","last_change":"2023-10-18T10:45:59.796520+0000","last_active":"2023-10-18T10:46:01.085021+0000","last_peered":"2023-10-18T10:46:01.085021+0000","last_clean":"2023-10-18T10:46:01.085021+0000","last_became_active":"2023-10-18T10:45:59.796056+0000","last_became_peered":"2023-10-18T10:45:59.796056+0000","last_unstale":"2023-10-18T10:46:01.085021+0000","last_undegraded":"2023-10-18T10:46:01.085021+0000","last_fullsized":"2023-10-18T10:46:01.085021+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:55:12.299657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085752+0000","last_change":"2023-10-18T10:45:55.302985+0000","last_active":"2023-10-18T10:46:01.085752+0000","last_peered":"2023-10-18T10:46:01.085752+0000","last_clean":"2023-10-18T10:46:01.085752+0000","last_became_active":"2023-10-18T10:45:55.302657+0000","last_became_peered":"2023-10-18T10:45:55.302657+0000","last_unstale":"2023-10-18T10:46:01.085752+0000","last_undegraded":"2023-10-18T10:46:01.085752+0000","last_fullsized":"2023-10-18T10:46:01.085752+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:55:03.412092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768360+0000","last_change":"2023-10-18T10:45:55.776143+0000","last_active":"2023-10-18T10:46:00.768360+0000","last_peered":"2023-10-18T10:46:00.768360+0000","last_clean":"2023-10-18T10:46:00.768360+0000","last_became_active":"2023-10-18T10:45:55.770969+0000","last_became_peered":"2023-10-18T10:45:55.770969+0000","last_unstale":"2023-10-18T10:46:00.768360+0000","last_undegraded":"2023-10-18T10:46:00.768360+0000","last_fullsized":"2023-10-18T10:46:00.768360+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:53:59.036438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085289+0000","last_change":"2023-10-18T10:45:59.810699+0000","last_active":"2023-10-18T10:46:01.085289+0000","last_peered":"2023-10-18T10:46:01.085289+0000","last_clean":"2023-10-18T10:46:01.085289+0000","last_became_active":"2023-10-18T10:45:59.810219+0000","last_became_peered":"2023-10-18T10:45:59.810219+0000","last_unstale":"2023-10-18T10:46:01.085289+0000","last_undegraded":"2023-10-18T10:46:01.085289+0000","last_fullsized":"2023-10-18T10:46:01.085289+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:06:30.723321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085485+0000","last_change":"2023-10-18T10:45:55.300287+0000","last_active":"2023-10-18T10:46:01.085485+0000","last_peered":"2023-10-18T10:46:01.085485+0000","last_clean":"2023-10-18T10:46:01.085485+0000","last_became_active":"2023-10-18T10:45:55.300014+0000","last_became_peered":"2023-10-18T10:45:55.300014+0000","last_unstale":"2023-10-18T10:46:01.085485+0000","last_undegraded":"2023-10-18T10:46:01.085485+0000","last_fullsized":"2023-10-18T10:46:01.085485+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:40:14.773073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768476+0000","last_change":"2023-10-18T10:45:55.772481+0000","last_active":"2023-10-18T10:46:00.768476+0000","last_peered":"2023-10-18T10:46:00.768476+0000","last_clean":"2023-10-18T10:46:00.768476+0000","last_became_active":"2023-10-18T10:45:55.770365+0000","last_became_peered":"2023-10-18T10:45:55.770365+0000","last_unstale":"2023-10-18T10:46:00.768476+0000","last_undegraded":"2023-10-18T10:46:00.768476+0000","last_fullsized":"2023-10-18T10:46:00.768476+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:21:41.323647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085226+0000","last_change":"2023-10-18T10:45:59.796654+0000","last_active":"2023-10-18T10:46:01.085226+0000","last_peered":"2023-10-18T10:46:01.085226+0000","last_clean":"2023-10-18T10:46:01.085226+0000","last_became_active":"2023-10-18T10:45:59.796115+0000","last_became_peered":"2023-10-18T10:45:59.796115+0000","last_unstale":"2023-10-18T10:46:01.085226+0000","last_undegraded":"2023-10-18T10:46:01.085226+0000","last_fullsized":"2023-10-18T10:46:01.085226+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:28:08.869152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086358+0000","last_change":"2023-10-18T10:45:55.302642+0000","last_active":"2023-10-18T10:46:01.086358+0000","last_peered":"2023-10-18T10:46:01.086358+0000","last_clean":"2023-10-18T10:46:01.086358+0000","last_became_active":"2023-10-18T10:45:55.302307+0000","last_became_peered":"2023-10-18T10:45:55.302307+0000","last_unstale":"2023-10-18T10:46:01.086358+0000","last_undegraded":"2023-10-18T10:46:01.086358+0000","last_fullsized":"2023-10-18T10:46:01.086358+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:40:52.185453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.767207+0000","last_change":"2023-10-18T10:45:55.765319+0000","last_active":"2023-10-18T10:46:00.767207+0000","last_peered":"2023-10-18T10:46:00.767207+0000","last_clean":"2023-10-18T10:46:00.767207+0000","last_became_active":"2023-10-18T10:45:55.764958+0000","last_became_peered":"2023-10-18T10:45:55.764958+0000","last_unstale":"2023-10-18T10:46:00.767207+0000","last_undegraded":"2023-10-18T10:46:00.767207+0000","last_fullsized":"2023-10-18T10:46:00.767207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:30:49.094396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768337+0000","last_change":"2023-10-18T10:45:59.791333+0000","last_active":"2023-10-18T10:46:00.768337+0000","last_peered":"2023-10-18T10:46:00.768337+0000","last_clean":"2023-10-18T10:46:00.768337+0000","last_became_active":"2023-10-18T10:45:59.790903+0000","last_became_peered":"2023-10-18T10:45:59.790903+0000","last_unstale":"2023-10-18T10:46:00.768337+0000","last_undegraded":"2023-10-18T10:46:00.768337+0000","last_fullsized":"2023-10-18T10:46:00.768337+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:52:13.857250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768597+0000","last_change":"2023-10-18T10:45:55.302330+0000","last_active":"2023-10-18T10:46:00.768597+0000","last_peered":"2023-10-18T10:46:00.768597+0000","last_clean":"2023-10-18T10:46:00.768597+0000","last_became_active":"2023-10-18T10:45:55.301709+0000","last_became_peered":"2023-10-18T10:45:55.301709+0000","last_unstale":"2023-10-18T10:46:00.768597+0000","last_undegraded":"2023-10-18T10:46:00.768597+0000","last_fullsized":"2023-10-18T10:46:00.768597+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:40:49.128175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902373+0000","last_change":"2023-10-18T10:45:55.767859+0000","last_active":"2023-10-18T10:46:00.902373+0000","last_peered":"2023-10-18T10:46:00.902373+0000","last_clean":"2023-10-18T10:46:00.902373+0000","last_became_active":"2023-10-18T10:45:55.767505+0000","last_became_peered":"2023-10-18T10:45:55.767505+0000","last_unstale":"2023-10-18T10:46:00.902373+0000","last_undegraded":"2023-10-18T10:46:00.902373+0000","last_fullsized":"2023-10-18T10:46:00.902373+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:12:23.654089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.770164+0000","last_change":"2023-10-18T10:45:59.794892+0000","last_active":"2023-10-18T10:46:00.770164+0000","last_peered":"2023-10-18T10:46:00.770164+0000","last_clean":"2023-10-18T10:46:00.770164+0000","last_became_active":"2023-10-18T10:45:59.794477+0000","last_became_peered":"2023-10-18T10:45:59.794477+0000","last_unstale":"2023-10-18T10:46:00.770164+0000","last_undegraded":"2023-10-18T10:46:00.770164+0000","last_fullsized":"2023-10-18T10:46:00.770164+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:04:07.040979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084932+0000","last_change":"2023-10-18T10:45:55.290491+0000","last_active":"2023-10-18T10:46:01.084932+0000","last_peered":"2023-10-18T10:46:01.084932+0000","last_clean":"2023-10-18T10:46:01.084932+0000","last_became_active":"2023-10-18T10:45:55.290134+0000","last_became_peered":"2023-10-18T10:45:55.290134+0000","last_unstale":"2023-10-18T10:46:01.084932+0000","last_undegraded":"2023-10-18T10:46:01.084932+0000","last_fullsized":"2023-10-18T10:46:01.084932+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:39.977541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086354+0000","last_change":"2023-10-18T10:45:55.765415+0000","last_active":"2023-10-18T10:46:01.086354+0000","last_peered":"2023-10-18T10:46:01.086354+0000","last_clean":"2023-10-18T10:46:01.086354+0000","last_became_active":"2023-10-18T10:45:55.764573+0000","last_became_peered":"2023-10-18T10:45:55.764573+0000","last_unstale":"2023-10-18T10:46:01.086354+0000","last_undegraded":"2023-10-18T10:46:01.086354+0000","last_fullsized":"2023-10-18T10:46:01.086354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:33:49.751994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086225+0000","last_change":"2023-10-18T10:45:59.801842+0000","last_active":"2023-10-18T10:46:01.086225+0000","last_peered":"2023-10-18T10:46:01.086225+0000","last_clean":"2023-10-18T10:46:01.086225+0000","last_became_active":"2023-10-18T10:45:59.801367+0000","last_became_peered":"2023-10-18T10:45:59.801367+0000","last_unstale":"2023-10-18T10:46:01.086225+0000","last_undegraded":"2023-10-18T10:46:01.086225+0000","last_fullsized":"2023-10-18T10:46:01.086225+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:56:46.630947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902154+0000","last_change":"2023-10-18T10:45:55.298611+0000","last_active":"2023-10-18T10:46:00.902154+0000","last_peered":"2023-10-18T10:46:00.902154+0000","last_clean":"2023-10-18T10:46:00.902154+0000","last_became_active":"2023-10-18T10:45:55.297980+0000","last_became_peered":"2023-10-18T10:45:55.297980+0000","last_unstale":"2023-10-18T10:46:00.902154+0000","last_undegraded":"2023-10-18T10:46:00.902154+0000","last_fullsized":"2023-10-18T10:46:00.902154+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:05:56.466612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.770285+0000","last_change":"2023-10-18T10:45:55.775435+0000","last_active":"2023-10-18T10:46:00.770285+0000","last_peered":"2023-10-18T10:46:00.770285+0000","last_clean":"2023-10-18T10:46:00.770285+0000","last_became_active":"2023-10-18T10:45:55.774841+0000","last_became_peered":"2023-10-18T10:45:55.774841+0000","last_unstale":"2023-10-18T10:46:00.770285+0000","last_undegraded":"2023-10-18T10:46:00.770285+0000","last_fullsized":"2023-10-18T10:46:00.770285+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:27:03.226674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.901875+0000","last_change":"2023-10-18T10:45:55.768221+0000","last_active":"2023-10-18T10:46:00.901875+0000","last_peered":"2023-10-18T10:46:00.901875+0000","last_clean":"2023-10-18T10:46:00.901875+0000","last_became_active":"2023-10-18T10:45:55.767975+0000","last_became_peered":"2023-10-18T10:45:55.767975+0000","last_unstale":"2023-10-18T10:46:00.901875+0000","last_undegraded":"2023-10-18T10:46:00.901875+0000","last_fullsized":"2023-10-18T10:46:00.901875+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:14:00.849685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086003+0000","last_change":"2023-10-18T10:45:55.291798+0000","last_active":"2023-10-18T10:46:01.086003+0000","last_peered":"2023-10-18T10:46:01.086003+0000","last_clean":"2023-10-18T10:46:01.086003+0000","last_became_active":"2023-10-18T10:45:55.291380+0000","last_became_peered":"2023-10-18T10:45:55.291380+0000","last_unstale":"2023-10-18T10:46:01.086003+0000","last_undegraded":"2023-10-18T10:46:01.086003+0000","last_fullsized":"2023-10-18T10:46:01.086003+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:49:45.673807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.769758+0000","last_change":"2023-10-18T10:45:59.794170+0000","last_active":"2023-10-18T10:46:00.769758+0000","last_peered":"2023-10-18T10:46:00.769758+0000","last_clean":"2023-10-18T10:46:00.769758+0000","last_became_active":"2023-10-18T10:45:59.793675+0000","last_became_peered":"2023-10-18T10:45:59.793675+0000","last_unstale":"2023-10-18T10:46:00.769758+0000","last_undegraded":"2023-10-18T10:46:00.769758+0000","last_fullsized":"2023-10-18T10:46:00.769758+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:08:00.737560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.901966+0000","last_change":"2023-10-18T10:45:55.765758+0000","last_active":"2023-10-18T10:46:00.901966+0000","last_peered":"2023-10-18T10:46:00.901966+0000","last_clean":"2023-10-18T10:46:00.901966+0000","last_became_active":"2023-10-18T10:45:55.765436+0000","last_became_peered":"2023-10-18T10:45:55.765436+0000","last_unstale":"2023-10-18T10:46:00.901966+0000","last_undegraded":"2023-10-18T10:46:00.901966+0000","last_fullsized":"2023-10-18T10:46:00.901966+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:40:53.192771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086132+0000","last_change":"2023-10-18T10:45:55.295720+0000","last_active":"2023-10-18T10:46:01.086132+0000","last_peered":"2023-10-18T10:46:01.086132+0000","last_clean":"2023-10-18T10:46:01.086132+0000","last_became_active":"2023-10-18T10:45:55.295401+0000","last_became_peered":"2023-10-18T10:45:55.295401+0000","last_unstale":"2023-10-18T10:46:01.086132+0000","last_undegraded":"2023-10-18T10:46:01.086132+0000","last_fullsized":"2023-10-18T10:46:01.086132+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:49:45.673807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084670+0000","last_change":"2023-10-18T10:45:59.798312+0000","last_active":"2023-10-18T10:46:01.084670+0000","last_peered":"2023-10-18T10:46:01.084670+0000","last_clean":"2023-10-18T10:46:01.084670+0000","last_became_active":"2023-10-18T10:45:59.797944+0000","last_became_peered":"2023-10-18T10:45:59.797944+0000","last_unstale":"2023-10-18T10:46:01.084670+0000","last_undegraded":"2023-10-18T10:46:01.084670+0000","last_fullsized":"2023-10-18T10:46:01.084670+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:51:40.857910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085288+0000","last_change":"2023-10-18T10:45:55.307908+0000","last_active":"2023-10-18T10:46:01.085288+0000","last_peered":"2023-10-18T10:46:01.085288+0000","last_clean":"2023-10-18T10:46:01.085288+0000","last_became_active":"2023-10-18T10:45:55.307593+0000","last_became_peered":"2023-10-18T10:45:55.307593+0000","last_unstale":"2023-10-18T10:46:01.085288+0000","last_undegraded":"2023-10-18T10:46:01.085288+0000","last_fullsized":"2023-10-18T10:46:01.085288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:20:45.634595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084888+0000","last_change":"2023-10-18T10:45:55.778356+0000","last_active":"2023-10-18T10:46:01.084888+0000","last_peered":"2023-10-18T10:46:01.084888+0000","last_clean":"2023-10-18T10:46:01.084888+0000","last_became_active":"2023-10-18T10:45:55.776465+0000","last_became_peered":"2023-10-18T10:45:55.776465+0000","last_unstale":"2023-10-18T10:46:01.084888+0000","last_undegraded":"2023-10-18T10:46:01.084888+0000","last_fullsized":"2023-10-18T10:46:01.084888+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:10:59.545961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902026+0000","last_change":"2023-10-18T10:45:59.788972+0000","last_active":"2023-10-18T10:46:00.902026+0000","last_peered":"2023-10-18T10:46:00.902026+0000","last_clean":"2023-10-18T10:46:00.902026+0000","last_became_active":"2023-10-18T10:45:59.788613+0000","last_became_peered":"2023-10-18T10:45:59.788613+0000","last_unstale":"2023-10-18T10:46:00.902026+0000","last_undegraded":"2023-10-18T10:46:00.902026+0000","last_fullsized":"2023-10-18T10:46:00.902026+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:33:04.407310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"27'9","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:06.342753+0000","last_change":"2023-10-18T10:45:55.298794+0000","last_active":"2023-10-18T10:46:06.342753+0000","last_peered":"2023-10-18T10:46:06.342753+0000","last_clean":"2023-10-18T10:46:06.342753+0000","last_became_active":"2023-10-18T10:45:55.297181+0000","last_became_peered":"2023-10-18T10:45:55.297181+0000","last_unstale":"2023-10-18T10:46:06.342753+0000","last_undegraded":"2023-10-18T10:46:06.342753+0000","last_fullsized":"2023-10-18T10:46:06.342753+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:16:43.562003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768593+0000","last_change":"2023-10-18T10:45:55.777329+0000","last_active":"2023-10-18T10:46:00.768593+0000","last_peered":"2023-10-18T10:46:00.768593+0000","last_clean":"2023-10-18T10:46:00.768593+0000","last_became_active":"2023-10-18T10:45:55.777047+0000","last_became_peered":"2023-10-18T10:45:55.777047+0000","last_unstale":"2023-10-18T10:46:00.768593+0000","last_undegraded":"2023-10-18T10:46:00.768593+0000","last_fullsized":"2023-10-18T10:46:00.768593+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:09:20.566242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084662+0000","last_change":"2023-10-18T10:45:59.804498+0000","last_active":"2023-10-18T10:46:01.084662+0000","last_peered":"2023-10-18T10:46:01.084662+0000","last_clean":"2023-10-18T10:46:01.084662+0000","last_became_active":"2023-10-18T10:45:59.804083+0000","last_became_peered":"2023-10-18T10:45:59.804083+0000","last_unstale":"2023-10-18T10:46:01.084662+0000","last_undegraded":"2023-10-18T10:46:01.084662+0000","last_fullsized":"2023-10-18T10:46:01.084662+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:04:58.822427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.769276+0000","last_change":"2023-10-18T10:45:55.774541+0000","last_active":"2023-10-18T10:46:00.769276+0000","last_peered":"2023-10-18T10:46:00.769276+0000","last_clean":"2023-10-18T10:46:00.769276+0000","last_became_active":"2023-10-18T10:45:55.774258+0000","last_became_peered":"2023-10-18T10:45:55.774258+0000","last_unstale":"2023-10-18T10:46:00.769276+0000","last_undegraded":"2023-10-18T10:46:00.769276+0000","last_fullsized":"2023-10-18T10:46:00.769276+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:11:42.497290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.770380+0000","last_change":"2023-10-18T10:45:50.252021+0000","last_active":"2023-10-18T10:46:00.770380+0000","last_peered":"2023-10-18T10:46:00.770380+0000","last_clean":"2023-10-18T10:46:00.770380+0000","last_became_active":"2023-10-18T10:45:50.251427+0000","last_became_peered":"2023-10-18T10:45:50.251427+0000","last_unstale":"2023-10-18T10:46:00.770380+0000","last_undegraded":"2023-10-18T10:46:00.770380+0000","last_fullsized":"2023-10-18T10:46:00.770380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:43.575003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"26'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086531+0000","last_change":"2023-10-18T10:45:55.296927+0000","last_active":"2023-10-18T10:46:01.086531+0000","last_peered":"2023-10-18T10:46:01.086531+0000","last_clean":"2023-10-18T10:46:01.086531+0000","last_became_active":"2023-10-18T10:45:55.296614+0000","last_became_peered":"2023-10-18T10:45:55.296614+0000","last_unstale":"2023-10-18T10:46:01.086531+0000","last_undegraded":"2023-10-18T10:46:01.086531+0000","last_fullsized":"2023-10-18T10:46:01.086531+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:05:56.466612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902246+0000","last_change":"2023-10-18T10:45:59.786937+0000","last_active":"2023-10-18T10:46:00.902246+0000","last_peered":"2023-10-18T10:46:00.902246+0000","last_clean":"2023-10-18T10:46:00.902246+0000","last_became_active":"2023-10-18T10:45:59.786436+0000","last_became_peered":"2023-10-18T10:45:59.786436+0000","last_unstale":"2023-10-18T10:46:00.902246+0000","last_undegraded":"2023-10-18T10:46:00.902246+0000","last_fullsized":"2023-10-18T10:46:00.902246+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:37:40.801018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902006+0000","last_change":"2023-10-18T10:45:55.766947+0000","last_active":"2023-10-18T10:46:00.902006+0000","last_peered":"2023-10-18T10:46:00.902006+0000","last_clean":"2023-10-18T10:46:00.902006+0000","last_became_active":"2023-10-18T10:45:55.765951+0000","last_became_peered":"2023-10-18T10:45:55.765951+0000","last_unstale":"2023-10-18T10:46:00.902006+0000","last_undegraded":"2023-10-18T10:46:00.902006+0000","last_fullsized":"2023-10-18T10:46:00.902006+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:06:27.237867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085276+0000","last_change":"2023-10-18T10:45:50.252876+0000","last_active":"2023-10-18T10:46:01.085276+0000","last_peered":"2023-10-18T10:46:01.085276+0000","last_clean":"2023-10-18T10:46:01.085276+0000","last_became_active":"2023-10-18T10:45:50.252559+0000","last_became_peered":"2023-10-18T10:45:50.252559+0000","last_unstale":"2023-10-18T10:46:01.085276+0000","last_undegraded":"2023-10-18T10:46:01.085276+0000","last_fullsized":"2023-10-18T10:46:01.085276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:55:15.913186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902858+0000","last_change":"2023-10-18T10:45:55.300421+0000","last_active":"2023-10-18T10:46:00.902858+0000","last_peered":"2023-10-18T10:46:00.902858+0000","last_clean":"2023-10-18T10:46:00.902858+0000","last_became_active":"2023-10-18T10:45:55.300170+0000","last_became_peered":"2023-10-18T10:45:55.300170+0000","last_unstale":"2023-10-18T10:46:00.902858+0000","last_undegraded":"2023-10-18T10:46:00.902858+0000","last_fullsized":"2023-10-18T10:46:00.902858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:52:09.364478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768261+0000","last_change":"2023-10-18T10:45:59.799805+0000","last_active":"2023-10-18T10:46:00.768261+0000","last_peered":"2023-10-18T10:46:00.768261+0000","last_clean":"2023-10-18T10:46:00.768261+0000","last_became_active":"2023-10-18T10:45:59.799112+0000","last_became_peered":"2023-10-18T10:45:59.799112+0000","last_unstale":"2023-10-18T10:46:00.768261+0000","last_undegraded":"2023-10-18T10:46:00.768261+0000","last_fullsized":"2023-10-18T10:46:00.768261+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:49:10.356115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086095+0000","last_change":"2023-10-18T10:45:55.774299+0000","last_active":"2023-10-18T10:46:01.086095+0000","last_peered":"2023-10-18T10:46:01.086095+0000","last_clean":"2023-10-18T10:46:01.086095+0000","last_became_active":"2023-10-18T10:45:55.773409+0000","last_became_peered":"2023-10-18T10:45:55.773409+0000","last_unstale":"2023-10-18T10:46:01.086095+0000","last_undegraded":"2023-10-18T10:46:01.086095+0000","last_fullsized":"2023-10-18T10:46:01.086095+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:37:37.315564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085581+0000","last_change":"2023-10-18T10:45:50.585224+0000","last_active":"2023-10-18T10:46:01.085581+0000","last_peered":"2023-10-18T10:46:01.085581+0000","last_clean":"2023-10-18T10:46:01.085581+0000","last_became_active":"2023-10-18T10:45:50.584872+0000","last_became_peered":"2023-10-18T10:45:50.584872+0000","last_unstale":"2023-10-18T10:46:01.085581+0000","last_undegraded":"2023-10-18T10:46:01.085581+0000","last_fullsized":"2023-10-18T10:46:01.085581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:07:41.932960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085445+0000","last_change":"2023-10-18T10:45:55.298601+0000","last_active":"2023-10-18T10:46:01.085445+0000","last_peered":"2023-10-18T10:46:01.085445+0000","last_clean":"2023-10-18T10:46:01.085445+0000","last_became_active":"2023-10-18T10:45:55.295209+0000","last_became_peered":"2023-10-18T10:45:55.295209+0000","last_unstale":"2023-10-18T10:46:01.085445+0000","last_undegraded":"2023-10-18T10:46:01.085445+0000","last_fullsized":"2023-10-18T10:46:01.085445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:55:07.806885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.769905+0000","last_change":"2023-10-18T10:45:59.787239+0000","last_active":"2023-10-18T10:46:00.769905+0000","last_peered":"2023-10-18T10:46:00.769905+0000","last_clean":"2023-10-18T10:46:00.769905+0000","last_became_active":"2023-10-18T10:45:59.786770+0000","last_became_peered":"2023-10-18T10:45:59.786770+0000","last_unstale":"2023-10-18T10:46:00.769905+0000","last_undegraded":"2023-10-18T10:46:00.769905+0000","last_fullsized":"2023-10-18T10:46:00.769905+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:09:26.681314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084957+0000","last_change":"2023-10-18T10:45:55.770838+0000","last_active":"2023-10-18T10:46:01.084957+0000","last_peered":"2023-10-18T10:46:01.084957+0000","last_clean":"2023-10-18T10:46:01.084957+0000","last_became_active":"2023-10-18T10:45:55.770308+0000","last_became_peered":"2023-10-18T10:45:55.770308+0000","last_unstale":"2023-10-18T10:46:01.084957+0000","last_undegraded":"2023-10-18T10:46:01.084957+0000","last_fullsized":"2023-10-18T10:46:01.084957+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:42:54.758946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085651+0000","last_change":"2023-10-18T10:45:50.584946+0000","last_active":"2023-10-18T10:46:01.085651+0000","last_peered":"2023-10-18T10:46:01.085651+0000","last_clean":"2023-10-18T10:46:01.085651+0000","last_became_active":"2023-10-18T10:45:50.584638+0000","last_became_peered":"2023-10-18T10:45:50.584638+0000","last_unstale":"2023-10-18T10:46:01.085651+0000","last_undegraded":"2023-10-18T10:46:01.085651+0000","last_fullsized":"2023-10-18T10:46:01.085651+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:40:09.742761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085828+0000","last_change":"2023-10-18T10:45:55.299483+0000","last_active":"2023-10-18T10:46:01.085828+0000","last_peered":"2023-10-18T10:46:01.085828+0000","last_clean":"2023-10-18T10:46:01.085828+0000","last_became_active":"2023-10-18T10:45:55.299195+0000","last_became_peered":"2023-10-18T10:45:55.299195+0000","last_unstale":"2023-10-18T10:46:01.085828+0000","last_undegraded":"2023-10-18T10:46:01.085828+0000","last_fullsized":"2023-10-18T10:46:01.085828+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:25:17.957984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085926+0000","last_change":"2023-10-18T10:45:59.801291+0000","last_active":"2023-10-18T10:46:01.085926+0000","last_peered":"2023-10-18T10:46:01.085926+0000","last_clean":"2023-10-18T10:46:01.085926+0000","last_became_active":"2023-10-18T10:45:59.800819+0000","last_became_peered":"2023-10-18T10:45:59.800819+0000","last_unstale":"2023-10-18T10:46:01.085926+0000","last_undegraded":"2023-10-18T10:46:01.085926+0000","last_fullsized":"2023-10-18T10:46:01.085926+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:06:30.412609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086211+0000","last_change":"2023-10-18T10:45:55.773950+0000","last_active":"2023-10-18T10:46:01.086211+0000","last_peered":"2023-10-18T10:46:01.086211+0000","last_clean":"2023-10-18T10:46:01.086211+0000","last_became_active":"2023-10-18T10:45:55.769581+0000","last_became_peered":"2023-10-18T10:45:55.769581+0000","last_unstale":"2023-10-18T10:46:01.086211+0000","last_undegraded":"2023-10-18T10:46:01.086211+0000","last_fullsized":"2023-10-18T10:46:01.086211+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:31:36.488372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768705+0000","last_change":"2023-10-18T10:45:55.297879+0000","last_active":"2023-10-18T10:46:00.768705+0000","last_peered":"2023-10-18T10:46:00.768705+0000","last_clean":"2023-10-18T10:46:00.768705+0000","last_became_active":"2023-10-18T10:45:55.297097+0000","last_became_peered":"2023-10-18T10:45:55.297097+0000","last_unstale":"2023-10-18T10:46:00.768705+0000","last_undegraded":"2023-10-18T10:46:00.768705+0000","last_fullsized":"2023-10-18T10:46:00.768705+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:27:53.149351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086078+0000","last_change":"2023-10-18T10:45:50.251505+0000","last_active":"2023-10-18T10:46:01.086078+0000","last_peered":"2023-10-18T10:46:01.086078+0000","last_clean":"2023-10-18T10:46:01.086078+0000","last_became_active":"2023-10-18T10:45:50.251111+0000","last_became_peered":"2023-10-18T10:45:50.251111+0000","last_unstale":"2023-10-18T10:46:01.086078+0000","last_undegraded":"2023-10-18T10:46:01.086078+0000","last_fullsized":"2023-10-18T10:46:01.086078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:04:22.817598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085546+0000","last_change":"2023-10-18T10:45:59.800297+0000","last_active":"2023-10-18T10:46:01.085546+0000","last_peered":"2023-10-18T10:46:01.085546+0000","last_clean":"2023-10-18T10:46:01.085546+0000","last_became_active":"2023-10-18T10:45:59.799434+0000","last_became_peered":"2023-10-18T10:45:59.799434+0000","last_unstale":"2023-10-18T10:46:01.085546+0000","last_undegraded":"2023-10-18T10:46:01.085546+0000","last_fullsized":"2023-10-18T10:46:01.085546+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:55:07.904865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086122+0000","last_change":"2023-10-18T10:45:55.774137+0000","last_active":"2023-10-18T10:46:01.086122+0000","last_peered":"2023-10-18T10:46:01.086122+0000","last_clean":"2023-10-18T10:46:01.086122+0000","last_became_active":"2023-10-18T10:45:55.773832+0000","last_became_peered":"2023-10-18T10:45:55.773832+0000","last_unstale":"2023-10-18T10:46:01.086122+0000","last_undegraded":"2023-10-18T10:46:01.086122+0000","last_fullsized":"2023-10-18T10:46:01.086122+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:31:21.466919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084731+0000","last_change":"2023-10-18T10:45:55.294225+0000","last_active":"2023-10-18T10:46:01.084731+0000","last_peered":"2023-10-18T10:46:01.084731+0000","last_clean":"2023-10-18T10:46:01.084731+0000","last_became_active":"2023-10-18T10:45:55.292869+0000","last_became_peered":"2023-10-18T10:45:55.292869+0000","last_unstale":"2023-10-18T10:46:01.084731+0000","last_undegraded":"2023-10-18T10:46:01.084731+0000","last_fullsized":"2023-10-18T10:46:01.084731+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:18:31.743377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.766604+0000","last_change":"2023-10-18T10:45:50.259019+0000","last_active":"2023-10-18T10:46:00.766604+0000","last_peered":"2023-10-18T10:46:00.766604+0000","last_clean":"2023-10-18T10:46:00.766604+0000","last_became_active":"2023-10-18T10:45:50.258607+0000","last_became_peered":"2023-10-18T10:45:50.258607+0000","last_unstale":"2023-10-18T10:46:00.766604+0000","last_undegraded":"2023-10-18T10:46:00.766604+0000","last_fullsized":"2023-10-18T10:46:00.766604+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:14:34.875058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.767128+0000","last_change":"2023-10-18T10:45:59.803747+0000","last_active":"2023-10-18T10:46:00.767128+0000","last_peered":"2023-10-18T10:46:00.767128+0000","last_clean":"2023-10-18T10:46:00.767128+0000","last_became_active":"2023-10-18T10:45:59.795250+0000","last_became_peered":"2023-10-18T10:45:59.795250+0000","last_unstale":"2023-10-18T10:46:00.767128+0000","last_undegraded":"2023-10-18T10:46:00.767128+0000","last_fullsized":"2023-10-18T10:46:00.767128+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:25.942126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086446+0000","last_change":"2023-10-18T10:45:55.763100+0000","last_active":"2023-10-18T10:46:01.086446+0000","last_peered":"2023-10-18T10:46:01.086446+0000","last_clean":"2023-10-18T10:46:01.086446+0000","last_became_active":"2023-10-18T10:45:55.762545+0000","last_became_peered":"2023-10-18T10:45:55.762545+0000","last_unstale":"2023-10-18T10:46:01.086446+0000","last_undegraded":"2023-10-18T10:46:01.086446+0000","last_fullsized":"2023-10-18T10:46:01.086446+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:33:00.921856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085280+0000","last_change":"2023-10-18T10:45:55.305049+0000","last_active":"2023-10-18T10:46:01.085280+0000","last_peered":"2023-10-18T10:46:01.085280+0000","last_clean":"2023-10-18T10:46:01.085280+0000","last_became_active":"2023-10-18T10:45:55.304746+0000","last_became_peered":"2023-10-18T10:45:55.304746+0000","last_unstale":"2023-10-18T10:46:01.085280+0000","last_undegraded":"2023-10-18T10:46:01.085280+0000","last_fullsized":"2023-10-18T10:46:01.085280+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:37:42.159482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084766+0000","last_change":"2023-10-18T10:45:50.251375+0000","last_active":"2023-10-18T10:46:01.084766+0000","last_peered":"2023-10-18T10:46:01.084766+0000","last_clean":"2023-10-18T10:46:01.084766+0000","last_became_active":"2023-10-18T10:45:50.251055+0000","last_became_peered":"2023-10-18T10:45:50.251055+0000","last_unstale":"2023-10-18T10:46:01.084766+0000","last_undegraded":"2023-10-18T10:46:01.084766+0000","last_fullsized":"2023-10-18T10:46:01.084766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:49:40.643495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085615+0000","last_change":"2023-10-18T10:45:59.791676+0000","last_active":"2023-10-18T10:46:01.085615+0000","last_peered":"2023-10-18T10:46:01.085615+0000","last_clean":"2023-10-18T10:46:01.085615+0000","last_became_active":"2023-10-18T10:45:59.791341+0000","last_became_peered":"2023-10-18T10:45:59.791341+0000","last_unstale":"2023-10-18T10:46:01.085615+0000","last_undegraded":"2023-10-18T10:46:01.085615+0000","last_fullsized":"2023-10-18T10:46:01.085615+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:15:44.664969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.770245+0000","last_change":"2023-10-18T10:45:55.777240+0000","last_active":"2023-10-18T10:46:00.770245+0000","last_peered":"2023-10-18T10:46:00.770245+0000","last_clean":"2023-10-18T10:46:00.770245+0000","last_became_active":"2023-10-18T10:45:55.775286+0000","last_became_peered":"2023-10-18T10:45:55.775286+0000","last_unstale":"2023-10-18T10:46:00.770245+0000","last_undegraded":"2023-10-18T10:46:00.770245+0000","last_fullsized":"2023-10-18T10:46:00.770245+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:52.500160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902463+0000","last_change":"2023-10-18T10:45:55.296440+0000","last_active":"2023-10-18T10:46:00.902463+0000","last_peered":"2023-10-18T10:46:00.902463+0000","last_clean":"2023-10-18T10:46:00.902463+0000","last_became_active":"2023-10-18T10:45:55.296124+0000","last_became_peered":"2023-10-18T10:45:55.296124+0000","last_unstale":"2023-10-18T10:46:00.902463+0000","last_undegraded":"2023-10-18T10:46:00.902463+0000","last_fullsized":"2023-10-18T10:46:00.902463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:43:19.116789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085874+0000","last_change":"2023-10-18T10:45:50.251310+0000","last_active":"2023-10-18T10:46:01.085874+0000","last_peered":"2023-10-18T10:46:01.085874+0000","last_clean":"2023-10-18T10:46:01.085874+0000","last_became_active":"2023-10-18T10:45:50.250823+0000","last_became_peered":"2023-10-18T10:45:50.250823+0000","last_unstale":"2023-10-18T10:46:01.085874+0000","last_undegraded":"2023-10-18T10:46:01.085874+0000","last_fullsized":"2023-10-18T10:46:01.085874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:33:28.469766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":183,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084881+0000","last_change":"2023-10-18T10:45:44.563531+0000","last_active":"2023-10-18T10:46:01.084881+0000","last_peered":"2023-10-18T10:46:01.084881+0000","last_clean":"2023-10-18T10:46:01.084881+0000","last_became_active":"2023-10-18T10:45:44.562504+0000","last_became_peered":"2023-10-18T10:45:44.562504+0000","last_unstale":"2023-10-18T10:46:01.084881+0000","last_undegraded":"2023-10-18T10:46:01.084881+0000","last_fullsized":"2023-10-18T10:46:01.084881+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:43.181461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:43.181461+0000","last_clean_scrub_stamp":"2023-10-18T10:45:43.181461+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:37:25.232224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084962+0000","last_change":"2023-10-18T10:45:59.807765+0000","last_active":"2023-10-18T10:46:01.084962+0000","last_peered":"2023-10-18T10:46:01.084962+0000","last_clean":"2023-10-18T10:46:01.084962+0000","last_became_active":"2023-10-18T10:45:59.807369+0000","last_became_peered":"2023-10-18T10:45:59.807369+0000","last_unstale":"2023-10-18T10:46:01.084962+0000","last_undegraded":"2023-10-18T10:46:01.084962+0000","last_fullsized":"2023-10-18T10:46:01.084962+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:14:04.335139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086027+0000","last_change":"2023-10-18T10:45:55.763363+0000","last_active":"2023-10-18T10:46:01.086027+0000","last_peered":"2023-10-18T10:46:01.086027+0000","last_clean":"2023-10-18T10:46:01.086027+0000","last_became_active":"2023-10-18T10:45:55.763155+0000","last_became_peered":"2023-10-18T10:45:55.763155+0000","last_unstale":"2023-10-18T10:46:01.086027+0000","last_undegraded":"2023-10-18T10:46:01.086027+0000","last_fullsized":"2023-10-18T10:46:01.086027+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:15:41.179515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902288+0000","last_change":"2023-10-18T10:45:55.304658+0000","last_active":"2023-10-18T10:46:00.902288+0000","last_peered":"2023-10-18T10:46:00.902288+0000","last_clean":"2023-10-18T10:46:00.902288+0000","last_became_active":"2023-10-18T10:45:55.303900+0000","last_became_peered":"2023-10-18T10:45:55.303900+0000","last_unstale":"2023-10-18T10:46:00.902288+0000","last_undegraded":"2023-10-18T10:46:00.902288+0000","last_fullsized":"2023-10-18T10:46:00.902288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:34:05.567387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768906+0000","last_change":"2023-10-18T10:45:59.792438+0000","last_active":"2023-10-18T10:46:00.768906+0000","last_peered":"2023-10-18T10:46:00.768906+0000","last_clean":"2023-10-18T10:46:00.768906+0000","last_became_active":"2023-10-18T10:45:59.791975+0000","last_became_peered":"2023-10-18T10:45:59.791975+0000","last_unstale":"2023-10-18T10:46:00.768906+0000","last_undegraded":"2023-10-18T10:46:00.768906+0000","last_fullsized":"2023-10-18T10:46:00.768906+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:19:39.692866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768991+0000","last_change":"2023-10-18T10:45:55.773190+0000","last_active":"2023-10-18T10:46:00.768991+0000","last_peered":"2023-10-18T10:46:00.768991+0000","last_clean":"2023-10-18T10:46:00.768991+0000","last_became_active":"2023-10-18T10:45:55.769619+0000","last_became_peered":"2023-10-18T10:45:55.769619+0000","last_unstale":"2023-10-18T10:46:00.768991+0000","last_undegraded":"2023-10-18T10:46:00.768991+0000","last_fullsized":"2023-10-18T10:46:00.768991+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:12:47.136700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.769371+0000","last_change":"2023-10-18T10:45:55.299320+0000","last_active":"2023-10-18T10:46:00.769371+0000","last_peered":"2023-10-18T10:46:00.769371+0000","last_clean":"2023-10-18T10:46:00.769371+0000","last_became_active":"2023-10-18T10:45:55.299008+0000","last_became_peered":"2023-10-18T10:45:55.299008+0000","last_unstale":"2023-10-18T10:46:00.769371+0000","last_undegraded":"2023-10-18T10:46:00.769371+0000","last_fullsized":"2023-10-18T10:46:00.769371+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:51.786437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085465+0000","last_change":"2023-10-18T10:45:59.801383+0000","last_active":"2023-10-18T10:46:01.085465+0000","last_peered":"2023-10-18T10:46:01.085465+0000","last_clean":"2023-10-18T10:46:01.085465+0000","last_became_active":"2023-10-18T10:45:59.801052+0000","last_became_peered":"2023-10-18T10:45:59.801052+0000","last_unstale":"2023-10-18T10:46:01.085465+0000","last_undegraded":"2023-10-18T10:46:01.085465+0000","last_fullsized":"2023-10-18T10:46:01.085465+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:57:31.735612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768713+0000","last_change":"2023-10-18T10:45:55.762399+0000","last_active":"2023-10-18T10:46:00.768713+0000","last_peered":"2023-10-18T10:46:00.768713+0000","last_clean":"2023-10-18T10:46:00.768713+0000","last_became_active":"2023-10-18T10:45:55.762074+0000","last_became_peered":"2023-10-18T10:45:55.762074+0000","last_unstale":"2023-10-18T10:46:00.768713+0000","last_undegraded":"2023-10-18T10:46:00.768713+0000","last_fullsized":"2023-10-18T10:46:00.768713+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:44:41.551817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.770066+0000","last_change":"2023-10-18T10:45:55.301204+0000","last_active":"2023-10-18T10:46:00.770066+0000","last_peered":"2023-10-18T10:46:00.770066+0000","last_clean":"2023-10-18T10:46:00.770066+0000","last_became_active":"2023-10-18T10:45:55.300895+0000","last_became_peered":"2023-10-18T10:45:55.300895+0000","last_unstale":"2023-10-18T10:46:00.770066+0000","last_undegraded":"2023-10-18T10:46:00.770066+0000","last_fullsized":"2023-10-18T10:46:00.770066+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:33:30.055966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084972+0000","last_change":"2023-10-18T10:45:59.794810+0000","last_active":"2023-10-18T10:46:01.084972+0000","last_peered":"2023-10-18T10:46:01.084972+0000","last_clean":"2023-10-18T10:46:01.084972+0000","last_became_active":"2023-10-18T10:45:59.794392+0000","last_became_peered":"2023-10-18T10:45:59.794392+0000","last_unstale":"2023-10-18T10:46:01.084972+0000","last_undegraded":"2023-10-18T10:46:01.084972+0000","last_fullsized":"2023-10-18T10:46:01.084972+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:05:09.364081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768556+0000","last_change":"2023-10-18T10:45:55.775591+0000","last_active":"2023-10-18T10:46:00.768556+0000","last_peered":"2023-10-18T10:46:00.768556+0000","last_clean":"2023-10-18T10:46:00.768556+0000","last_became_active":"2023-10-18T10:45:55.770394+0000","last_became_peered":"2023-10-18T10:45:55.770394+0000","last_unstale":"2023-10-18T10:46:00.768556+0000","last_undegraded":"2023-10-18T10:46:00.768556+0000","last_fullsized":"2023-10-18T10:46:00.768556+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:03:08.191047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902332+0000","last_change":"2023-10-18T10:45:55.299012+0000","last_active":"2023-10-18T10:46:00.902332+0000","last_peered":"2023-10-18T10:46:00.902332+0000","last_clean":"2023-10-18T10:46:00.902332+0000","last_became_active":"2023-10-18T10:45:55.296941+0000","last_became_peered":"2023-10-18T10:45:55.296941+0000","last_unstale":"2023-10-18T10:46:00.902332+0000","last_undegraded":"2023-10-18T10:46:00.902332+0000","last_fullsized":"2023-10-18T10:46:00.902332+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:07:46.963272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.766976+0000","last_change":"2023-10-18T10:45:59.797124+0000","last_active":"2023-10-18T10:46:00.766976+0000","last_peered":"2023-10-18T10:46:00.766976+0000","last_clean":"2023-10-18T10:46:00.766976+0000","last_became_active":"2023-10-18T10:45:59.796721+0000","last_became_peered":"2023-10-18T10:45:59.796721+0000","last_unstale":"2023-10-18T10:46:00.766976+0000","last_undegraded":"2023-10-18T10:46:00.766976+0000","last_fullsized":"2023-10-18T10:46:00.766976+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:11:32.963010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085998+0000","last_change":"2023-10-18T10:45:55.767007+0000","last_active":"2023-10-18T10:46:01.085998+0000","last_peered":"2023-10-18T10:46:01.085998+0000","last_clean":"2023-10-18T10:46:01.085998+0000","last_became_active":"2023-10-18T10:45:55.766727+0000","last_became_peered":"2023-10-18T10:45:55.766727+0000","last_unstale":"2023-10-18T10:46:01.085998+0000","last_undegraded":"2023-10-18T10:46:01.085998+0000","last_fullsized":"2023-10-18T10:46:01.085998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:51:37.372456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085606+0000","last_change":"2023-10-18T10:45:55.293762+0000","last_active":"2023-10-18T10:46:01.085606+0000","last_peered":"2023-10-18T10:46:01.085606+0000","last_clean":"2023-10-18T10:46:01.085606+0000","last_became_active":"2023-10-18T10:45:55.293349+0000","last_became_peered":"2023-10-18T10:45:55.293349+0000","last_unstale":"2023-10-18T10:46:01.085606+0000","last_undegraded":"2023-10-18T10:46:01.085606+0000","last_fullsized":"2023-10-18T10:46:01.085606+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:31:20.459601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.766635+0000","last_change":"2023-10-18T10:45:59.790292+0000","last_active":"2023-10-18T10:46:00.766635+0000","last_peered":"2023-10-18T10:46:00.766635+0000","last_clean":"2023-10-18T10:46:00.766635+0000","last_became_active":"2023-10-18T10:45:59.789901+0000","last_became_peered":"2023-10-18T10:45:59.789901+0000","last_unstale":"2023-10-18T10:46:00.766635+0000","last_undegraded":"2023-10-18T10:46:00.766635+0000","last_fullsized":"2023-10-18T10:46:00.766635+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:26:52.984157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.769853+0000","last_change":"2023-10-18T10:45:55.768064+0000","last_active":"2023-10-18T10:46:00.769853+0000","last_peered":"2023-10-18T10:46:00.769853+0000","last_clean":"2023-10-18T10:46:00.769853+0000","last_became_active":"2023-10-18T10:45:55.767773+0000","last_became_peered":"2023-10-18T10:45:55.767773+0000","last_unstale":"2023-10-18T10:46:00.769853+0000","last_undegraded":"2023-10-18T10:46:00.769853+0000","last_fullsized":"2023-10-18T10:46:00.769853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:30:00.014556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085414+0000","last_change":"2023-10-18T10:45:55.303880+0000","last_active":"2023-10-18T10:46:01.085414+0000","last_peered":"2023-10-18T10:46:01.085414+0000","last_clean":"2023-10-18T10:46:01.085414+0000","last_became_active":"2023-10-18T10:45:55.303577+0000","last_became_peered":"2023-10-18T10:45:55.303577+0000","last_unstale":"2023-10-18T10:46:01.085414+0000","last_undegraded":"2023-10-18T10:46:01.085414+0000","last_fullsized":"2023-10-18T10:46:01.085414+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:02:55.960777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.766839+0000","last_change":"2023-10-18T10:45:59.789006+0000","last_active":"2023-10-18T10:46:00.766839+0000","last_peered":"2023-10-18T10:46:00.766839+0000","last_clean":"2023-10-18T10:46:00.766839+0000","last_became_active":"2023-10-18T10:45:59.788544+0000","last_became_peered":"2023-10-18T10:45:59.788544+0000","last_unstale":"2023-10-18T10:46:00.766839+0000","last_undegraded":"2023-10-18T10:46:00.766839+0000","last_fullsized":"2023-10-18T10:46:00.766839+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:32:31.878208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.769128+0000","last_change":"2023-10-18T10:45:55.775207+0000","last_active":"2023-10-18T10:46:00.769128+0000","last_peered":"2023-10-18T10:46:00.769128+0000","last_clean":"2023-10-18T10:46:00.769128+0000","last_became_active":"2023-10-18T10:45:55.774909+0000","last_became_peered":"2023-10-18T10:45:55.774909+0000","last_unstale":"2023-10-18T10:46:00.769128+0000","last_undegraded":"2023-10-18T10:46:00.769128+0000","last_fullsized":"2023-10-18T10:46:00.769128+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:07:30.694604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085978+0000","last_change":"2023-10-18T10:45:55.302100+0000","last_active":"2023-10-18T10:46:01.085978+0000","last_peered":"2023-10-18T10:46:01.085978+0000","last_clean":"2023-10-18T10:46:01.085978+0000","last_became_active":"2023-10-18T10:45:55.301652+0000","last_became_peered":"2023-10-18T10:45:55.301652+0000","last_unstale":"2023-10-18T10:46:01.085978+0000","last_undegraded":"2023-10-18T10:46:01.085978+0000","last_fullsized":"2023-10-18T10:46:01.085978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:12:22.646771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.766536+0000","last_change":"2023-10-18T10:45:59.794997+0000","last_active":"2023-10-18T10:46:00.766536+0000","last_peered":"2023-10-18T10:46:00.766536+0000","last_clean":"2023-10-18T10:46:00.766536+0000","last_became_active":"2023-10-18T10:45:59.794587+0000","last_became_peered":"2023-10-18T10:45:59.794587+0000","last_unstale":"2023-10-18T10:46:00.766536+0000","last_undegraded":"2023-10-18T10:46:00.766536+0000","last_fullsized":"2023-10-18T10:46:00.766536+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:22:23.799739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084896+0000","last_change":"2023-10-18T10:45:55.772538+0000","last_active":"2023-10-18T10:46:01.084896+0000","last_peered":"2023-10-18T10:46:01.084896+0000","last_clean":"2023-10-18T10:46:01.084896+0000","last_became_active":"2023-10-18T10:45:55.772207+0000","last_became_peered":"2023-10-18T10:45:55.772207+0000","last_unstale":"2023-10-18T10:46:01.084896+0000","last_undegraded":"2023-10-18T10:46:01.084896+0000","last_fullsized":"2023-10-18T10:46:01.084896+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:55:21.950816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086120+0000","last_change":"2023-10-18T10:45:55.305858+0000","last_active":"2023-10-18T10:46:01.086120+0000","last_peered":"2023-10-18T10:46:01.086120+0000","last_clean":"2023-10-18T10:46:01.086120+0000","last_became_active":"2023-10-18T10:45:55.305591+0000","last_became_peered":"2023-10-18T10:45:55.305591+0000","last_unstale":"2023-10-18T10:46:01.086120+0000","last_undegraded":"2023-10-18T10:46:01.086120+0000","last_fullsized":"2023-10-18T10:46:01.086120+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:43:19.116789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084960+0000","last_change":"2023-10-18T10:45:59.799125+0000","last_active":"2023-10-18T10:46:01.084960+0000","last_peered":"2023-10-18T10:46:01.084960+0000","last_clean":"2023-10-18T10:46:01.084960+0000","last_became_active":"2023-10-18T10:45:59.798734+0000","last_became_peered":"2023-10-18T10:45:59.798734+0000","last_unstale":"2023-10-18T10:46:01.084960+0000","last_undegraded":"2023-10-18T10:46:01.084960+0000","last_fullsized":"2023-10-18T10:46:01.084960+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:56:46.630947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085751+0000","last_change":"2023-10-18T10:45:59.794340+0000","last_active":"2023-10-18T10:46:01.085751+0000","last_peered":"2023-10-18T10:46:01.085751+0000","last_clean":"2023-10-18T10:46:01.085751+0000","last_became_active":"2023-10-18T10:45:59.793965+0000","last_became_peered":"2023-10-18T10:45:59.793965+0000","last_unstale":"2023-10-18T10:46:01.085751+0000","last_undegraded":"2023-10-18T10:46:01.085751+0000","last_fullsized":"2023-10-18T10:46:01.085751+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:29.702107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.767968+0000","last_change":"2023-10-18T10:45:55.292258+0000","last_active":"2023-10-18T10:46:00.767968+0000","last_peered":"2023-10-18T10:46:00.767968+0000","last_clean":"2023-10-18T10:46:00.767968+0000","last_became_active":"2023-10-18T10:45:55.291007+0000","last_became_peered":"2023-10-18T10:45:55.291007+0000","last_unstale":"2023-10-18T10:46:00.767968+0000","last_undegraded":"2023-10-18T10:46:00.767968+0000","last_fullsized":"2023-10-18T10:46:00.767968+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:06:37.792273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902848+0000","last_change":"2023-10-18T10:45:55.765261+0000","last_active":"2023-10-18T10:46:00.902848+0000","last_peered":"2023-10-18T10:46:00.902848+0000","last_clean":"2023-10-18T10:46:00.902848+0000","last_became_active":"2023-10-18T10:45:55.765025+0000","last_became_peered":"2023-10-18T10:45:55.765025+0000","last_unstale":"2023-10-18T10:46:00.902848+0000","last_undegraded":"2023-10-18T10:46:00.902848+0000","last_fullsized":"2023-10-18T10:46:00.902848+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:49:06.870660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902759+0000","last_change":"2023-10-18T10:45:59.795015+0000","last_active":"2023-10-18T10:46:00.902759+0000","last_peered":"2023-10-18T10:46:00.902759+0000","last_clean":"2023-10-18T10:46:00.902759+0000","last_became_active":"2023-10-18T10:45:59.794646+0000","last_became_peered":"2023-10-18T10:45:59.794646+0000","last_unstale":"2023-10-18T10:46:00.902759+0000","last_undegraded":"2023-10-18T10:46:00.902759+0000","last_fullsized":"2023-10-18T10:46:00.902759+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:08:21.298342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085153+0000","last_change":"2023-10-18T10:45:55.305369+0000","last_active":"2023-10-18T10:46:01.085153+0000","last_peered":"2023-10-18T10:46:01.085153+0000","last_clean":"2023-10-18T10:46:01.085153+0000","last_became_active":"2023-10-18T10:45:55.304821+0000","last_became_peered":"2023-10-18T10:45:55.304821+0000","last_unstale":"2023-10-18T10:46:01.085153+0000","last_undegraded":"2023-10-18T10:46:01.085153+0000","last_fullsized":"2023-10-18T10:46:01.085153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:32:59.914538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086514+0000","last_change":"2023-10-18T10:45:55.763301+0000","last_active":"2023-10-18T10:46:01.086514+0000","last_peered":"2023-10-18T10:46:01.086514+0000","last_clean":"2023-10-18T10:46:01.086514+0000","last_became_active":"2023-10-18T10:45:55.762910+0000","last_became_peered":"2023-10-18T10:45:55.762910+0000","last_unstale":"2023-10-18T10:46:01.086514+0000","last_undegraded":"2023-10-18T10:46:01.086514+0000","last_fullsized":"2023-10-18T10:46:01.086514+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:06:27.237867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085615+0000","last_change":"2023-10-18T10:45:55.302395+0000","last_active":"2023-10-18T10:46:01.085615+0000","last_peered":"2023-10-18T10:46:01.085615+0000","last_clean":"2023-10-18T10:46:01.085615+0000","last_became_active":"2023-10-18T10:45:55.302090+0000","last_became_peered":"2023-10-18T10:45:55.302090+0000","last_unstale":"2023-10-18T10:46:01.085615+0000","last_undegraded":"2023-10-18T10:46:01.085615+0000","last_fullsized":"2023-10-18T10:46:01.085615+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:25:17.957984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085682+0000","last_change":"2023-10-18T10:45:55.776022+0000","last_active":"2023-10-18T10:46:01.085682+0000","last_peered":"2023-10-18T10:46:01.085682+0000","last_clean":"2023-10-18T10:46:01.085682+0000","last_became_active":"2023-10-18T10:45:55.775711+0000","last_became_peered":"2023-10-18T10:45:55.775711+0000","last_unstale":"2023-10-18T10:46:01.085682+0000","last_undegraded":"2023-10-18T10:46:01.085682+0000","last_fullsized":"2023-10-18T10:46:01.085682+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:31:36.488372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902232+0000","last_change":"2023-10-18T10:45:59.798164+0000","last_active":"2023-10-18T10:46:00.902232+0000","last_peered":"2023-10-18T10:46:00.902232+0000","last_clean":"2023-10-18T10:46:00.902232+0000","last_became_active":"2023-10-18T10:45:59.797723+0000","last_became_peered":"2023-10-18T10:45:59.797723+0000","last_unstale":"2023-10-18T10:46:00.902232+0000","last_undegraded":"2023-10-18T10:46:00.902232+0000","last_fullsized":"2023-10-18T10:46:00.902232+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:03:00.453549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085393+0000","last_change":"2023-10-18T10:45:55.306329+0000","last_active":"2023-10-18T10:46:01.085393+0000","last_peered":"2023-10-18T10:46:01.085393+0000","last_clean":"2023-10-18T10:46:01.085393+0000","last_became_active":"2023-10-18T10:45:55.306033+0000","last_became_peered":"2023-10-18T10:45:55.306033+0000","last_unstale":"2023-10-18T10:46:01.085393+0000","last_undegraded":"2023-10-18T10:46:01.085393+0000","last_fullsized":"2023-10-18T10:46:01.085393+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:25:36.844395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902485+0000","last_change":"2023-10-18T10:45:55.768454+0000","last_active":"2023-10-18T10:46:00.902485+0000","last_peered":"2023-10-18T10:46:00.902485+0000","last_clean":"2023-10-18T10:46:00.902485+0000","last_became_active":"2023-10-18T10:45:55.767547+0000","last_became_peered":"2023-10-18T10:45:55.767547+0000","last_unstale":"2023-10-18T10:46:00.902485+0000","last_undegraded":"2023-10-18T10:46:00.902485+0000","last_fullsized":"2023-10-18T10:46:00.902485+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:19:20.955967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085524+0000","last_change":"2023-10-18T10:45:59.790629+0000","last_active":"2023-10-18T10:46:01.085524+0000","last_peered":"2023-10-18T10:46:01.085524+0000","last_clean":"2023-10-18T10:46:01.085524+0000","last_became_active":"2023-10-18T10:45:59.790163+0000","last_became_peered":"2023-10-18T10:45:59.790163+0000","last_unstale":"2023-10-18T10:46:01.085524+0000","last_undegraded":"2023-10-18T10:46:01.085524+0000","last_fullsized":"2023-10-18T10:46:01.085524+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:10:56.439036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.767840+0000","last_change":"2023-10-18T10:45:55.759248+0000","last_active":"2023-10-18T10:46:00.767840+0000","last_peered":"2023-10-18T10:46:00.767840+0000","last_clean":"2023-10-18T10:46:00.767840+0000","last_became_active":"2023-10-18T10:45:55.758926+0000","last_became_peered":"2023-10-18T10:45:55.758926+0000","last_unstale":"2023-10-18T10:46:00.767840+0000","last_undegraded":"2023-10-18T10:46:00.767840+0000","last_fullsized":"2023-10-18T10:46:00.767840+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:37:00.951309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.901903+0000","last_change":"2023-10-18T10:45:55.777594+0000","last_active":"2023-10-18T10:46:00.901903+0000","last_peered":"2023-10-18T10:46:00.901903+0000","last_clean":"2023-10-18T10:46:00.901903+0000","last_became_active":"2023-10-18T10:45:55.777126+0000","last_became_peered":"2023-10-18T10:45:55.777126+0000","last_unstale":"2023-10-18T10:46:00.901903+0000","last_undegraded":"2023-10-18T10:46:00.901903+0000","last_fullsized":"2023-10-18T10:46:00.901903+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:33:49.751994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768469+0000","last_change":"2023-10-18T10:45:55.770662+0000","last_active":"2023-10-18T10:46:00.768469+0000","last_peered":"2023-10-18T10:46:00.768469+0000","last_clean":"2023-10-18T10:46:00.768469+0000","last_became_active":"2023-10-18T10:45:55.769816+0000","last_became_peered":"2023-10-18T10:45:55.769816+0000","last_unstale":"2023-10-18T10:46:00.768469+0000","last_undegraded":"2023-10-18T10:46:00.768469+0000","last_fullsized":"2023-10-18T10:46:00.768469+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:40:54.772659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902183+0000","last_change":"2023-10-18T10:45:55.764434+0000","last_active":"2023-10-18T10:46:00.902183+0000","last_peered":"2023-10-18T10:46:00.902183+0000","last_clean":"2023-10-18T10:46:00.902183+0000","last_became_active":"2023-10-18T10:45:55.764164+0000","last_became_peered":"2023-10-18T10:45:55.764164+0000","last_unstale":"2023-10-18T10:46:00.902183+0000","last_undegraded":"2023-10-18T10:46:00.902183+0000","last_fullsized":"2023-10-18T10:46:00.902183+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:25:37.851713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085549+0000","last_change":"2023-10-18T10:45:55.773793+0000","last_active":"2023-10-18T10:46:01.085549+0000","last_peered":"2023-10-18T10:46:01.085549+0000","last_clean":"2023-10-18T10:46:01.085549+0000","last_became_active":"2023-10-18T10:45:55.769530+0000","last_became_peered":"2023-10-18T10:45:55.769530+0000","last_unstale":"2023-10-18T10:46:01.085549+0000","last_undegraded":"2023-10-18T10:46:01.085549+0000","last_fullsized":"2023-10-18T10:46:01.085549+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:57:28.250158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085697+0000","last_change":"2023-10-18T10:45:55.780769+0000","last_active":"2023-10-18T10:46:01.085697+0000","last_peered":"2023-10-18T10:46:01.085697+0000","last_clean":"2023-10-18T10:46:01.085697+0000","last_became_active":"2023-10-18T10:45:55.780434+0000","last_became_peered":"2023-10-18T10:45:55.780434+0000","last_unstale":"2023-10-18T10:46:01.085697+0000","last_undegraded":"2023-10-18T10:46:01.085697+0000","last_fullsized":"2023-10-18T10:46:01.085697+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:37:43.166800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085181+0000","last_change":"2023-10-18T10:45:55.770696+0000","last_active":"2023-10-18T10:46:01.085181+0000","last_peered":"2023-10-18T10:46:01.085181+0000","last_clean":"2023-10-18T10:46:01.085181+0000","last_became_active":"2023-10-18T10:45:55.769723+0000","last_became_peered":"2023-10-18T10:45:55.769723+0000","last_unstale":"2023-10-18T10:46:01.085181+0000","last_undegraded":"2023-10-18T10:46:01.085181+0000","last_fullsized":"2023-10-18T10:46:01.085181+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:59:15.197613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085919+0000","last_change":"2023-10-18T10:45:55.766491+0000","last_active":"2023-10-18T10:46:01.085919+0000","last_peered":"2023-10-18T10:46:01.085919+0000","last_clean":"2023-10-18T10:46:01.085919+0000","last_became_active":"2023-10-18T10:45:55.766215+0000","last_became_peered":"2023-10-18T10:45:55.766215+0000","last_unstale":"2023-10-18T10:46:01.085919+0000","last_undegraded":"2023-10-18T10:46:01.085919+0000","last_fullsized":"2023-10-18T10:46:01.085919+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:10:52.953582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085494+0000","last_change":"2023-10-18T10:45:55.767712+0000","last_active":"2023-10-18T10:46:01.085494+0000","last_peered":"2023-10-18T10:46:01.085494+0000","last_clean":"2023-10-18T10:46:01.085494+0000","last_became_active":"2023-10-18T10:45:55.765004+0000","last_became_peered":"2023-10-18T10:45:55.765004+0000","last_unstale":"2023-10-18T10:46:01.085494+0000","last_undegraded":"2023-10-18T10:46:01.085494+0000","last_fullsized":"2023-10-18T10:46:01.085494+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:34:06.574705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.901824+0000","last_change":"2023-10-18T10:45:55.757810+0000","last_active":"2023-10-18T10:46:00.901824+0000","last_peered":"2023-10-18T10:46:00.901824+0000","last_clean":"2023-10-18T10:46:00.901824+0000","last_became_active":"2023-10-18T10:45:55.756492+0000","last_became_peered":"2023-10-18T10:45:55.756492+0000","last_unstale":"2023-10-18T10:46:00.901824+0000","last_undegraded":"2023-10-18T10:46:00.901824+0000","last_fullsized":"2023-10-18T10:46:00.901824+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:56:43.145493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085491+0000","last_change":"2023-10-18T10:45:55.767803+0000","last_active":"2023-10-18T10:46:01.085491+0000","last_peered":"2023-10-18T10:46:01.085491+0000","last_clean":"2023-10-18T10:46:01.085491+0000","last_became_active":"2023-10-18T10:45:55.767525+0000","last_became_peered":"2023-10-18T10:45:55.767525+0000","last_unstale":"2023-10-18T10:46:01.085491+0000","last_undegraded":"2023-10-18T10:46:01.085491+0000","last_fullsized":"2023-10-18T10:46:01.085491+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:26.216653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085266+0000","last_change":"2023-10-18T10:45:55.769366+0000","last_active":"2023-10-18T10:46:01.085266+0000","last_peered":"2023-10-18T10:46:01.085266+0000","last_clean":"2023-10-18T10:46:01.085266+0000","last_became_active":"2023-10-18T10:45:55.769083+0000","last_became_peered":"2023-10-18T10:45:55.769083+0000","last_unstale":"2023-10-18T10:46:01.085266+0000","last_undegraded":"2023-10-18T10:46:01.085266+0000","last_fullsized":"2023-10-18T10:46:01.085266+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:05:57.473930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768585+0000","last_change":"2023-10-18T10:45:55.770711+0000","last_active":"2023-10-18T10:46:00.768585+0000","last_peered":"2023-10-18T10:46:00.768585+0000","last_clean":"2023-10-18T10:46:00.768585+0000","last_became_active":"2023-10-18T10:45:55.770147+0000","last_became_peered":"2023-10-18T10:45:55.770147+0000","last_unstale":"2023-10-18T10:46:00.768585+0000","last_undegraded":"2023-10-18T10:46:00.768585+0000","last_fullsized":"2023-10-18T10:46:00.768585+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:20:18.473140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.767102+0000","last_change":"2023-10-18T10:45:55.758011+0000","last_active":"2023-10-18T10:46:00.767102+0000","last_peered":"2023-10-18T10:46:00.767102+0000","last_clean":"2023-10-18T10:46:00.767102+0000","last_became_active":"2023-10-18T10:45:55.757668+0000","last_became_peered":"2023-10-18T10:45:55.757668+0000","last_unstale":"2023-10-18T10:46:00.767102+0000","last_undegraded":"2023-10-18T10:46:00.767102+0000","last_fullsized":"2023-10-18T10:46:00.767102+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:41:48.881411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.769446+0000","last_change":"2023-10-18T10:45:55.763084+0000","last_active":"2023-10-18T10:46:00.769446+0000","last_peered":"2023-10-18T10:46:00.769446+0000","last_clean":"2023-10-18T10:46:00.769446+0000","last_became_active":"2023-10-18T10:45:55.762776+0000","last_became_peered":"2023-10-18T10:45:55.762776+0000","last_unstale":"2023-10-18T10:46:00.769446+0000","last_undegraded":"2023-10-18T10:46:00.769446+0000","last_fullsized":"2023-10-18T10:46:00.769446+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:28:02.667670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085807+0000","last_change":"2023-10-18T10:45:55.762254+0000","last_active":"2023-10-18T10:46:01.085807+0000","last_peered":"2023-10-18T10:46:01.085807+0000","last_clean":"2023-10-18T10:46:01.085807+0000","last_became_active":"2023-10-18T10:45:55.761870+0000","last_became_peered":"2023-10-18T10:45:55.761870+0000","last_unstale":"2023-10-18T10:46:01.085807+0000","last_undegraded":"2023-10-18T10:46:01.085807+0000","last_fullsized":"2023-10-18T10:46:01.085807+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:43:01.654769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768118+0000","last_change":"2023-10-18T10:45:55.765743+0000","last_active":"2023-10-18T10:46:00.768118+0000","last_peered":"2023-10-18T10:46:00.768118+0000","last_clean":"2023-10-18T10:46:00.768118+0000","last_became_active":"2023-10-18T10:45:55.765470+0000","last_became_peered":"2023-10-18T10:45:55.765470+0000","last_unstale":"2023-10-18T10:46:00.768118+0000","last_undegraded":"2023-10-18T10:46:00.768118+0000","last_fullsized":"2023-10-18T10:46:00.768118+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:19:20.955967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085124+0000","last_change":"2023-10-18T10:45:55.781643+0000","last_active":"2023-10-18T10:46:01.085124+0000","last_peered":"2023-10-18T10:46:01.085124+0000","last_clean":"2023-10-18T10:46:01.085124+0000","last_became_active":"2023-10-18T10:45:55.781252+0000","last_became_peered":"2023-10-18T10:45:55.781252+0000","last_unstale":"2023-10-18T10:46:01.085124+0000","last_undegraded":"2023-10-18T10:46:01.085124+0000","last_fullsized":"2023-10-18T10:46:01.085124+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:40:53.192771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085958+0000","last_change":"2023-10-18T10:45:55.774683+0000","last_active":"2023-10-18T10:46:01.085958+0000","last_peered":"2023-10-18T10:46:01.085958+0000","last_clean":"2023-10-18T10:46:01.085958+0000","last_became_active":"2023-10-18T10:45:55.773664+0000","last_became_peered":"2023-10-18T10:45:55.773664+0000","last_unstale":"2023-10-18T10:46:01.085958+0000","last_undegraded":"2023-10-18T10:46:01.085958+0000","last_fullsized":"2023-10-18T10:46:01.085958+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:06:26.927155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.766819+0000","last_change":"2023-10-18T10:45:55.755077+0000","last_active":"2023-10-18T10:46:00.766819+0000","last_peered":"2023-10-18T10:46:00.766819+0000","last_clean":"2023-10-18T10:46:00.766819+0000","last_became_active":"2023-10-18T10:45:55.754723+0000","last_became_peered":"2023-10-18T10:45:55.754723+0000","last_unstale":"2023-10-18T10:46:00.766819+0000","last_undegraded":"2023-10-18T10:46:00.766819+0000","last_fullsized":"2023-10-18T10:46:00.766819+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:17:50.477786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.767162+0000","last_change":"2023-10-18T10:45:55.771484+0000","last_active":"2023-10-18T10:46:00.767162+0000","last_peered":"2023-10-18T10:46:00.767162+0000","last_clean":"2023-10-18T10:46:00.767162+0000","last_became_active":"2023-10-18T10:45:55.771108+0000","last_became_peered":"2023-10-18T10:45:55.771108+0000","last_unstale":"2023-10-18T10:46:00.767162+0000","last_undegraded":"2023-10-18T10:46:00.767162+0000","last_fullsized":"2023-10-18T10:46:00.767162+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:18.131261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768708+0000","last_change":"2023-10-18T10:45:55.778320+0000","last_active":"2023-10-18T10:46:00.768708+0000","last_peered":"2023-10-18T10:46:00.768708+0000","last_clean":"2023-10-18T10:46:00.768708+0000","last_became_active":"2023-10-18T10:45:55.777767+0000","last_became_peered":"2023-10-18T10:45:55.777767+0000","last_unstale":"2023-10-18T10:46:00.768708+0000","last_undegraded":"2023-10-18T10:46:00.768708+0000","last_fullsized":"2023-10-18T10:46:00.768708+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:05:56.688630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086126+0000","last_change":"2023-10-18T10:45:55.771747+0000","last_active":"2023-10-18T10:46:01.086126+0000","last_peered":"2023-10-18T10:46:01.086126+0000","last_clean":"2023-10-18T10:46:01.086126+0000","last_became_active":"2023-10-18T10:45:55.771297+0000","last_became_peered":"2023-10-18T10:45:55.771297+0000","last_unstale":"2023-10-18T10:46:01.086126+0000","last_undegraded":"2023-10-18T10:46:01.086126+0000","last_fullsized":"2023-10-18T10:46:01.086126+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:51:37.372456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085611+0000","last_change":"2023-10-18T10:45:55.756546+0000","last_active":"2023-10-18T10:46:01.085611+0000","last_peered":"2023-10-18T10:46:01.085611+0000","last_clean":"2023-10-18T10:46:01.085611+0000","last_became_active":"2023-10-18T10:45:55.756277+0000","last_became_peered":"2023-10-18T10:45:55.756277+0000","last_unstale":"2023-10-18T10:46:01.085611+0000","last_undegraded":"2023-10-18T10:46:01.085611+0000","last_fullsized":"2023-10-18T10:46:01.085611+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:33:34.507396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084785+0000","last_change":"2023-10-18T10:45:55.778442+0000","last_active":"2023-10-18T10:46:01.084785+0000","last_peered":"2023-10-18T10:46:01.084785+0000","last_clean":"2023-10-18T10:46:01.084785+0000","last_became_active":"2023-10-18T10:45:55.778198+0000","last_became_peered":"2023-10-18T10:45:55.778198+0000","last_unstale":"2023-10-18T10:46:01.084785+0000","last_undegraded":"2023-10-18T10:46:01.084785+0000","last_fullsized":"2023-10-18T10:46:01.084785+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:28:05.383698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085042+0000","last_change":"2023-10-18T10:45:55.779285+0000","last_active":"2023-10-18T10:46:01.085042+0000","last_peered":"2023-10-18T10:46:01.085042+0000","last_clean":"2023-10-18T10:46:01.085042+0000","last_became_active":"2023-10-18T10:45:55.778975+0000","last_became_peered":"2023-10-18T10:45:55.778975+0000","last_unstale":"2023-10-18T10:46:01.085042+0000","last_undegraded":"2023-10-18T10:46:01.085042+0000","last_fullsized":"2023-10-18T10:46:01.085042+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:33:49.751994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768170+0000","last_change":"2023-10-18T10:45:55.770892+0000","last_active":"2023-10-18T10:46:00.768170+0000","last_peered":"2023-10-18T10:46:00.768170+0000","last_clean":"2023-10-18T10:46:00.768170+0000","last_became_active":"2023-10-18T10:45:55.770181+0000","last_became_peered":"2023-10-18T10:45:55.770181+0000","last_unstale":"2023-10-18T10:46:00.768170+0000","last_undegraded":"2023-10-18T10:46:00.768170+0000","last_fullsized":"2023-10-18T10:46:00.768170+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:06:44.169699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086251+0000","last_change":"2023-10-18T10:45:55.759114+0000","last_active":"2023-10-18T10:46:01.086251+0000","last_peered":"2023-10-18T10:46:01.086251+0000","last_clean":"2023-10-18T10:46:01.086251+0000","last_became_active":"2023-10-18T10:45:55.758825+0000","last_became_peered":"2023-10-18T10:45:55.758825+0000","last_unstale":"2023-10-18T10:46:01.086251+0000","last_undegraded":"2023-10-18T10:46:01.086251+0000","last_fullsized":"2023-10-18T10:46:01.086251+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:14:00.849685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.770488+0000","last_change":"2023-10-18T10:45:55.763407+0000","last_active":"2023-10-18T10:46:00.770488+0000","last_peered":"2023-10-18T10:46:00.770488+0000","last_clean":"2023-10-18T10:46:00.770488+0000","last_became_active":"2023-10-18T10:45:55.763083+0000","last_became_peered":"2023-10-18T10:45:55.763083+0000","last_unstale":"2023-10-18T10:46:00.770488+0000","last_undegraded":"2023-10-18T10:46:00.770488+0000","last_fullsized":"2023-10-18T10:46:00.770488+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:30.982488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768596+0000","last_change":"2023-10-18T10:45:55.770469+0000","last_active":"2023-10-18T10:46:00.768596+0000","last_peered":"2023-10-18T10:46:00.768596+0000","last_clean":"2023-10-18T10:46:00.768596+0000","last_became_active":"2023-10-18T10:45:55.769998+0000","last_became_peered":"2023-10-18T10:45:55.769998+0000","last_unstale":"2023-10-18T10:46:00.768596+0000","last_undegraded":"2023-10-18T10:46:00.768596+0000","last_fullsized":"2023-10-18T10:46:00.768596+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:00:14.580995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085929+0000","last_change":"2023-10-18T10:45:55.781899+0000","last_active":"2023-10-18T10:46:01.085929+0000","last_peered":"2023-10-18T10:46:01.085929+0000","last_clean":"2023-10-18T10:46:01.085929+0000","last_became_active":"2023-10-18T10:45:55.781624+0000","last_became_peered":"2023-10-18T10:45:55.781624+0000","last_unstale":"2023-10-18T10:46:01.085929+0000","last_undegraded":"2023-10-18T10:46:01.085929+0000","last_fullsized":"2023-10-18T10:46:01.085929+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:43:01.654769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.767009+0000","last_change":"2023-10-18T10:45:55.757924+0000","last_active":"2023-10-18T10:46:00.767009+0000","last_peered":"2023-10-18T10:46:00.767009+0000","last_clean":"2023-10-18T10:46:00.767009+0000","last_became_active":"2023-10-18T10:45:55.757644+0000","last_became_peered":"2023-10-18T10:45:55.757644+0000","last_unstale":"2023-10-18T10:46:00.767009+0000","last_undegraded":"2023-10-18T10:46:00.767009+0000","last_fullsized":"2023-10-18T10:46:00.767009+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:08:54.769872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343932,"statfs":{"total":96636764160,"available":96608186368,"internally_reserved":0,"allocated":913408,"data_stored":714326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":120553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":311296,"data_stored":123798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":119991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":888832,"data_stored":710851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:46:15.270 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-18T10:46:15.271 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-18T10:46:15.271 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-18T10:46:15.272 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-18T10:46:15.272 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-18T10:46:15.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.446+0000 7f8e8ede8640 1 Processor -- start 2023-10-18T10:46:15.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.446+0000 7f8e8ede8640 1 -- start start 2023-10-18T10:46:15.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.446+0000 7f8e8ede8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e8815c2f0 0x7f8e880b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:15.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.446+0000 7f8e8ede8640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8e880d0270 0x7f8e880d06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:15.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.446+0000 7f8e8ede8640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e8815d1b0 con 0x7f8e8815c6d0 2023-10-18T10:46:15.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.446+0000 7f8e8ede8640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e8815d490 con 0x7f8e880d0270 2023-10-18T10:46:15.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.446+0000 7f8e8ede8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8815d770 con 0x7f8e8815c2f0 2023-10-18T10:46:15.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.446+0000 7f8e87fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8e880d0270 0x7f8e880d06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:15.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.446+0000 7f8e87fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8e880d0270 0x7f8e880d06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45074/0 (socket says 172.21.15.89:45074) 2023-10-18T10:46:15.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.446+0000 7f8e87fff640 1 -- 172.21.15.89:0/2440898629 learned_addr learned my addr 172.21.15.89:0/2440898629 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:15.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.446+0000 7f8e8cb84640 1 --2- 172.21.15.89:0/2440898629 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e8815c2f0 0x7f8e880b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:15.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.446+0000 7f8e87fff640 1 -- 172.21.15.89:0/2440898629 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e8815c2f0 msgr2=0x7f8e880b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:15.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.446+0000 7f8e87fff640 1 --2- 172.21.15.89:0/2440898629 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e8815c2f0 0x7f8e880b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:15.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.446+0000 7f8e87fff640 1 -- 172.21.15.89:0/2440898629 >> v1:172.21.15.154:6789/0 conn(0x7f8e8815c6d0 legacy=0x7f8e88160780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:15.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.446+0000 7f8e87fff640 1 -- 172.21.15.89:0/2440898629 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e8815da50 con 0x7f8e880d0270 2023-10-18T10:46:15.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.450+0000 7f8e87fff640 1 --2- 172.21.15.89:0/2440898629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e880d0270 0x7f8e880d06d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8e74009870 tx=0x7f8e7401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de30a5a51eeb490 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:15.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.450+0000 7f8e877fe640 1 -- 172.21.15.89:0/2440898629 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e740a95f0 con 0x7f8e880d0270 2023-10-18T10:46:15.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.450+0000 7f8e877fe640 1 -- 172.21.15.89:0/2440898629 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e740a9750 con 0x7f8e880d0270 2023-10-18T10:46:15.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.450+0000 7f8e877fe640 1 -- 172.21.15.89:0/2440898629 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e740a9ae0 con 0x7f8e880d0270 2023-10-18T10:46:15.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.450+0000 7f8e8ede8640 1 -- 172.21.15.89:0/2440898629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e880d0270 msgr2=0x7f8e880d06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:15.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.450+0000 7f8e8ede8640 1 --2- 172.21.15.89:0/2440898629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e880d0270 0x7f8e880d06d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8e74009870 tx=0x7f8e7401c510 comp rx=0 tx=0).stop 2023-10-18T10:46:15.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.450+0000 7f8e8ede8640 1 -- 172.21.15.89:0/2440898629 shutdown_connections 2023-10-18T10:46:15.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.450+0000 7f8e8ede8640 1 --2- 172.21.15.89:0/2440898629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e880d0270 0x7f8e880d06d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:15.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.450+0000 7f8e8ede8640 1 --2- 172.21.15.89:0/2440898629 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e8815c2f0 0x7f8e880b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:15.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.450+0000 7f8e8ede8640 1 -- 172.21.15.89:0/2440898629 >> 172.21.15.89:0/2440898629 conn(0x7f8e88069ad0 msgr2=0x7f8e88069eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:15.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.450+0000 7f8e8ede8640 1 -- 172.21.15.89:0/2440898629 shutdown_connections 2023-10-18T10:46:15.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.450+0000 7f8e8ede8640 1 -- 172.21.15.89:0/2440898629 wait complete. 2023-10-18T10:46:15.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.450+0000 7f8e8ede8640 1 Processor -- start 2023-10-18T10:46:15.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.450+0000 7f8e8ede8640 1 -- start start 2023-10-18T10:46:15.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.454+0000 7f8e8ede8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e880d0270 0x7f8e88159da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:15.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.454+0000 7f8e8ede8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e8815c2f0 0x7f8e880cbc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:15.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.454+0000 7f8e8ede8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e8815c6d0 0x7f8e880cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:15.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.454+0000 7f8e8ede8640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e880cb680 con 0x7f8e880d0270 2023-10-18T10:46:15.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.454+0000 7f8e8ede8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e880cb960 con 0x7f8e8815c2f0 2023-10-18T10:46:15.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.454+0000 7f8e8ede8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8815ade0 con 0x7f8e8815c6d0 2023-10-18T10:46:15.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.454+0000 7f8e8cb84640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e880d0270 0x7f8e88159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:15.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.454+0000 7f8e8cb84640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e880d0270 0x7f8e88159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45086/0 (socket says 172.21.15.89:45086) 2023-10-18T10:46:15.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.454+0000 7f8e8cb84640 1 -- 172.21.15.89:0/3649221829 learned_addr learned my addr 172.21.15.89:0/3649221829 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:15.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.454+0000 7f8e87fff640 1 --2- 172.21.15.89:0/3649221829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e8815c2f0 0x7f8e880cbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:15.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.454+0000 7f8e8d385640 1 --2- 172.21.15.89:0/3649221829 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e8815c6d0 0x7f8e880cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:15.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.454+0000 7f8e8cb84640 1 -- 172.21.15.89:0/3649221829 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e8815c6d0 msgr2=0x7f8e880cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:15.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.454+0000 7f8e8cb84640 1 --2- 172.21.15.89:0/3649221829 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e8815c6d0 0x7f8e880cc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:15.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.454+0000 7f8e8cb84640 1 -- 172.21.15.89:0/3649221829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e8815c2f0 msgr2=0x7f8e880cbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:15.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.454+0000 7f8e8cb84640 1 --2- 172.21.15.89:0/3649221829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e8815c2f0 0x7f8e880cbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:15.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.454+0000 7f8e8cb84640 1 -- 172.21.15.89:0/3649221829 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e740094d0 con 0x7f8e880d0270 2023-10-18T10:46:15.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.454+0000 7f8e87fff640 1 --2- 172.21.15.89:0/3649221829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e8815c2f0 0x7f8e880cbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:15.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.454+0000 7f8e8cb84640 1 --2- 172.21.15.89:0/3649221829 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e880d0270 0x7f8e88159da0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c00e930 tx=0x7f8e7c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:15.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.454+0000 7f8e857fa640 1 -- 172.21.15.89:0/3649221829 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c009690 con 0x7f8e880d0270 2023-10-18T10:46:15.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.454+0000 7f8e857fa640 1 -- 172.21.15.89:0/3649221829 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e7c00ca00 con 0x7f8e880d0270 2023-10-18T10:46:15.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.454+0000 7f8e8ede8640 1 -- 172.21.15.89:0/3649221829 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e8815b0c0 con 0x7f8e880d0270 2023-10-18T10:46:15.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.454+0000 7f8e857fa640 1 -- 172.21.15.89:0/3649221829 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c00cd70 con 0x7f8e880d0270 2023-10-18T10:46:15.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.454+0000 7f8e8ede8640 1 -- 172.21.15.89:0/3649221829 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e8815b680 con 0x7f8e880d0270 2023-10-18T10:46:15.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.454+0000 7f8e8ede8640 1 -- 172.21.15.89:0/3649221829 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e88066ca0 con 0x7f8e880d0270 2023-10-18T10:46:15.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.458+0000 7f8e857fa640 1 -- 172.21.15.89:0/3649221829 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f8e7c013470 con 0x7f8e880d0270 2023-10-18T10:46:15.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.458+0000 7f8e857fa640 1 --2- 172.21.15.89:0/3649221829 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8e6c065680 0x7f8e6c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:15.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.458+0000 7f8e857fa640 1 -- 172.21.15.89:0/3649221829 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8e7c083420 con 0x7f8e880d0270 2023-10-18T10:46:15.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.458+0000 7f8e87fff640 1 --2- 172.21.15.89:0/3649221829 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8e6c065680 0x7f8e6c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:15.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.458+0000 7f8e87fff640 1 --2- 172.21.15.89:0/3649221829 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8e6c065680 0x7f8e6c067b40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8e740ae000 tx=0x7f8e740b3040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:15.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.462+0000 7f8e857fa640 1 -- 172.21.15.89:0/3649221829 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f8e7c050c10 con 0x7f8e880d0270 2023-10-18T10:46:15.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.642+0000 7f8e8ede8640 1 -- 172.21.15.89:0/3649221829 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f8e881668a0 con 0x7f8e880d0270 2023-10-18T10:46:15.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.646+0000 7f8e857fa640 1 -- 172.21.15.89:0/3649221829 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+162898 (secure 0 0 0) 0x7f8e7c054230 con 0x7f8e880d0270 2023-10-18T10:46:15.649 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:15.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.650+0000 7f8e8ede8640 1 -- 172.21.15.89:0/3649221829 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8e6c065680 msgr2=0x7f8e6c067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:15.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.650+0000 7f8e8ede8640 1 --2- 172.21.15.89:0/3649221829 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8e6c065680 0x7f8e6c067b40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8e740ae000 tx=0x7f8e740b3040 comp rx=0 tx=0).stop 2023-10-18T10:46:15.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.650+0000 7f8e8ede8640 1 -- 172.21.15.89:0/3649221829 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e880d0270 msgr2=0x7f8e88159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:15.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.650+0000 7f8e8ede8640 1 --2- 172.21.15.89:0/3649221829 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e880d0270 0x7f8e88159da0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c00e930 tx=0x7f8e7c00edf0 comp rx=0 tx=0).stop 2023-10-18T10:46:15.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.650+0000 7f8e8ede8640 1 -- 172.21.15.89:0/3649221829 shutdown_connections 2023-10-18T10:46:15.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.650+0000 7f8e8ede8640 1 --2- 172.21.15.89:0/3649221829 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8e6c065680 0x7f8e6c067b40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:15.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.650+0000 7f8e8ede8640 1 --2- 172.21.15.89:0/3649221829 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e8815c6d0 0x7f8e880cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:15.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.650+0000 7f8e8ede8640 1 --2- 172.21.15.89:0/3649221829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e8815c2f0 0x7f8e880cbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:15.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.650+0000 7f8e8ede8640 1 --2- 172.21.15.89:0/3649221829 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e880d0270 0x7f8e88159da0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:15.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.650+0000 7f8e8ede8640 1 -- 172.21.15.89:0/3649221829 >> 172.21.15.89:0/3649221829 conn(0x7f8e88069ad0 msgr2=0x7f8e8815dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:15.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.650+0000 7f8e8ede8640 1 -- 172.21.15.89:0/3649221829 shutdown_connections 2023-10-18T10:46:15.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.650+0000 7f8e8ede8640 1 -- 172.21.15.89:0/3649221829 wait complete. 2023-10-18T10:46:15.669 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":5,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1086608968},{"type":"v1","addr":"172.21.15.154:6833","nonce":1086608968}]},"active_addr":"172.21.15.154:6833/1086608968","active_change":"2023-10-18T10:45:35.037212+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4119,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.154:0","nonce":3670381254}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.154:0","nonce":1408412613}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.154:0","nonce":214366221}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.154:0","nonce":2707888803}]}]} 2023-10-18T10:46:15.674 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-18T10:46:15.674 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-18T10:46:15.675 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:46:15.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.814+0000 7efeca6d8640 1 Processor -- start 2023-10-18T10:46:15.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.814+0000 7efeca6d8640 1 -- start start 2023-10-18T10:46:15.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.818+0000 7efeca6d8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efec40d0760 0x7efec40d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:15.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.818+0000 7efeca6d8640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efec40cd160 0x7efec40cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:15.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.818+0000 7efeca6d8640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efec40d1100 con 0x7efec415c940 2023-10-18T10:46:15.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.818+0000 7efeca6d8640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efec415da50 con 0x7efec40cd160 2023-10-18T10:46:15.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.818+0000 7efeca6d8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec40d3110 con 0x7efec40d0760 2023-10-18T10:46:15.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.818+0000 7efec8c75640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efec40cd160 0x7efec40cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:15.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.818+0000 7efec37fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efec40d0760 0x7efec40d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:15.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.818+0000 7efec3fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7efec415c940 0x7efec40d2a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.89:53658/0 (socket says 172.21.15.89:53658) 2023-10-18T10:46:15.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.818+0000 7efec8c75640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efec40cd160 0x7efec40cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:34082/0 (socket says 172.21.15.89:34082) 2023-10-18T10:46:15.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.818+0000 7efec3fff640 1 -- 172.21.15.89:0/1094054541 learned_addr learned my addr 172.21.15.89:0/1094054541 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:15.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.818+0000 7efec37fe640 1 -- 172.21.15.89:0/1094054541 >> v2:172.21.15.154:3300/0 conn(0x7efec40cd160 msgr2=0x7efec40cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:15.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.818+0000 7efec37fe640 1 --2- 172.21.15.89:0/1094054541 >> v2:172.21.15.154:3300/0 conn(0x7efec40cd160 0x7efec40cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:15.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.818+0000 7efec37fe640 1 -- 172.21.15.89:0/1094054541 >> v1:172.21.15.154:6789/0 conn(0x7efec415c940 legacy=0x7efec40d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:15.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.818+0000 7efec37fe640 1 -- 172.21.15.89:0/1094054541 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec415ce40 con 0x7efec40d0760 2023-10-18T10:46:15.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.818+0000 7efec2ffd640 1 -- 172.21.15.89:0/1094054541 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3383388189 0 0) 0x7efec40d1100 con 0x7efec415c940 2023-10-18T10:46:15.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.822+0000 7efec37fe640 1 --2- 172.21.15.89:0/1094054541 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efec40d0760 0x7efec40d0bc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7efeb80097c0 tx=0x7efeb801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7299f743ea0b0374 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:15.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.822+0000 7efec2ffd640 1 -- 172.21.15.89:0/1094054541 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb80ab650 con 0x7efec40d0760 2023-10-18T10:46:15.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.822+0000 7efec2ffd640 1 -- 172.21.15.89:0/1094054541 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeb80ab7b0 con 0x7efec40d0760 2023-10-18T10:46:15.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.822+0000 7efec2ffd640 1 -- 172.21.15.89:0/1094054541 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb80abb40 con 0x7efec40d0760 2023-10-18T10:46:15.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.822+0000 7efeca6d8640 1 -- 172.21.15.89:0/1094054541 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efec40d0760 msgr2=0x7efec40d0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:15.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.822+0000 7efeca6d8640 1 --2- 172.21.15.89:0/1094054541 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efec40d0760 0x7efec40d0bc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7efeb80097c0 tx=0x7efeb801c450 comp rx=0 tx=0).stop 2023-10-18T10:46:15.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.822+0000 7efeca6d8640 1 -- 172.21.15.89:0/1094054541 shutdown_connections 2023-10-18T10:46:15.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.822+0000 7efeca6d8640 1 --2- 172.21.15.89:0/1094054541 >> v2:172.21.15.154:3300/0 conn(0x7efec40cd160 0x7efec40cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:15.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.822+0000 7efeca6d8640 1 --2- 172.21.15.89:0/1094054541 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efec40d0760 0x7efec40d0bc0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:15.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.822+0000 7efeca6d8640 1 -- 172.21.15.89:0/1094054541 >> 172.21.15.89:0/1094054541 conn(0x7efec4089fd0 msgr2=0x7efec409b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:15.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.822+0000 7efeca6d8640 1 -- 172.21.15.89:0/1094054541 shutdown_connections 2023-10-18T10:46:15.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.822+0000 7efeca6d8640 1 -- 172.21.15.89:0/1094054541 wait complete. 2023-10-18T10:46:15.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.822+0000 7efeca6d8640 1 Processor -- start 2023-10-18T10:46:15.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.822+0000 7efeca6d8640 1 -- start start 2023-10-18T10:46:15.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.826+0000 7efeca6d8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efec40cd160 0x7efec4154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:15.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.826+0000 7efeca6d8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efec40d0760 0x7efec40b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:15.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.826+0000 7efeca6d8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efec4157830 0x7efec4158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:15.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.826+0000 7efeca6d8640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec4157cb0 con 0x7efec40cd160 2023-10-18T10:46:15.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.826+0000 7efeca6d8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec4154ab0 con 0x7efec40d0760 2023-10-18T10:46:15.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.826+0000 7efeca6d8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec4154d90 con 0x7efec4157830 2023-10-18T10:46:15.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.826+0000 7efec3fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efec40cd160 0x7efec4154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:15.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.826+0000 7efec3fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efec40cd160 0x7efec4154350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45104/0 (socket says 172.21.15.89:45104) 2023-10-18T10:46:15.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.826+0000 7efec3fff640 1 -- 172.21.15.89:0/2948000440 learned_addr learned my addr 172.21.15.89:0/2948000440 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:15.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.826+0000 7efec3fff640 1 -- 172.21.15.89:0/2948000440 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efec4157830 msgr2=0x7efec4158f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:15.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.826+0000 7efec3fff640 1 --2- 172.21.15.89:0/2948000440 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efec4157830 0x7efec4158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:15.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.826+0000 7efec37fe640 1 --2- 172.21.15.89:0/2948000440 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efec40d0760 0x7efec40b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:15.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.826+0000 7efec3fff640 1 -- 172.21.15.89:0/2948000440 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efec40d0760 msgr2=0x7efec40b57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:15.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.826+0000 7efec3fff640 1 --2- 172.21.15.89:0/2948000440 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efec40d0760 0x7efec40b57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:15.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.826+0000 7efec3fff640 1 -- 172.21.15.89:0/2948000440 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb8009420 con 0x7efec40cd160 2023-10-18T10:46:15.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.826+0000 7efec37fe640 1 --2- 172.21.15.89:0/2948000440 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efec40d0760 0x7efec40b57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:15.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.826+0000 7efec3fff640 1 --2- 172.21.15.89:0/2948000440 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efec40cd160 0x7efec4154350 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efeb400c400 tx=0x7efeb400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:15.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.826+0000 7efec0ff9640 1 -- 172.21.15.89:0/2948000440 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb40074e0 con 0x7efec40cd160 2023-10-18T10:46:15.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.826+0000 7efec0ff9640 1 -- 172.21.15.89:0/2948000440 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeb4007660 con 0x7efec40cd160 2023-10-18T10:46:15.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.826+0000 7efec0ff9640 1 -- 172.21.15.89:0/2948000440 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb4004a30 con 0x7efec40cd160 2023-10-18T10:46:15.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.826+0000 7efeca6d8640 1 -- 172.21.15.89:0/2948000440 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec4155070 con 0x7efec40cd160 2023-10-18T10:46:15.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.826+0000 7efeca6d8640 1 -- 172.21.15.89:0/2948000440 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec40b5ec0 con 0x7efec40cd160 2023-10-18T10:46:15.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.826+0000 7efeca6d8640 1 -- 172.21.15.89:0/2948000440 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efec404dbe0 con 0x7efec40cd160 2023-10-18T10:46:15.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.826+0000 7efec0ff9640 1 -- 172.21.15.89:0/2948000440 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7efeb4004b90 con 0x7efec40cd160 2023-10-18T10:46:15.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.830+0000 7efec0ff9640 1 --2- 172.21.15.89:0/2948000440 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efeb00656b0 0x7efeb0067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:15.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.830+0000 7efec0ff9640 1 -- 172.21.15.89:0/2948000440 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7efeb4082f30 con 0x7efec40cd160 2023-10-18T10:46:15.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.830+0000 7efec37fe640 1 --2- 172.21.15.89:0/2948000440 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efeb00656b0 0x7efeb0067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:15.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.830+0000 7efec37fe640 1 --2- 172.21.15.89:0/2948000440 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efeb00656b0 0x7efeb0067b70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efeb80af000 tx=0x7efeb8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:15.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.834+0000 7efec0ff9640 1 -- 172.21.15.89:0/2948000440 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7efeb40540f0 con 0x7efec40cd160 2023-10-18T10:46:15.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.982+0000 7efeca6d8640 1 -- 172.21.15.89:0/2948000440 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efec40c9590 con 0x7efec40cd160 2023-10-18T10:46:15.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.986+0000 7efec0ff9640 1 -- 172.21.15.89:0/2948000440 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20096 (secure 0 0 0) 0x7efeb4053c80 con 0x7efec40cd160 2023-10-18T10:46:15.987 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:15.987 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":27,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:46:00.166702+0000","last_up_change":"2023-10-18T10:45:42.164688+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:45:53.309122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-18T10:45:54.659763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:45:57.796514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6824","nonce":773078364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6827","nonce":773078364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6831","nonce":773078364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6829","nonce":773078364}]},"public_addr":"172.21.15.154:6824/773078364","cluster_addr":"172.21.15.154:6827/773078364","heartbeat_back_addr":"172.21.15.154:6831/773078364","heartbeat_front_addr":"172.21.15.154:6829/773078364","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:46:15.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.986+0000 7efeca6d8640 1 -- 172.21.15.89:0/2948000440 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efeb00656b0 msgr2=0x7efeb0067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:15.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.986+0000 7efeca6d8640 1 --2- 172.21.15.89:0/2948000440 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efeb00656b0 0x7efeb0067b70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efeb80af000 tx=0x7efeb8008040 comp rx=0 tx=0).stop 2023-10-18T10:46:15.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.986+0000 7efeca6d8640 1 -- 172.21.15.89:0/2948000440 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efec40cd160 msgr2=0x7efec4154350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:15.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.986+0000 7efeca6d8640 1 --2- 172.21.15.89:0/2948000440 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efec40cd160 0x7efec4154350 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efeb400c400 tx=0x7efeb400c8c0 comp rx=0 tx=0).stop 2023-10-18T10:46:15.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.986+0000 7efeca6d8640 1 -- 172.21.15.89:0/2948000440 shutdown_connections 2023-10-18T10:46:15.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.986+0000 7efeca6d8640 1 --2- 172.21.15.89:0/2948000440 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efeb00656b0 0x7efeb0067b70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:15.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.986+0000 7efeca6d8640 1 --2- 172.21.15.89:0/2948000440 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efec4157830 0x7efec4158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:15.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.986+0000 7efeca6d8640 1 --2- 172.21.15.89:0/2948000440 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efec40d0760 0x7efec40b57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:15.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.986+0000 7efeca6d8640 1 --2- 172.21.15.89:0/2948000440 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efec40cd160 0x7efec4154350 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:15.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.986+0000 7efeca6d8640 1 -- 172.21.15.89:0/2948000440 >> 172.21.15.89:0/2948000440 conn(0x7efec4089fd0 msgr2=0x7efec40cf040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:15.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.986+0000 7efeca6d8640 1 -- 172.21.15.89:0/2948000440 shutdown_connections 2023-10-18T10:46:15.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:15.986+0000 7efeca6d8640 1 -- 172.21.15.89:0/2948000440 wait complete. 2023-10-18T10:46:16.008 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-18T10:46:16.008 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:46:16.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.146+0000 7f904efe2640 1 Processor -- start 2023-10-18T10:46:16.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.146+0000 7f904efe2640 1 -- start start 2023-10-18T10:46:16.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.146+0000 7f904efe2640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f904815c940 0x7f90480d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.146+0000 7f904efe2640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f90480d0e70 0x7f90480cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.146+0000 7f904efe2640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f904815da50 con 0x7f90480d2f40 2023-10-18T10:46:16.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.146+0000 7f904efe2640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f904815ce40 con 0x7f90480d0e70 2023-10-18T10:46:16.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.146+0000 7f904efe2640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f904815d120 con 0x7f904815c940 2023-10-18T10:46:16.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.146+0000 7f9047fff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f90480d2f40 0x7f90480d0760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35524/0 (socket says 172.21.15.89:35524) 2023-10-18T10:46:16.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.146+0000 7f9047fff640 1 -- 172.21.15.89:0/1462958189 learned_addr learned my addr 172.21.15.89:0/1462958189 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:16.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.146+0000 7f904d57f640 1 --2- 172.21.15.89:0/1462958189 >> v2:172.21.15.89:3300/0 conn(0x7f90480d0e70 0x7f90480cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.146+0000 7f904d57f640 1 -- 172.21.15.89:0/1462958189 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f904815c940 msgr2=0x7f90480d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:16.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.146+0000 7f904d57f640 1 --2- 172.21.15.89:0/1462958189 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f904815c940 0x7f90480d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.146+0000 7f904d57f640 1 -- 172.21.15.89:0/1462958189 >> v1:172.21.15.89:6789/0 conn(0x7f90480d2f40 legacy=0x7f90480d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.146+0000 7f904d57f640 1 -- 172.21.15.89:0/1462958189 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f904815d400 con 0x7f90480d0e70 2023-10-18T10:46:16.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.150+0000 7f904d57f640 1 --2- 172.21.15.89:0/1462958189 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90480d0e70 0x7f90480cd160 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9040009dc0 tx=0x7f904001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb731769f459927 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:16.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.150+0000 7f90477fe640 1 -- 172.21.15.89:0/1462958189 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90400a5960 con 0x7f90480d0e70 2023-10-18T10:46:16.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.150+0000 7f90477fe640 1 -- 172.21.15.89:0/1462958189 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90400a5ac0 con 0x7f90480d0e70 2023-10-18T10:46:16.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.150+0000 7f90477fe640 1 -- 172.21.15.89:0/1462958189 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90400a5e50 con 0x7f90480d0e70 2023-10-18T10:46:16.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.150+0000 7f904efe2640 1 -- 172.21.15.89:0/1462958189 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90480d0e70 msgr2=0x7f90480cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.150+0000 7f904efe2640 1 --2- 172.21.15.89:0/1462958189 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90480d0e70 0x7f90480cd160 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9040009dc0 tx=0x7f904001c900 comp rx=0 tx=0).stop 2023-10-18T10:46:16.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.150+0000 7f904efe2640 1 -- 172.21.15.89:0/1462958189 shutdown_connections 2023-10-18T10:46:16.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.150+0000 7f904efe2640 1 --2- 172.21.15.89:0/1462958189 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90480d0e70 0x7f90480cd160 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.150+0000 7f904efe2640 1 --2- 172.21.15.89:0/1462958189 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f904815c940 0x7f90480d2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.150+0000 7f904efe2640 1 -- 172.21.15.89:0/1462958189 >> 172.21.15.89:0/1462958189 conn(0x7f9048089fd0 msgr2=0x7f904809b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:16.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.150+0000 7f904efe2640 1 -- 172.21.15.89:0/1462958189 shutdown_connections 2023-10-18T10:46:16.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.150+0000 7f904efe2640 1 -- 172.21.15.89:0/1462958189 wait complete. 2023-10-18T10:46:16.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.154+0000 7f904efe2640 1 Processor -- start 2023-10-18T10:46:16.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.154+0000 7f904efe2640 1 -- start start 2023-10-18T10:46:16.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.154+0000 7f904efe2640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90480d0e70 0x7f904815c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.154+0000 7f904efe2640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9048152c80 0x7f9048152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.154+0000 7f904efe2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9048152640 0x7f9048154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.154+0000 7f904efe2640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90480c8ed0 con 0x7f9048152c80 2023-10-18T10:46:16.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.154+0000 7f904efe2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904806b020 con 0x7f9048152640 2023-10-18T10:46:16.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.154+0000 7f904efe2640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f904806b300 con 0x7f90480d0e70 2023-10-18T10:46:16.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.154+0000 7f9047fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9048152c80 0x7f9048152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.154+0000 7f9047fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9048152c80 0x7f9048152100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45112/0 (socket says 172.21.15.89:45112) 2023-10-18T10:46:16.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.154+0000 7f9047fff640 1 -- 172.21.15.89:0/480124491 learned_addr learned my addr 172.21.15.89:0/480124491 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:16.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.154+0000 7f904d57f640 1 --2- 172.21.15.89:0/480124491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9048152640 0x7f9048154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.154+0000 7f904cd7e640 1 --2- 172.21.15.89:0/480124491 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90480d0e70 0x7f904815c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.154+0000 7f9047fff640 1 -- 172.21.15.89:0/480124491 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90480d0e70 msgr2=0x7f904815c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.154+0000 7f9047fff640 1 --2- 172.21.15.89:0/480124491 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90480d0e70 0x7f904815c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.154+0000 7f9047fff640 1 -- 172.21.15.89:0/480124491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9048152640 msgr2=0x7f9048154500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.154+0000 7f9047fff640 1 --2- 172.21.15.89:0/480124491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9048152640 0x7f9048154500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.154+0000 7f9047fff640 1 -- 172.21.15.89:0/480124491 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9040009a70 con 0x7f9048152c80 2023-10-18T10:46:16.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.154+0000 7f9047fff640 1 --2- 172.21.15.89:0/480124491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9048152c80 0x7f9048152100 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f903800c420 tx=0x7f903800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:16.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.154+0000 7f90457fa640 1 -- 172.21.15.89:0/480124491 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9038005cf0 con 0x7f9048152c80 2023-10-18T10:46:16.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.154+0000 7f90457fa640 1 -- 172.21.15.89:0/480124491 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9038005e50 con 0x7f9048152c80 2023-10-18T10:46:16.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.154+0000 7f90457fa640 1 -- 172.21.15.89:0/480124491 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90380072f0 con 0x7f9048152c80 2023-10-18T10:46:16.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.154+0000 7f904efe2640 1 -- 172.21.15.89:0/480124491 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90480c8880 con 0x7f9048152c80 2023-10-18T10:46:16.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.154+0000 7f904efe2640 1 -- 172.21.15.89:0/480124491 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9048154a40 con 0x7f9048152c80 2023-10-18T10:46:16.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.154+0000 7f904efe2640 1 -- 172.21.15.89:0/480124491 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f904806b470 con 0x7f9048152c80 2023-10-18T10:46:16.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.158+0000 7f90457fa640 1 -- 172.21.15.89:0/480124491 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f9038007450 con 0x7f9048152c80 2023-10-18T10:46:16.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.158+0000 7f90457fa640 1 --2- 172.21.15.89:0/480124491 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f90280656b0 0x7f9028067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.158+0000 7f90457fa640 1 -- 172.21.15.89:0/480124491 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9038082690 con 0x7f9048152c80 2023-10-18T10:46:16.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.158+0000 7f904cd7e640 1 --2- 172.21.15.89:0/480124491 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f90280656b0 0x7f9028067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.158+0000 7f904cd7e640 1 --2- 172.21.15.89:0/480124491 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f90280656b0 0x7f9028067b70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9030006fd0 tx=0x7f9030008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:16.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.162+0000 7f90457fa640 1 -- 172.21.15.89:0/480124491 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f903804fe50 con 0x7f9048152c80 2023-10-18T10:46:16.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.310+0000 7f904efe2640 1 -- 172.21.15.89:0/480124491 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9048152a50 con 0x7f9048152c80 2023-10-18T10:46:16.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.310+0000 7f90457fa640 1 -- 172.21.15.89:0/480124491 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20096 (secure 0 0 0) 0x7f9038053470 con 0x7f9048152c80 2023-10-18T10:46:16.314 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:16.314 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":27,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:46:00.166702+0000","last_up_change":"2023-10-18T10:45:42.164688+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:45:53.309122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-18T10:45:54.659763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:45:57.796514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6824","nonce":773078364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6827","nonce":773078364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6831","nonce":773078364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6829","nonce":773078364}]},"public_addr":"172.21.15.154:6824/773078364","cluster_addr":"172.21.15.154:6827/773078364","heartbeat_back_addr":"172.21.15.154:6831/773078364","heartbeat_front_addr":"172.21.15.154:6829/773078364","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:46:16.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.314+0000 7f904efe2640 1 -- 172.21.15.89:0/480124491 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f90280656b0 msgr2=0x7f9028067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.314+0000 7f904efe2640 1 --2- 172.21.15.89:0/480124491 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f90280656b0 0x7f9028067b70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9030006fd0 tx=0x7f9030008040 comp rx=0 tx=0).stop 2023-10-18T10:46:16.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.314+0000 7f904efe2640 1 -- 172.21.15.89:0/480124491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9048152c80 msgr2=0x7f9048152100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.314+0000 7f904efe2640 1 --2- 172.21.15.89:0/480124491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9048152c80 0x7f9048152100 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f903800c420 tx=0x7f903800c8e0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.314+0000 7f904efe2640 1 -- 172.21.15.89:0/480124491 shutdown_connections 2023-10-18T10:46:16.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.314+0000 7f904efe2640 1 --2- 172.21.15.89:0/480124491 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f90280656b0 0x7f9028067b70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.314+0000 7f904efe2640 1 --2- 172.21.15.89:0/480124491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9048152640 0x7f9048154500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.314+0000 7f904efe2640 1 --2- 172.21.15.89:0/480124491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9048152c80 0x7f9048152100 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.314+0000 7f904efe2640 1 --2- 172.21.15.89:0/480124491 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90480d0e70 0x7f904815c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.314+0000 7f904efe2640 1 -- 172.21.15.89:0/480124491 >> 172.21.15.89:0/480124491 conn(0x7f9048089fd0 msgr2=0x7f90480b7400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:16.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.314+0000 7f904efe2640 1 -- 172.21.15.89:0/480124491 shutdown_connections 2023-10-18T10:46:16.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.314+0000 7f904efe2640 1 -- 172.21.15.89:0/480124491 wait complete. 2023-10-18T10:46:16.335 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-18T10:46:16.336 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-18T10:46:16.337 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-18T10:46:16.337 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-18T10:46:16.338 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-18T10:46:16.338 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-18T10:46:16.339 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-18T10:46:16.339 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-18T10:46:16.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.578+0000 7f3dd932c640 1 Processor -- start 2023-10-18T10:46:16.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.578+0000 7f3dd932c640 1 -- start start 2023-10-18T10:46:16.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.578+0000 7f1afa25f640 1 Processor -- start 2023-10-18T10:46:16.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.578+0000 7f3dd932c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3dd40c8bc0 0x7f3dd40c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.578+0000 7f1afa25f640 1 -- start start 2023-10-18T10:46:16.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.578+0000 7f1afa25f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1af40d0700 0x7f1af4160780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.578+0000 7f1afa25f640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1af415d1b0 con 0x7f1af40d0270 2023-10-18T10:46:16.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.578+0000 7f1afa25f640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1af415d490 con 0x7f1af415c2f0 2023-10-18T10:46:16.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.578+0000 7f1afa25f640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1af40d0b50 con 0x7f1af40d0700 2023-10-18T10:46:16.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.578+0000 7f3dd932c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dd4160bb0 0x7f3dd40ace30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.578+0000 7f3dd932c640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dd40ac290 con 0x7f3dd406df60 2023-10-18T10:46:16.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.578+0000 7f3dd932c640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dd406cd10 con 0x7f3dd40c8bc0 2023-10-18T10:46:16.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.578+0000 7f3dd932c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd406cfb0 con 0x7f3dd4160bb0 2023-10-18T10:46:16.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.578+0000 7f1af2ffd640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1af40d0270 0x7f1af415c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35538/0 (socket says 172.21.15.89:35538) 2023-10-18T10:46:16.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.578+0000 7f1af2ffd640 1 -- 172.21.15.89:0/646768265 learned_addr learned my addr 172.21.15.89:0/646768265 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:16.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.578+0000 7f3dd27fc640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f3dd406df60 0x7f3dd40ac720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35532/0 (socket says 172.21.15.89:35532) 2023-10-18T10:46:16.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.578+0000 7f1af3fff640 1 --2- 172.21.15.89:0/646768265 >> v2:172.21.15.89:3300/0 conn(0x7f1af40d0700 0x7f1af4160780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.578+0000 7f3dd37fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dd4160bb0 0x7f3dd40ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.578+0000 7f3dd37fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dd4160bb0 0x7f3dd40ace30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:37170/0 (socket says 172.21.15.89:37170) 2023-10-18T10:46:16.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.578+0000 7f3dd37fe640 1 -- 172.21.15.89:0/4015131991 learned_addr learned my addr 172.21.15.89:0/4015131991 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:16.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.578+0000 7f3dd2ffd640 1 --2- 172.21.15.89:0/4015131991 >> v2:172.21.15.89:3300/0 conn(0x7f3dd40c8bc0 0x7f3dd40c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.582+0000 7f1af27fc640 1 -- 172.21.15.89:0/646768265 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2078547152 0 0) 0x7f1af415d1b0 con 0x7f1af40d0270 2023-10-18T10:46:16.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.582+0000 7f1af27fc640 1 -- 172.21.15.89:0/646768265 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1adc003530 con 0x7f1af40d0270 2023-10-18T10:46:16.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.582+0000 7f1af27fc640 1 -- 172.21.15.89:0/646768265 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 407113740 0 0) 0x7f1af415d490 con 0x7f1af415c2f0 2023-10-18T10:46:16.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.582+0000 7f1af27fc640 1 -- 172.21.15.89:0/646768265 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1af415d1b0 con 0x7f1af415c2f0 2023-10-18T10:46:16.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.582+0000 7f1af3fff640 1 -- 172.21.15.89:0/646768265 >> v1:172.21.15.154:6789/0 conn(0x7f1af415c2f0 legacy=0x7f1af40b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.582+0000 7f1af3fff640 1 -- 172.21.15.89:0/646768265 >> v1:172.21.15.89:6789/0 conn(0x7f1af40d0270 legacy=0x7f1af415c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.582+0000 7f1af3fff640 1 -- 172.21.15.89:0/646768265 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af415d770 con 0x7f1af40d0700 2023-10-18T10:46:16.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.582+0000 7f3dd1ffb640 1 -- 172.21.15.89:0/4015131991 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2175145149 0 0) 0x7f3dd40ac290 con 0x7f3dd406df60 2023-10-18T10:46:16.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.582+0000 7f3dd1ffb640 1 -- 172.21.15.89:0/4015131991 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3db8003530 con 0x7f3dd406df60 2023-10-18T10:46:16.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.582+0000 7f3dd1ffb640 1 -- 172.21.15.89:0/4015131991 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1971095911 0 0) 0x7f3db8003530 con 0x7f3dd406df60 2023-10-18T10:46:16.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.582+0000 7f3dd1ffb640 1 -- 172.21.15.89:0/4015131991 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dd4160bb0 msgr2=0x7f3dd40ace30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:16.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.582+0000 7f3dd1ffb640 1 --2- 172.21.15.89:0/4015131991 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dd4160bb0 0x7f3dd40ace30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.582+0000 7f3dd1ffb640 1 -- 172.21.15.89:0/4015131991 >> v2:172.21.15.89:3300/0 conn(0x7f3dd40c8bc0 msgr2=0x7f3dd40c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:16.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.582+0000 7f3dd1ffb640 1 --2- 172.21.15.89:0/4015131991 >> v2:172.21.15.89:3300/0 conn(0x7f3dd40c8bc0 0x7f3dd40c8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.582+0000 7f3dd1ffb640 1 -- 172.21.15.89:0/4015131991 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd40ab960 con 0x7f3dd406df60 2023-10-18T10:46:16.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.582+0000 7f3dd1ffb640 1 -- 172.21.15.89:0/4015131991 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7f3dc0001680 con 0x7f3dd406df60 2023-10-18T10:46:16.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.582+0000 7f3dd1ffb640 1 -- 172.21.15.89:0/4015131991 >> v1:172.21.15.89:6789/0 conn(0x7f3dd406df60 legacy=0x7f3dd40ac720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.582+0000 7f3dd1ffb640 1 --2- 172.21.15.89:0/4015131991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3db8003980 0x7f3db8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.582+0000 7f3dd1ffb640 1 --2- 172.21.15.89:0/4015131991 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3db80047d0 0x7f3db8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.582+0000 7f3dd1ffb640 1 --2- 172.21.15.89:0/4015131991 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3db8005190 0x7f3db800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.582+0000 7f3dd1ffb640 1 -- 172.21.15.89:0/4015131991 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd406cd10 con 0x7f3db8003980 2023-10-18T10:46:16.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.582+0000 7f3dd1ffb640 1 -- 172.21.15.89:0/4015131991 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd406cfb0 con 0x7f3db80047d0 2023-10-18T10:46:16.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.582+0000 7f3dd1ffb640 1 -- 172.21.15.89:0/4015131991 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db80147c0 con 0x7f3db8005190 2023-10-18T10:46:16.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.582+0000 7f3dd27fc640 1 --2- 172.21.15.89:0/4015131991 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3db80047d0 0x7f3db8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.586+0000 7f3dd27fc640 1 -- 172.21.15.89:0/4015131991 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3db8005190 msgr2=0x7f3db800b990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T10:46:16.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.586+0000 7f3dd27fc640 1 --2- 172.21.15.89:0/4015131991 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3db8005190 0x7f3db800b990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.586+0000 7f3dd27fc640 1 -- 172.21.15.89:0/4015131991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3db8003980 msgr2=0x7f3db8003da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T10:46:16.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.586+0000 7f3dd27fc640 1 --2- 172.21.15.89:0/4015131991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3db8003980 0x7f3db8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.586+0000 7f3dd27fc640 1 -- 172.21.15.89:0/4015131991 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db8003530 con 0x7f3db80047d0 2023-10-18T10:46:16.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.586+0000 7f1af3fff640 1 --2- 172.21.15.89:0/646768265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1af40d0700 0x7f1af4160780 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1ae40097f0 tx=0x7f1ae401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e98322339b9c23 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:16.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.586+0000 7f3dd37fe640 1 --2- 172.21.15.89:0/4015131991 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dd4160bb0 0x7f3dd40ace30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:16.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.586+0000 7f1af27fc640 1 -- 172.21.15.89:0/646768265 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae40ab600 con 0x7f1af40d0700 2023-10-18T10:46:16.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.586+0000 7f1af27fc640 1 -- 172.21.15.89:0/646768265 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae40a7050 con 0x7f1af40d0700 2023-10-18T10:46:16.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.586+0000 7f1af27fc640 1 -- 172.21.15.89:0/646768265 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae40ab980 con 0x7f1af40d0700 2023-10-18T10:46:16.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.590+0000 7f3dd27fc640 1 --2- 172.21.15.89:0/4015131991 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3db80047d0 0x7f3db8004c50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000c190 tx=0x7f3dc001cbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ccf62fcf39c8086 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:16.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.590+0000 7f1afa25f640 1 -- 172.21.15.89:0/646768265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1af40d0700 msgr2=0x7f1af4160780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.590+0000 7f1afa25f640 1 --2- 172.21.15.89:0/646768265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1af40d0700 0x7f1af4160780 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1ae40097f0 tx=0x7f1ae401c480 comp rx=0 tx=0).stop 2023-10-18T10:46:16.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.590+0000 7f1afa25f640 1 -- 172.21.15.89:0/646768265 shutdown_connections 2023-10-18T10:46:16.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.590+0000 7f1afa25f640 1 --2- 172.21.15.89:0/646768265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1af40d0700 0x7f1af4160780 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.590+0000 7f1afa25f640 1 -- 172.21.15.89:0/646768265 >> 172.21.15.89:0/646768265 conn(0x7f1af4069ad0 msgr2=0x7f1af4069eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:16.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.590+0000 7f3dd1ffb640 1 -- 172.21.15.89:0/4015131991 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0007110 con 0x7f3db80047d0 2023-10-18T10:46:16.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.590+0000 7fdf8eacb640 1 Processor -- start 2023-10-18T10:46:16.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.590+0000 7f3dd1ffb640 1 -- 172.21.15.89:0/4015131991 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc0007270 con 0x7f3db80047d0 2023-10-18T10:46:16.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.590+0000 7f3dd1ffb640 1 -- 172.21.15.89:0/4015131991 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0007620 con 0x7f3db80047d0 2023-10-18T10:46:16.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.590+0000 7f1afa25f640 1 -- 172.21.15.89:0/646768265 shutdown_connections 2023-10-18T10:46:16.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.590+0000 7f1afa25f640 1 -- 172.21.15.89:0/646768265 wait complete. 2023-10-18T10:46:16.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.590+0000 7f3dd932c640 1 -- 172.21.15.89:0/4015131991 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3db80047d0 msgr2=0x7f3db8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.590+0000 7f3dd932c640 1 --2- 172.21.15.89:0/4015131991 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3db80047d0 0x7f3db8004c50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000c190 tx=0x7f3dc001cbb0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.590+0000 7f3dd37fe640 1 -- 172.21.15.89:0/4015131991 reap_dead start 2023-10-18T10:46:16.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.590+0000 7f3dd932c640 1 -- 172.21.15.89:0/4015131991 shutdown_connections 2023-10-18T10:46:16.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.590+0000 7f3dd932c640 1 -- 172.21.15.89:0/4015131991 >> 172.21.15.89:0/4015131991 conn(0x7f3dd40afe00 msgr2=0x7f3dd40b0210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:16.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.594+0000 7f1afa25f640 1 Processor -- start 2023-10-18T10:46:16.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.594+0000 7f3dd932c640 1 -- 172.21.15.89:0/4015131991 shutdown_connections 2023-10-18T10:46:16.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.594+0000 7fd0cd1f3640 1 Processor -- start 2023-10-18T10:46:16.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.594+0000 7f3dd932c640 1 -- 172.21.15.89:0/4015131991 wait complete. 2023-10-18T10:46:16.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.590+0000 7fdf8eacb640 1 -- start start 2023-10-18T10:46:16.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.590+0000 7fdf8eacb640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdf88070780 0x7fdf880d1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.594+0000 7fdf8c867640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdf88070780 0x7fdf880d1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.594+0000 7fdf8c867640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdf88070780 0x7fdf880d1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45148/0 (socket says 172.21.15.89:45148) 2023-10-18T10:46:16.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.594+0000 7f1afa25f640 1 -- start start 2023-10-18T10:46:16.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.594+0000 7fd0cd1f3640 1 -- start start 2023-10-18T10:46:16.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.594+0000 7fdf8eacb640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdf880b6dd0 0x7fdf880b4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.594+0000 7fdf8eacb640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf881652d0 con 0x7fdf880b5640 2023-10-18T10:46:16.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.594+0000 7fdf8eacb640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf881646c0 con 0x7fdf88070780 2023-10-18T10:46:16.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.594+0000 7fdf8eacb640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf881649a0 con 0x7fdf880b6dd0 2023-10-18T10:46:16.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.594+0000 7fdf8c867640 1 -- 172.21.15.89:0/589463525 learned_addr learned my addr 172.21.15.89:0/589463525 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:16.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.594+0000 7fdf87fff640 1 --2- 172.21.15.89:0/589463525 >> v2:172.21.15.154:3300/0 conn(0x7fdf880b6dd0 0x7fdf880b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.594+0000 7fd0cd1f3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0c80d0700 0x7fd0c8160780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.594+0000 7fd0cd1f3640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0c815d1b0 con 0x7fd0c80d0270 2023-10-18T10:46:16.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.594+0000 7fd0cd1f3640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0c815d490 con 0x7fd0c815c2f0 2023-10-18T10:46:16.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.594+0000 7fd0cd1f3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c80d0b50 con 0x7fd0c80d0700 2023-10-18T10:46:16.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.594+0000 7fd0c759e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0c80d0700 0x7fd0c8160780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.594+0000 7f1afa25f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1af40d0270 0x7f1af40c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.594+0000 7f1afa25f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1af40d0700 0x7f1af40cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.594+0000 7fd0c659c640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd0c80d0270 0x7fd0c815c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35556/0 (socket says 172.21.15.89:35556) 2023-10-18T10:46:16.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.594+0000 7fd0c659c640 1 -- 172.21.15.89:0/1888568942 learned_addr learned my addr 172.21.15.89:0/1888568942 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:16.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.594+0000 7fdf8c867640 1 -- 172.21.15.89:0/589463525 >> v2:172.21.15.154:3300/0 conn(0x7fdf880b6dd0 msgr2=0x7fdf880b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:16.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.594+0000 7fdf8c867640 1 --2- 172.21.15.89:0/589463525 >> v2:172.21.15.154:3300/0 conn(0x7fdf880b6dd0 0x7fdf880b4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.594+0000 7fdf8c867640 1 -- 172.21.15.89:0/589463525 >> v1:172.21.15.89:6789/0 conn(0x7fdf880b5640 legacy=0x7fdf880cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:16.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.594+0000 7fdf8c867640 1 -- 172.21.15.89:0/589463525 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf88164c80 con 0x7fdf88070780 2023-10-18T10:46:16.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.594+0000 7f1afa25f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1af415c2f0 0x7f1af4147550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.594+0000 7f1afa25f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af40c9160 con 0x7f1af40d0270 2023-10-18T10:46:16.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.594+0000 7f1afa25f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af40cbb00 con 0x7f1af415c2f0 2023-10-18T10:46:16.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.594+0000 7f1afa25f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af40cbde0 con 0x7f1af40d0700 2023-10-18T10:46:16.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7fd0c759e640 1 -- 172.21.15.89:0/1888568942 >> v1:172.21.15.154:6789/0 conn(0x7fd0c815c2f0 legacy=0x7fd0c80b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7fd0c5d9b640 1 -- 172.21.15.89:0/1888568942 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2921344553 0 0) 0x7fd0c815d490 con 0x7fd0c815c2f0 2023-10-18T10:46:16.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7fd0c759e640 1 -- 172.21.15.89:0/1888568942 >> v1:172.21.15.89:6789/0 conn(0x7fd0c80d0270 legacy=0x7fd0c815c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7fd0c759e640 1 -- 172.21.15.89:0/1888568942 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c815d770 con 0x7fd0c80d0700 2023-10-18T10:46:16.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7fdd7d38f640 1 Processor -- start 2023-10-18T10:46:16.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7fdd7d38f640 1 -- start start 2023-10-18T10:46:16.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.594+0000 7f1af2ffd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1af40d0700 0x7f1af40cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7f1af37fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1af40d0270 0x7f1af40c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7f1af2ffd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1af40d0700 0x7f1af40cb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:37186/0 (socket says 172.21.15.89:37186) 2023-10-18T10:46:16.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7f1af2ffd640 1 -- 172.21.15.89:0/3136184549 learned_addr learned my addr 172.21.15.89:0/3136184549 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:16.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7f1af3fff640 1 --2- 172.21.15.89:0/3136184549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1af415c2f0 0x7f1af4147550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7f1af2ffd640 1 -- 172.21.15.89:0/3136184549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1af415c2f0 msgr2=0x7f1af4147550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7f1af2ffd640 1 --2- 172.21.15.89:0/3136184549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1af415c2f0 0x7f1af4147550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7f1af2ffd640 1 -- 172.21.15.89:0/3136184549 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1af40d0270 msgr2=0x7f1af40c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7f1af2ffd640 1 --2- 172.21.15.89:0/3136184549 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1af40d0270 0x7f1af40c8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7f1af2ffd640 1 -- 172.21.15.89:0/3136184549 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae4009420 con 0x7f1af40d0700 2023-10-18T10:46:16.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7f3dd932c640 1 Processor -- start 2023-10-18T10:46:16.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7f1af2ffd640 1 --2- 172.21.15.89:0/3136184549 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1af40d0700 0x7f1af40cb1d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1ae800c4d0 tx=0x7f1ae800c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:16.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7f1adbfff640 1 -- 172.21.15.89:0/3136184549 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8005cd0 con 0x7f1af40d0700 2023-10-18T10:46:16.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7f1adbfff640 1 -- 172.21.15.89:0/3136184549 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae8005e30 con 0x7f1af40d0700 2023-10-18T10:46:16.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7f1adbfff640 1 -- 172.21.15.89:0/3136184549 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8007270 con 0x7f1af40d0700 2023-10-18T10:46:16.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7f1afa25f640 1 -- 172.21.15.89:0/3136184549 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1af40cc0c0 con 0x7f1af40d0700 2023-10-18T10:46:16.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7f1afa25f640 1 -- 172.21.15.89:0/3136184549 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1af41458a0 con 0x7f1af40d0700 2023-10-18T10:46:16.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7f3dd932c640 1 -- start start 2023-10-18T10:46:16.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7fdf87fff640 1 --2- 172.21.15.89:0/589463525 >> v2:172.21.15.154:3300/0 conn(0x7fdf880b6dd0 0x7fdf880b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:46:16.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7f22544e5640 1 Processor -- start 2023-10-18T10:46:16.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7f22544e5640 1 -- start start 2023-10-18T10:46:16.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7fdd7d38f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdd7815c2f0 0x7fdd780b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7fdd7d38f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd780d0270 0x7fdd780d06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7fdd7d38f640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd7815d1b0 con 0x7fdd7815c6d0 2023-10-18T10:46:16.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7fdd7d38f640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd7815d490 con 0x7fdd7815c2f0 2023-10-18T10:46:16.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7fdd7d38f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7815d770 con 0x7fdd780d0270 2023-10-18T10:46:16.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7fdd76ffd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdd7815c2f0 0x7fdd780b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7f3dd932c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3dd41587b0 0x7f3dd40c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7fdd777fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fdd7815c6d0 0x7fdd78160780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35566/0 (socket says 172.21.15.89:35566) 2023-10-18T10:46:16.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7fdd777fe640 1 -- 172.21.15.89:0/2761635048 learned_addr learned my addr 172.21.15.89:0/2761635048 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:16.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.598+0000 7fdd6e7fc640 1 --2- 172.21.15.89:0/2761635048 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd780d0270 0x7fdd780d06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7f1afa25f640 1 -- 172.21.15.89:0/3136184549 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1af416a340 con 0x7f1af40d0700 2023-10-18T10:46:16.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7fdd767fc640 1 -- 172.21.15.89:0/2761635048 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3811124088 0 0) 0x7fdd7815d1b0 con 0x7fdd7815c6d0 2023-10-18T10:46:16.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7fdd767fc640 1 -- 172.21.15.89:0/2761635048 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd64003530 con 0x7fdd7815c6d0 2023-10-18T10:46:16.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7fdd76ffd640 1 -- 172.21.15.89:0/2761635048 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd780d0270 msgr2=0x7fdd780d06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:16.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7fdd76ffd640 1 --2- 172.21.15.89:0/2761635048 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd780d0270 0x7fdd780d06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7fdd76ffd640 1 -- 172.21.15.89:0/2761635048 >> v1:172.21.15.89:6789/0 conn(0x7fdd7815c6d0 legacy=0x7fdd78160780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7fdd76ffd640 1 -- 172.21.15.89:0/2761635048 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd7815da50 con 0x7fdd7815c2f0 2023-10-18T10:46:16.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7fbd063e1640 1 Processor -- start 2023-10-18T10:46:16.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7f3dd932c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3dd40b3a20 0x7f3dd415f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7fdf8c867640 1 --2- 172.21.15.89:0/589463525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf88070780 0x7fdf880d1d90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdf88164620 tx=0x7fdf7c01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e689d627ac71dc87 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:16.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7f3dd2ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3dd41587b0 0x7f3dd40c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7f3dd2ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3dd41587b0 0x7f3dd40c7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:34132/0 (socket says 172.21.15.89:34132) 2023-10-18T10:46:16.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7fbd063e1640 1 -- start start 2023-10-18T10:46:16.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7f3dd27fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3dd40b3a20 0x7f3dd415f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7f3dd27fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3dd40b3a20 0x7f3dd415f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45176/0 (socket says 172.21.15.89:45176) 2023-10-18T10:46:16.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7f3dd932c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dd40b20d0 0x7f3dd415fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7f3dd932c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd41f43c0 con 0x7f3dd40b3a20 2023-10-18T10:46:16.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7f3dd932c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd41f46a0 con 0x7f3dd41587b0 2023-10-18T10:46:16.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7f3dd932c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd41f4980 con 0x7f3dd40b20d0 2023-10-18T10:46:16.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7f3dd2ffd640 1 -- 172.21.15.89:0/3538787601 learned_addr learned my addr 172.21.15.89:0/3538787601 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:16.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7f3dd37fe640 1 --2- 172.21.15.89:0/3538787601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dd40b20d0 0x7f3dd415fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7f3dd27fc640 1 -- 172.21.15.89:0/3538787601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dd40b20d0 msgr2=0x7f3dd415fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7fd0c759e640 1 --2- 172.21.15.89:0/1888568942 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0c80d0700 0x7fd0c8160780 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0009df0 tx=0x7fd0b001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be77d59ce0f5e6f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:16.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7f22544e5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f224c0c94b0 0x7f224c0afd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7f22544e5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f224c0b02d0 0x7f224c158e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7fdf877fe640 1 -- 172.21.15.89:0/589463525 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c0a5650 con 0x7fdf88070780 2023-10-18T10:46:16.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7fdf877fe640 1 -- 172.21.15.89:0/589463525 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf7c0a57b0 con 0x7fdf88070780 2023-10-18T10:46:16.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7fdf877fe640 1 -- 172.21.15.89:0/589463525 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c0a5b40 con 0x7fdf88070780 2023-10-18T10:46:16.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7fd0c5d9b640 1 -- 172.21.15.89:0/1888568942 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b00a5830 con 0x7fd0c80d0700 2023-10-18T10:46:16.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7fd0c5d9b640 1 -- 172.21.15.89:0/1888568942 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b00a5990 con 0x7fd0c80d0700 2023-10-18T10:46:16.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7fd0c5d9b640 1 -- 172.21.15.89:0/1888568942 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b00a5d20 con 0x7fd0c80d0700 2023-10-18T10:46:16.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7f3dd27fc640 1 --2- 172.21.15.89:0/3538787601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dd40b20d0 0x7f3dd415fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7f3dd27fc640 1 -- 172.21.15.89:0/3538787601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3dd41587b0 msgr2=0x7f3dd40c7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7f3dd27fc640 1 --2- 172.21.15.89:0/3538787601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3dd41587b0 0x7f3dd40c7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7f3dd27fc640 1 -- 172.21.15.89:0/3538787601 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd40cc060 con 0x7f3dd40b3a20 2023-10-18T10:46:16.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7f3dd2ffd640 1 --2- 172.21.15.89:0/3538787601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3dd41587b0 0x7f3dd40c7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:46:16.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7f3dd27fc640 1 --2- 172.21.15.89:0/3538787601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3dd40b3a20 0x7f3dd415f7a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0006000 tx=0x7f3db8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:16.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7fd0cd1f3640 1 -- 172.21.15.89:0/1888568942 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0c80d0700 msgr2=0x7fd0c8160780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7fd0cd1f3640 1 --2- 172.21.15.89:0/1888568942 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0c80d0700 0x7fd0c8160780 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0009df0 tx=0x7fd0b001c970 comp rx=0 tx=0).stop 2023-10-18T10:46:16.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7fdf8eacb640 1 -- 172.21.15.89:0/589463525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf88070780 msgr2=0x7fdf880d1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7fdf8eacb640 1 --2- 172.21.15.89:0/589463525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf88070780 0x7fdf880d1d90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdf88164620 tx=0x7fdf7c01c9b0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7f1adbfff640 1 -- 172.21.15.89:0/3136184549 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f1ae80073d0 con 0x7f1af40d0700 2023-10-18T10:46:16.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7f1adbfff640 1 --2- 172.21.15.89:0/3136184549 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1ae0065680 0x7f1ae0067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7f1adbfff640 1 -- 172.21.15.89:0/3136184549 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1ae8093620 con 0x7f1af40d0700 2023-10-18T10:46:16.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7fbd063e1640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbd0006dd10 0x7fbd0006cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7fbd063e1640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd0015d4f0 con 0x7fbd000b0920 2023-10-18T10:46:16.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7fbd063e1640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd000cbee0 con 0x7fbd0015cfa0 2023-10-18T10:46:16.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.602+0000 7fbd063e1640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd000ccd60 con 0x7fbd0006dd10 2023-10-18T10:46:16.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7f1af37fe640 1 --2- 172.21.15.89:0/3136184549 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1ae0065680 0x7f1ae0067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7fd0cd1f3640 1 -- 172.21.15.89:0/1888568942 shutdown_connections 2023-10-18T10:46:16.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7fd0cd1f3640 1 --2- 172.21.15.89:0/1888568942 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0c80d0700 0x7fd0c8160780 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7fd0cd1f3640 1 -- 172.21.15.89:0/1888568942 >> 172.21.15.89:0/1888568942 conn(0x7fd0c8069ad0 msgr2=0x7fd0c8069eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:16.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7fdf8eacb640 1 -- 172.21.15.89:0/589463525 shutdown_connections 2023-10-18T10:46:16.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7fdf8eacb640 1 --2- 172.21.15.89:0/589463525 >> v2:172.21.15.154:3300/0 conn(0x7fdf880b6dd0 0x7fdf880b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7fdf8eacb640 1 --2- 172.21.15.89:0/589463525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf88070780 0x7fdf880d1d90 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7fdf8eacb640 1 -- 172.21.15.89:0/589463525 >> 172.21.15.89:0/589463525 conn(0x7fdf880cf120 msgr2=0x7fdf880cf530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:16.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7f1adbfff640 1 --2- 172.21.15.89:0/3136184549 >> [v2:172.21.15.89:6800/308765932,v1:172.21.15.89:6801/308765932] conn(0x7f1ae006f2e0 0x7f1ae0071720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7f1adbfff640 1 -- 172.21.15.89:0/3136184549 --> [v2:172.21.15.89:6800/308765932,v1:172.21.15.89:6801/308765932] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1ae0071df0 con 0x7f1ae006f2e0 2023-10-18T10:46:16.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7f1adbfff640 1 -- 172.21.15.89:0/3136184549 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ae8095ce0 con 0x7f1af40d0700 2023-10-18T10:46:16.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7f1af3fff640 1 --2- 172.21.15.89:0/3136184549 >> [v2:172.21.15.89:6800/308765932,v1:172.21.15.89:6801/308765932] conn(0x7f1ae006f2e0 0x7f1ae0071720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7f1af37fe640 1 --2- 172.21.15.89:0/3136184549 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1ae0065680 0x7f1ae0067b40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1aec007fd0 tx=0x7f1aec009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:16.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7f3dcb7fe640 1 -- 172.21.15.89:0/3538787601 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0001640 con 0x7f3dd40b3a20 2023-10-18T10:46:16.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7f3dd932c640 1 -- 172.21.15.89:0/3538787601 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dd41f4c60 con 0x7f3dd40b3a20 2023-10-18T10:46:16.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7f3dcb7fe640 1 -- 172.21.15.89:0/3538787601 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc00047f0 con 0x7f3dd40b3a20 2023-10-18T10:46:16.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7f3dd932c640 1 -- 172.21.15.89:0/3538787601 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dd40c8360 con 0x7f3dd40b3a20 2023-10-18T10:46:16.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7f3dcb7fe640 1 -- 172.21.15.89:0/3538787601 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0001640 con 0x7f3dd40b3a20 2023-10-18T10:46:16.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7f3dc97fa640 1 -- 172.21.15.89:0/3538787601 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3db4000fd0 con 0x7f3dd40b3a20 2023-10-18T10:46:16.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7f2251a80640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f224c0b02d0 0x7f224c158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7f2252281640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f224c0c94b0 0x7f224c0afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7f2252281640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f224c0c94b0 0x7f224c0afd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:34136/0 (socket says 172.21.15.89:34136) 2023-10-18T10:46:16.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7f2251a80640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f224c0b02d0 0x7f224c158e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:37212/0 (socket says 172.21.15.89:37212) 2023-10-18T10:46:16.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7f22544e5640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f224c15cc20 con 0x7f224c159360 2023-10-18T10:46:16.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7f22544e5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f224c0b0750 con 0x7f224c0c94b0 2023-10-18T10:46:16.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7f22544e5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f224c159b90 con 0x7f224c0b02d0 2023-10-18T10:46:16.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7f2252281640 1 -- 172.21.15.89:0/4216993840 learned_addr learned my addr 172.21.15.89:0/4216993840 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:16.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7fd0cd1f3640 1 -- 172.21.15.89:0/1888568942 shutdown_connections 2023-10-18T10:46:16.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7fd0cd1f3640 1 -- 172.21.15.89:0/1888568942 wait complete. 2023-10-18T10:46:16.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7fdf8eacb640 1 -- 172.21.15.89:0/589463525 shutdown_connections 2023-10-18T10:46:16.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7fdf8eacb640 1 -- 172.21.15.89:0/589463525 wait complete. 2023-10-18T10:46:16.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7fd456d05640 1 Processor -- start 2023-10-18T10:46:16.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7fd456d05640 1 -- start start 2023-10-18T10:46:16.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7fbd0497e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbd0006dd10 0x7fbd0006cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7fbd0497e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbd0006dd10 0x7fbd0006cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:34150/0 (socket says 172.21.15.89:34150) 2023-10-18T10:46:16.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7fbd0497e640 1 -- 172.21.15.89:0/2955890159 learned_addr learned my addr 172.21.15.89:0/2955890159 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:16.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7fbd0497e640 1 -- 172.21.15.89:0/2955890159 >> v1:172.21.15.154:6789/0 conn(0x7fbd0015cfa0 legacy=0x7fbd000cc650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T10:46:16.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7fbd0497e640 1 -- 172.21.15.89:0/2955890159 >> v1:172.21.15.89:6789/0 conn(0x7fbd000b0920 legacy=0x7fbd00158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:16.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7fbd0497e640 1 -- 172.21.15.89:0/2955890159 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd000cc1c0 con 0x7fbd0006dd10 2023-10-18T10:46:16.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7f2252281640 1 -- 172.21.15.89:0/4216993840 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f224c0b02d0 msgr2=0x7f224c158e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:16.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7f2252281640 1 --2- 172.21.15.89:0/4216993840 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f224c0b02d0 0x7f224c158e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7f2252281640 1 -- 172.21.15.89:0/4216993840 >> v1:172.21.15.89:6789/0 conn(0x7f224c159360 legacy=0x7f224c06b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:16.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7f2252281640 1 -- 172.21.15.89:0/4216993840 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f224c15c2f0 con 0x7f224c0c94b0 2023-10-18T10:46:16.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7fd456d05640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd45015c310 0x7fd4500b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7f1af3fff640 1 --2- 172.21.15.89:0/3136184549 >> [v2:172.21.15.89:6800/308765932,v1:172.21.15.89:6801/308765932] conn(0x7f1ae006f2e0 0x7f1ae0071720 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:16.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7fd456d05640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd4500cf950 0x7fd4500d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7fd456d05640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4500d1470 con 0x7fd4500cd4e0 2023-10-18T10:46:16.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7fd456d05640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4500d0f80 con 0x7fd45015c310 2023-10-18T10:46:16.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7fd456d05640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4500d1190 con 0x7fd4500cf950 2023-10-18T10:46:16.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7fdf8eacb640 1 Processor -- start 2023-10-18T10:46:16.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7fdf8eacb640 1 -- start start 2023-10-18T10:46:16.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7fd456504640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd4500cf950 0x7fd4500d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7fd456504640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd4500cf950 0x7fd4500d0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:34166/0 (socket says 172.21.15.89:34166) 2023-10-18T10:46:16.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.606+0000 7fd456504640 1 -- 172.21.15.89:0/192584431 learned_addr learned my addr 172.21.15.89:0/192584431 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:16.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fd456504640 1 -- 172.21.15.89:0/192584431 >> v2:172.21.15.89:3300/0 conn(0x7fd45015c310 msgr2=0x7fd4500b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:16.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fd0cd1f3640 1 Processor -- start 2023-10-18T10:46:16.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fd455d03640 1 --2- 172.21.15.89:0/192584431 >> v2:172.21.15.89:3300/0 conn(0x7fd45015c310 0x7fd4500b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fd456504640 1 --2- 172.21.15.89:0/192584431 >> v2:172.21.15.89:3300/0 conn(0x7fd45015c310 0x7fd4500b53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fd456504640 1 -- 172.21.15.89:0/192584431 >> v1:172.21.15.89:6789/0 conn(0x7fd4500cd4e0 legacy=0x7fd4500cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fd456504640 1 -- 172.21.15.89:0/192584431 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4500d1960 con 0x7fd4500cf950 2023-10-18T10:46:16.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fd455d03640 1 --2- 172.21.15.89:0/192584431 >> v2:172.21.15.89:3300/0 conn(0x7fd45015c310 0x7fd4500b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:16.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7f3dcb7fe640 1 -- 172.21.15.89:0/3538787601 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f3dc0004960 con 0x7f3dd40b3a20 2023-10-18T10:46:16.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fdf8eacb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf88070780 0x7fdf881602d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fdf8eacb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf880b5640 0x7fdf88154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fdf8eacb640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf880b6dd0 0x7fdf880b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fdf8eacb640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf880c54a0 con 0x7fdf880b6dd0 2023-10-18T10:46:16.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fdf8eacb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf881568e0 con 0x7fdf880b5640 2023-10-18T10:46:16.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fdf8eacb640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf88156b80 con 0x7fdf88070780 2023-10-18T10:46:16.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fbd0497e640 1 --2- 172.21.15.89:0/2955890159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd0006dd10 0x7fbd0006cd10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0009e50 tx=0x7fbcf001c960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7587586e3b61dc3e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:16.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fd0cd1f3640 1 -- start start 2023-10-18T10:46:16.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fd0cd1f3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0c80d0270 0x7fd0c80caf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fd0cd1f3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0c80d0700 0x7fd0c80c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fd0cd1f3640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0c815c2f0 0x7fd0c80c6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fd0cd1f3640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c80c7930 con 0x7fd0c815c2f0 2023-10-18T10:46:16.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fd0cd1f3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c80caa30 con 0x7fd0c80d0700 2023-10-18T10:46:16.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fd0cd1f3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c80c6d20 con 0x7fd0c80d0270 2023-10-18T10:46:16.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fdf87fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf880b5640 0x7fdf88154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fdf87fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf880b5640 0x7fdf88154b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:34170/0 (socket says 172.21.15.89:34170) 2023-10-18T10:46:16.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fdf87fff640 1 -- 172.21.15.89:0/3377746371 learned_addr learned my addr 172.21.15.89:0/3377746371 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:16.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fdd76ffd640 1 --2- 172.21.15.89:0/2761635048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd7815c2f0 0x7fdd780b18e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdd680097c0 tx=0x7fdd6801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a561fd15f2d82131 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:16.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fdd767fc640 1 -- 172.21.15.89:0/2761635048 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd680ab670 con 0x7fdd7815c2f0 2023-10-18T10:46:16.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fdd767fc640 1 -- 172.21.15.89:0/2761635048 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd680ab7d0 con 0x7fdd7815c2f0 2023-10-18T10:46:16.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fdd767fc640 1 -- 172.21.15.89:0/2761635048 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd680abb60 con 0x7fdd7815c2f0 2023-10-18T10:46:16.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fbcfeffd640 1 -- 172.21.15.89:0/2955890159 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf00a5730 con 0x7fbd0006dd10 2023-10-18T10:46:16.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fbcfeffd640 1 -- 172.21.15.89:0/2955890159 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf00a5890 con 0x7fbd0006dd10 2023-10-18T10:46:16.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fbcfeffd640 1 -- 172.21.15.89:0/2955890159 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf00a5c20 con 0x7fbd0006dd10 2023-10-18T10:46:16.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fdf87fff640 1 -- 172.21.15.89:0/3377746371 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf88070780 msgr2=0x7fdf881602d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-18T10:46:16.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fdf87fff640 1 --2- 172.21.15.89:0/3377746371 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf88070780 0x7fdf881602d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fdf87fff640 1 -- 172.21.15.89:0/3377746371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf880b6dd0 msgr2=0x7fdf880b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fdd7d38f640 1 -- 172.21.15.89:0/2761635048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd7815c2f0 msgr2=0x7fdd780b18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fdd7d38f640 1 --2- 172.21.15.89:0/2761635048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd7815c2f0 0x7fdd780b18e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdd680097c0 tx=0x7fdd6801c470 comp rx=0 tx=0).stop 2023-10-18T10:46:16.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7f3dcb7fe640 1 --2- 172.21.15.89:0/3538787601 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3dbc0656e0 0x7f3dbc067ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fd0c659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0c80d0700 0x7fd0c80c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fd0c659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0c80d0700 0x7fd0c80c61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:34172/0 (socket says 172.21.15.89:34172) 2023-10-18T10:46:16.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fd0c659c640 1 -- 172.21.15.89:0/1492844345 learned_addr learned my addr 172.21.15.89:0/1492844345 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:16.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fdf87fff640 1 --2- 172.21.15.89:0/3377746371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf880b6dd0 0x7fdf880b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fdf87fff640 1 -- 172.21.15.89:0/3377746371 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf74009420 con 0x7fdf880b5640 2023-10-18T10:46:16.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7f3dcb7fe640 1 -- 172.21.15.89:0/3538787601 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3dc00f1760 con 0x7f3dd40b3a20 2023-10-18T10:46:16.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7f3dcb7fe640 1 --2- 172.21.15.89:0/3538787601 >> [v2:172.21.15.89:6816/2220030573,v1:172.21.15.89:6817/2220030573] conn(0x7f3dbc06f340 0x7f3dbc071780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7f3dcb7fe640 1 -- 172.21.15.89:0/3538787601 --> [v2:172.21.15.89:6816/2220030573,v1:172.21.15.89:6817/2220030573] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3dbc071e50 con 0x7f3dbc06f340 2023-10-18T10:46:16.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7f3dcb7fe640 1 -- 172.21.15.89:0/3538787601 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3dc00e9d90 con 0x7f3dd40b3a20 2023-10-18T10:46:16.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7f3dd2ffd640 1 --2- 172.21.15.89:0/3538787601 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3dbc0656e0 0x7f3dbc067ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7f2252281640 1 --2- 172.21.15.89:0/4216993840 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f224c0c94b0 0x7f224c0afd90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2244006380 tx=0x7f22440250a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3162884fb2679ec server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:16.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7f225127f640 1 -- 172.21.15.89:0/4216993840 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2244010ce0 con 0x7f224c0c94b0 2023-10-18T10:46:16.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7f225127f640 1 -- 172.21.15.89:0/4216993840 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22440b0050 con 0x7f224c0c94b0 2023-10-18T10:46:16.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7f225127f640 1 -- 172.21.15.89:0/4216993840 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22440122f0 con 0x7f224c0c94b0 2023-10-18T10:46:16.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fd0c6d9d640 1 --2- 172.21.15.89:0/1492844345 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0c80d0270 0x7fd0c80caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fd0c659c640 1 -- 172.21.15.89:0/1492844345 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0c80d0270 msgr2=0x7fd0c80caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fd0c659c640 1 --2- 172.21.15.89:0/1492844345 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0c80d0270 0x7fd0c80caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fd0c659c640 1 -- 172.21.15.89:0/1492844345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0c815c2f0 msgr2=0x7fd0c80c6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fd0c659c640 1 --2- 172.21.15.89:0/1492844345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0c815c2f0 0x7fd0c80c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fd0c659c640 1 -- 172.21.15.89:0/1492844345 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b0009a70 con 0x7fd0c80d0700 2023-10-18T10:46:16.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fdf87fff640 1 --2- 172.21.15.89:0/3377746371 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf880b5640 0x7fdf88154b60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdf7400e980 tx=0x7fdf7400ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:16.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fbd063e1640 1 -- 172.21.15.89:0/2955890159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd0006dd10 msgr2=0x7fbd0006cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fbd063e1640 1 --2- 172.21.15.89:0/2955890159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd0006dd10 0x7fbd0006cd10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0009e50 tx=0x7fbcf001c960 comp rx=0 tx=0).stop 2023-10-18T10:46:16.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fbd063e1640 1 -- 172.21.15.89:0/2955890159 shutdown_connections 2023-10-18T10:46:16.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fbd063e1640 1 --2- 172.21.15.89:0/2955890159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd0006dd10 0x7fbd0006cd10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fbd063e1640 1 -- 172.21.15.89:0/2955890159 >> 172.21.15.89:0/2955890159 conn(0x7fbd000afe00 msgr2=0x7fbd000b0210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:16.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fdd7d38f640 1 -- 172.21.15.89:0/2761635048 shutdown_connections 2023-10-18T10:46:16.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fdd7d38f640 1 --2- 172.21.15.89:0/2761635048 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd780d0270 0x7fdd780d06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fdd7d38f640 1 --2- 172.21.15.89:0/2761635048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd7815c2f0 0x7fdd780b18e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.610+0000 7fdd7d38f640 1 -- 172.21.15.89:0/2761635048 >> 172.21.15.89:0/2761635048 conn(0x7fdd78069ad0 msgr2=0x7fdd78069eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:16.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fdf857fa640 1 -- 172.21.15.89:0/3377746371 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7400ca00 con 0x7fdf880b5640 2023-10-18T10:46:16.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fdf8eacb640 1 -- 172.21.15.89:0/3377746371 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf7c009b30 con 0x7fdf880b5640 2023-10-18T10:46:16.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fdd7d38f640 1 -- 172.21.15.89:0/2761635048 shutdown_connections 2023-10-18T10:46:16.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fdf857fa640 1 -- 172.21.15.89:0/3377746371 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf7400cb60 con 0x7fdf880b5640 2023-10-18T10:46:16.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fd0c659c640 1 --2- 172.21.15.89:0/1492844345 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0c80d0700 0x7fd0c80c61d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd0b800c500 tx=0x7fd0b800c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:16.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fbd063e1640 1 -- 172.21.15.89:0/2955890159 shutdown_connections 2023-10-18T10:46:16.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7f3dd37fe640 1 --2- 172.21.15.89:0/3538787601 >> [v2:172.21.15.89:6816/2220030573,v1:172.21.15.89:6817/2220030573] conn(0x7f3dbc06f340 0x7f3dbc071780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fdf8eacb640 1 -- 172.21.15.89:0/3377746371 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf88157140 con 0x7fdf880b5640 2023-10-18T10:46:16.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7f1adbfff640 1 -- 172.21.15.89:0/3136184549 <== osd.0 v2:172.21.15.89:6800/308765932 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f1ae0071df0 con 0x7f1ae006f2e0 2023-10-18T10:46:16.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fdd7d38f640 1 -- 172.21.15.89:0/2761635048 wait complete. 2023-10-18T10:46:16.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fd0bf7fe640 1 -- 172.21.15.89:0/1492844345 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b8005cf0 con 0x7fd0c80d0700 2023-10-18T10:46:16.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fd0bf7fe640 1 -- 172.21.15.89:0/1492844345 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b8005e50 con 0x7fd0c80d0700 2023-10-18T10:46:16.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fd0bf7fe640 1 -- 172.21.15.89:0/1492844345 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b80072f0 con 0x7fd0c80d0700 2023-10-18T10:46:16.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7f3dd2ffd640 1 --2- 172.21.15.89:0/3538787601 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3dbc0656e0 0x7f3dbc067ba0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4011380 tx=0x7f3dc40096b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:16.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fd456504640 1 --2- 172.21.15.89:0/192584431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4500cf950 0x7fd4500d0a40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd4500b1e50 tx=0x7fd44801c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f8b2cfeb6429541 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:16.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fd454d01640 1 -- 172.21.15.89:0/192584431 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4480a57d0 con 0x7fd4500cf950 2023-10-18T10:46:16.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fd454d01640 1 -- 172.21.15.89:0/192584431 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4480a5930 con 0x7fd4500cf950 2023-10-18T10:46:16.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fd454d01640 1 -- 172.21.15.89:0/192584431 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4480a5cc0 con 0x7fd4500cf950 2023-10-18T10:46:16.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7f3dd37fe640 1 --2- 172.21.15.89:0/3538787601 >> [v2:172.21.15.89:6816/2220030573,v1:172.21.15.89:6817/2220030573] conn(0x7f3dbc06f340 0x7f3dbc071780 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:16.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7f22544e5640 1 -- 172.21.15.89:0/4216993840 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f224c0c94b0 msgr2=0x7f224c0afd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7f22544e5640 1 --2- 172.21.15.89:0/4216993840 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f224c0c94b0 0x7f224c0afd90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2244006380 tx=0x7f22440250a0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fd456d05640 1 -- 172.21.15.89:0/192584431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4500cf950 msgr2=0x7fd4500d0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fd456d05640 1 --2- 172.21.15.89:0/192584431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4500cf950 0x7fd4500d0a40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd4500b1e50 tx=0x7fd44801c910 comp rx=0 tx=0).stop 2023-10-18T10:46:16.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fd0cd1f3640 1 -- 172.21.15.89:0/1492844345 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0c80c7000 con 0x7fd0c80d0700 2023-10-18T10:46:16.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fd0cd1f3640 1 -- 172.21.15.89:0/1492844345 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0c80c75c0 con 0x7fd0c80d0700 2023-10-18T10:46:16.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fdf857fa640 1 -- 172.21.15.89:0/3377746371 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7400ca00 con 0x7fdf880b5640 2023-10-18T10:46:16.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fdf8eacb640 1 -- 172.21.15.89:0/3377746371 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdf8816da00 con 0x7fdf880b5640 2023-10-18T10:46:16.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fbd063e1640 1 -- 172.21.15.89:0/2955890159 wait complete. 2023-10-18T10:46:16.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fd456d05640 1 -- 172.21.15.89:0/192584431 shutdown_connections 2023-10-18T10:46:16.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fd456d05640 1 --2- 172.21.15.89:0/192584431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4500cf950 0x7fd4500d0a40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fd456d05640 1 --2- 172.21.15.89:0/192584431 >> v2:172.21.15.89:3300/0 conn(0x7fd45015c310 0x7fd4500b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fbd063e1640 1 Processor -- start 2023-10-18T10:46:16.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fd456d05640 1 -- 172.21.15.89:0/192584431 >> 172.21.15.89:0/192584431 conn(0x7fd450070120 msgr2=0x7fd450070530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:16.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fd456d05640 1 -- 172.21.15.89:0/192584431 shutdown_connections 2023-10-18T10:46:16.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7f22544e5640 1 -- 172.21.15.89:0/4216993840 shutdown_connections 2023-10-18T10:46:16.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7f22544e5640 1 --2- 172.21.15.89:0/4216993840 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f224c0b02d0 0x7f224c158e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7f22544e5640 1 --2- 172.21.15.89:0/4216993840 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f224c0c94b0 0x7f224c0afd90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7f22544e5640 1 -- 172.21.15.89:0/4216993840 >> 172.21.15.89:0/4216993840 conn(0x7f224c069f00 msgr2=0x7f224c0cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:16.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fd456d05640 1 -- 172.21.15.89:0/192584431 wait complete. 2023-10-18T10:46:16.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fdd7d38f640 1 Processor -- start 2023-10-18T10:46:16.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.614+0000 7fdf857fa640 1 -- 172.21.15.89:0/3377746371 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fdf7400ccd0 con 0x7fdf880b5640 2023-10-18T10:46:16.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.618+0000 7f22544e5640 1 -- 172.21.15.89:0/4216993840 shutdown_connections 2023-10-18T10:46:16.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.618+0000 7fdf857fa640 1 --2- 172.21.15.89:0/3377746371 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdf6c0656e0 0x7fdf6c067ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.618+0000 7fdf857fa640 1 -- 172.21.15.89:0/3377746371 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdf74083090 con 0x7fdf880b5640 2023-10-18T10:46:16.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.618+0000 7fd0cd1f3640 1 -- 172.21.15.89:0/1492844345 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd0c816a340 con 0x7fd0c80d0700 2023-10-18T10:46:16.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.618+0000 7fdf857fa640 1 --2- 172.21.15.89:0/3377746371 >> [v2:172.21.15.154:6800/2025560293,v1:172.21.15.154:6801/2025560293] conn(0x7fdf6c06f340 0x7fdf6c071780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.618+0000 7fdf857fa640 1 -- 172.21.15.89:0/3377746371 --> [v2:172.21.15.154:6800/2025560293,v1:172.21.15.154:6801/2025560293] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdf6c071e50 con 0x7fdf6c06f340 2023-10-18T10:46:16.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.618+0000 7fdf857fa640 1 -- 172.21.15.89:0/3377746371 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf74085670 con 0x7fdf880b5640 2023-10-18T10:46:16.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.618+0000 7fdf8c867640 1 --2- 172.21.15.89:0/3377746371 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdf6c0656e0 0x7fdf6c067ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.618+0000 7fd0bf7fe640 1 -- 172.21.15.89:0/1492844345 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fd0b8007450 con 0x7fd0c80d0700 2023-10-18T10:46:16.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.618+0000 7f22544e5640 1 -- 172.21.15.89:0/4216993840 wait complete. 2023-10-18T10:46:16.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.618+0000 7fdf8c867640 1 --2- 172.21.15.89:0/3377746371 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdf6c0656e0 0x7fdf6c067ba0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0ae000 tx=0x7fdf7c0a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:16.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.618+0000 7fbd063e1640 1 -- start start 2023-10-18T10:46:16.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.618+0000 7f3dcb7fe640 1 -- 172.21.15.89:0/3538787601 <== osd.1 v2:172.21.15.89:6816/2220030573 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f3dbc071e50 con 0x7f3dbc06f340 2023-10-18T10:46:16.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.618+0000 7fd0bf7fe640 1 --2- 172.21.15.89:0/1492844345 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd0a80656d0 0x7fd0a8067b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.618+0000 7fd0bf7fe640 1 -- 172.21.15.89:0/1492844345 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd0b80825b0 con 0x7fd0c80d0700 2023-10-18T10:46:16.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.618+0000 7f22544e5640 1 Processor -- start 2023-10-18T10:46:16.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.618+0000 7fd0bf7fe640 1 --2- 172.21.15.89:0/1492844345 >> [v2:172.21.15.89:6822/2771730391,v1:172.21.15.89:6824/2771730391] conn(0x7fd0a806f3b0 0x7fd0a80717f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.618+0000 7fd0bf7fe640 1 -- 172.21.15.89:0/1492844345 --> [v2:172.21.15.89:6822/2771730391,v1:172.21.15.89:6824/2771730391] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd0a8071ec0 con 0x7fd0a806f3b0 2023-10-18T10:46:16.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.618+0000 7fd0bf7fe640 1 -- 172.21.15.89:0/1492844345 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0b8084b90 con 0x7fd0c80d0700 2023-10-18T10:46:16.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.618+0000 7fdd7d38f640 1 -- start start 2023-10-18T10:46:16.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.618+0000 7fdf8d068640 1 --2- 172.21.15.89:0/3377746371 >> [v2:172.21.15.154:6800/2025560293,v1:172.21.15.154:6801/2025560293] conn(0x7fdf6c06f340 0x7fdf6c071780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.618+0000 7fd0c6d9d640 1 --2- 172.21.15.89:0/1492844345 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd0a80656d0 0x7fd0a8067b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.618+0000 7f22544e5640 1 -- start start 2023-10-18T10:46:16.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fbd063e1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd0006dd10 0x7fbd000c4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fbd063e1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd000b0920 0x7fbd000b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fbd063e1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd0015cfa0 0x7fbd001678e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.618+0000 7fdd7d38f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd780d0270 0x7fdd78154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.618+0000 7fdd7d38f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd7815c2f0 0x7fdd780c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fdd7d38f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd7815c6d0 0x7fdd781534c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fdd7d38f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd78153a00 con 0x7fdd7815c2f0 2023-10-18T10:46:16.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fdd7d38f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd78157ff0 con 0x7fdd780d0270 2023-10-18T10:46:16.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fdd7d38f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd781582d0 con 0x7fdd7815c6d0 2023-10-18T10:46:16.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fd456d05640 1 Processor -- start 2023-10-18T10:46:16.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fbcff7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd000b0920 0x7fbd000b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fbcfffff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd0006dd10 0x7fbd000c4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fbcfffff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd0006dd10 0x7fbd000c4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:37244/0 (socket says 172.21.15.89:37244) 2023-10-18T10:46:16.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fdf8d068640 1 --2- 172.21.15.89:0/3377746371 >> [v2:172.21.15.154:6800/2025560293,v1:172.21.15.154:6801/2025560293] conn(0x7fdf6c06f340 0x7fdf6c071780 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:16.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fbcfffff640 1 -- 172.21.15.89:0/876752675 learned_addr learned my addr 172.21.15.89:0/876752675 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:16.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fbd0497e640 1 --2- 172.21.15.89:0/876752675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd0015cfa0 0x7fbd001678e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fbd063e1640 1 -- 172.21.15.89:0/876752675 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd000b4150 con 0x7fbd0015cfa0 2023-10-18T10:46:16.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fbd063e1640 1 -- 172.21.15.89:0/876752675 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd00168080 con 0x7fbd000b0920 2023-10-18T10:46:16.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fbd063e1640 1 -- 172.21.15.89:0/876752675 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd000b3780 con 0x7fbd0006dd10 2023-10-18T10:46:16.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7f22544e5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f224c0b02d0 0x7f224c14a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7f22544e5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f224c0c94b0 0x7f224c151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7f22544e5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f224c159360 0x7f224c145880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7f22544e5640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f224c0c6f00 con 0x7f224c0b02d0 2023-10-18T10:46:16.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7f22544e5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f224c152140 con 0x7f224c159360 2023-10-18T10:46:16.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7f22544e5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f224c14b160 con 0x7f224c0c94b0 2023-10-18T10:46:16.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fdd76ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd780d0270 0x7fdd78154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fdd76ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd780d0270 0x7fdd78154f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:34186/0 (socket says 172.21.15.89:34186) 2023-10-18T10:46:16.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fdd76ffd640 1 -- 172.21.15.89:0/4157337148 learned_addr learned my addr 172.21.15.89:0/4157337148 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:16.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fdd777fe640 1 --2- 172.21.15.89:0/4157337148 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd7815c6d0 0x7fdd781534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fdd6e7fc640 1 --2- 172.21.15.89:0/4157337148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd7815c2f0 0x7fdd780c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fdd777fe640 1 -- 172.21.15.89:0/4157337148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd780d0270 msgr2=0x7fdd78154f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fdd777fe640 1 --2- 172.21.15.89:0/4157337148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd780d0270 0x7fdd78154f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fdd777fe640 1 -- 172.21.15.89:0/4157337148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd7815c2f0 msgr2=0x7fdd780c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fdd777fe640 1 --2- 172.21.15.89:0/4157337148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd7815c2f0 0x7fdd780c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fdd777fe640 1 -- 172.21.15.89:0/4157337148 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd68009420 con 0x7fdd7815c6d0 2023-10-18T10:46:16.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fdd777fe640 1 --2- 172.21.15.89:0/4157337148 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd7815c6d0 0x7fdd781534c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000c390 tx=0x7fdd7000c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:16.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.618+0000 7fd0c759e640 1 --2- 172.21.15.89:0/1492844345 >> [v2:172.21.15.89:6822/2771730391,v1:172.21.15.89:6824/2771730391] conn(0x7fd0a806f3b0 0x7fd0a80717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fd0c6d9d640 1 --2- 172.21.15.89:0/1492844345 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd0a80656d0 0x7fd0a8067b90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0000f70 tx=0x7fd0c0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:16.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fd0c759e640 1 --2- 172.21.15.89:0/1492844345 >> [v2:172.21.15.89:6822/2771730391,v1:172.21.15.89:6824/2771730391] conn(0x7fd0a806f3b0 0x7fd0a80717f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:16.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7f2251a80640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f224c0c94b0 0x7f224c151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7f2251a80640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f224c0c94b0 0x7f224c151c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:37256/0 (socket says 172.21.15.89:37256) 2023-10-18T10:46:16.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7f2251a80640 1 -- 172.21.15.89:0/3937605769 learned_addr learned my addr 172.21.15.89:0/3937605769 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:16.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7f2252a82640 1 --2- 172.21.15.89:0/3937605769 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f224c159360 0x7f224c145880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fdd6ffff640 1 -- 172.21.15.89:0/4157337148 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70005cf0 con 0x7fdd7815c6d0 2023-10-18T10:46:16.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fdf857fa640 1 -- 172.21.15.89:0/3377746371 <== osd.4 v2:172.21.15.154:6800/2025560293 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fdf6c071e50 con 0x7fdf6c06f340 2023-10-18T10:46:16.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7f2252281640 1 --2- 172.21.15.89:0/3937605769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f224c0b02d0 0x7f224c14a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fbcff7fe640 1 -- 172.21.15.89:0/876752675 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd0006dd10 msgr2=0x7fbd000c4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fbcff7fe640 1 --2- 172.21.15.89:0/876752675 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd0006dd10 0x7fbd000c4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fbcff7fe640 1 -- 172.21.15.89:0/876752675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd0015cfa0 msgr2=0x7fbd001678e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fbcff7fe640 1 --2- 172.21.15.89:0/876752675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd0015cfa0 0x7fbd001678e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fbcff7fe640 1 -- 172.21.15.89:0/876752675 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf0009b30 con 0x7fbd000b0920 2023-10-18T10:46:16.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7f2252a82640 1 -- 172.21.15.89:0/3937605769 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f224c0c94b0 msgr2=0x7f224c151c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7f2252a82640 1 --2- 172.21.15.89:0/3937605769 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f224c0c94b0 0x7f224c151c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7f2252a82640 1 -- 172.21.15.89:0/3937605769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f224c0b02d0 msgr2=0x7f224c14a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7f2252a82640 1 --2- 172.21.15.89:0/3937605769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f224c0b02d0 0x7f224c14a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7f2252a82640 1 -- 172.21.15.89:0/3937605769 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2244006060 con 0x7f224c159360 2023-10-18T10:46:16.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7f2252281640 1 --2- 172.21.15.89:0/3937605769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f224c0b02d0 0x7f224c14a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:16.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7f2252a82640 1 --2- 172.21.15.89:0/3937605769 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f224c159360 0x7f224c145880 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f22480093f0 tx=0x7f22480098b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:16.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fbcff7fe640 1 --2- 172.21.15.89:0/876752675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd000b0920 0x7fbd000b3c10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbcf400b3e0 tx=0x7fbcf400b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:16.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fbcfcff9640 1 -- 172.21.15.89:0/876752675 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4011830 con 0x7fbd000b0920 2023-10-18T10:46:16.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fdd6ffff640 1 -- 172.21.15.89:0/4157337148 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd70005e50 con 0x7fdd7815c6d0 2023-10-18T10:46:16.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fdd6ffff640 1 -- 172.21.15.89:0/4157337148 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd700072f0 con 0x7fdd7815c6d0 2023-10-18T10:46:16.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fdd7d38f640 1 -- 172.21.15.89:0/4157337148 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd781585b0 con 0x7fdd7815c6d0 2023-10-18T10:46:16.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7f2242ffd640 1 -- 172.21.15.89:0/3937605769 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22480107b0 con 0x7f224c159360 2023-10-18T10:46:16.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fbcfcff9640 1 -- 172.21.15.89:0/876752675 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf40119b0 con 0x7fbd000b0920 2023-10-18T10:46:16.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.622+0000 7fdd7d38f640 1 -- 172.21.15.89:0/4157337148 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd78158b70 con 0x7fdd7815c6d0 2023-10-18T10:46:16.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.626+0000 7f2242ffd640 1 -- 172.21.15.89:0/3937605769 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2248010930 con 0x7f224c159360 2023-10-18T10:46:16.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.626+0000 7f2242ffd640 1 -- 172.21.15.89:0/3937605769 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2248010ce0 con 0x7f224c159360 2023-10-18T10:46:16.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.626+0000 7fbd063e1640 1 -- 172.21.15.89:0/876752675 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd00167e20 con 0x7fbd000b0920 2023-10-18T10:46:16.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.626+0000 7fdd7d38f640 1 -- 172.21.15.89:0/4157337148 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdd780b1e20 con 0x7fdd7815c6d0 2023-10-18T10:46:16.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.626+0000 7fd0bf7fe640 1 -- 172.21.15.89:0/1492844345 <== osd.3 v2:172.21.15.89:6822/2771730391 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fd0a8071ec0 con 0x7fd0a806f3b0 2023-10-18T10:46:16.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.626+0000 7fbd063e1640 1 -- 172.21.15.89:0/876752675 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd000b3130 con 0x7fbd000b0920 2023-10-18T10:46:16.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.626+0000 7f22544e5640 1 -- 172.21.15.89:0/3937605769 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f224c14b440 con 0x7f224c159360 2023-10-18T10:46:16.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.626+0000 7fbcfcff9640 1 -- 172.21.15.89:0/876752675 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4011d60 con 0x7fbd000b0920 2023-10-18T10:46:16.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.626+0000 7f22544e5640 1 -- 172.21.15.89:0/3937605769 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f224c1460f0 con 0x7f224c159360 2023-10-18T10:46:16.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.626+0000 7fd456d05640 1 -- start start 2023-10-18T10:46:16.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.626+0000 7fd456d05640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4500cd4e0 0x7fd4500cc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.626+0000 7fd456d05640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4500cf950 0x7fd450139e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.626+0000 7fd456d05640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd45015c310 0x7fd45015f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.626+0000 7fd456d05640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4501568a0 con 0x7fd4500cf950 2023-10-18T10:46:16.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.626+0000 7fd456d05640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd450155f40 con 0x7fd4500cd4e0 2023-10-18T10:46:16.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.626+0000 7fd456d05640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4500b33d0 con 0x7fd45015c310 2023-10-18T10:46:16.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.626+0000 7f22544e5640 1 -- 172.21.15.89:0/3937605769 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f224c15d0b0 con 0x7f224c159360 2023-10-18T10:46:16.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.626+0000 7fd456504640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd45015c310 0x7fd45015f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.626+0000 7fd456504640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd45015c310 0x7fd45015f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:37262/0 (socket says 172.21.15.89:37262) 2023-10-18T10:46:16.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.626+0000 7fd456504640 1 -- 172.21.15.89:0/1029608725 learned_addr learned my addr 172.21.15.89:0/1029608725 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:16.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.626+0000 7fd455d03640 1 --2- 172.21.15.89:0/1029608725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4500cd4e0 0x7fd4500cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.626+0000 7fd456504640 1 -- 172.21.15.89:0/1029608725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4500cd4e0 msgr2=0x7fd4500cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.626+0000 7fd456504640 1 --2- 172.21.15.89:0/1029608725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4500cd4e0 0x7fd4500cc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.626+0000 7fd456504640 1 -- 172.21.15.89:0/1029608725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4500cf950 msgr2=0x7fd450139e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.626+0000 7fd456504640 1 --2- 172.21.15.89:0/1029608725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4500cf950 0x7fd450139e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.626+0000 7fd456504640 1 -- 172.21.15.89:0/1029608725 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd448009b30 con 0x7fd45015c310 2023-10-18T10:46:16.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.626+0000 7fd455d03640 1 --2- 172.21.15.89:0/1029608725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4500cd4e0 0x7fd4500cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:46:16.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.626+0000 7fdd6ffff640 1 -- 172.21.15.89:0/4157337148 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fdd70007450 con 0x7fdd7815c6d0 2023-10-18T10:46:16.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.630+0000 7fd456504640 1 --2- 172.21.15.89:0/1029608725 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd45015c310 0x7fd45015f150 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd4480af040 tx=0x7fd4480196b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:16.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.630+0000 7f2242ffd640 1 -- 172.21.15.89:0/3937605769 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f2248005920 con 0x7f224c159360 2023-10-18T10:46:16.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.630+0000 7fdd6ffff640 1 --2- 172.21.15.89:0/4157337148 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdd58065750 0x7fdd58067c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.626+0000 7fbd063e1640 1 -- 172.21.15.89:0/876752675 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbd0004dbe0 con 0x7fbd000b0920 2023-10-18T10:46:16.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.630+0000 7fdd6ffff640 1 -- 172.21.15.89:0/4157337148 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdd700824c0 con 0x7fdd7815c6d0 2023-10-18T10:46:16.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.630+0000 7fdd6ffff640 1 --2- 172.21.15.89:0/4157337148 >> [v2:172.21.15.154:6822/773078364,v1:172.21.15.154:6824/773078364] conn(0x7fdd5806f3b0 0x7fdd580717f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.630+0000 7fdd6ffff640 1 -- 172.21.15.89:0/4157337148 --> [v2:172.21.15.154:6822/773078364,v1:172.21.15.154:6824/773078364] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdd58071ec0 con 0x7fdd5806f3b0 2023-10-18T10:46:16.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.630+0000 7f2242ffd640 1 --2- 172.21.15.89:0/3937605769 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2230065680 0x7f2230067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.630+0000 7f2242ffd640 1 -- 172.21.15.89:0/3937605769 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2248082f90 con 0x7f224c159360 2023-10-18T10:46:16.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.630+0000 7f2242ffd640 1 --2- 172.21.15.89:0/3937605769 >> [v2:172.21.15.89:6808/4229386532,v1:172.21.15.89:6809/4229386532] conn(0x7f223006f2e0 0x7f2230071720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.630+0000 7f2242ffd640 1 -- 172.21.15.89:0/3937605769 --> [v2:172.21.15.89:6808/4229386532,v1:172.21.15.89:6809/4229386532] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2230071df0 con 0x7f223006f2e0 2023-10-18T10:46:16.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.630+0000 7f2242ffd640 1 -- 172.21.15.89:0/3937605769 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22480833e0 con 0x7f224c159360 2023-10-18T10:46:16.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.630+0000 7fdd76ffd640 1 --2- 172.21.15.89:0/4157337148 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdd58065750 0x7fdd58067c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.630+0000 7fdd6e7fc640 1 --2- 172.21.15.89:0/4157337148 >> [v2:172.21.15.154:6822/773078364,v1:172.21.15.154:6824/773078364] conn(0x7fdd5806f3b0 0x7fdd580717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.630+0000 7fbcfcff9640 1 -- 172.21.15.89:0/876752675 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fbcf401a400 con 0x7fbd000b0920 2023-10-18T10:46:16.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.630+0000 7f2252281640 1 --2- 172.21.15.89:0/3937605769 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2230065680 0x7f2230067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.630+0000 7f2251a80640 1 --2- 172.21.15.89:0/3937605769 >> [v2:172.21.15.89:6808/4229386532,v1:172.21.15.89:6809/4229386532] conn(0x7f223006f2e0 0x7f2230071720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.630+0000 7fdd6ffff640 1 -- 172.21.15.89:0/4157337148 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd7004fdb0 con 0x7fdd7815c6d0 2023-10-18T10:46:16.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.630+0000 7f2251a80640 1 --2- 172.21.15.89:0/3937605769 >> [v2:172.21.15.89:6808/4229386532,v1:172.21.15.89:6809/4229386532] conn(0x7f223006f2e0 0x7f2230071720 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:16.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.630+0000 7fdd6e7fc640 1 --2- 172.21.15.89:0/4157337148 >> [v2:172.21.15.154:6822/773078364,v1:172.21.15.154:6824/773078364] conn(0x7fdd5806f3b0 0x7fdd580717f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:16.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.630+0000 7fbcfcff9640 1 --2- 172.21.15.89:0/876752675 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fbcec065750 0x7fbcec067c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.630+0000 7fdd76ffd640 1 --2- 172.21.15.89:0/4157337148 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdd58065750 0x7fdd58067c10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdd6801a0a0 tx=0x7fdd680a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:16.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.634+0000 7f2252281640 1 --2- 172.21.15.89:0/3937605769 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2230065680 0x7f2230067b40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f22440b6000 tx=0x7f22440af000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:16.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.634+0000 7fd4467fc640 1 -- 172.21.15.89:0/1029608725 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4480b8670 con 0x7fd45015c310 2023-10-18T10:46:16.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.634+0000 7fd4467fc640 1 -- 172.21.15.89:0/1029608725 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4480b87f0 con 0x7fd45015c310 2023-10-18T10:46:16.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.634+0000 7fd4467fc640 1 -- 172.21.15.89:0/1029608725 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4480b8ba0 con 0x7fd45015c310 2023-10-18T10:46:16.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.634+0000 7fbcfcff9640 1 -- 172.21.15.89:0/876752675 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fbcf4083030 con 0x7fbd000b0920 2023-10-18T10:46:16.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.634+0000 7fd456d05640 1 -- 172.21.15.89:0/1029608725 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4501561e0 con 0x7fd45015c310 2023-10-18T10:46:16.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.634+0000 7fbcfffff640 1 --2- 172.21.15.89:0/876752675 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fbcec065750 0x7fbcec067c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.634+0000 7fdd6ffff640 1 -- 172.21.15.89:0/4157337148 <== osd.6 v2:172.21.15.154:6822/773078364 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fdd58071ec0 con 0x7fdd5806f3b0 2023-10-18T10:46:16.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.634+0000 7fbcfcff9640 1 --2- 172.21.15.89:0/876752675 >> [v2:172.21.15.154:6808/357281174,v1:172.21.15.154:6809/357281174] conn(0x7fbcec06f430 0x7fbcec071870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.634+0000 7f2242ffd640 1 -- 172.21.15.89:0/3937605769 <== osd.2 v2:172.21.15.89:6808/4229386532 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f2230071df0 con 0x7f223006f2e0 2023-10-18T10:46:16.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.634+0000 7fbd0497e640 1 --2- 172.21.15.89:0/876752675 >> [v2:172.21.15.154:6808/357281174,v1:172.21.15.154:6809/357281174] conn(0x7fbcec06f430 0x7fbcec071870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.634+0000 7fbcfcff9640 1 -- 172.21.15.89:0/876752675 --> [v2:172.21.15.154:6808/357281174,v1:172.21.15.154:6809/357281174] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbcec071f40 con 0x7fbcec06f430 2023-10-18T10:46:16.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.634+0000 7fbcfcff9640 1 -- 172.21.15.89:0/876752675 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcf40855f0 con 0x7fbd000b0920 2023-10-18T10:46:16.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.634+0000 7fbcfffff640 1 --2- 172.21.15.89:0/876752675 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fbcec065750 0x7fbcec067c10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbd000c9740 tx=0x7fbcf8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:16.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.634+0000 7fd456d05640 1 -- 172.21.15.89:0/1029608725 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd45013a3d0 con 0x7fd45015c310 2023-10-18T10:46:16.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.634+0000 7fbd0497e640 1 --2- 172.21.15.89:0/876752675 >> [v2:172.21.15.154:6808/357281174,v1:172.21.15.154:6809/357281174] conn(0x7fbcec06f430 0x7fbcec071870 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:16.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.634+0000 7fbcfcff9640 1 -- 172.21.15.89:0/876752675 <== osd.5 v2:172.21.15.154:6808/357281174 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fbcec071f40 con 0x7fbcec06f430 2023-10-18T10:46:16.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.634+0000 7f1afa25f640 1 -- 172.21.15.89:0/3136184549 --> [v2:172.21.15.89:6800/308765932,v1:172.21.15.89:6801/308765932] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1af4066ca0 con 0x7f1ae006f2e0 2023-10-18T10:46:16.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.638+0000 7f1adbfff640 1 -- 172.21.15.89:0/3136184549 <== osd.0 v2:172.21.15.89:6800/308765932 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1af4066ca0 con 0x7f1ae006f2e0 2023-10-18T10:46:16.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.638+0000 7fd456d05640 1 -- 172.21.15.89:0/1029608725 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd4500cd8f0 con 0x7fd45015c310 2023-10-18T10:46:16.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.638+0000 7fd4467fc640 1 -- 172.21.15.89:0/1029608725 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fd4480b8d00 con 0x7fd45015c310 2023-10-18T10:46:16.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.638+0000 7f1afa25f640 1 -- 172.21.15.89:0/3136184549 >> [v2:172.21.15.89:6800/308765932,v1:172.21.15.89:6801/308765932] conn(0x7f1ae006f2e0 msgr2=0x7f1ae0071720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.638+0000 7f1afa25f640 1 --2- 172.21.15.89:0/3136184549 >> [v2:172.21.15.89:6800/308765932,v1:172.21.15.89:6801/308765932] conn(0x7f1ae006f2e0 0x7f1ae0071720 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.638+0000 7f1afa25f640 1 -- 172.21.15.89:0/3136184549 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1ae0065680 msgr2=0x7f1ae0067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.638+0000 7f1afa25f640 1 --2- 172.21.15.89:0/3136184549 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1ae0065680 0x7f1ae0067b40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1aec007fd0 tx=0x7f1aec009040 comp rx=0 tx=0).stop 2023-10-18T10:46:16.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.638+0000 7f1afa25f640 1 -- 172.21.15.89:0/3136184549 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1af40d0700 msgr2=0x7f1af40cb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.638+0000 7f1afa25f640 1 --2- 172.21.15.89:0/3136184549 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1af40d0700 0x7f1af40cb1d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1ae800c4d0 tx=0x7f1ae800c990 comp rx=0 tx=0).stop 2023-10-18T10:46:16.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.638+0000 7fd4467fc640 1 --2- 172.21.15.89:0/1029608725 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd434065750 0x7fd434067c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.638+0000 7fd4467fc640 1 -- 172.21.15.89:0/1029608725 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd44811e930 con 0x7fd45015c310 2023-10-18T10:46:16.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.638+0000 7fd4467fc640 1 --2- 172.21.15.89:0/1029608725 >> [v2:172.21.15.154:6815/4139186586,v1:172.21.15.154:6816/4139186586] conn(0x7fd43406f3b0 0x7fd4340717f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:16.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.642+0000 7f1af3fff640 1 -- 172.21.15.89:0/3136184549 reap_dead start 2023-10-18T10:46:16.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.642+0000 7fd4467fc640 1 -- 172.21.15.89:0/1029608725 --> [v2:172.21.15.154:6815/4139186586,v1:172.21.15.154:6816/4139186586] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd434071ec0 con 0x7fd43406f3b0 2023-10-18T10:46:16.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.642+0000 7f1afa25f640 1 -- 172.21.15.89:0/3136184549 shutdown_connections 2023-10-18T10:46:16.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.642+0000 7f1afa25f640 1 -- 172.21.15.89:0/3136184549 >> 172.21.15.89:0/3136184549 conn(0x7f1af4069ad0 msgr2=0x7f1af40d2ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:16.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.642+0000 7fd455d03640 1 --2- 172.21.15.89:0/1029608725 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd434065750 0x7fd434067c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.642+0000 7f1afa25f640 1 -- 172.21.15.89:0/3136184549 shutdown_connections 2023-10-18T10:46:16.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.642+0000 7f1afa25f640 1 -- 172.21.15.89:0/3136184549 wait complete. 2023-10-18T10:46:16.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.642+0000 7fd455502640 1 --2- 172.21.15.89:0/1029608725 >> [v2:172.21.15.154:6815/4139186586,v1:172.21.15.154:6816/4139186586] conn(0x7fd43406f3b0 0x7fd4340717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:16.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.642+0000 7fd455d03640 1 --2- 172.21.15.89:0/1029608725 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd434065750 0x7fd434067c10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd44c00af30 tx=0x7fd44c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:16.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.642+0000 7fd4467fc640 1 -- 172.21.15.89:0/1029608725 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4480ec290 con 0x7fd45015c310 2023-10-18T10:46:16.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.642+0000 7fd455502640 1 --2- 172.21.15.89:0/1029608725 >> [v2:172.21.15.154:6815/4139186586,v1:172.21.15.154:6816/4139186586] conn(0x7fd43406f3b0 0x7fd4340717f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:16.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.642+0000 7fd4467fc640 1 -- 172.21.15.89:0/1029608725 <== osd.7 v2:172.21.15.154:6815/4139186586 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fd434071ec0 con 0x7fd43406f3b0 2023-10-18T10:46:16.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.646+0000 7f3dc97fa640 1 -- 172.21.15.89:0/3538787601 --> [v2:172.21.15.89:6816/2220030573,v1:172.21.15.89:6817/2220030573] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3db4002cc0 con 0x7f3dbc06f340 2023-10-18T10:46:16.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.646+0000 7fdf8eacb640 1 -- 172.21.15.89:0/3377746371 --> [v2:172.21.15.154:6800/2025560293,v1:172.21.15.154:6801/2025560293] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdf88156cf0 con 0x7fdf6c06f340 2023-10-18T10:46:16.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.646+0000 7f3dcb7fe640 1 -- 172.21.15.89:0/3538787601 <== osd.1 v2:172.21.15.89:6816/2220030573 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3db4002cc0 con 0x7f3dbc06f340 2023-10-18T10:46:16.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.650+0000 7f3dc97fa640 1 -- 172.21.15.89:0/3538787601 >> [v2:172.21.15.89:6816/2220030573,v1:172.21.15.89:6817/2220030573] conn(0x7f3dbc06f340 msgr2=0x7f3dbc071780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.650+0000 7f3dc97fa640 1 --2- 172.21.15.89:0/3538787601 >> [v2:172.21.15.89:6816/2220030573,v1:172.21.15.89:6817/2220030573] conn(0x7f3dbc06f340 0x7f3dbc071780 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.650+0000 7f3dc97fa640 1 -- 172.21.15.89:0/3538787601 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3dbc0656e0 msgr2=0x7f3dbc067ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.650+0000 7f3dc97fa640 1 --2- 172.21.15.89:0/3538787601 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3dbc0656e0 0x7f3dbc067ba0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4011380 tx=0x7f3dc40096b0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.650+0000 7f3dc97fa640 1 -- 172.21.15.89:0/3538787601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3dd40b3a20 msgr2=0x7f3dd415f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.650+0000 7f3dc97fa640 1 --2- 172.21.15.89:0/3538787601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3dd40b3a20 0x7f3dd415f7a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0006000 tx=0x7f3db8005800 comp rx=0 tx=0).stop 2023-10-18T10:46:16.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.650+0000 7f3dd37fe640 1 -- 172.21.15.89:0/3538787601 reap_dead start 2023-10-18T10:46:16.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.650+0000 7fdf857fa640 1 -- 172.21.15.89:0/3377746371 <== osd.4 v2:172.21.15.154:6800/2025560293 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdf88156cf0 con 0x7fdf6c06f340 2023-10-18T10:46:16.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.650+0000 7fdf8eacb640 1 -- 172.21.15.89:0/3377746371 >> [v2:172.21.15.154:6800/2025560293,v1:172.21.15.154:6801/2025560293] conn(0x7fdf6c06f340 msgr2=0x7fdf6c071780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.650+0000 7fdf8eacb640 1 --2- 172.21.15.89:0/3377746371 >> [v2:172.21.15.154:6800/2025560293,v1:172.21.15.154:6801/2025560293] conn(0x7fdf6c06f340 0x7fdf6c071780 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.650+0000 7fdf8eacb640 1 -- 172.21.15.89:0/3377746371 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdf6c0656e0 msgr2=0x7fdf6c067ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.650+0000 7fdf8eacb640 1 --2- 172.21.15.89:0/3377746371 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdf6c0656e0 0x7fdf6c067ba0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0ae000 tx=0x7fdf7c0a7070 comp rx=0 tx=0).stop 2023-10-18T10:46:16.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.650+0000 7fdf8eacb640 1 -- 172.21.15.89:0/3377746371 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf880b5640 msgr2=0x7fdf88154b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.650+0000 7fdf8eacb640 1 --2- 172.21.15.89:0/3377746371 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf880b5640 0x7fdf88154b60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdf7400e980 tx=0x7fdf7400ee40 comp rx=0 tx=0).stop 2023-10-18T10:46:16.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.650+0000 7fdf8d068640 1 -- 172.21.15.89:0/3377746371 reap_dead start 2023-10-18T10:46:16.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.650+0000 7f3dc97fa640 1 -- 172.21.15.89:0/3538787601 shutdown_connections 2023-10-18T10:46:16.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.650+0000 7f3dc97fa640 1 -- 172.21.15.89:0/3538787601 >> 172.21.15.89:0/3538787601 conn(0x7f3dd40afe00 msgr2=0x7f3dd40c8fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:16.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.650+0000 7fdf8eacb640 1 -- 172.21.15.89:0/3377746371 shutdown_connections 2023-10-18T10:46:16.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.650+0000 7fdf8eacb640 1 -- 172.21.15.89:0/3377746371 >> 172.21.15.89:0/3377746371 conn(0x7fdf880cf120 msgr2=0x7fdf88161f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:16.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.650+0000 7fdf8eacb640 1 -- 172.21.15.89:0/3377746371 shutdown_connections 2023-10-18T10:46:16.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.650+0000 7fdf8eacb640 1 -- 172.21.15.89:0/3377746371 wait complete. 2023-10-18T10:46:16.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.654+0000 7f3dc97fa640 1 -- 172.21.15.89:0/3538787601 shutdown_connections 2023-10-18T10:46:16.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.654+0000 7f3dc97fa640 1 -- 172.21.15.89:0/3538787601 wait complete. 2023-10-18T10:46:16.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.658+0000 7fd0cd1f3640 1 -- 172.21.15.89:0/1492844345 --> [v2:172.21.15.89:6822/2771730391,v1:172.21.15.89:6824/2771730391] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd0c80c7aa0 con 0x7fd0a806f3b0 2023-10-18T10:46:16.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.662+0000 7fd0bf7fe640 1 -- 172.21.15.89:0/1492844345 <== osd.3 v2:172.21.15.89:6822/2771730391 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd0c80c7aa0 con 0x7fd0a806f3b0 2023-10-18T10:46:16.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.662+0000 7fbd063e1640 1 -- 172.21.15.89:0/876752675 --> [v2:172.21.15.154:6808/357281174,v1:172.21.15.154:6809/357281174] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbd000b38c0 con 0x7fbcec06f430 2023-10-18T10:46:16.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.662+0000 7fbcfcff9640 1 -- 172.21.15.89:0/876752675 <== osd.5 v2:172.21.15.154:6808/357281174 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbd000b38c0 con 0x7fbcec06f430 2023-10-18T10:46:16.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.662+0000 7fbd063e1640 1 -- 172.21.15.89:0/876752675 >> [v2:172.21.15.154:6808/357281174,v1:172.21.15.154:6809/357281174] conn(0x7fbcec06f430 msgr2=0x7fbcec071870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.662+0000 7fbd063e1640 1 --2- 172.21.15.89:0/876752675 >> [v2:172.21.15.154:6808/357281174,v1:172.21.15.154:6809/357281174] conn(0x7fbcec06f430 0x7fbcec071870 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.662+0000 7fbd063e1640 1 -- 172.21.15.89:0/876752675 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fbcec065750 msgr2=0x7fbcec067c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.662+0000 7fbd063e1640 1 --2- 172.21.15.89:0/876752675 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fbcec065750 0x7fbcec067c10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbd000c9740 tx=0x7fbcf8009040 comp rx=0 tx=0).stop 2023-10-18T10:46:16.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.662+0000 7fbd063e1640 1 -- 172.21.15.89:0/876752675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd000b0920 msgr2=0x7fbd000b3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.662+0000 7fdd7d38f640 1 -- 172.21.15.89:0/4157337148 --> [v2:172.21.15.154:6822/773078364,v1:172.21.15.154:6824/773078364] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdd78158d90 con 0x7fdd5806f3b0 2023-10-18T10:46:16.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.662+0000 7fdd6ffff640 1 -- 172.21.15.89:0/4157337148 <== osd.6 v2:172.21.15.154:6822/773078364 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdd78158d90 con 0x7fdd5806f3b0 2023-10-18T10:46:16.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.662+0000 7fdd7d38f640 1 -- 172.21.15.89:0/4157337148 >> [v2:172.21.15.154:6822/773078364,v1:172.21.15.154:6824/773078364] conn(0x7fdd5806f3b0 msgr2=0x7fdd580717f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.662+0000 7fdd7d38f640 1 --2- 172.21.15.89:0/4157337148 >> [v2:172.21.15.154:6822/773078364,v1:172.21.15.154:6824/773078364] conn(0x7fdd5806f3b0 0x7fdd580717f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.662+0000 7fdd7d38f640 1 -- 172.21.15.89:0/4157337148 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdd58065750 msgr2=0x7fdd58067c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.662+0000 7fdd7d38f640 1 --2- 172.21.15.89:0/4157337148 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdd58065750 0x7fdd58067c10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdd6801a0a0 tx=0x7fdd680a7000 comp rx=0 tx=0).stop 2023-10-18T10:46:16.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.662+0000 7fdd7d38f640 1 -- 172.21.15.89:0/4157337148 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd7815c6d0 msgr2=0x7fdd781534c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.662+0000 7fdd7d38f640 1 --2- 172.21.15.89:0/4157337148 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdd7815c6d0 0x7fdd781534c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000c390 tx=0x7fdd7000c850 comp rx=0 tx=0).stop 2023-10-18T10:46:16.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.662+0000 7fdd777fe640 1 -- 172.21.15.89:0/4157337148 reap_dead start 2023-10-18T10:46:16.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.662+0000 7f22544e5640 1 -- 172.21.15.89:0/3937605769 --> [v2:172.21.15.89:6808/4229386532,v1:172.21.15.89:6809/4229386532] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f224c0709f0 con 0x7f223006f2e0 2023-10-18T10:46:16.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.662+0000 7fbd063e1640 1 --2- 172.21.15.89:0/876752675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd000b0920 0x7fbd000b3c10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbcf400b3e0 tx=0x7fbcf400b8a0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.662+0000 7fbd0497e640 1 -- 172.21.15.89:0/876752675 reap_dead start 2023-10-18T10:46:16.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.662+0000 7fd0cd1f3640 1 -- 172.21.15.89:0/1492844345 >> [v2:172.21.15.89:6822/2771730391,v1:172.21.15.89:6824/2771730391] conn(0x7fd0a806f3b0 msgr2=0x7fd0a80717f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.662+0000 7fd0cd1f3640 1 --2- 172.21.15.89:0/1492844345 >> [v2:172.21.15.89:6822/2771730391,v1:172.21.15.89:6824/2771730391] conn(0x7fd0a806f3b0 0x7fd0a80717f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.662+0000 7fd0cd1f3640 1 -- 172.21.15.89:0/1492844345 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd0a80656d0 msgr2=0x7fd0a8067b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.662+0000 7fd0cd1f3640 1 --2- 172.21.15.89:0/1492844345 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd0a80656d0 0x7fd0a8067b90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0000f70 tx=0x7fd0c0009040 comp rx=0 tx=0).stop 2023-10-18T10:46:16.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.662+0000 7fd0cd1f3640 1 -- 172.21.15.89:0/1492844345 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0c80d0700 msgr2=0x7fd0c80c61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.662+0000 7fd0cd1f3640 1 --2- 172.21.15.89:0/1492844345 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0c80d0700 0x7fd0c80c61d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd0b800c500 tx=0x7fd0b800c9c0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.662+0000 7fd0c759e640 1 -- 172.21.15.89:0/1492844345 reap_dead start 2023-10-18T10:46:16.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.662+0000 7fd0cd1f3640 1 -- 172.21.15.89:0/1492844345 shutdown_connections 2023-10-18T10:46:16.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.662+0000 7fd0cd1f3640 1 -- 172.21.15.89:0/1492844345 >> 172.21.15.89:0/1492844345 conn(0x7fd0c8069ad0 msgr2=0x7fd0c815dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:16.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.662+0000 7fbd063e1640 1 -- 172.21.15.89:0/876752675 shutdown_connections 2023-10-18T10:46:16.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.662+0000 7fbd063e1640 1 -- 172.21.15.89:0/876752675 >> 172.21.15.89:0/876752675 conn(0x7fbd000afe00 msgr2=0x7fbd0015d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:16.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.666+0000 7fbd063e1640 1 -- 172.21.15.89:0/876752675 shutdown_connections 2023-10-18T10:46:16.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.666+0000 7fd0cd1f3640 1 -- 172.21.15.89:0/1492844345 shutdown_connections 2023-10-18T10:46:16.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.666+0000 7fd0cd1f3640 1 -- 172.21.15.89:0/1492844345 wait complete. 2023-10-18T10:46:16.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.666+0000 7fbd063e1640 1 -- 172.21.15.89:0/876752675 wait complete. 2023-10-18T10:46:16.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.666+0000 7fdd7d38f640 1 -- 172.21.15.89:0/4157337148 shutdown_connections 2023-10-18T10:46:16.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.666+0000 7fdd7d38f640 1 -- 172.21.15.89:0/4157337148 >> 172.21.15.89:0/4157337148 conn(0x7fdd78069ad0 msgr2=0x7fdd780b1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:16.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.666+0000 7f2242ffd640 1 -- 172.21.15.89:0/3937605769 <== osd.2 v2:172.21.15.89:6808/4229386532 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2230071df0 con 0x7f223006f2e0 2023-10-18T10:46:16.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.666+0000 7fdd7d38f640 1 -- 172.21.15.89:0/4157337148 shutdown_connections 2023-10-18T10:46:16.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.666+0000 7fdd7d38f640 1 -- 172.21.15.89:0/4157337148 wait complete. 2023-10-18T10:46:16.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.666+0000 7f22544e5640 1 -- 172.21.15.89:0/3937605769 >> [v2:172.21.15.89:6808/4229386532,v1:172.21.15.89:6809/4229386532] conn(0x7f223006f2e0 msgr2=0x7f2230071720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.666+0000 7f22544e5640 1 --2- 172.21.15.89:0/3937605769 >> [v2:172.21.15.89:6808/4229386532,v1:172.21.15.89:6809/4229386532] conn(0x7f223006f2e0 0x7f2230071720 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.666+0000 7f22544e5640 1 -- 172.21.15.89:0/3937605769 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2230065680 msgr2=0x7f2230067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.666+0000 7f22544e5640 1 --2- 172.21.15.89:0/3937605769 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2230065680 0x7f2230067b40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f22440b6000 tx=0x7f22440af000 comp rx=0 tx=0).stop 2023-10-18T10:46:16.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.666+0000 7f22544e5640 1 -- 172.21.15.89:0/3937605769 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f224c159360 msgr2=0x7f224c145880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.666+0000 7f22544e5640 1 --2- 172.21.15.89:0/3937605769 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f224c159360 0x7f224c145880 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f22480093f0 tx=0x7f22480098b0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.666+0000 7f2252a82640 1 -- 172.21.15.89:0/3937605769 reap_dead start 2023-10-18T10:46:16.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.666+0000 7f22544e5640 1 -- 172.21.15.89:0/3937605769 shutdown_connections 2023-10-18T10:46:16.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.666+0000 7f22544e5640 1 -- 172.21.15.89:0/3937605769 >> 172.21.15.89:0/3937605769 conn(0x7f224c069f00 msgr2=0x7f224c158960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:16.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.666+0000 7f22544e5640 1 -- 172.21.15.89:0/3937605769 shutdown_connections 2023-10-18T10:46:16.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.666+0000 7f22544e5640 1 -- 172.21.15.89:0/3937605769 wait complete. 2023-10-18T10:46:16.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.674+0000 7fd456d05640 1 -- 172.21.15.89:0/1029608725 --> [v2:172.21.15.154:6815/4139186586,v1:172.21.15.154:6816/4139186586] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd4501569e0 con 0x7fd43406f3b0 2023-10-18T10:46:16.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.674+0000 7fd4467fc640 1 -- 172.21.15.89:0/1029608725 <== osd.7 v2:172.21.15.154:6815/4139186586 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd4501569e0 con 0x7fd43406f3b0 2023-10-18T10:46:16.769 INFO:teuthology.orchestra.run.smithi089.stdout:55834574857 2023-10-18T10:46:16.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.674+0000 7fd456d05640 1 -- 172.21.15.89:0/1029608725 >> [v2:172.21.15.154:6815/4139186586,v1:172.21.15.154:6816/4139186586] conn(0x7fd43406f3b0 msgr2=0x7fd4340717f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.674+0000 7fd456d05640 1 --2- 172.21.15.89:0/1029608725 >> [v2:172.21.15.154:6815/4139186586,v1:172.21.15.154:6816/4139186586] conn(0x7fd43406f3b0 0x7fd4340717f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.770 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-18T10:46:16.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.674+0000 7fd456d05640 1 -- 172.21.15.89:0/1029608725 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd434065750 msgr2=0x7fd434067c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.674+0000 7fd456d05640 1 --2- 172.21.15.89:0/1029608725 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd434065750 0x7fd434067c10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd44c00af30 tx=0x7fd44c008040 comp rx=0 tx=0).stop 2023-10-18T10:46:16.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.674+0000 7fd456d05640 1 -- 172.21.15.89:0/1029608725 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd45015c310 msgr2=0x7fd45015f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:16.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.674+0000 7fd456d05640 1 --2- 172.21.15.89:0/1029608725 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd45015c310 0x7fd45015f150 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd4480af040 tx=0x7fd4480196b0 comp rx=0 tx=0).stop 2023-10-18T10:46:16.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.674+0000 7fd456504640 1 -- 172.21.15.89:0/1029608725 reap_dead start 2023-10-18T10:46:16.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.674+0000 7fd456d05640 1 -- 172.21.15.89:0/1029608725 shutdown_connections 2023-10-18T10:46:16.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.674+0000 7fd456d05640 1 -- 172.21.15.89:0/1029608725 >> 172.21.15.89:0/1029608725 conn(0x7fd450070120 msgr2=0x7fd4500ccab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:16.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.682+0000 7fd456d05640 1 -- 172.21.15.89:0/1029608725 shutdown_connections 2023-10-18T10:46:16.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:16.682+0000 7fd456d05640 1 -- 172.21.15.89:0/1029608725 wait complete. 2023-10-18T10:46:16.773 INFO:teuthology.orchestra.run.smithi089.stdout:55834574857 2023-10-18T10:46:16.773 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-18T10:46:16.774 INFO:teuthology.orchestra.run.smithi089.stdout:55834574857 2023-10-18T10:46:16.775 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-18T10:46:16.775 INFO:teuthology.orchestra.run.smithi089.stdout:55834574857 2023-10-18T10:46:16.776 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-18T10:46:16.777 INFO:teuthology.orchestra.run.smithi089.stdout:55834574858 2023-10-18T10:46:16.777 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-18T10:46:16.778 INFO:teuthology.orchestra.run.smithi089.stdout:55834574857 2023-10-18T10:46:16.778 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-18T10:46:16.779 INFO:teuthology.orchestra.run.smithi089.stdout:55834574856 2023-10-18T10:46:16.780 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-18T10:46:16.781 INFO:teuthology.orchestra.run.smithi089.stdout:55834574857 2023-10-18T10:46:16.781 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-18T10:46:17.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.050+0000 7fc1b6ffa640 1 Processor -- start 2023-10-18T10:46:17.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.050+0000 7fc1b6ffa640 1 -- start start 2023-10-18T10:46:17.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.054+0000 7fc1b6ffa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1a80df210 0x7fc1a8054800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.054+0000 7fc1b6ffa640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc1a80df5f0 0x7fc1a804f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.054+0000 7fc1b6ffa640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc1a8006ec0 0x7fc1a804fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.054+0000 7fc1b6ffa640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1a8007440 con 0x7fc1a80df5f0 2023-10-18T10:46:17.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.054+0000 7fc1b6ffa640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1a8007720 con 0x7fc1a8006ec0 2023-10-18T10:46:17.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.054+0000 7fc1b6ffa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a8007a00 con 0x7fc1a80df210 2023-10-18T10:46:17.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.054+0000 7fc1affff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1a80df210 0x7fc1a8054800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.054+0000 7fc1b4d96640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc1a8006ec0 0x7fc1a804fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.054+0000 7fc1b4d96640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc1a8006ec0 0x7fc1a804fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:34214/0 (socket says 172.21.15.89:34214) 2023-10-18T10:46:17.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.054+0000 7fc1b4d96640 1 -- 172.21.15.89:0/2007428388 learned_addr learned my addr 172.21.15.89:0/2007428388 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:17.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.058+0000 7fc1b5d98640 1 --2- 172.21.15.89:0/2007428388 >> v2:172.21.15.89:3300/0 conn(0x7fc1a80df5f0 0x7fc1a804f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.058+0000 7fc1b4d96640 1 -- 172.21.15.89:0/2007428388 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1a80df210 msgr2=0x7fc1a8054800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:17.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.058+0000 7fc1b4d96640 1 --2- 172.21.15.89:0/2007428388 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1a80df210 0x7fc1a8054800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.058+0000 7fc1b4d96640 1 -- 172.21.15.89:0/2007428388 >> v2:172.21.15.89:3300/0 conn(0x7fc1a80df5f0 msgr2=0x7fc1a804f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:17.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.058+0000 7fc1b4d96640 1 --2- 172.21.15.89:0/2007428388 >> v2:172.21.15.89:3300/0 conn(0x7fc1a80df5f0 0x7fc1a804f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.058+0000 7fc1b4d96640 1 -- 172.21.15.89:0/2007428388 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a8007ce0 con 0x7fc1a8006ec0 2023-10-18T10:46:17.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.058+0000 7fc1affff640 1 --2- 172.21.15.89:0/2007428388 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1a80df210 0x7fc1a8054800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:46:17.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.058+0000 7faf5b59e640 1 Processor -- start 2023-10-18T10:46:17.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.058+0000 7faf5b59e640 1 -- start start 2023-10-18T10:46:17.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.058+0000 7faf5b59e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faf5c158810 0x7faf5c165840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.062+0000 7faf5b59e640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf5c0ccb60 con 0x7faf5c0cb660 2023-10-18T10:46:17.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.062+0000 7faf5ad9d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faf5c158810 0x7faf5c165840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.062+0000 7faf5ad9d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faf5c158810 0x7faf5c165840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45258/0 (socket says 172.21.15.89:45258) 2023-10-18T10:46:17.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.062+0000 7faf5ad9d640 1 -- 172.21.15.89:0/2247099915 learned_addr learned my addr 172.21.15.89:0/2247099915 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:17.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.062+0000 7faf5b59e640 1 -- 172.21.15.89:0/2247099915 --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf5c0cbc90 con 0x7faf5c158390 2023-10-18T10:46:17.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.062+0000 7faf5b59e640 1 -- 172.21.15.89:0/2247099915 --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf5c0cbf50 con 0x7faf5c158810 2023-10-18T10:46:17.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.062+0000 7faf5ad9d640 1 -- 172.21.15.89:0/2247099915 >> v1:172.21.15.154:6789/0 conn(0x7faf5c158390 legacy=0x7faf5c06cbf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T10:46:17.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.062+0000 7faf5ad9d640 1 -- 172.21.15.89:0/2247099915 >> v1:172.21.15.89:6789/0 conn(0x7faf5c0cb660 legacy=0x7faf5c0cd8d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T10:46:17.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.062+0000 7faf5ad9d640 1 -- 172.21.15.89:0/2247099915 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf5c0cc230 con 0x7faf5c158810 2023-10-18T10:46:17.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.062+0000 7fc1b4d96640 1 --2- 172.21.15.89:0/2007428388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1a8006ec0 0x7fc1a804fed0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8055e90 tx=0x7fc19c01f080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39501b790f379a10 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:17.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.062+0000 7fc1af7fe640 1 -- 172.21.15.89:0/2007428388 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c0a8e50 con 0x7fc1a8006ec0 2023-10-18T10:46:17.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.062+0000 7fc1af7fe640 1 -- 172.21.15.89:0/2007428388 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19c0ac070 con 0x7fc1a8006ec0 2023-10-18T10:46:17.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.062+0000 7fc1af7fe640 1 -- 172.21.15.89:0/2007428388 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c0b2620 con 0x7fc1a8006ec0 2023-10-18T10:46:17.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.066+0000 7fc1b6ffa640 1 -- 172.21.15.89:0/2007428388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1a8006ec0 msgr2=0x7fc1a804fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.066+0000 7fc1b6ffa640 1 --2- 172.21.15.89:0/2007428388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1a8006ec0 0x7fc1a804fed0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8055e90 tx=0x7fc19c01f080 comp rx=0 tx=0).stop 2023-10-18T10:46:17.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.066+0000 7fc1b6ffa640 1 -- 172.21.15.89:0/2007428388 shutdown_connections 2023-10-18T10:46:17.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.066+0000 7fc1b6ffa640 1 --2- 172.21.15.89:0/2007428388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1a8006ec0 0x7fc1a804fed0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.066+0000 7fc1b6ffa640 1 --2- 172.21.15.89:0/2007428388 >> v2:172.21.15.89:3300/0 conn(0x7fc1a80df5f0 0x7fc1a804f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.066+0000 7fc1b6ffa640 1 --2- 172.21.15.89:0/2007428388 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1a80df210 0x7fc1a8054800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.066+0000 7fc1b6ffa640 1 -- 172.21.15.89:0/2007428388 >> 172.21.15.89:0/2007428388 conn(0x7fc1a8055130 msgr2=0x7fc1a8055540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:17.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.066+0000 7fc1b6ffa640 1 -- 172.21.15.89:0/2007428388 shutdown_connections 2023-10-18T10:46:17.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.066+0000 7fc1b6ffa640 1 -- 172.21.15.89:0/2007428388 wait complete. 2023-10-18T10:46:17.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.066+0000 7faf5ad9d640 1 --2- 172.21.15.89:0/2247099915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf5c158810 0x7faf5c165840 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7faf50009870 tx=0x7faf5001ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b212cfdd99d13260 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:17.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.066+0000 7f92da8ca640 1 Processor -- start 2023-10-18T10:46:17.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.066+0000 7f043dffa640 1 Processor -- start 2023-10-18T10:46:17.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.066+0000 7faf5959a640 1 -- 172.21.15.89:0/2247099915 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf500a89b0 con 0x7faf5c158810 2023-10-18T10:46:17.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.066+0000 7f92da8ca640 1 -- start start 2023-10-18T10:46:17.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.066+0000 7fc1b6ffa640 1 Processor -- start 2023-10-18T10:46:17.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.066+0000 7f043dffa640 1 -- start start 2023-10-18T10:46:17.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.066+0000 7faf5959a640 1 -- 172.21.15.89:0/2247099915 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf500a8b10 con 0x7faf5c158810 2023-10-18T10:46:17.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.066+0000 7faf5959a640 1 -- 172.21.15.89:0/2247099915 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf500a8ea0 con 0x7faf5c158810 2023-10-18T10:46:17.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7fc1b6ffa640 1 -- start start 2023-10-18T10:46:17.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7faf5b59e640 1 -- 172.21.15.89:0/2247099915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf5c158810 msgr2=0x7faf5c165840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7faf5b59e640 1 --2- 172.21.15.89:0/2247099915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf5c158810 0x7faf5c165840 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7faf50009870 tx=0x7faf5001ecb0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7f043dffa640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f04380ad2b0 0x7f04380ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7f92da8ca640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92d40d0050 0x7f92d40d1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7f92da8ca640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f92d40d1e70 0x7f92d40b5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7f043dffa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f043806c630 0x7f04380cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7f043dffa640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f043806bd10 con 0x7f043806c1a0 2023-10-18T10:46:17.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7f043dffa640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f043806ae20 con 0x7f04380ad2b0 2023-10-18T10:46:17.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7f043dffa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f043806b100 con 0x7f043806c630 2023-10-18T10:46:17.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7faf5b59e640 1 -- 172.21.15.89:0/2247099915 shutdown_connections 2023-10-18T10:46:17.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7faf5b59e640 1 --2- 172.21.15.89:0/2247099915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf5c158810 0x7faf5c165840 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7faf5b59e640 1 -- 172.21.15.89:0/2247099915 >> 172.21.15.89:0/2247099915 conn(0x7faf5c0cab40 msgr2=0x7faf5c0caf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:17.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7f04377fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f04380ad2b0 0x7f04380ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7f04377fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f04380ad2b0 0x7f04380ca1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45270/0 (socket says 172.21.15.89:45270) 2023-10-18T10:46:17.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7f04377fe640 1 -- 172.21.15.89:0/1017352591 learned_addr learned my addr 172.21.15.89:0/1017352591 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:17.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7f0437fff640 1 --2- 172.21.15.89:0/1017352591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f043806c630 0x7f04380cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7faf5b59e640 1 -- 172.21.15.89:0/2247099915 shutdown_connections 2023-10-18T10:46:17.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7f92d98c8640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f92d415c560 0x7f92d40b4350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35616/0 (socket says 172.21.15.89:35616) 2023-10-18T10:46:17.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7f92d98c8640 1 -- 172.21.15.89:0/3758792332 learned_addr learned my addr 172.21.15.89:0/3758792332 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:17.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7fc1b6ffa640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1a8006ec0 0x7fc1a817b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7fc1b6ffa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1a80df210 0x7fc1a8168a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7fc1b6ffa640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1a80df5f0 0x7fc1a8168f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7fc1b6ffa640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a80d8ff0 con 0x7fc1a80df5f0 2023-10-18T10:46:17.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7fc1b6ffa640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a80d8370 con 0x7fc1a8006ec0 2023-10-18T10:46:17.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7fc1b6ffa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a80d8650 con 0x7fc1a80df210 2023-10-18T10:46:17.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7f0437fff640 1 -- 172.21.15.89:0/1017352591 >> v2:172.21.15.89:3300/0 conn(0x7f04380ad2b0 msgr2=0x7f04380ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:17.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7f0437fff640 1 --2- 172.21.15.89:0/1017352591 >> v2:172.21.15.89:3300/0 conn(0x7f04380ad2b0 0x7f04380ca1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7f0437fff640 1 -- 172.21.15.89:0/1017352591 >> v1:172.21.15.154:6789/0 conn(0x7f043806c1a0 legacy=0x7f0438160fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:17.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7f92da8ca640 1 -- 172.21.15.89:0/3758792332 --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92d40d2290 con 0x7f92d415c560 2023-10-18T10:46:17.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7f92da8ca640 1 -- 172.21.15.89:0/3758792332 --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92d40b4a60 con 0x7f92d40d1e70 2023-10-18T10:46:17.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7f92da8ca640 1 -- 172.21.15.89:0/3758792332 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d415c940 con 0x7f92d40d0050 2023-10-18T10:46:17.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7f92d90c7640 1 --2- 172.21.15.89:0/3758792332 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92d40d0050 0x7f92d40d1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7f0437fff640 1 -- 172.21.15.89:0/1017352591 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f043806b3e0 con 0x7f043806c630 2023-10-18T10:46:17.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7faf5b59e640 1 -- 172.21.15.89:0/2247099915 wait complete. 2023-10-18T10:46:17.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7f92da0c9640 1 --2- 172.21.15.89:0/3758792332 >> v2:172.21.15.89:3300/0 conn(0x7f92d40d1e70 0x7f92d40b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7fc1affff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1a8006ec0 0x7fc1a817b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7fc1b5d98640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1a80df210 0x7fc1a8168a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7fc1affff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1a8006ec0 0x7fc1a817b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:34228/0 (socket says 172.21.15.89:34228) 2023-10-18T10:46:17.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7fc1affff640 1 -- 172.21.15.89:0/1307729326 learned_addr learned my addr 172.21.15.89:0/1307729326 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:17.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7fc1b5d98640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1a80df210 0x7fc1a8168a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:37292/0 (socket says 172.21.15.89:37292) 2023-10-18T10:46:17.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7fc1affff640 1 -- 172.21.15.89:0/1307729326 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1a80df210 msgr2=0x7fc1a8168a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7fc1affff640 1 --2- 172.21.15.89:0/1307729326 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1a80df210 0x7fc1a8168a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7fc1affff640 1 -- 172.21.15.89:0/1307729326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1a80df5f0 msgr2=0x7fc1a8168f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7fc1affff640 1 --2- 172.21.15.89:0/1307729326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1a80df5f0 0x7fc1a8168f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.074+0000 7fc1affff640 1 -- 172.21.15.89:0/1307729326 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a400d8c0 con 0x7fc1a8006ec0 2023-10-18T10:46:17.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.074+0000 7faf5b59e640 1 Processor -- start 2023-10-18T10:46:17.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7f92d90c7640 1 -- 172.21.15.89:0/3758792332 >> v2:172.21.15.89:3300/0 conn(0x7f92d40d1e70 msgr2=0x7f92d40b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:17.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7f92d90c7640 1 --2- 172.21.15.89:0/3758792332 >> v2:172.21.15.89:3300/0 conn(0x7f92d40d1e70 0x7f92d40b5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7f92d90c7640 1 -- 172.21.15.89:0/3758792332 >> v1:172.21.15.89:6789/0 conn(0x7f92d415c560 legacy=0x7f92d40b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:17.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.070+0000 7f92d90c7640 1 -- 172.21.15.89:0/3758792332 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d40d0550 con 0x7f92d40d0050 2023-10-18T10:46:17.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.074+0000 7fc1b5d98640 1 --2- 172.21.15.89:0/1307729326 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1a80df210 0x7fc1a8168a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:17.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.074+0000 7faf5b59e640 1 -- start start 2023-10-18T10:46:17.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.074+0000 7fc1affff640 1 --2- 172.21.15.89:0/1307729326 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1a8006ec0 0x7fc1a817b080 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc1a400bf00 tx=0x7fc1a400f480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:17.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.074+0000 7fc1ad7fa640 1 -- 172.21.15.89:0/1307729326 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a400de80 con 0x7fc1a8006ec0 2023-10-18T10:46:17.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.074+0000 7fc1ad7fa640 1 -- 172.21.15.89:0/1307729326 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a400fb90 con 0x7fc1a8006ec0 2023-10-18T10:46:17.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.074+0000 7fc1ad7fa640 1 -- 172.21.15.89:0/1307729326 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4016420 con 0x7fc1a8006ec0 2023-10-18T10:46:17.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.074+0000 7fc1b6ffa640 1 -- 172.21.15.89:0/1307729326 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc19c00a570 con 0x7fc1a8006ec0 2023-10-18T10:46:17.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.074+0000 7fc1b6ffa640 1 -- 172.21.15.89:0/1307729326 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a80d8c10 con 0x7fc1a8006ec0 2023-10-18T10:46:17.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.074+0000 7faf5b59e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf5c0cb660 0x7faf5c0c15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.074+0000 7faf5b59e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf5c158390 0x7faf5c14c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.074+0000 7f0437fff640 1 --2- 172.21.15.89:0/1017352591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f043806c630 0x7f04380cbaa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f04240097c0 tx=0x7f042401c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3aa3c73f424dd64e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:17.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.074+0000 7faf5a59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf5c0cb660 0x7faf5c0c15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7f04367fc640 1 -- 172.21.15.89:0/1017352591 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04240ac630 con 0x7f043806c630 2023-10-18T10:46:17.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7f04367fc640 1 -- 172.21.15.89:0/1017352591 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04240ac790 con 0x7f043806c630 2023-10-18T10:46:17.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7f04367fc640 1 -- 172.21.15.89:0/1017352591 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04240acb20 con 0x7f043806c630 2023-10-18T10:46:17.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7faf5a59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf5c0cb660 0x7faf5c0c15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45296/0 (socket says 172.21.15.89:45296) 2023-10-18T10:46:17.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7faf5a59c640 1 -- 172.21.15.89:0/3649717764 learned_addr learned my addr 172.21.15.89:0/3649717764 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:17.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7faf5b59e640 1 --2- 172.21.15.89:0/3649717764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf5c158810 0x7faf5c14c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7faf59d9b640 1 --2- 172.21.15.89:0/3649717764 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf5c158390 0x7faf5c14c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7f043dffa640 1 -- 172.21.15.89:0/1017352591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f043806c630 msgr2=0x7f04380cbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7f043dffa640 1 --2- 172.21.15.89:0/1017352591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f043806c630 0x7f04380cbaa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f04240097c0 tx=0x7f042401c410 comp rx=0 tx=0).stop 2023-10-18T10:46:17.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7faf5b59e640 1 -- 172.21.15.89:0/3649717764 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c1f2020 con 0x7faf5c0cb660 2023-10-18T10:46:17.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7faf5b59e640 1 -- 172.21.15.89:0/3649717764 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c15bc10 con 0x7faf5c158810 2023-10-18T10:46:17.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.074+0000 7fc1b6ffa640 1 -- 172.21.15.89:0/1307729326 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1a8050fb0 con 0x7fc1a8006ec0 2023-10-18T10:46:17.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7faf5b59e640 1 -- 172.21.15.89:0/3649717764 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c15b930 con 0x7faf5c158390 2023-10-18T10:46:17.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7f92d90c7640 1 --2- 172.21.15.89:0/3758792332 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92d40d0050 0x7f92d40d1930 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f92c40097c0 tx=0x7f92c401c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c36d4688e8b195e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:17.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7faf5ad9d640 1 --2- 172.21.15.89:0/3649717764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf5c158810 0x7faf5c14c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7f043dffa640 1 -- 172.21.15.89:0/1017352591 shutdown_connections 2023-10-18T10:46:17.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7f043dffa640 1 --2- 172.21.15.89:0/1017352591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f043806c630 0x7f04380cbaa0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7f043dffa640 1 --2- 172.21.15.89:0/1017352591 >> v2:172.21.15.89:3300/0 conn(0x7f04380ad2b0 0x7f04380ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7f043dffa640 1 -- 172.21.15.89:0/1017352591 >> 172.21.15.89:0/1017352591 conn(0x7f04380ca9e0 msgr2=0x7f04380cadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:17.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7f92d88c6640 1 -- 172.21.15.89:0/3758792332 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c40ab600 con 0x7f92d40d0050 2023-10-18T10:46:17.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7fc1ad7fa640 1 -- 172.21.15.89:0/1307729326 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fc1a4014070 con 0x7fc1a8006ec0 2023-10-18T10:46:17.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7f92d88c6640 1 -- 172.21.15.89:0/3758792332 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c40a7050 con 0x7f92d40d0050 2023-10-18T10:46:17.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7f92d88c6640 1 -- 172.21.15.89:0/3758792332 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c40ab980 con 0x7f92d40d0050 2023-10-18T10:46:17.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7f043dffa640 1 -- 172.21.15.89:0/1017352591 shutdown_connections 2023-10-18T10:46:17.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7f043dffa640 1 -- 172.21.15.89:0/1017352591 wait complete. 2023-10-18T10:46:17.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7faf59d9b640 1 -- 172.21.15.89:0/3649717764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf5c158810 msgr2=0x7faf5c14c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7faf59d9b640 1 --2- 172.21.15.89:0/3649717764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf5c158810 0x7faf5c14c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7faf59d9b640 1 -- 172.21.15.89:0/3649717764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf5c0cb660 msgr2=0x7faf5c0c15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7faf59d9b640 1 --2- 172.21.15.89:0/3649717764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf5c0cb660 0x7faf5c0c15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7faf59d9b640 1 -- 172.21.15.89:0/3649717764 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf500094d0 con 0x7faf5c158390 2023-10-18T10:46:17.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7faf5a59c640 1 --2- 172.21.15.89:0/3649717764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf5c0cb660 0x7faf5c0c15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:46:17.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7f1dd28ca640 1 Processor -- start 2023-10-18T10:46:17.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7f1dd28ca640 1 -- start start 2023-10-18T10:46:17.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7faf59d9b640 1 --2- 172.21.15.89:0/3649717764 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf5c158390 0x7faf5c14c330 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7faf4c00b3c0 tx=0x7faf4c00b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:17.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7fc1ad7fa640 1 --2- 172.21.15.89:0/1307729326 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc194065680 0x7fc194067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7faf4affd640 1 -- 172.21.15.89:0/3649717764 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c011830 con 0x7faf5c158390 2023-10-18T10:46:17.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7faf4affd640 1 -- 172.21.15.89:0/3649717764 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4c011990 con 0x7faf5c158390 2023-10-18T10:46:17.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7faf4affd640 1 -- 172.21.15.89:0/3649717764 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c011d80 con 0x7faf5c158390 2023-10-18T10:46:17.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7faf5b59e640 1 -- 172.21.15.89:0/3649717764 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf5c14cf00 con 0x7faf5c158390 2023-10-18T10:46:17.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7f92da8ca640 1 -- 172.21.15.89:0/3758792332 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92d40d0050 msgr2=0x7f92d40d1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7f92da8ca640 1 --2- 172.21.15.89:0/3758792332 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92d40d0050 0x7f92d40d1930 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f92c40097c0 tx=0x7f92c401c440 comp rx=0 tx=0).stop 2023-10-18T10:46:17.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7faf5b59e640 1 -- 172.21.15.89:0/3649717764 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf5c14d4c0 con 0x7faf5c158390 2023-10-18T10:46:17.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7fc1ad7fa640 1 -- 172.21.15.89:0/1307729326 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc1a40829e0 con 0x7fc1a8006ec0 2023-10-18T10:46:17.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7f92da8ca640 1 -- 172.21.15.89:0/3758792332 shutdown_connections 2023-10-18T10:46:17.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7f92da8ca640 1 --2- 172.21.15.89:0/3758792332 >> v2:172.21.15.89:3300/0 conn(0x7f92d40d1e70 0x7f92d40b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7f92da8ca640 1 --2- 172.21.15.89:0/3758792332 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92d40d0050 0x7f92d40d1930 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.078+0000 7f92da8ca640 1 -- 172.21.15.89:0/3758792332 >> 172.21.15.89:0/3758792332 conn(0x7f92d40d2510 msgr2=0x7f92d40d2920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:17.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7fc1b5d98640 1 --2- 172.21.15.89:0/1307729326 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc194065680 0x7fc194067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f1dd28ca640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1dcc15c940 0x7f1dcc0d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f1dd28ca640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1dcc0d2f40 0x7f1dcc0d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f1dd28ca640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1dcc0d0ca0 0x7f1dcc0cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f1dd28ca640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dcc0d1130 con 0x7f1dcc0d0ca0 2023-10-18T10:46:17.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f1dd28ca640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dcc15da50 con 0x7f1dcc15c940 2023-10-18T10:46:17.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f1dd28ca640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc15ce40 con 0x7f1dcc0d2f40 2023-10-18T10:46:17.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f1979f05640 1 Processor -- start 2023-10-18T10:46:17.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f92da8ca640 1 -- 172.21.15.89:0/3758792332 shutdown_connections 2023-10-18T10:46:17.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f043dffa640 1 Processor -- start 2023-10-18T10:46:17.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f92da8ca640 1 -- 172.21.15.89:0/3758792332 wait complete. 2023-10-18T10:46:17.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f1dcbfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1dcc15c940 0x7f1dcc0d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f1dcbfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1dcc15c940 0x7f1dcc0d2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:34246/0 (socket says 172.21.15.89:34246) 2023-10-18T10:46:17.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f1dcbfff640 1 -- 172.21.15.89:0/925709225 learned_addr learned my addr 172.21.15.89:0/925709225 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:17.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f1dcbfff640 1 -- 172.21.15.89:0/925709225 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1dcc0d2f40 msgr2=0x7f1dcc0d0760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-18T10:46:17.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f1dcbfff640 1 --2- 172.21.15.89:0/925709225 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1dcc0d2f40 0x7f1dcc0d0760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f1dcbfff640 1 -- 172.21.15.89:0/925709225 >> v2:172.21.15.89:3300/0 conn(0x7f1dcc0d0ca0 msgr2=0x7f1dcc0cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:17.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f1dcbfff640 1 --2- 172.21.15.89:0/925709225 >> v2:172.21.15.89:3300/0 conn(0x7f1dcc0d0ca0 0x7f1dcc0cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f1dcbfff640 1 -- 172.21.15.89:0/925709225 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dcc15d120 con 0x7f1dcc15c940 2023-10-18T10:46:17.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f1979f05640 1 -- start start 2023-10-18T10:46:17.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7faf5b59e640 1 -- 172.21.15.89:0/3649717764 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf5c073b50 con 0x7faf5c158390 2023-10-18T10:46:17.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7fc1b5d98640 1 --2- 172.21.15.89:0/1307729326 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc194065680 0x7fc194067b40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc1b00684d0 tx=0x7fc1b008b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:17.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f1979f05640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19740cd1d0 0x7f1974161120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f1979f05640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1974165390 0x7f1974161660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f1979f05640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1974161ba0 con 0x7f197415b060 2023-10-18T10:46:17.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f1979f05640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19741620c0 con 0x7f1974165390 2023-10-18T10:46:17.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f1979f05640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1974161de0 con 0x7f19740cd1d0 2023-10-18T10:46:17.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f5b7666a640 1 Processor -- start 2023-10-18T10:46:17.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f5b7666a640 1 -- start start 2023-10-18T10:46:17.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f043dffa640 1 -- start start 2023-10-18T10:46:17.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f043dffa640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f043806c1a0 0x7f0438153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f043dffa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f043806c630 0x7f0438070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f043dffa640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04380ad2b0 0x7f0438151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f043dffa640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0438070960 con 0x7f043806c1a0 2023-10-18T10:46:17.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f043dffa640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0438153ae0 con 0x7f04380ad2b0 2023-10-18T10:46:17.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f043dffa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0438070030 con 0x7f043806c630 2023-10-18T10:46:17.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f92da8ca640 1 Processor -- start 2023-10-18T10:46:17.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f92da8ca640 1 -- start start 2023-10-18T10:46:17.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7f0436ffd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f043806c630 0x7f0438070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7faf4affd640 1 -- 172.21.15.89:0/3649717764 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7faf4c01b460 con 0x7faf5c158390 2023-10-18T10:46:17.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.082+0000 7fa88c89e640 1 Processor -- start 2023-10-18T10:46:17.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7fc1ad7fa640 1 -- 172.21.15.89:0/1307729326 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fc1a4050110 con 0x7fc1a8006ec0 2023-10-18T10:46:17.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7fa88c89e640 1 -- start start 2023-10-18T10:46:17.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f5b7666a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b700cc9d0 0x7f5b7006d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f5b7666a640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b70162140 con 0x7f5b700cbee0 2023-10-18T10:46:17.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f5b7666a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b7006ce40 con 0x7f5b70160e10 2023-10-18T10:46:17.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f5b7666a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b700c9070 con 0x7f5b700cc9d0 2023-10-18T10:46:17.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f1978f03640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f197415b060 0x7f19740ccac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35624/0 (socket says 172.21.15.89:35624) 2023-10-18T10:46:17.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f1978f03640 1 -- 172.21.15.89:0/2060516543 learned_addr learned my addr 172.21.15.89:0/2060516543 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:17.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f1973fff640 1 --2- 172.21.15.89:0/2060516543 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19740cd1d0 0x7f1974161120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f5b6f7fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f5b70160e10 0x7f5b700cc2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.89:53704/0 (socket says 172.21.15.89:53704) 2023-10-18T10:46:17.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f5b6f7fe640 1 -- 172.21.15.89:0/2368886690 learned_addr learned my addr 172.21.15.89:0/2368886690 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:17.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f0437fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04380ad2b0 0x7f0438151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f0436ffd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f043806c630 0x7f0438070df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:37316/0 (socket says 172.21.15.89:37316) 2023-10-18T10:46:17.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f0436ffd640 1 -- 172.21.15.89:0/1664031242 learned_addr learned my addr 172.21.15.89:0/1664031242 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:17.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f04377fe640 1 --2- 172.21.15.89:0/1664031242 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f043806c1a0 0x7f0438153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f92da8ca640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92d40d0050 0x7f92d40b3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f92da8ca640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92d40d1e70 0x7f92d40b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f92da8ca640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92d415c560 0x7f92d41f7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f92da8ca640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d4157d00 con 0x7f92d40d1e70 2023-10-18T10:46:17.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f92da8ca640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d41502c0 con 0x7f92d415c560 2023-10-18T10:46:17.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f92da8ca640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d4157080 con 0x7f92d40d0050 2023-10-18T10:46:17.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f92da0c9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92d415c560 0x7f92d41f7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f92da0c9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92d415c560 0x7f92d41f7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:34252/0 (socket says 172.21.15.89:34252) 2023-10-18T10:46:17.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f92da0c9640 1 -- 172.21.15.89:0/86409863 learned_addr learned my addr 172.21.15.89:0/86409863 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:17.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f92d98c8640 1 --2- 172.21.15.89:0/86409863 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92d40d0050 0x7f92d40b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f0436ffd640 1 -- 172.21.15.89:0/1664031242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04380ad2b0 msgr2=0x7f0438151fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f0436ffd640 1 --2- 172.21.15.89:0/1664031242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04380ad2b0 0x7f0438151fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f0436ffd640 1 -- 172.21.15.89:0/1664031242 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f043806c1a0 msgr2=0x7f0438153290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f0436ffd640 1 --2- 172.21.15.89:0/1664031242 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f043806c1a0 0x7f0438153290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f0436ffd640 1 -- 172.21.15.89:0/1664031242 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0424009420 con 0x7f043806c630 2023-10-18T10:46:17.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f1973fff640 1 -- 172.21.15.89:0/2060516543 >> v2:172.21.15.89:3300/0 conn(0x7f1974165390 msgr2=0x7f1974161660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T10:46:17.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f5b74c07640 1 --2- 172.21.15.89:0/2368886690 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b700cc9d0 0x7f5b7006d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f92d90c7640 1 --2- 172.21.15.89:0/86409863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92d40d1e70 0x7f92d40b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f1979704640 1 --2- 172.21.15.89:0/2060516543 >> v2:172.21.15.89:3300/0 conn(0x7f1974165390 0x7f1974161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f0436ffd640 1 --2- 172.21.15.89:0/1664031242 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f043806c630 0x7f0438070df0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f042c00c450 tx=0x7f042c00c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:17.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f5b74c07640 1 -- 172.21.15.89:0/2368886690 >> v1:172.21.15.154:6789/0 conn(0x7f5b70160e10 legacy=0x7f5b700cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f5b74c07640 1 -- 172.21.15.89:0/2368886690 >> v1:172.21.15.89:6789/0 conn(0x7f5b700cbee0 legacy=0x7f5b700c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:17.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f5b74c07640 1 -- 172.21.15.89:0/2368886690 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b700c1ab0 con 0x7f5b700cc9d0 2023-10-18T10:46:17.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7fa88c89e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8880b3f10 0x7fa8880b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7fa88c89e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa8880b5690 0x7fa8880b5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f92da0c9640 1 -- 172.21.15.89:0/86409863 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92d40d0050 msgr2=0x7f92d40b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f92da0c9640 1 --2- 172.21.15.89:0/86409863 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92d40d0050 0x7f92d40b3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f92da0c9640 1 -- 172.21.15.89:0/86409863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92d40d1e70 msgr2=0x7f92d40b36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f1dcbfff640 1 --2- 172.21.15.89:0/925709225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1dcc15c940 0x7f1dcc0d2a00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc0725b0 tx=0x7f1dbc01c230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2e346dc62f8ca75 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:17.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7fa88c89e640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8880d12f0 con 0x7fa88815c570 2023-10-18T10:46:17.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7fa88c89e640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8880d04d0 con 0x7fa8880b5690 2023-10-18T10:46:17.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7fa88c89e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8880b4890 con 0x7fa8880b3f10 2023-10-18T10:46:17.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f92da0c9640 1 --2- 172.21.15.89:0/86409863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92d40d1e70 0x7f92d40b36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f92da0c9640 1 -- 172.21.15.89:0/86409863 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c4009420 con 0x7f92d415c560 2023-10-18T10:46:17.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f92d90c7640 1 --2- 172.21.15.89:0/86409863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92d40d1e70 0x7f92d40b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:46:17.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7fa886ffd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8880b3f10 0x7fa8880b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7fa886ffd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8880b3f10 0x7fa8880b4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:37368/0 (socket says 172.21.15.89:37368) 2023-10-18T10:46:17.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7fa886ffd640 1 -- 172.21.15.89:0/2729892601 learned_addr learned my addr 172.21.15.89:0/2729892601 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:17.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7fa887fff640 1 --2- 172.21.15.89:0/2729892601 >> v2:172.21.15.89:3300/0 conn(0x7fa8880b5690 0x7fa8880b5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7faf4affd640 1 --2- 172.21.15.89:0/3649717764 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7faf3c065750 0x7faf3c067c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7faf4affd640 1 -- 172.21.15.89:0/3649717764 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7faf4c082e60 con 0x7faf5c158390 2023-10-18T10:46:17.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7faf5a59c640 1 --2- 172.21.15.89:0/3649717764 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7faf3c065750 0x7faf3c067c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f1dcb7fe640 1 -- 172.21.15.89:0/925709225 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc0ab600 con 0x7f1dcc15c940 2023-10-18T10:46:17.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f1dcb7fe640 1 -- 172.21.15.89:0/925709225 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dbc0a7050 con 0x7f1dcc15c940 2023-10-18T10:46:17.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f1dcb7fe640 1 -- 172.21.15.89:0/925709225 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc0ab980 con 0x7f1dcc15c940 2023-10-18T10:46:17.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f92da0c9640 1 --2- 172.21.15.89:0/86409863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92d415c560 0x7f92d41f7e70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f92cc00cd70 tx=0x7f92cc00a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:17.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f1973fff640 1 --2- 172.21.15.89:0/2060516543 >> v2:172.21.15.89:3300/0 conn(0x7f1974165390 0x7f1974161660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f1973fff640 1 -- 172.21.15.89:0/2060516543 >> v1:172.21.15.89:6789/0 conn(0x7f197415b060 legacy=0x7f19740ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.086+0000 7f1973fff640 1 -- 172.21.15.89:0/2060516543 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19741648b0 con 0x7f19740cd1d0 2023-10-18T10:46:17.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.090+0000 7fa887fff640 1 -- 172.21.15.89:0/2729892601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8880b3f10 msgr2=0x7fa8880b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:17.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.090+0000 7fa887fff640 1 --2- 172.21.15.89:0/2729892601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8880b3f10 0x7fa8880b4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.090+0000 7fa887fff640 1 -- 172.21.15.89:0/2729892601 >> v1:172.21.15.89:6789/0 conn(0x7fa88815c570 legacy=0x7fa8880d18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.090+0000 7fa887fff640 1 -- 172.21.15.89:0/2729892601 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa88815c950 con 0x7fa8880b5690 2023-10-18T10:46:17.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.090+0000 7f92ca7fc640 1 -- 172.21.15.89:0/86409863 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92cc003ba0 con 0x7f92d415c560 2023-10-18T10:46:17.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.090+0000 7faf5a59c640 1 --2- 172.21.15.89:0/3649717764 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7faf3c065750 0x7faf3c067c10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7faf5400f3c0 tx=0x7faf54003e70 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:17.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.090+0000 7f92ca7fc640 1 -- 172.21.15.89:0/86409863 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92cc003d00 con 0x7f92d415c560 2023-10-18T10:46:17.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.090+0000 7f92ca7fc640 1 -- 172.21.15.89:0/86409863 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92cc004240 con 0x7f92d415c560 2023-10-18T10:46:17.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.090+0000 7f92da8ca640 1 -- 172.21.15.89:0/86409863 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92d4157360 con 0x7f92d415c560 2023-10-18T10:46:17.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.090+0000 7f92da8ca640 1 -- 172.21.15.89:0/86409863 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92d4157920 con 0x7f92d415c560 2023-10-18T10:46:17.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.090+0000 7f1dd28ca640 1 -- 172.21.15.89:0/925709225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1dcc15c940 msgr2=0x7f1dcc0d2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.090+0000 7f1dd28ca640 1 --2- 172.21.15.89:0/925709225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1dcc15c940 0x7f1dcc0d2a00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc0725b0 tx=0x7f1dbc01c230 comp rx=0 tx=0).stop 2023-10-18T10:46:17.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.090+0000 7f1dd28ca640 1 -- 172.21.15.89:0/925709225 shutdown_connections 2023-10-18T10:46:17.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.090+0000 7f1dd28ca640 1 --2- 172.21.15.89:0/925709225 >> v2:172.21.15.89:3300/0 conn(0x7f1dcc0d0ca0 0x7f1dcc0cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.090+0000 7f1dd28ca640 1 --2- 172.21.15.89:0/925709225 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1dcc0d2f40 0x7f1dcc0d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.090+0000 7f1dd28ca640 1 --2- 172.21.15.89:0/925709225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1dcc15c940 0x7f1dcc0d2a00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.090+0000 7f1dd28ca640 1 -- 172.21.15.89:0/925709225 >> 172.21.15.89:0/925709225 conn(0x7f1dcc089fd0 msgr2=0x7f1dcc09b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:17.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.090+0000 7f92da8ca640 1 -- 172.21.15.89:0/86409863 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92d4157e40 con 0x7f92d415c560 2023-10-18T10:46:17.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.090+0000 7f1dd28ca640 1 -- 172.21.15.89:0/925709225 shutdown_connections 2023-10-18T10:46:17.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.090+0000 7f1dd28ca640 1 -- 172.21.15.89:0/925709225 wait complete. 2023-10-18T10:46:17.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.090+0000 7f5b74c07640 1 --2- 172.21.15.89:0/2368886690 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b700cc9d0 0x7f5b7006d320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5b640097f0 tx=0x7f5b6401c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=795c8e7b71ac26ee server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:17.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.090+0000 7f0423fff640 1 -- 172.21.15.89:0/1664031242 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c012830 con 0x7f043806c630 2023-10-18T10:46:17.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.090+0000 7faf4affd640 1 -- 172.21.15.89:0/3649717764 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7faf4c050630 con 0x7faf5c158390 2023-10-18T10:46:17.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.090+0000 7f0423fff640 1 -- 172.21.15.89:0/1664031242 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042c0129b0 con 0x7f043806c630 2023-10-18T10:46:17.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.090+0000 7f043dffa640 1 -- 172.21.15.89:0/1664031242 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0438070310 con 0x7f043806c630 2023-10-18T10:46:17.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.090+0000 7f5b6effd640 1 -- 172.21.15.89:0/2368886690 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b640ab600 con 0x7f5b700cc9d0 2023-10-18T10:46:17.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.090+0000 7f5b6effd640 1 -- 172.21.15.89:0/2368886690 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b640a7050 con 0x7f5b700cc9d0 2023-10-18T10:46:17.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.090+0000 7f0423fff640 1 -- 172.21.15.89:0/1664031242 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c012d60 con 0x7f043806c630 2023-10-18T10:46:17.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.090+0000 7f043dffa640 1 -- 172.21.15.89:0/1664031242 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04381537d0 con 0x7f043806c630 2023-10-18T10:46:17.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.090+0000 7f5b6effd640 1 -- 172.21.15.89:0/2368886690 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b640ab600 con 0x7f5b700cc9d0 2023-10-18T10:46:17.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7f92ca7fc640 1 -- 172.21.15.89:0/86409863 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f92cc0043a0 con 0x7f92d415c560 2023-10-18T10:46:17.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.090+0000 7f1973fff640 1 --2- 172.21.15.89:0/2060516543 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19740cd1d0 0x7f1974161120 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f195c0097a0 tx=0x7f195c01c310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=377754e818f1ff41 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:17.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7f19737fe640 1 -- 172.21.15.89:0/2060516543 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c0ab600 con 0x7f19740cd1d0 2023-10-18T10:46:17.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7f19737fe640 1 -- 172.21.15.89:0/2060516543 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195c0a7050 con 0x7f19740cd1d0 2023-10-18T10:46:17.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7f19737fe640 1 -- 172.21.15.89:0/2060516543 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c0ab980 con 0x7f19740cd1d0 2023-10-18T10:46:17.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7f5b7666a640 1 -- 172.21.15.89:0/2368886690 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b700cc9d0 msgr2=0x7f5b7006d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7f5b7666a640 1 --2- 172.21.15.89:0/2368886690 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b700cc9d0 0x7f5b7006d320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5b640097f0 tx=0x7f5b6401c3f0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7f5b7666a640 1 -- 172.21.15.89:0/2368886690 shutdown_connections 2023-10-18T10:46:17.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7f1dd28ca640 1 Processor -- start 2023-10-18T10:46:17.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7f5b7666a640 1 --2- 172.21.15.89:0/2368886690 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b700cc9d0 0x7f5b7006d320 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7f5b7666a640 1 -- 172.21.15.89:0/2368886690 >> 172.21.15.89:0/2368886690 conn(0x7f5b700afe00 msgr2=0x7f5b700b0210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:17.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7f043dffa640 1 -- 172.21.15.89:0/1664031242 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f043804e430 con 0x7f043806c630 2023-10-18T10:46:17.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7f5b7666a640 1 -- 172.21.15.89:0/2368886690 shutdown_connections 2023-10-18T10:46:17.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7f1dd28ca640 1 -- start start 2023-10-18T10:46:17.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7fa887fff640 1 --2- 172.21.15.89:0/2729892601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8880b5690 0x7fa8880b5b20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa880009d90 tx=0x7fa88001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed63e57abd685ba server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:17.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7f92ca7fc640 1 --2- 172.21.15.89:0/86409863 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f92b8065750 0x7f92b8067c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7f5b7666a640 1 -- 172.21.15.89:0/2368886690 wait complete. 2023-10-18T10:46:17.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7f92ca7fc640 1 -- 172.21.15.89:0/86409863 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f92cc012070 con 0x7f92d415c560 2023-10-18T10:46:17.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7f1979f05640 1 -- 172.21.15.89:0/2060516543 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19740cd1d0 msgr2=0x7f1974161120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7f1979f05640 1 --2- 172.21.15.89:0/2060516543 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19740cd1d0 0x7f1974161120 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f195c0097a0 tx=0x7f195c01c310 comp rx=0 tx=0).stop 2023-10-18T10:46:17.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7f92d98c8640 1 --2- 172.21.15.89:0/86409863 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f92b8065750 0x7f92b8067c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7f1979f05640 1 -- 172.21.15.89:0/2060516543 shutdown_connections 2023-10-18T10:46:17.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7f1979f05640 1 --2- 172.21.15.89:0/2060516543 >> v2:172.21.15.89:3300/0 conn(0x7f1974165390 0x7f1974161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7f1979f05640 1 --2- 172.21.15.89:0/2060516543 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19740cd1d0 0x7f1974161120 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7f1979f05640 1 -- 172.21.15.89:0/2060516543 >> 172.21.15.89:0/2060516543 conn(0x7f19740ceef0 msgr2=0x7f19740cf300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:17.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7f1979f05640 1 -- 172.21.15.89:0/2060516543 shutdown_connections 2023-10-18T10:46:17.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7fa8867fc640 1 -- 172.21.15.89:0/2729892601 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8800a59e0 con 0x7fa8880b5690 2023-10-18T10:46:17.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7fa8867fc640 1 -- 172.21.15.89:0/2729892601 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8800a5b40 con 0x7fa8880b5690 2023-10-18T10:46:17.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7fa8867fc640 1 -- 172.21.15.89:0/2729892601 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8800ad440 con 0x7fa8880b5690 2023-10-18T10:46:17.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7f1979f05640 1 -- 172.21.15.89:0/2060516543 wait complete. 2023-10-18T10:46:17.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7f1dd28ca640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1dcc0d0ca0 0x7f1dcc153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7fa88c89e640 1 -- 172.21.15.89:0/2729892601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8880b5690 msgr2=0x7fa8880b5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7fa88c89e640 1 --2- 172.21.15.89:0/2729892601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8880b5690 0x7fa8880b5b20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa880009d90 tx=0x7fa88001c940 comp rx=0 tx=0).stop 2023-10-18T10:46:17.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7f1dd28ca640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1dcc0b57f0 0x7f1dcc0c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7f1dd28ca640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dcc0b5c50 0x7f1dcc0c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7f1dd28ca640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc153af0 con 0x7f1dcc0b5c50 2023-10-18T10:46:17.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7fa88c89e640 1 -- 172.21.15.89:0/2729892601 shutdown_connections 2023-10-18T10:46:17.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7fa88c89e640 1 --2- 172.21.15.89:0/2729892601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8880b5690 0x7fa8880b5b20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7fa88c89e640 1 --2- 172.21.15.89:0/2729892601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8880b3f10 0x7fa8880b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.094+0000 7fa88c89e640 1 -- 172.21.15.89:0/2729892601 >> 172.21.15.89:0/2729892601 conn(0x7fa8880cef10 msgr2=0x7fa8880cf320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:17.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7fa88c89e640 1 -- 172.21.15.89:0/2729892601 shutdown_connections 2023-10-18T10:46:17.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7fa88c89e640 1 -- 172.21.15.89:0/2729892601 wait complete. 2023-10-18T10:46:17.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7f1dc37fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1dcc0b57f0 0x7f1dcc0c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7f1dcbfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1dcc0d0ca0 0x7f1dcc153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7f1dc37fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1dcc0b57f0 0x7f1dcc0c83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:37370/0 (socket says 172.21.15.89:37370) 2023-10-18T10:46:17.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7f1dc37fe640 1 -- 172.21.15.89:0/1367012935 learned_addr learned my addr 172.21.15.89:0/1367012935 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:17.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7f1dd28ca640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc0c8f60 con 0x7f1dcc0d0ca0 2023-10-18T10:46:17.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7f1dd28ca640 1 -- 172.21.15.89:0/1367012935 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc0c9240 con 0x7f1dcc0b57f0 2023-10-18T10:46:17.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7f92d98c8640 1 --2- 172.21.15.89:0/86409863 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f92b8065750 0x7f92b8067c10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f92d0007ca0 tx=0x7f92d0007bf0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:17.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7f1dd0e67640 1 --2- 172.21.15.89:0/1367012935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dcc0b5c50 0x7f1dcc0c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7f5b7666a640 1 Processor -- start 2023-10-18T10:46:17.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7f1dcbfff640 1 -- 172.21.15.89:0/1367012935 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1dcc0b57f0 msgr2=0x7f1dcc0c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7f1dcbfff640 1 --2- 172.21.15.89:0/1367012935 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1dcc0b57f0 0x7f1dcc0c83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7f1dcbfff640 1 -- 172.21.15.89:0/1367012935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dcc0b5c50 msgr2=0x7f1dcc0c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7f1dcbfff640 1 --2- 172.21.15.89:0/1367012935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dcc0b5c50 0x7f1dcc0c88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7f1dcbfff640 1 -- 172.21.15.89:0/1367012935 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dbc0094d0 con 0x7f1dcc0d0ca0 2023-10-18T10:46:17.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7f0423fff640 1 -- 172.21.15.89:0/1664031242 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f042c01c460 con 0x7f043806c630 2023-10-18T10:46:17.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7f5b7666a640 1 -- start start 2023-10-18T10:46:17.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7f1979f05640 1 Processor -- start 2023-10-18T10:46:17.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7f1979f05640 1 -- start start 2023-10-18T10:46:17.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7f92ca7fc640 1 -- 172.21.15.89:0/86409863 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f92cc0512a0 con 0x7f92d415c560 2023-10-18T10:46:17.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7f1dcbfff640 1 --2- 172.21.15.89:0/1367012935 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1dcc0d0ca0 0x7f1dcc153340 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc0ab000 tx=0x7f1dbc018fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:17.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7f0423fff640 1 --2- 172.21.15.89:0/1664031242 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f041c065750 0x7f041c067c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7f0423fff640 1 -- 172.21.15.89:0/1664031242 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f042c082e60 con 0x7f043806c630 2023-10-18T10:46:17.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7fa88c89e640 1 Processor -- start 2023-10-18T10:46:17.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7f5b7666a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b700cbee0 0x7f5b70154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7f5b7666a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b700cc9d0 0x7f5b701584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7f5b7666a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b70160e10 0x7f5b70153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7f5b7666a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b70154450 con 0x7f5b700cbee0 2023-10-18T10:46:17.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7f5b7666a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b70157720 con 0x7f5b700cc9d0 2023-10-18T10:46:17.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7f5b7666a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b70157a00 con 0x7f5b70160e10 2023-10-18T10:46:17.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7f1dc97fa640 1 -- 172.21.15.89:0/1367012935 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc0b5600 con 0x7f1dcc0d0ca0 2023-10-18T10:46:17.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7f1dc97fa640 1 -- 172.21.15.89:0/1367012935 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dbc0abe90 con 0x7f1dcc0d0ca0 2023-10-18T10:46:17.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7f1dc97fa640 1 -- 172.21.15.89:0/1367012935 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc0b59a0 con 0x7f1dcc0d0ca0 2023-10-18T10:46:17.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7f1dd28ca640 1 -- 172.21.15.89:0/1367012935 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dcc153d00 con 0x7f1dcc0d0ca0 2023-10-18T10:46:17.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7f1dd28ca640 1 -- 172.21.15.89:0/1367012935 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dcc153880 con 0x7f1dcc0d0ca0 2023-10-18T10:46:17.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7f04377fe640 1 --2- 172.21.15.89:0/1664031242 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f041c065750 0x7f041c067c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f1979f05640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19740cd1d0 0x7f1974152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f5b6f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b700cc9d0 0x7f5b701584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f04377fe640 1 --2- 172.21.15.89:0/1664031242 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f041c065750 0x7f041c067c10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0430000bb0 tx=0x7f043000b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:17.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f1979f05640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f197415b060 0x7f19740cb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f1979f05640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1974165390 0x7f19740c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f5b74c07640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b70160e10 0x7f5b70153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f5b74c07640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b70160e10 0x7f5b70153c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:37376/0 (socket says 172.21.15.89:37376) 2023-10-18T10:46:17.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f5b74c07640 1 -- 172.21.15.89:0/2806537723 learned_addr learned my addr 172.21.15.89:0/2806537723 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:17.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f1979f05640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197415a720 con 0x7f1974165390 2023-10-18T10:46:17.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f1979f05640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19740c8aa0 con 0x7f197415b060 2023-10-18T10:46:17.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f1979f05640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f197415a160 con 0x7f19740cd1d0 2023-10-18T10:46:17.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f1978f03640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19740cd1d0 0x7f1974152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f1973fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f197415b060 0x7f19740cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.098+0000 7fa88c89e640 1 -- start start 2023-10-18T10:46:17.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7fa88c89e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8880b3f10 0x7fa888151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7fa88c89e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8880b5690 0x7fa88814ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7fa88c89e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa88815c570 0x7fa88814c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7fa88c89e640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88814d830 con 0x7fa8880b5690 2023-10-18T10:46:17.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7fa88c89e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88814f5d0 con 0x7fa88815c570 2023-10-18T10:46:17.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f5b6ffff640 1 --2- 172.21.15.89:0/2806537723 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b700cbee0 0x7f5b70154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7fa88c89e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa88814d170 con 0x7fa8880b3f10 2023-10-18T10:46:17.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f1978f03640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19740cd1d0 0x7f1974152870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:37382/0 (socket says 172.21.15.89:37382) 2023-10-18T10:46:17.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f1978f03640 1 -- 172.21.15.89:0/1412789726 learned_addr learned my addr 172.21.15.89:0/1412789726 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:17.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f1973fff640 1 -- 172.21.15.89:0/1412789726 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19740cd1d0 msgr2=0x7f1974152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f1973fff640 1 --2- 172.21.15.89:0/1412789726 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19740cd1d0 0x7f1974152870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f1973fff640 1 -- 172.21.15.89:0/1412789726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1974165390 msgr2=0x7f19740c5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f1973fff640 1 --2- 172.21.15.89:0/1412789726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1974165390 0x7f19740c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f1973fff640 1 -- 172.21.15.89:0/1412789726 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f195c0093b0 con 0x7f197415b060 2023-10-18T10:46:17.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f1978f03640 1 --2- 172.21.15.89:0/1412789726 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19740cd1d0 0x7f1974152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:17.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f1973fff640 1 --2- 172.21.15.89:0/1412789726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f197415b060 0x7f19740cb970 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1974161f20 tx=0x7f195c0190b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:17.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f5b74c07640 1 -- 172.21.15.89:0/2806537723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b700cc9d0 msgr2=0x7f5b701584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f5b74c07640 1 --2- 172.21.15.89:0/2806537723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b700cc9d0 0x7f5b701584e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f5b74c07640 1 -- 172.21.15.89:0/2806537723 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b700cbee0 msgr2=0x7f5b70154de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f5b74c07640 1 --2- 172.21.15.89:0/2806537723 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b700cbee0 0x7f5b70154de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f5b74c07640 1 -- 172.21.15.89:0/2806537723 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b64009420 con 0x7f5b70160e10 2023-10-18T10:46:17.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f5b6ffff640 1 --2- 172.21.15.89:0/2806537723 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b700cbee0 0x7f5b70154de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:17.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f1dd28ca640 1 -- 172.21.15.89:0/1367012935 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dcc04dbe0 con 0x7f1dcc0d0ca0 2023-10-18T10:46:17.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7fa887fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa88815c570 0x7fa88814c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7fa887fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa88815c570 0x7fa88814c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:34280/0 (socket says 172.21.15.89:34280) 2023-10-18T10:46:17.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7fa887fff640 1 -- 172.21.15.89:0/1675872758 learned_addr learned my addr 172.21.15.89:0/1675872758 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:17.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f19717fa640 1 -- 172.21.15.89:0/1412789726 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c0059a0 con 0x7f197415b060 2023-10-18T10:46:17.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f5b74c07640 1 --2- 172.21.15.89:0/2806537723 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b70160e10 0x7f5b70153c10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5b64009740 tx=0x7f5b64019190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:17.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f1979f05640 1 -- 172.21.15.89:0/1412789726 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f197415b5c0 con 0x7f197415b060 2023-10-18T10:46:17.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f1979f05640 1 -- 172.21.15.89:0/1412789726 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19740c8d80 con 0x7f197415b060 2023-10-18T10:46:17.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f0423fff640 1 -- 172.21.15.89:0/1664031242 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f042c0506a0 con 0x7f043806c630 2023-10-18T10:46:17.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f5b6cff9640 1 -- 172.21.15.89:0/2806537723 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b64003c70 con 0x7f5b70160e10 2023-10-18T10:46:17.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7fa8877fe640 1 --2- 172.21.15.89:0/1675872758 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8880b3f10 0x7fa888151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7fa887fff640 1 -- 172.21.15.89:0/1675872758 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8880b3f10 msgr2=0x7fa888151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7fa887fff640 1 --2- 172.21.15.89:0/1675872758 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8880b3f10 0x7fa888151fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7fa887fff640 1 -- 172.21.15.89:0/1675872758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8880b5690 msgr2=0x7fa88814ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f5b6cff9640 1 -- 172.21.15.89:0/2806537723 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b64003dd0 con 0x7f5b70160e10 2023-10-18T10:46:17.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f5b6cff9640 1 -- 172.21.15.89:0/2806537723 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b640b4640 con 0x7f5b70160e10 2023-10-18T10:46:17.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f1dc97fa640 1 -- 172.21.15.89:0/1367012935 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f1dbc0b5b00 con 0x7f1dcc0d0ca0 2023-10-18T10:46:17.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7fa887fff640 1 --2- 172.21.15.89:0/1675872758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8880b5690 0x7fa88814ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7fa887fff640 1 -- 172.21.15.89:0/1675872758 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa880009a70 con 0x7fa88815c570 2023-10-18T10:46:17.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7fa8877fe640 1 --2- 172.21.15.89:0/1675872758 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8880b3f10 0x7fa888151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:17.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f19717fa640 1 -- 172.21.15.89:0/1412789726 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195c005b00 con 0x7f197415b060 2023-10-18T10:46:17.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.102+0000 7f19717fa640 1 -- 172.21.15.89:0/1412789726 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c005e90 con 0x7f197415b060 2023-10-18T10:46:17.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.106+0000 7f1979f05640 1 -- 172.21.15.89:0/1412789726 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1974073b50 con 0x7f197415b060 2023-10-18T10:46:17.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.106+0000 7f1dc97fa640 1 --2- 172.21.15.89:0/1367012935 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1dac0656d0 0x7f1dac067b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.106+0000 7f1dc97fa640 1 -- 172.21.15.89:0/1367012935 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1dbc11d1d0 con 0x7f1dcc0d0ca0 2023-10-18T10:46:17.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.106+0000 7f1dc37fe640 1 --2- 172.21.15.89:0/1367012935 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1dac0656d0 0x7f1dac067b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.106+0000 7f5b7666a640 1 -- 172.21.15.89:0/2806537723 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b7006ddf0 con 0x7f5b70160e10 2023-10-18T10:46:17.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.106+0000 7f5b7666a640 1 -- 172.21.15.89:0/2806537723 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b70157fc0 con 0x7f5b70160e10 2023-10-18T10:46:17.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.106+0000 7f1dc37fe640 1 --2- 172.21.15.89:0/1367012935 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1dac0656d0 0x7f1dac067b90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc15fba0 tx=0x7f1db4009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:17.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.106+0000 7f5b7666a640 1 -- 172.21.15.89:0/2806537723 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b700ccec0 con 0x7f5b70160e10 2023-10-18T10:46:17.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.106+0000 7fa887fff640 1 --2- 172.21.15.89:0/1675872758 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa88815c570 0x7fa88814c4a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa88001aff0 tx=0x7fa8800196e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:17.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.106+0000 7f19717fa640 1 -- 172.21.15.89:0/1412789726 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f195c0b2040 con 0x7f197415b060 2023-10-18T10:46:17.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.110+0000 7f19717fa640 1 --2- 172.21.15.89:0/1412789726 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1954065680 0x7f1954067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.110+0000 7f19717fa640 1 -- 172.21.15.89:0/1412789726 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f195c11cf70 con 0x7f197415b060 2023-10-18T10:46:17.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.110+0000 7f1978f03640 1 --2- 172.21.15.89:0/1412789726 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1954065680 0x7f1954067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.110+0000 7f1dc97fa640 1 -- 172.21.15.89:0/1367012935 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f1dbc0eab10 con 0x7f1dcc0d0ca0 2023-10-18T10:46:17.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.110+0000 7f5b6cff9640 1 -- 172.21.15.89:0/2806537723 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f5b640b2040 con 0x7f5b70160e10 2023-10-18T10:46:17.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.110+0000 7f1978f03640 1 --2- 172.21.15.89:0/1412789726 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1954065680 0x7f1954067b40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f196c000c10 tx=0x7f196c007d70 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:17.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.110+0000 7f5b6cff9640 1 --2- 172.21.15.89:0/2806537723 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5b5c0656d0 0x7f5b5c067b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.110+0000 7f5b6cff9640 1 -- 172.21.15.89:0/2806537723 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5b6411cef0 con 0x7f5b70160e10 2023-10-18T10:46:17.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.110+0000 7f5b6ffff640 1 --2- 172.21.15.89:0/2806537723 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5b5c0656d0 0x7f5b5c067b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.114+0000 7f5b6ffff640 1 --2- 172.21.15.89:0/2806537723 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5b5c0656d0 0x7f5b5c067b90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5b7015e3c0 tx=0x7f5b680074c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:17.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.114+0000 7f5b6cff9640 1 -- 172.21.15.89:0/2806537723 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f5b640ea8c0 con 0x7f5b70160e10 2023-10-18T10:46:17.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.114+0000 7fa86ffff640 1 -- 172.21.15.89:0/1675872758 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8800aec30 con 0x7fa88815c570 2023-10-18T10:46:17.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.114+0000 7fa86ffff640 1 -- 172.21.15.89:0/1675872758 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8800aedb0 con 0x7fa88815c570 2023-10-18T10:46:17.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.114+0000 7fa86ffff640 1 -- 172.21.15.89:0/1675872758 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8800b7650 con 0x7fa88815c570 2023-10-18T10:46:17.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.118+0000 7fa88c89e640 1 -- 172.21.15.89:0/1675872758 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa88814d450 con 0x7fa88815c570 2023-10-18T10:46:17.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.118+0000 7f19717fa640 1 -- 172.21.15.89:0/1412789726 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f195c0ea730 con 0x7f197415b060 2023-10-18T10:46:17.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.118+0000 7fa88c89e640 1 -- 172.21.15.89:0/1675872758 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa88814b4b0 con 0x7fa88815c570 2023-10-18T10:46:17.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.118+0000 7fa88c89e640 1 -- 172.21.15.89:0/1675872758 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa88814d970 con 0x7fa88815c570 2023-10-18T10:46:17.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.122+0000 7fa86ffff640 1 -- 172.21.15.89:0/1675872758 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fa8800b4050 con 0x7fa88815c570 2023-10-18T10:46:17.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.122+0000 7fa86ffff640 1 --2- 172.21.15.89:0/1675872758 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa874065700 0x7fa874067bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:17.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.122+0000 7fa8877fe640 1 --2- 172.21.15.89:0/1675872758 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa874065700 0x7fa874067bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:17.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.122+0000 7fa86ffff640 1 -- 172.21.15.89:0/1675872758 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa8800efd60 con 0x7fa88815c570 2023-10-18T10:46:17.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.122+0000 7fa8877fe640 1 --2- 172.21.15.89:0/1675872758 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa874065700 0x7fa874067bc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa87c000f70 tx=0x7fa87c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:17.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.126+0000 7fa86ffff640 1 -- 172.21.15.89:0/1675872758 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fa8800ec1a0 con 0x7fa88815c570 2023-10-18T10:46:17.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.410+0000 7f92da8ca640 1 -- 172.21.15.89:0/86409863 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f92d4073b50 con 0x7f92d415c560 2023-10-18T10:46:17.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.410+0000 7fc1b6ffa640 1 -- 172.21.15.89:0/1307729326 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc1a80d9130 con 0x7fc1a8006ec0 2023-10-18T10:46:17.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.414+0000 7f92ca7fc640 1 -- 172.21.15.89:0/86409863 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f92cc0548c0 con 0x7f92d415c560 2023-10-18T10:46:17.414 INFO:teuthology.orchestra.run.smithi089.stdout:55834574856 2023-10-18T10:46:17.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.414+0000 7fc1ad7fa640 1 -- 172.21.15.89:0/1307729326 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc1a80d9130 con 0x7fc1a8006ec0 2023-10-18T10:46:17.417 INFO:teuthology.orchestra.run.smithi089.stdout:55834574856 2023-10-18T10:46:17.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.414+0000 7fc1b6ffa640 1 -- 172.21.15.89:0/1307729326 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc194065680 msgr2=0x7fc194067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.414+0000 7fc1b6ffa640 1 --2- 172.21.15.89:0/1307729326 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc194065680 0x7fc194067b40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc1b00684d0 tx=0x7fc1b008b040 comp rx=0 tx=0).stop 2023-10-18T10:46:17.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.414+0000 7fc1b6ffa640 1 -- 172.21.15.89:0/1307729326 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1a8006ec0 msgr2=0x7fc1a817b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.414+0000 7fc1b6ffa640 1 --2- 172.21.15.89:0/1307729326 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1a8006ec0 0x7fc1a817b080 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc1a400bf00 tx=0x7fc1a400f480 comp rx=0 tx=0).stop 2023-10-18T10:46:17.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.418+0000 7fc1b6ffa640 1 -- 172.21.15.89:0/1307729326 shutdown_connections 2023-10-18T10:46:17.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.418+0000 7fc1b6ffa640 1 --2- 172.21.15.89:0/1307729326 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc194065680 0x7fc194067b40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.418+0000 7fc1b6ffa640 1 --2- 172.21.15.89:0/1307729326 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1a80df210 0x7fc1a8168a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.418+0000 7fc1b6ffa640 1 --2- 172.21.15.89:0/1307729326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1a80df5f0 0x7fc1a8168f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.418+0000 7fc1b6ffa640 1 --2- 172.21.15.89:0/1307729326 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1a8006ec0 0x7fc1a817b080 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.418+0000 7fc1b6ffa640 1 -- 172.21.15.89:0/1307729326 >> 172.21.15.89:0/1307729326 conn(0x7fc1a8055130 msgr2=0x7fc1a80539f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:17.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.418+0000 7fc1b6ffa640 1 -- 172.21.15.89:0/1307729326 shutdown_connections 2023-10-18T10:46:17.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.418+0000 7fc1b6ffa640 1 -- 172.21.15.89:0/1307729326 wait complete. 2023-10-18T10:46:17.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.418+0000 7f92da8ca640 1 -- 172.21.15.89:0/86409863 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f92b8065750 msgr2=0x7f92b8067c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.418+0000 7f92da8ca640 1 --2- 172.21.15.89:0/86409863 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f92b8065750 0x7f92b8067c10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f92d0007ca0 tx=0x7f92d0007bf0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.418+0000 7faf5b59e640 1 -- 172.21.15.89:0/3649717764 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7faf5c06c760 con 0x7faf5c158390 2023-10-18T10:46:17.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.418+0000 7f92da8ca640 1 -- 172.21.15.89:0/86409863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92d415c560 msgr2=0x7f92d41f7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.418+0000 7f92da8ca640 1 --2- 172.21.15.89:0/86409863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92d415c560 0x7f92d41f7e70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f92cc00cd70 tx=0x7f92cc00a660 comp rx=0 tx=0).stop 2023-10-18T10:46:17.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.418+0000 7f92da8ca640 1 -- 172.21.15.89:0/86409863 shutdown_connections 2023-10-18T10:46:17.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.418+0000 7f92da8ca640 1 --2- 172.21.15.89:0/86409863 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f92b8065750 0x7f92b8067c10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.418+0000 7f92da8ca640 1 --2- 172.21.15.89:0/86409863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92d415c560 0x7f92d41f7e70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.418+0000 7f92da8ca640 1 --2- 172.21.15.89:0/86409863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92d40d1e70 0x7f92d40b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.418+0000 7f92da8ca640 1 --2- 172.21.15.89:0/86409863 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92d40d0050 0x7f92d40b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.418+0000 7f92da8ca640 1 -- 172.21.15.89:0/86409863 >> 172.21.15.89:0/86409863 conn(0x7f92d40d2510 msgr2=0x7f92d415b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:17.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.418+0000 7f92da8ca640 1 -- 172.21.15.89:0/86409863 shutdown_connections 2023-10-18T10:46:17.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.418+0000 7f92da8ca640 1 -- 172.21.15.89:0/86409863 wait complete. 2023-10-18T10:46:17.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.418+0000 7faf4affd640 1 -- 172.21.15.89:0/3649717764 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faf4c053c50 con 0x7faf5c158390 2023-10-18T10:46:17.424 INFO:teuthology.orchestra.run.smithi089.stdout:55834574856 2023-10-18T10:46:17.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.422+0000 7faf5b59e640 1 -- 172.21.15.89:0/3649717764 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7faf3c065750 msgr2=0x7faf3c067c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.422+0000 7faf5b59e640 1 --2- 172.21.15.89:0/3649717764 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7faf3c065750 0x7faf3c067c10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7faf5400f3c0 tx=0x7faf54003e70 comp rx=0 tx=0).stop 2023-10-18T10:46:17.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.422+0000 7faf5b59e640 1 -- 172.21.15.89:0/3649717764 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf5c158390 msgr2=0x7faf5c14c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.422+0000 7faf5b59e640 1 --2- 172.21.15.89:0/3649717764 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf5c158390 0x7faf5c14c330 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7faf4c00b3c0 tx=0x7faf4c00b880 comp rx=0 tx=0).stop 2023-10-18T10:46:17.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.422+0000 7faf5b59e640 1 -- 172.21.15.89:0/3649717764 shutdown_connections 2023-10-18T10:46:17.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.422+0000 7faf5b59e640 1 --2- 172.21.15.89:0/3649717764 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7faf3c065750 0x7faf3c067c10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.422+0000 7faf5b59e640 1 --2- 172.21.15.89:0/3649717764 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faf5c158390 0x7faf5c14c330 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.422+0000 7faf5b59e640 1 --2- 172.21.15.89:0/3649717764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf5c158810 0x7faf5c14c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.422+0000 7faf5b59e640 1 --2- 172.21.15.89:0/3649717764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf5c0cb660 0x7faf5c0c15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.422+0000 7faf5b59e640 1 -- 172.21.15.89:0/3649717764 >> 172.21.15.89:0/3649717764 conn(0x7faf5c0cab40 msgr2=0x7faf5c0acc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:17.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.426+0000 7faf5b59e640 1 -- 172.21.15.89:0/3649717764 shutdown_connections 2023-10-18T10:46:17.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.426+0000 7faf5b59e640 1 -- 172.21.15.89:0/3649717764 wait complete. 2023-10-18T10:46:17.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.430+0000 7f043dffa640 1 -- 172.21.15.89:0/1664031242 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f0438070170 con 0x7f043806c630 2023-10-18T10:46:17.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.430+0000 7f0423fff640 1 -- 172.21.15.89:0/1664031242 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0438070170 con 0x7f043806c630 2023-10-18T10:46:17.433 INFO:teuthology.orchestra.run.smithi089.stdout:55834574856 2023-10-18T10:46:17.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.434+0000 7f043dffa640 1 -- 172.21.15.89:0/1664031242 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f041c065750 msgr2=0x7f041c067c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.434+0000 7f043dffa640 1 --2- 172.21.15.89:0/1664031242 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f041c065750 0x7f041c067c10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0430000bb0 tx=0x7f043000b040 comp rx=0 tx=0).stop 2023-10-18T10:46:17.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.434+0000 7f043dffa640 1 -- 172.21.15.89:0/1664031242 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f043806c630 msgr2=0x7f0438070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.434+0000 7f043dffa640 1 --2- 172.21.15.89:0/1664031242 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f043806c630 0x7f0438070df0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f042c00c450 tx=0x7f042c00c910 comp rx=0 tx=0).stop 2023-10-18T10:46:17.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.434+0000 7f1dd28ca640 1 -- 172.21.15.89:0/1367012935 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1dcc0b85d0 con 0x7f1dcc0d0ca0 2023-10-18T10:46:17.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.434+0000 7f043dffa640 1 -- 172.21.15.89:0/1664031242 shutdown_connections 2023-10-18T10:46:17.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.434+0000 7f043dffa640 1 --2- 172.21.15.89:0/1664031242 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f041c065750 0x7f041c067c10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.434+0000 7f043dffa640 1 --2- 172.21.15.89:0/1664031242 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f043806c630 0x7f0438070df0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.434+0000 7f043dffa640 1 --2- 172.21.15.89:0/1664031242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04380ad2b0 0x7f0438151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.434+0000 7f043dffa640 1 --2- 172.21.15.89:0/1664031242 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f043806c1a0 0x7f0438153290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.434+0000 7f043dffa640 1 -- 172.21.15.89:0/1664031242 >> 172.21.15.89:0/1664031242 conn(0x7f04380ca9e0 msgr2=0x7f043815a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:17.440 INFO:teuthology.orchestra.run.smithi089.stdout:55834574857 2023-10-18T10:46:17.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.434+0000 7f1dc97fa640 1 -- 172.21.15.89:0/1367012935 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1dbc0ee130 con 0x7f1dcc0d0ca0 2023-10-18T10:46:17.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.438+0000 7f5b7666a640 1 -- 172.21.15.89:0/2806537723 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f5b7004dbe0 con 0x7f5b70160e10 2023-10-18T10:46:17.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.438+0000 7f043dffa640 1 -- 172.21.15.89:0/1664031242 shutdown_connections 2023-10-18T10:46:17.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.438+0000 7f043dffa640 1 -- 172.21.15.89:0/1664031242 wait complete. 2023-10-18T10:46:17.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.438+0000 7f1dd28ca640 1 -- 172.21.15.89:0/1367012935 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1dac0656d0 msgr2=0x7f1dac067b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.438+0000 7f1dd28ca640 1 --2- 172.21.15.89:0/1367012935 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1dac0656d0 0x7f1dac067b90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc15fba0 tx=0x7f1db4009190 comp rx=0 tx=0).stop 2023-10-18T10:46:17.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.438+0000 7f1dd28ca640 1 -- 172.21.15.89:0/1367012935 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1dcc0d0ca0 msgr2=0x7f1dcc153340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.438+0000 7f1dd28ca640 1 --2- 172.21.15.89:0/1367012935 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1dcc0d0ca0 0x7f1dcc153340 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc0ab000 tx=0x7f1dbc018fd0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.438+0000 7f5b6cff9640 1 -- 172.21.15.89:0/2806537723 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5b640b0090 con 0x7f5b70160e10 2023-10-18T10:46:17.443 INFO:teuthology.orchestra.run.smithi089.stdout:55834574856 2023-10-18T10:46:17.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.438+0000 7f1dd28ca640 1 -- 172.21.15.89:0/1367012935 shutdown_connections 2023-10-18T10:46:17.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.438+0000 7f1dd28ca640 1 --2- 172.21.15.89:0/1367012935 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1dac0656d0 0x7f1dac067b90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.438+0000 7f1dd28ca640 1 --2- 172.21.15.89:0/1367012935 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1dcc0b57f0 0x7f1dcc0c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.438+0000 7f1dd28ca640 1 --2- 172.21.15.89:0/1367012935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dcc0b5c50 0x7f1dcc0c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.438+0000 7f1dd28ca640 1 --2- 172.21.15.89:0/1367012935 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1dcc0d0ca0 0x7f1dcc153340 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.438+0000 7f1dd28ca640 1 -- 172.21.15.89:0/1367012935 >> 172.21.15.89:0/1367012935 conn(0x7f1dcc089fd0 msgr2=0x7f1dcc15dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:17.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.442+0000 7f1979f05640 1 -- 172.21.15.89:0/1412789726 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f19740b1790 con 0x7f197415b060 2023-10-18T10:46:17.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.442+0000 7f1dd28ca640 1 -- 172.21.15.89:0/1367012935 shutdown_connections 2023-10-18T10:46:17.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.442+0000 7f19717fa640 1 -- 172.21.15.89:0/1412789726 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f195c0b8020 con 0x7f197415b060 2023-10-18T10:46:17.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.442+0000 7f1dd28ca640 1 -- 172.21.15.89:0/1367012935 wait complete. 2023-10-18T10:46:17.447 INFO:teuthology.orchestra.run.smithi089.stdout:55834574855 2023-10-18T10:46:17.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.446+0000 7f5b7666a640 1 -- 172.21.15.89:0/2806537723 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5b5c0656d0 msgr2=0x7f5b5c067b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.446+0000 7f5b7666a640 1 --2- 172.21.15.89:0/2806537723 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5b5c0656d0 0x7f5b5c067b90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5b7015e3c0 tx=0x7f5b680074c0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.446+0000 7f1979f05640 1 -- 172.21.15.89:0/1412789726 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1954065680 msgr2=0x7f1954067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.446+0000 7f1979f05640 1 --2- 172.21.15.89:0/1412789726 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1954065680 0x7f1954067b40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f196c000c10 tx=0x7f196c007d70 comp rx=0 tx=0).stop 2023-10-18T10:46:17.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.446+0000 7f1979f05640 1 -- 172.21.15.89:0/1412789726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f197415b060 msgr2=0x7f19740cb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.446+0000 7f1979f05640 1 --2- 172.21.15.89:0/1412789726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f197415b060 0x7f19740cb970 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1974161f20 tx=0x7f195c0190b0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.446+0000 7f5b7666a640 1 -- 172.21.15.89:0/2806537723 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b70160e10 msgr2=0x7f5b70153c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.446+0000 7f5b7666a640 1 --2- 172.21.15.89:0/2806537723 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b70160e10 0x7f5b70153c10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5b64009740 tx=0x7f5b64019190 comp rx=0 tx=0).stop 2023-10-18T10:46:17.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.446+0000 7f5b7666a640 1 -- 172.21.15.89:0/2806537723 shutdown_connections 2023-10-18T10:46:17.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.446+0000 7f5b7666a640 1 --2- 172.21.15.89:0/2806537723 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5b5c0656d0 0x7f5b5c067b90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.446+0000 7f5b7666a640 1 --2- 172.21.15.89:0/2806537723 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b70160e10 0x7f5b70153c10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.446+0000 7f5b7666a640 1 --2- 172.21.15.89:0/2806537723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b700cc9d0 0x7f5b701584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.446+0000 7f5b7666a640 1 --2- 172.21.15.89:0/2806537723 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b700cbee0 0x7f5b70154de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.446+0000 7f5b7666a640 1 -- 172.21.15.89:0/2806537723 >> 172.21.15.89:0/2806537723 conn(0x7f5b700afe00 msgr2=0x7f5b70162330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:17.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.446+0000 7f5b7666a640 1 -- 172.21.15.89:0/2806537723 shutdown_connections 2023-10-18T10:46:17.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.446+0000 7f1979f05640 1 -- 172.21.15.89:0/1412789726 shutdown_connections 2023-10-18T10:46:17.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.446+0000 7f1979f05640 1 --2- 172.21.15.89:0/1412789726 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1954065680 0x7f1954067b40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.446+0000 7f1979f05640 1 --2- 172.21.15.89:0/1412789726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1974165390 0x7f19740c5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.446+0000 7f1979f05640 1 --2- 172.21.15.89:0/1412789726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f197415b060 0x7f19740cb970 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.446+0000 7f1979f05640 1 --2- 172.21.15.89:0/1412789726 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19740cd1d0 0x7f1974152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.446+0000 7f1979f05640 1 -- 172.21.15.89:0/1412789726 >> 172.21.15.89:0/1412789726 conn(0x7f19740ceef0 msgr2=0x7f19741535f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:17.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.446+0000 7f5b7666a640 1 -- 172.21.15.89:0/2806537723 wait complete. 2023-10-18T10:46:17.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.446+0000 7f1979f05640 1 -- 172.21.15.89:0/1412789726 shutdown_connections 2023-10-18T10:46:17.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.446+0000 7f1979f05640 1 -- 172.21.15.89:0/1412789726 wait complete. 2023-10-18T10:46:17.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.454+0000 7fa88c89e640 1 -- 172.21.15.89:0/1675872758 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fa88814d970 con 0x7fa88815c570 2023-10-18T10:46:17.459 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.1 2023-10-18T10:46:17.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.458+0000 7fa86ffff640 1 -- 172.21.15.89:0/1675872758 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa8800b2070 con 0x7fa88815c570 2023-10-18T10:46:17.459 INFO:teuthology.orchestra.run.smithi089.stdout:55834574856 2023-10-18T10:46:17.462 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.0 2023-10-18T10:46:17.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.462+0000 7fa88c89e640 1 -- 172.21.15.89:0/1675872758 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa874065700 msgr2=0x7fa874067bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.462+0000 7fa88c89e640 1 --2- 172.21.15.89:0/1675872758 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa874065700 0x7fa874067bc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa87c000f70 tx=0x7fa87c009040 comp rx=0 tx=0).stop 2023-10-18T10:46:17.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.462+0000 7fa88c89e640 1 -- 172.21.15.89:0/1675872758 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa88815c570 msgr2=0x7fa88814c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:17.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.462+0000 7fa88c89e640 1 --2- 172.21.15.89:0/1675872758 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa88815c570 0x7fa88814c4a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa88001aff0 tx=0x7fa8800196e0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.462+0000 7fa88c89e640 1 -- 172.21.15.89:0/1675872758 shutdown_connections 2023-10-18T10:46:17.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.462+0000 7fa88c89e640 1 --2- 172.21.15.89:0/1675872758 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa874065700 0x7fa874067bc0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.462+0000 7fa88c89e640 1 --2- 172.21.15.89:0/1675872758 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa88815c570 0x7fa88814c4a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.462+0000 7fa88c89e640 1 --2- 172.21.15.89:0/1675872758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8880b5690 0x7fa88814ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.462+0000 7fa88c89e640 1 --2- 172.21.15.89:0/1675872758 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8880b3f10 0x7fa888151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:17.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.462+0000 7fa88c89e640 1 -- 172.21.15.89:0/1675872758 >> 172.21.15.89:0/1675872758 conn(0x7fa8880cef10 msgr2=0x7fa888159280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:17.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.462+0000 7fa88c89e640 1 -- 172.21.15.89:0/1675872758 shutdown_connections 2023-10-18T10:46:17.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:17.462+0000 7fa88c89e640 1 -- 172.21.15.89:0/1675872758 wait complete. 2023-10-18T10:46:17.466 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.4 2023-10-18T10:46:17.480 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.2 2023-10-18T10:46:17.483 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.7 2023-10-18T10:46:17.484 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.3 2023-10-18T10:46:17.484 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.5 2023-10-18T10:46:17.492 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.6 2023-10-18T10:46:17.860 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:46:17.865 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:46:18.459 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-18T10:46:18.463 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-18T10:46:18.468 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-18T10:46:18.481 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-18T10:46:18.484 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-18T10:46:18.484 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-18T10:46:18.485 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-18T10:46:18.493 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-18T10:46:18.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.674+0000 7ff75cb2d640 1 Processor -- start 2023-10-18T10:46:18.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.674+0000 7ff75cb2d640 1 -- start start 2023-10-18T10:46:18.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.678+0000 7ff75cb2d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7581587e0 0x7ff75806fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.678+0000 7ff75cb2d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff7580cd680 0x7ff7580c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.678+0000 7ff75cb2d640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff75806bdb0 con 0x7ff7580701e0 2023-10-18T10:46:18.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.678+0000 7ff75cb2d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff75806de20 con 0x7ff7580cd680 2023-10-18T10:46:18.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.678+0000 7ff75cb2d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7580cddf0 con 0x7ff7581587e0 2023-10-18T10:46:18.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.678+0000 7ff7577fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7581587e0 0x7ff75806fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.678+0000 7ff7577fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7581587e0 0x7ff75806fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:37400/0 (socket says 172.21.15.89:37400) 2023-10-18T10:46:18.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.678+0000 7ff7577fe640 1 -- 172.21.15.89:0/4127834485 learned_addr learned my addr 172.21.15.89:0/4127834485 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:18.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.678+0000 7ff756ffd640 1 --2- 172.21.15.89:0/4127834485 >> v2:172.21.15.154:3300/0 conn(0x7ff7580cd680 0x7ff7580c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.678+0000 7ff7577fe640 1 -- 172.21.15.89:0/4127834485 >> v2:172.21.15.154:3300/0 conn(0x7ff7580cd680 msgr2=0x7ff7580c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:18.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.678+0000 7ff7577fe640 1 --2- 172.21.15.89:0/4127834485 >> v2:172.21.15.154:3300/0 conn(0x7ff7580cd680 0x7ff7580c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.678+0000 7ff7577fe640 1 -- 172.21.15.89:0/4127834485 >> v1:172.21.15.89:6789/0 conn(0x7ff7580701e0 legacy=0x7ff75806cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:18.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.678+0000 7ff7577fe640 1 -- 172.21.15.89:0/4127834485 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75806bfe0 con 0x7ff7581587e0 2023-10-18T10:46:18.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.690+0000 7ff7577fe640 1 --2- 172.21.15.89:0/4127834485 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7581587e0 0x7ff75806fc50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff748009870 tx=0x7ff74801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca4022b7c8514f7d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:18.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.690+0000 7ff7567fc640 1 -- 172.21.15.89:0/4127834485 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7480ad640 con 0x7ff7581587e0 2023-10-18T10:46:18.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.690+0000 7ff7567fc640 1 -- 172.21.15.89:0/4127834485 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7480ad7a0 con 0x7ff7581587e0 2023-10-18T10:46:18.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.690+0000 7ff7567fc640 1 -- 172.21.15.89:0/4127834485 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7480adb30 con 0x7ff7581587e0 2023-10-18T10:46:18.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.690+0000 7ff75cb2d640 1 -- 172.21.15.89:0/4127834485 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7581587e0 msgr2=0x7ff75806fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:18.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.690+0000 7ff75cb2d640 1 --2- 172.21.15.89:0/4127834485 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7581587e0 0x7ff75806fc50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff748009870 tx=0x7ff74801c480 comp rx=0 tx=0).stop 2023-10-18T10:46:18.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.690+0000 7ff75cb2d640 1 -- 172.21.15.89:0/4127834485 shutdown_connections 2023-10-18T10:46:18.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.690+0000 7ff75cb2d640 1 --2- 172.21.15.89:0/4127834485 >> v2:172.21.15.154:3300/0 conn(0x7ff7580cd680 0x7ff7580c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.690+0000 7ff75cb2d640 1 --2- 172.21.15.89:0/4127834485 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7581587e0 0x7ff75806fc50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.690+0000 7ff75cb2d640 1 -- 172.21.15.89:0/4127834485 >> 172.21.15.89:0/4127834485 conn(0x7ff7580cab10 msgr2=0x7ff7580caf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:18.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.690+0000 7ff75cb2d640 1 -- 172.21.15.89:0/4127834485 shutdown_connections 2023-10-18T10:46:18.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.690+0000 7ff75cb2d640 1 -- 172.21.15.89:0/4127834485 wait complete. 2023-10-18T10:46:18.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.694+0000 7ff75cb2d640 1 Processor -- start 2023-10-18T10:46:18.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.694+0000 7ff75cb2d640 1 -- start start 2023-10-18T10:46:18.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.694+0000 7ff75cb2d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7580701e0 0x7ff758148070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.698+0000 7ff75cb2d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7580cd680 0x7ff75814f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.698+0000 7ff75cb2d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7581587e0 0x7ff75814fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.698+0000 7ff75cb2d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7580b2460 con 0x7ff7580701e0 2023-10-18T10:46:18.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.698+0000 7ff75cb2d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7580c5200 con 0x7ff7580cd680 2023-10-18T10:46:18.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.698+0000 7ff75cb2d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7580c54e0 con 0x7ff7581587e0 2023-10-18T10:46:18.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.698+0000 7ff7577fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7580701e0 0x7ff758148070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.698+0000 7ff7577fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7580701e0 0x7ff758148070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45396/0 (socket says 172.21.15.89:45396) 2023-10-18T10:46:18.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.698+0000 7ff7577fe640 1 -- 172.21.15.89:0/4211970223 learned_addr learned my addr 172.21.15.89:0/4211970223 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:18.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.698+0000 7ff757fff640 1 --2- 172.21.15.89:0/4211970223 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7581587e0 0x7ff75814fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.698+0000 7ff756ffd640 1 --2- 172.21.15.89:0/4211970223 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7580cd680 0x7ff75814f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.698+0000 7ff7577fe640 1 -- 172.21.15.89:0/4211970223 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7581587e0 msgr2=0x7ff75814fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:18.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.698+0000 7ff7577fe640 1 --2- 172.21.15.89:0/4211970223 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7581587e0 0x7ff75814fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.698+0000 7ff7577fe640 1 -- 172.21.15.89:0/4211970223 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7580cd680 msgr2=0x7ff75814f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:18.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.698+0000 7ff7577fe640 1 --2- 172.21.15.89:0/4211970223 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7580cd680 0x7ff75814f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.698+0000 7ff7577fe640 1 -- 172.21.15.89:0/4211970223 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7480094d0 con 0x7ff7580701e0 2023-10-18T10:46:18.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.702+0000 7ff7577fe640 1 --2- 172.21.15.89:0/4211970223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7580701e0 0x7ff758148070 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff748019090 tx=0x7ff748019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:18.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.702+0000 7ff73ffff640 1 -- 172.21.15.89:0/4211970223 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7480b7640 con 0x7ff7580701e0 2023-10-18T10:46:18.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.702+0000 7ff73ffff640 1 -- 172.21.15.89:0/4211970223 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7480a6040 con 0x7ff7580701e0 2023-10-18T10:46:18.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.702+0000 7ff73ffff640 1 -- 172.21.15.89:0/4211970223 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7480b79c0 con 0x7ff7580701e0 2023-10-18T10:46:18.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.702+0000 7ff75cb2d640 1 -- 172.21.15.89:0/4211970223 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7580c57c0 con 0x7ff7580701e0 2023-10-18T10:46:18.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.702+0000 7ff75cb2d640 1 -- 172.21.15.89:0/4211970223 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7581485b0 con 0x7ff7580701e0 2023-10-18T10:46:18.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.702+0000 7ff75cb2d640 1 -- 172.21.15.89:0/4211970223 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff758073b50 con 0x7ff7580701e0 2023-10-18T10:46:18.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.706+0000 7ff73ffff640 1 -- 172.21.15.89:0/4211970223 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7ff7480b7b20 con 0x7ff7580701e0 2023-10-18T10:46:18.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.706+0000 7ff73ffff640 1 --2- 172.21.15.89:0/4211970223 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff7440656d0 0x7ff744067b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.706+0000 7ff756ffd640 1 --2- 172.21.15.89:0/4211970223 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff7440656d0 0x7ff744067b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.710+0000 7ff756ffd640 1 --2- 172.21.15.89:0/4211970223 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff7440656d0 0x7ff744067b90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff7580c1590 tx=0x7ff74c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:18.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.714+0000 7ff73ffff640 1 -- 172.21.15.89:0/4211970223 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff7480ef160 con 0x7ff7580701e0 2023-10-18T10:46:18.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.714+0000 7ff73ffff640 1 -- 172.21.15.89:0/4211970223 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7ff7480eae10 con 0x7ff7580701e0 2023-10-18T10:46:18.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.750+0000 7fe1b4d34640 1 Processor -- start 2023-10-18T10:46:18.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.750+0000 7fe1b4d34640 1 -- start start 2023-10-18T10:46:18.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.750+0000 7fe1b4d34640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1b015a4e0 0x7fe1b00b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.750+0000 7fe1b4d34640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe1b00cb660 0x7fe1b0071740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.750+0000 7fe1b4d34640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1b00cc9d0 con 0x7fe1b006cbf0 2023-10-18T10:46:18.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.750+0000 7fe1b4d34640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1b00cbaa0 con 0x7fe1b00cb660 2023-10-18T10:46:18.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.750+0000 7fe1b4d34640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b0071c80 con 0x7fe1b015a4e0 2023-10-18T10:46:18.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.750+0000 7fe1a6ffd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe1b00cb660 0x7fe1b0071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.750+0000 7fe1a6ffd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe1b00cb660 0x7fe1b0071740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45408/0 (socket says 172.21.15.89:45408) 2023-10-18T10:46:18.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.750+0000 7fe1a6ffd640 1 -- 172.21.15.89:0/3905653328 learned_addr learned my addr 172.21.15.89:0/3905653328 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:18.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.754+0000 7fe1aeffd640 1 -- 172.21.15.89:0/3905653328 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2420041356 0 0) 0x7fe1b00cc9d0 con 0x7fe1b006cbf0 2023-10-18T10:46:18.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.754+0000 7fe1aeffd640 1 -- 172.21.15.89:0/3905653328 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe19c003530 con 0x7fe1b006cbf0 2023-10-18T10:46:18.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.754+0000 7fe1a6ffd640 1 -- 172.21.15.89:0/3905653328 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1b015a4e0 msgr2=0x7fe1b00b0f30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-18T10:46:18.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.754+0000 7fe1a6ffd640 1 --2- 172.21.15.89:0/3905653328 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1b015a4e0 0x7fe1b00b0f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.754+0000 7fe1a6ffd640 1 -- 172.21.15.89:0/3905653328 >> v1:172.21.15.154:6789/0 conn(0x7fe1b006cbf0 legacy=0x7fe1b00cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:18.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.754+0000 7fe1a6ffd640 1 -- 172.21.15.89:0/3905653328 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b00cbdc0 con 0x7fe1b00cb660 2023-10-18T10:46:18.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.758+0000 7f5ca018f640 1 Processor -- start 2023-10-18T10:46:18.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.758+0000 7f5ca018f640 1 -- start start 2023-10-18T10:46:18.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.758+0000 7fe1a6ffd640 1 --2- 172.21.15.89:0/3905653328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1b00cb660 0x7fe1b0071740 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe198009870 tx=0x7fe198020870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05be81da25b86d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:18.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.758+0000 7f6faf59e640 1 Processor -- start 2023-10-18T10:46:18.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.758+0000 7f96e759e640 1 Processor -- start 2023-10-18T10:46:18.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.758+0000 7f6faf59e640 1 -- start start 2023-10-18T10:46:18.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.758+0000 7f96e759e640 1 -- start start 2023-10-18T10:46:18.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.758+0000 7f5ca018f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c980c8bc0 0x7f5c980c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.758+0000 7f5ca018f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5c98160bb0 0x7f5c980ace30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.758+0000 7f5ca018f640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c980ac290 con 0x7f5c9806df60 2023-10-18T10:46:18.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.758+0000 7f5ca018f640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c9806cd10 con 0x7f5c98160bb0 2023-10-18T10:46:18.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.758+0000 7f5ca018f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9806cfb0 con 0x7f5c980c8bc0 2023-10-18T10:46:18.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.758+0000 7fe1aeffd640 1 -- 172.21.15.89:0/3905653328 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1980a95f0 con 0x7fe1b00cb660 2023-10-18T10:46:18.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.758+0000 7f5c9df2b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c980c8bc0 0x7f5c980c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.758+0000 7f5c9df2b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c980c8bc0 0x7f5c980c8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:37422/0 (socket says 172.21.15.89:37422) 2023-10-18T10:46:18.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.758+0000 7f5c9df2b640 1 -- 172.21.15.89:0/3538541474 learned_addr learned my addr 172.21.15.89:0/3538541474 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:18.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.758+0000 7fe1aeffd640 1 -- 172.21.15.89:0/3905653328 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1980a9750 con 0x7fe1b00cb660 2023-10-18T10:46:18.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.758+0000 7fe1aeffd640 1 -- 172.21.15.89:0/3905653328 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1980a9ae0 con 0x7fe1b00cb660 2023-10-18T10:46:18.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.762+0000 7f5c9e72c640 1 --2- 172.21.15.89:0/3538541474 >> v2:172.21.15.154:3300/0 conn(0x7f5c98160bb0 0x7f5c980ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.762+0000 7f5c9df2b640 1 -- 172.21.15.89:0/3538541474 >> v2:172.21.15.154:3300/0 conn(0x7f5c98160bb0 msgr2=0x7f5c980ace30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:18.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.762+0000 7f5c9df2b640 1 --2- 172.21.15.89:0/3538541474 >> v2:172.21.15.154:3300/0 conn(0x7f5c98160bb0 0x7f5c980ace30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.762+0000 7f5c9df2b640 1 -- 172.21.15.89:0/3538541474 >> v1:172.21.15.89:6789/0 conn(0x7f5c9806df60 legacy=0x7f5c980ac720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:18.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.762+0000 7f5c9df2b640 1 -- 172.21.15.89:0/3538541474 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c980ab960 con 0x7f5c980c8bc0 2023-10-18T10:46:18.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.762+0000 7f256df37640 1 Processor -- start 2023-10-18T10:46:18.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.762+0000 7f256df37640 1 -- start start 2023-10-18T10:46:18.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.762+0000 7f96e759e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f96e80cf950 0x7f96e80b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.762+0000 7f96e759e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f96e80cd4e0 0x7f96e80d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.762+0000 7f96e759e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96e80d1470 con 0x7f96e815c310 2023-10-18T10:46:18.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.762+0000 7f96e759e640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96e80d0f80 con 0x7f96e80cf950 2023-10-18T10:46:18.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.762+0000 7f96e759e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96e80cd990 con 0x7f96e80cd4e0 2023-10-18T10:46:18.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.762+0000 7f6faf59e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6fa8033d10 0x7f6fa80eb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.762+0000 7f6faf59e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6fa8047bd0 0x7f6fa8052ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.762+0000 7f6faf59e640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fa80ebda0 con 0x7f6fa8048200 2023-10-18T10:46:18.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.762+0000 7f6faf59e640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fa80ec020 con 0x7f6fa8047bd0 2023-10-18T10:46:18.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.762+0000 7f6faf59e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa804f200 con 0x7f6fa8033d10 2023-10-18T10:46:18.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.762+0000 7f96e5d9b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f96e80cf950 0x7f96e80b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.762+0000 7f96e6d9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f96e80cd4e0 0x7f96e80d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.762+0000 7f96e6d9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f96e80cd4e0 0x7f96e80d0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:34310/0 (socket says 172.21.15.89:34310) 2023-10-18T10:46:18.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.762+0000 7f96e6d9d640 1 -- 172.21.15.89:0/2247443488 learned_addr learned my addr 172.21.15.89:0/2247443488 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:18.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.762+0000 7f96e6d9d640 1 -- 172.21.15.89:0/2247443488 >> v2:172.21.15.89:3300/0 conn(0x7f96e80cf950 msgr2=0x7f96e80b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:18.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.762+0000 7f96e6d9d640 1 --2- 172.21.15.89:0/2247443488 >> v2:172.21.15.89:3300/0 conn(0x7f96e80cf950 0x7f96e80b53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.762+0000 7f96e6d9d640 1 -- 172.21.15.89:0/2247443488 >> v1:172.21.15.154:6789/0 conn(0x7f96e815c310 legacy=0x7f96e80cf240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-18T10:46:18.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.762+0000 7f96e6d9d640 1 -- 172.21.15.89:0/2247443488 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e80d1190 con 0x7f96e80cd4e0 2023-10-18T10:46:18.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.762+0000 7fe1b4d34640 1 -- 172.21.15.89:0/3905653328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1b00cb660 msgr2=0x7fe1b0071740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:18.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.762+0000 7fe1b4d34640 1 --2- 172.21.15.89:0/3905653328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1b00cb660 0x7fe1b0071740 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe198009870 tx=0x7fe198020870 comp rx=0 tx=0).stop 2023-10-18T10:46:18.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.766+0000 7f6fae59c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6fa8033d10 0x7f6fa80eb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.766+0000 7f6faed9d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6fa8047bd0 0x7f6fa8052ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.766+0000 7f6fadd9b640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6fa8048200 0x7f6fa80eb690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35674/0 (socket says 172.21.15.89:35674) 2023-10-18T10:46:18.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.766+0000 7f6fadd9b640 1 -- 172.21.15.89:0/1909541564 learned_addr learned my addr 172.21.15.89:0/1909541564 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:18.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.762+0000 7f256df37640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25680c94b0 0x7f25680afd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.762+0000 7f256df37640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f256815cc20 con 0x7f25680b02d0 2023-10-18T10:46:18.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.762+0000 7f256df37640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2568159b90 con 0x7f2568159530 2023-10-18T10:46:18.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.762+0000 7f256df37640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25680b0750 con 0x7f25680c94b0 2023-10-18T10:46:18.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.766+0000 7fe1b4d34640 1 -- 172.21.15.89:0/3905653328 shutdown_connections 2023-10-18T10:46:18.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.766+0000 7fe1b4d34640 1 --2- 172.21.15.89:0/3905653328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1b00cb660 0x7fe1b0071740 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.766+0000 7fe1b4d34640 1 --2- 172.21.15.89:0/3905653328 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1b015a4e0 0x7fe1b00b0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.766+0000 7fe1b4d34640 1 -- 172.21.15.89:0/3905653328 >> 172.21.15.89:0/3905653328 conn(0x7fe1b00cab40 msgr2=0x7fe1b00caf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:18.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.766+0000 7fe1b4d34640 1 -- 172.21.15.89:0/3905653328 shutdown_connections 2023-10-18T10:46:18.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.766+0000 7fe1b4d34640 1 -- 172.21.15.89:0/3905653328 wait complete. 2023-10-18T10:46:18.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.766+0000 7f6fae59c640 1 -- 172.21.15.89:0/1909541564 >> v2:172.21.15.89:3300/0 conn(0x7f6fa8047bd0 msgr2=0x7f6fa8052ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:18.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.766+0000 7f6fae59c640 1 --2- 172.21.15.89:0/1909541564 >> v2:172.21.15.89:3300/0 conn(0x7f6fa8047bd0 0x7f6fa8052ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.766+0000 7f6fae59c640 1 -- 172.21.15.89:0/1909541564 >> v1:172.21.15.89:6789/0 conn(0x7f6fa8048200 legacy=0x7f6fa80eb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:18.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.766+0000 7f25677fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25680c94b0 0x7f25680afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.766+0000 7f25677fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25680c94b0 0x7f25680afd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:37434/0 (socket says 172.21.15.89:37434) 2023-10-18T10:46:18.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.766+0000 7f25677fe640 1 -- 172.21.15.89:0/154892589 learned_addr learned my addr 172.21.15.89:0/154892589 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:18.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.766+0000 7f5c9df2b640 1 --2- 172.21.15.89:0/3538541474 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c980c8bc0 0x7f5c980c8fa0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5c88009870 tx=0x7f5c8801c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f5c583371d593dc server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:18.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.766+0000 7f25667fc640 1 -- 172.21.15.89:0/154892589 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 707662625 0 0) 0x7f256815cc20 con 0x7f25680b02d0 2023-10-18T10:46:18.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.766+0000 7f25667fc640 1 -- 172.21.15.89:0/154892589 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2550003530 con 0x7f25680b02d0 2023-10-18T10:46:18.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.766+0000 7f25677fe640 1 -- 172.21.15.89:0/154892589 >> v1:172.21.15.154:6789/0 conn(0x7f2568159530 legacy=0x7f256806f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:18.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.766+0000 7f25677fe640 1 -- 172.21.15.89:0/154892589 >> v1:172.21.15.89:6789/0 conn(0x7f25680b02d0 legacy=0x7f2568158e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:18.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.766+0000 7f25677fe640 1 -- 172.21.15.89:0/154892589 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f256815c2f0 con 0x7f25680c94b0 2023-10-18T10:46:18.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.766+0000 7fe1b4d34640 1 Processor -- start 2023-10-18T10:46:18.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.766+0000 7f6fae59c640 1 -- 172.21.15.89:0/1909541564 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa804f4e0 con 0x7f6fa8033d10 2023-10-18T10:46:18.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.766+0000 7f6faed9d640 1 --2- 172.21.15.89:0/1909541564 >> v2:172.21.15.89:3300/0 conn(0x7f6fa8047bd0 0x7f6fa8052ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:18.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.766+0000 7f5c9cf29640 1 -- 172.21.15.89:0/3538541474 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c880ac640 con 0x7f5c980c8bc0 2023-10-18T10:46:18.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7f5c9cf29640 1 -- 172.21.15.89:0/3538541474 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c880ac7a0 con 0x7f5c980c8bc0 2023-10-18T10:46:18.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7f5c9cf29640 1 -- 172.21.15.89:0/3538541474 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c880acb30 con 0x7f5c980c8bc0 2023-10-18T10:46:18.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7f5ca018f640 1 -- 172.21.15.89:0/3538541474 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c980c8bc0 msgr2=0x7f5c980c8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:18.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7f5ca018f640 1 --2- 172.21.15.89:0/3538541474 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c980c8bc0 0x7f5c980c8fa0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5c88009870 tx=0x7f5c8801c440 comp rx=0 tx=0).stop 2023-10-18T10:46:18.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7f96e6d9d640 1 --2- 172.21.15.89:0/2247443488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96e80cd4e0 0x7f96e80d0a40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f96e80ce6a0 tx=0x7f96e001c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7295e39752cc0e13 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:18.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7f5ca018f640 1 -- 172.21.15.89:0/3538541474 shutdown_connections 2023-10-18T10:46:18.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7f5ca018f640 1 --2- 172.21.15.89:0/3538541474 >> v2:172.21.15.154:3300/0 conn(0x7f5c98160bb0 0x7f5c980ace30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7f5ca018f640 1 --2- 172.21.15.89:0/3538541474 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c980c8bc0 0x7f5c980c8fa0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7f5ca018f640 1 -- 172.21.15.89:0/3538541474 >> 172.21.15.89:0/3538541474 conn(0x7f5c980afe00 msgr2=0x7f5c980b0210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:18.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7fe1b4d34640 1 -- start start 2023-10-18T10:46:18.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7fe1b4d34640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1b006cbf0 0x7fe1b0157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7fe1b4d34640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1b00cb660 0x7fe1b01fb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7fe1b4d34640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1b015a4e0 0x7fe1b02017d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7fe1b4d34640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b0202890 con 0x7fe1b006cbf0 2023-10-18T10:46:18.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7fe1b4d34640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b01fbce0 con 0x7fe1b015a4e0 2023-10-18T10:46:18.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7fe1b4d34640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b0201ff0 con 0x7fe1b00cb660 2023-10-18T10:46:18.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7f96e559a640 1 -- 172.21.15.89:0/2247443488 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e00a5770 con 0x7f96e80cd4e0 2023-10-18T10:46:18.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7f96e559a640 1 -- 172.21.15.89:0/2247443488 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e00a58d0 con 0x7f96e80cd4e0 2023-10-18T10:46:18.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7f96e559a640 1 -- 172.21.15.89:0/2247443488 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e00a5c60 con 0x7f96e80cd4e0 2023-10-18T10:46:18.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7fe1affff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1b015a4e0 0x7fe1b02017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7fe1affff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1b015a4e0 0x7fe1b02017d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:34316/0 (socket says 172.21.15.89:34316) 2023-10-18T10:46:18.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7fe1affff640 1 -- 172.21.15.89:0/40217697 learned_addr learned my addr 172.21.15.89:0/40217697 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:18.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7f5ca018f640 1 -- 172.21.15.89:0/3538541474 shutdown_connections 2023-10-18T10:46:18.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7f96e759e640 1 -- 172.21.15.89:0/2247443488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96e80cd4e0 msgr2=0x7f96e80d0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:18.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7f96e759e640 1 --2- 172.21.15.89:0/2247443488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96e80cd4e0 0x7f96e80d0a40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f96e80ce6a0 tx=0x7f96e001c8b0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7fe1a6ffd640 1 --2- 172.21.15.89:0/40217697 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1b00cb660 0x7fe1b01fb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7fe1af7fe640 1 --2- 172.21.15.89:0/40217697 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1b006cbf0 0x7fe1b0157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7f96e759e640 1 -- 172.21.15.89:0/2247443488 shutdown_connections 2023-10-18T10:46:18.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7f96e759e640 1 --2- 172.21.15.89:0/2247443488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96e80cd4e0 0x7f96e80d0a40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7f96e759e640 1 --2- 172.21.15.89:0/2247443488 >> v2:172.21.15.89:3300/0 conn(0x7f96e80cf950 0x7f96e80b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7f96e759e640 1 -- 172.21.15.89:0/2247443488 >> 172.21.15.89:0/2247443488 conn(0x7f96e8070120 msgr2=0x7f96e8070530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:18.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7f96e759e640 1 -- 172.21.15.89:0/2247443488 shutdown_connections 2023-10-18T10:46:18.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7f6fae59c640 1 --2- 172.21.15.89:0/1909541564 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6fa8033d10 0x7f6fa80eb150 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6fa00097f0 tx=0x7f6fa001c310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76d21ec45e7d346d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:18.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7f96e759e640 1 -- 172.21.15.89:0/2247443488 wait complete. 2023-10-18T10:46:18.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7fe1affff640 1 -- 172.21.15.89:0/40217697 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1b00cb660 msgr2=0x7fe1b01fb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:18.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7f25677fe640 1 --2- 172.21.15.89:0/154892589 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25680c94b0 0x7f25680afd90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f25580088a0 tx=0x7f255801e8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3463cf8c3855a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:18.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7fe1affff640 1 --2- 172.21.15.89:0/40217697 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1b00cb660 0x7fe1b01fb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7fe1affff640 1 -- 172.21.15.89:0/40217697 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1b006cbf0 msgr2=0x7fe1b0157ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:18.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7fe1affff640 1 --2- 172.21.15.89:0/40217697 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1b006cbf0 0x7fe1b0157ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7fe1affff640 1 -- 172.21.15.89:0/40217697 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1980094d0 con 0x7fe1b015a4e0 2023-10-18T10:46:18.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f6fad59a640 1 -- 172.21.15.89:0/1909541564 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa00ab600 con 0x7f6fa8033d10 2023-10-18T10:46:18.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7fe1af7fe640 1 --2- 172.21.15.89:0/40217697 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1b006cbf0 0x7fe1b0157ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:18.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f25667fc640 1 -- 172.21.15.89:0/154892589 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25580a7930 con 0x7f25680c94b0 2023-10-18T10:46:18.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f25667fc640 1 -- 172.21.15.89:0/154892589 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25580ab070 con 0x7f25680c94b0 2023-10-18T10:46:18.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f25667fc640 1 -- 172.21.15.89:0/154892589 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25580a7cb0 con 0x7f25680c94b0 2023-10-18T10:46:18.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f6fad59a640 1 -- 172.21.15.89:0/1909541564 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa00a7050 con 0x7f6fa8033d10 2023-10-18T10:46:18.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f6fad59a640 1 -- 172.21.15.89:0/1909541564 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa00ab980 con 0x7f6fa8033d10 2023-10-18T10:46:18.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f5cb3f2c640 1 Processor -- start 2023-10-18T10:46:18.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f5cb3f2c640 1 -- start start 2023-10-18T10:46:18.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.770+0000 7f5ca018f640 1 -- 172.21.15.89:0/3538541474 wait complete. 2023-10-18T10:46:18.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7fe1affff640 1 --2- 172.21.15.89:0/40217697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1b015a4e0 0x7fe1b02017d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe1a800c390 tx=0x7fe1a800c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:18.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f5ca018f640 1 Processor -- start 2023-10-18T10:46:18.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f256df37640 1 -- 172.21.15.89:0/154892589 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25680c94b0 msgr2=0x7f25680afd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:18.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f256df37640 1 --2- 172.21.15.89:0/154892589 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25680c94b0 0x7f25680afd90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f25580088a0 tx=0x7f255801e8c0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f256df37640 1 -- 172.21.15.89:0/154892589 shutdown_connections 2023-10-18T10:46:18.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f256df37640 1 --2- 172.21.15.89:0/154892589 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25680c94b0 0x7f25680afd90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f256df37640 1 -- 172.21.15.89:0/154892589 >> 172.21.15.89:0/154892589 conn(0x7f2568069f00 msgr2=0x7f25680cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:18.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f256df37640 1 -- 172.21.15.89:0/154892589 shutdown_connections 2023-10-18T10:46:18.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f256df37640 1 -- 172.21.15.89:0/154892589 wait complete. 2023-10-18T10:46:18.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7fe1acff9640 1 -- 172.21.15.89:0/40217697 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a80054e0 con 0x7fe1b015a4e0 2023-10-18T10:46:18.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f6faf59e640 1 -- 172.21.15.89:0/1909541564 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6fa8033d10 msgr2=0x7f6fa80eb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:18.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f6faf59e640 1 --2- 172.21.15.89:0/1909541564 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6fa8033d10 0x7f6fa80eb150 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6fa00097f0 tx=0x7f6fa001c310 comp rx=0 tx=0).stop 2023-10-18T10:46:18.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7fe1acff9640 1 -- 172.21.15.89:0/40217697 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a8003600 con 0x7fe1b015a4e0 2023-10-18T10:46:18.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7fe1acff9640 1 -- 172.21.15.89:0/40217697 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8003990 con 0x7fe1b015a4e0 2023-10-18T10:46:18.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f6faf59e640 1 -- 172.21.15.89:0/1909541564 shutdown_connections 2023-10-18T10:46:18.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f6faf59e640 1 --2- 172.21.15.89:0/1909541564 >> v2:172.21.15.89:3300/0 conn(0x7f6fa8047bd0 0x7f6fa8052ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f6faf59e640 1 --2- 172.21.15.89:0/1909541564 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6fa8033d10 0x7f6fa80eb150 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f6faf59e640 1 -- 172.21.15.89:0/1909541564 >> 172.21.15.89:0/1909541564 conn(0x7f6fa8054b10 msgr2=0x7f6fa8054f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:18.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f5cb3f2c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5cac06ff00 0x7f5cac15e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f5cb3f2c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5cac0ca340 0x7f5cac15b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f5cb3f2c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cac15b680 0x7f5cac0c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f5cb3f2c640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cac15ef30 con 0x7f5cac0ca340 2023-10-18T10:46:18.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f5cb3f2c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cac15aa10 con 0x7f5cac06ff00 2023-10-18T10:46:18.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f5cb3f2c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac15acb0 con 0x7f5cac15b680 2023-10-18T10:46:18.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7fe1b4d34640 1 -- 172.21.15.89:0/40217697 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b02022d0 con 0x7fe1b015a4e0 2023-10-18T10:46:18.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7fe1b4d34640 1 -- 172.21.15.89:0/40217697 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b00c6550 con 0x7fe1b015a4e0 2023-10-18T10:46:18.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f6faf59e640 1 -- 172.21.15.89:0/1909541564 shutdown_connections 2023-10-18T10:46:18.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f6faf59e640 1 -- 172.21.15.89:0/1909541564 wait complete. 2023-10-18T10:46:18.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f5ca018f640 1 -- start start 2023-10-18T10:46:18.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f5ca018f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c9806df60 0x7f5c980c0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f5ca018f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c980c8bc0 0x7f5c980c2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f5ca018f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c98160bb0 0x7f5c980c5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f5ca018f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c980c3eb0 con 0x7f5c980c8bc0 2023-10-18T10:46:18.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f5ca018f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c980c32a0 con 0x7f5c9806df60 2023-10-18T10:46:18.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f5ca018f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c980c3580 con 0x7f5c98160bb0 2023-10-18T10:46:18.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f96e759e640 1 Processor -- start 2023-10-18T10:46:18.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f72031ba640 1 Processor -- start 2023-10-18T10:46:18.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f5c9e72c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c98160bb0 0x7f5c980c5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f5cb1cc8640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5cac06ff00 0x7f5cac15e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f5cb1cc8640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5cac06ff00 0x7f5cac15e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:34330/0 (socket says 172.21.15.89:34330) 2023-10-18T10:46:18.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.774+0000 7f5cb1cc8640 1 -- 172.21.15.89:0/823941791 learned_addr learned my addr 172.21.15.89:0/823941791 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:18.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f5cb24c9640 1 --2- 172.21.15.89:0/823941791 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cac15b680 0x7f5cac0c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f5c9e72c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c98160bb0 0x7f5c980c5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:37454/0 (socket says 172.21.15.89:37454) 2023-10-18T10:46:18.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f5cb14c7640 1 --2- 172.21.15.89:0/823941791 >> v2:172.21.15.89:3300/0 conn(0x7f5cac0ca340 0x7f5cac15b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f5c9e72c640 1 -- 172.21.15.89:0/1159290063 learned_addr learned my addr 172.21.15.89:0/1159290063 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:18.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f256df37640 1 Processor -- start 2023-10-18T10:46:18.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f5c9d72a640 1 --2- 172.21.15.89:0/1159290063 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c980c8bc0 0x7f5c980c2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f72031ba640 1 -- start start 2023-10-18T10:46:18.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f5cb1cc8640 1 -- 172.21.15.89:0/823941791 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cac15b680 msgr2=0x7f5cac0c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:18.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f5cb1cc8640 1 --2- 172.21.15.89:0/823941791 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cac15b680 0x7f5cac0c8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f5cb1cc8640 1 -- 172.21.15.89:0/823941791 >> v2:172.21.15.89:3300/0 conn(0x7f5cac0ca340 msgr2=0x7f5cac15b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:18.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f5cb1cc8640 1 --2- 172.21.15.89:0/823941791 >> v2:172.21.15.89:3300/0 conn(0x7f5cac0ca340 0x7f5cac15b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f5cb1cc8640 1 -- 172.21.15.89:0/823941791 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cac0702e0 con 0x7f5cac06ff00 2023-10-18T10:46:18.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f5cb14c7640 1 --2- 172.21.15.89:0/823941791 >> v2:172.21.15.89:3300/0 conn(0x7f5cac0ca340 0x7f5cac15b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:18.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7fe1b4d34640 1 -- 172.21.15.89:0/40217697 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1b0073b50 con 0x7fe1b015a4e0 2023-10-18T10:46:18.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f5c9e72c640 1 -- 172.21.15.89:0/1159290063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c9806df60 msgr2=0x7f5c980c0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:18.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f5c9e72c640 1 --2- 172.21.15.89:0/1159290063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c9806df60 0x7f5c980c0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f5cb24c9640 1 --2- 172.21.15.89:0/823941791 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cac15b680 0x7f5cac0c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:18.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f5c9e72c640 1 -- 172.21.15.89:0/1159290063 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c980c8bc0 msgr2=0x7f5c980c2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:18.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f5c9e72c640 1 --2- 172.21.15.89:0/1159290063 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c980c8bc0 0x7f5c980c2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f96e759e640 1 -- start start 2023-10-18T10:46:18.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f256df37640 1 -- start start 2023-10-18T10:46:18.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f5c9e72c640 1 -- 172.21.15.89:0/1159290063 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c880094d0 con 0x7f5c98160bb0 2023-10-18T10:46:18.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f5c9d72a640 1 --2- 172.21.15.89:0/1159290063 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c980c8bc0 0x7f5c980c2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:18.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f96e759e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96e80cd4e0 0x7f96e814fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f96e759e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96e80cf950 0x7f96e814f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f96e759e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96e815c310 0x7f96e80cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f96e759e640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e8159ac0 con 0x7f96e80cd4e0 2023-10-18T10:46:18.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f96e759e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e8159420 con 0x7f96e80cf950 2023-10-18T10:46:18.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f96e759e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96e814fad0 con 0x7f96e815c310 2023-10-18T10:46:18.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f96e659c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96e80cd4e0 0x7f96e814fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f96e6d9d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96e815c310 0x7f96e80cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f96e6d9d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96e815c310 0x7f96e80cb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:37466/0 (socket says 172.21.15.89:37466) 2023-10-18T10:46:18.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f96e6d9d640 1 -- 172.21.15.89:0/3336568431 learned_addr learned my addr 172.21.15.89:0/3336568431 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:18.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f96e6d9d640 1 -- 172.21.15.89:0/3336568431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96e80cf950 msgr2=0x7f96e814f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:18.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f6faf59e640 1 Processor -- start 2023-10-18T10:46:18.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f256df37640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25680b02d0 0x7f25680c4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f96e6d9d640 1 --2- 172.21.15.89:0/3336568431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96e80cf950 0x7f96e814f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f96e6d9d640 1 -- 172.21.15.89:0/3336568431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96e80cd4e0 msgr2=0x7f96e814fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:18.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f96e6d9d640 1 --2- 172.21.15.89:0/3336568431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96e80cd4e0 0x7f96e814fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f96e6d9d640 1 -- 172.21.15.89:0/3336568431 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e0009b30 con 0x7f96e815c310 2023-10-18T10:46:18.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f6faf59e640 1 -- start start 2023-10-18T10:46:18.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7f5c9e72c640 1 --2- 172.21.15.89:0/1159290063 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c98160bb0 0x7f5c980c5ce0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5c9400e7f0 tx=0x7f5c9400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:18.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.778+0000 7fe1acff9640 1 -- 172.21.15.89:0/40217697 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fe1a8003af0 con 0x7fe1b015a4e0 2023-10-18T10:46:18.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f256df37640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25680c94b0 0x7f25680c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f96e6d9d640 1 --2- 172.21.15.89:0/3336568431 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96e815c310 0x7f96e80cb8a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f96e001af60 tx=0x7f96e0019650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:18.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f72031ba640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f71fc15c2f0 0x7f71fc0b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f256df37640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2568159530 0x7f25680c7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f256df37640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256813e8b0 con 0x7f2568159530 2023-10-18T10:46:18.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f256df37640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25680c4d50 con 0x7f25680c94b0 2023-10-18T10:46:18.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f256df37640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2568150da0 con 0x7f25680b02d0 2023-10-18T10:46:18.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f25677fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25680b02d0 0x7f25680c4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f5c8e7fc640 1 -- 172.21.15.89:0/1159290063 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c94009530 con 0x7f5c98160bb0 2023-10-18T10:46:18.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f5c8e7fc640 1 -- 172.21.15.89:0/1159290063 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9400eea0 con 0x7f5c98160bb0 2023-10-18T10:46:18.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f5c8e7fc640 1 -- 172.21.15.89:0/1159290063 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9400cb60 con 0x7f5c98160bb0 2023-10-18T10:46:18.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f5ca018f640 1 -- 172.21.15.89:0/1159290063 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c980cc060 con 0x7f5c98160bb0 2023-10-18T10:46:18.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f25677fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25680b02d0 0x7f25680c4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:37472/0 (socket says 172.21.15.89:37472) 2023-10-18T10:46:18.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f25677fe640 1 -- 172.21.15.89:0/3900946924 learned_addr learned my addr 172.21.15.89:0/3900946924 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:18.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f2567fff640 1 --2- 172.21.15.89:0/3900946924 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2568159530 0x7f25680c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f5ca018f640 1 -- 172.21.15.89:0/1159290063 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c980c3b40 con 0x7f5c98160bb0 2023-10-18T10:46:18.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f2566ffd640 1 --2- 172.21.15.89:0/3900946924 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25680c94b0 0x7f25680c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f6faf59e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6fa8033d10 0x7f6fa80ea490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f6faf59e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fa8047bd0 0x7f6fa80e5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f6faf59e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6fa8048200 0x7f6fa80e60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f6faf59e640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa8048b20 con 0x7f6fa8047bd0 2023-10-18T10:46:18.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f6faf59e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa8048e00 con 0x7f6fa8048200 2023-10-18T10:46:18.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f6faf59e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa80490e0 con 0x7f6fa8033d10 2023-10-18T10:46:18.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f6faed9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6fa8048200 0x7f6fa80e60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f25677fe640 1 -- 172.21.15.89:0/3900946924 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25680c94b0 msgr2=0x7f25680c7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:18.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f25677fe640 1 --2- 172.21.15.89:0/3900946924 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25680c94b0 0x7f25680c7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f25677fe640 1 -- 172.21.15.89:0/3900946924 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2568159530 msgr2=0x7f25680c7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:18.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f25677fe640 1 --2- 172.21.15.89:0/3900946924 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2568159530 0x7f25680c7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f25677fe640 1 -- 172.21.15.89:0/3900946924 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2558008550 con 0x7f25680b02d0 2023-10-18T10:46:18.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f6fadd9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fa8047bd0 0x7f6fa80e5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f6fadd9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fa8047bd0 0x7f6fa80e5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45500/0 (socket says 172.21.15.89:45500) 2023-10-18T10:46:18.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f6fadd9b640 1 -- 172.21.15.89:0/905788006 learned_addr learned my addr 172.21.15.89:0/905788006 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:18.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f96d6ffd640 1 -- 172.21.15.89:0/3336568431 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e00a5770 con 0x7f96e815c310 2023-10-18T10:46:18.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f96d6ffd640 1 -- 172.21.15.89:0/3336568431 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e00b8670 con 0x7f96e815c310 2023-10-18T10:46:18.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f96d6ffd640 1 -- 172.21.15.89:0/3336568431 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e00b8a00 con 0x7f96e815c310 2023-10-18T10:46:18.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f96e759e640 1 -- 172.21.15.89:0/3336568431 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e81554a0 con 0x7f96e815c310 2023-10-18T10:46:18.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f96e759e640 1 -- 172.21.15.89:0/3336568431 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e815b060 con 0x7f96e815c310 2023-10-18T10:46:18.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f7200f56640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f71fc15c2f0 0x7f71fc0b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f72031ba640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f71fc0d0270 0x7f71fc0d0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f72031ba640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71fc15d1b0 con 0x7f71fc15c6d0 2023-10-18T10:46:18.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f72031ba640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71fc15d490 con 0x7f71fc15c2f0 2023-10-18T10:46:18.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f72031ba640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71fc15d770 con 0x7f71fc0d0270 2023-10-18T10:46:18.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7fe1acff9640 1 --2- 172.21.15.89:0/40217697 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe190065680 0x7fe190067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7fe1acff9640 1 -- 172.21.15.89:0/40217697 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe1a8082330 con 0x7fe1b015a4e0 2023-10-18T10:46:18.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f7200f56640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f71fc15c2f0 0x7f71fc0b18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45478/0 (socket says 172.21.15.89:45478) 2023-10-18T10:46:18.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f7200f56640 1 -- 172.21.15.89:0/3592660081 learned_addr learned my addr 172.21.15.89:0/3592660081 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:18.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f71f3fff640 1 --2- 172.21.15.89:0/3592660081 >> v2:172.21.15.154:3300/0 conn(0x7f71fc0d0270 0x7f71fc0d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f5cb1cc8640 1 --2- 172.21.15.89:0/823941791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cac06ff00 0x7f5cac15e000 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5cac06a830 tx=0x7f5c9c01c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d268bd0a6b0bcf8 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:18.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f6fae59c640 1 --2- 172.21.15.89:0/905788006 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6fa8033d10 0x7f6fa80ea490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7fe1af7fe640 1 --2- 172.21.15.89:0/40217697 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe190065680 0x7fe190067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f7200f56640 1 -- 172.21.15.89:0/3592660081 >> v2:172.21.15.154:3300/0 conn(0x7f71fc0d0270 msgr2=0x7f71fc0d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:18.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f7200f56640 1 --2- 172.21.15.89:0/3592660081 >> v2:172.21.15.154:3300/0 conn(0x7f71fc0d0270 0x7f71fc0d0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f7200f56640 1 -- 172.21.15.89:0/3592660081 >> v1:172.21.15.154:6789/0 conn(0x7f71fc15c6d0 legacy=0x7f71fc160780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:18.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f7200f56640 1 -- 172.21.15.89:0/3592660081 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71fc15da50 con 0x7f71fc15c2f0 2023-10-18T10:46:18.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f6fadd9b640 1 -- 172.21.15.89:0/905788006 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6fa8033d10 msgr2=0x7f6fa80ea490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:18.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f6fadd9b640 1 --2- 172.21.15.89:0/905788006 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6fa8033d10 0x7f6fa80ea490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f6fadd9b640 1 -- 172.21.15.89:0/905788006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6fa8048200 msgr2=0x7f6fa80e60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:18.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f6fadd9b640 1 --2- 172.21.15.89:0/905788006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6fa8048200 0x7f6fa80e60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f6fadd9b640 1 -- 172.21.15.89:0/905788006 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa0009420 con 0x7f6fa8047bd0 2023-10-18T10:46:18.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f25677fe640 1 --2- 172.21.15.89:0/3900946924 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25680b02d0 0x7f25680c4810 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f25580b3040 tx=0x7f255801d100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:18.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f6faed9d640 1 --2- 172.21.15.89:0/905788006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6fa8048200 0x7f6fa80e60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:18.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.782+0000 7f96e759e640 1 -- 172.21.15.89:0/3336568431 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96e8159c00 con 0x7f96e815c310 2023-10-18T10:46:18.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.786+0000 7f6fadd9b640 1 --2- 172.21.15.89:0/905788006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fa8047bd0 0x7f6fa80e5b70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4009f30 tx=0x7f6fa400c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:18.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.786+0000 7f5ca018f640 1 -- 172.21.15.89:0/1159290063 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c9806c810 con 0x7f5c98160bb0 2023-10-18T10:46:18.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.786+0000 7f254ffff640 1 -- 172.21.15.89:0/3900946924 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255800b150 con 0x7f25680b02d0 2023-10-18T10:46:18.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.786+0000 7f254ffff640 1 -- 172.21.15.89:0/3900946924 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f255800b2b0 con 0x7f25680b02d0 2023-10-18T10:46:18.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.786+0000 7f254ffff640 1 -- 172.21.15.89:0/3900946924 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255800b660 con 0x7f25680b02d0 2023-10-18T10:46:18.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.786+0000 7f256df37640 1 -- 172.21.15.89:0/3900946924 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25680b2440 con 0x7f25680b02d0 2023-10-18T10:46:18.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.786+0000 7f256df37640 1 -- 172.21.15.89:0/3900946924 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25680b2a00 con 0x7f25680b02d0 2023-10-18T10:46:18.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.786+0000 7f5c8e7fc640 1 -- 172.21.15.89:0/1159290063 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f5c9400ccc0 con 0x7f5c98160bb0 2023-10-18T10:46:18.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.786+0000 7f6f9effd640 1 -- 172.21.15.89:0/905788006 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4007af0 con 0x7f6fa8047bd0 2023-10-18T10:46:18.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.786+0000 7f6f9effd640 1 -- 172.21.15.89:0/905788006 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa4007c50 con 0x7f6fa8047bd0 2023-10-18T10:46:18.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.786+0000 7f6f9effd640 1 -- 172.21.15.89:0/905788006 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa400ce70 con 0x7f6fa8047bd0 2023-10-18T10:46:18.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.786+0000 7f6faf59e640 1 -- 172.21.15.89:0/905788006 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fa80493c0 con 0x7f6fa8047bd0 2023-10-18T10:46:18.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.786+0000 7f6faf59e640 1 -- 172.21.15.89:0/905788006 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fa80df760 con 0x7f6fa8047bd0 2023-10-18T10:46:18.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.786+0000 7f5cb0cc6640 1 -- 172.21.15.89:0/823941791 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c0ac640 con 0x7f5cac06ff00 2023-10-18T10:46:18.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.786+0000 7f5cb0cc6640 1 -- 172.21.15.89:0/823941791 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9c0ac7a0 con 0x7f5cac06ff00 2023-10-18T10:46:18.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.786+0000 7f5cb0cc6640 1 -- 172.21.15.89:0/823941791 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c0ac640 con 0x7f5cac06ff00 2023-10-18T10:46:18.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.786+0000 7f96d6ffd640 1 -- 172.21.15.89:0/3336568431 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f96e00b8b60 con 0x7f96e815c310 2023-10-18T10:46:18.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.786+0000 7f256df37640 1 -- 172.21.15.89:0/3900946924 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f256813e9f0 con 0x7f25680b02d0 2023-10-18T10:46:18.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.786+0000 7f5c8e7fc640 1 --2- 172.21.15.89:0/1159290063 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5c7c0656d0 0x7f5c7c067b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.786+0000 7f5c8e7fc640 1 -- 172.21.15.89:0/1159290063 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5c94083580 con 0x7f5c98160bb0 2023-10-18T10:46:18.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.786+0000 7fe1af7fe640 1 --2- 172.21.15.89:0/40217697 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe190065680 0x7fe190067b40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe1a000c420 tx=0x7fe1a0009570 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:18.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.786+0000 7fe1acff9640 1 -- 172.21.15.89:0/40217697 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fe1a804fb10 con 0x7fe1b015a4e0 2023-10-18T10:46:18.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.786+0000 7f6faf59e640 1 -- 172.21.15.89:0/905788006 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f94005260 con 0x7f6fa8047bd0 2023-10-18T10:46:18.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.786+0000 7f5c9df2b640 1 --2- 172.21.15.89:0/1159290063 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5c7c0656d0 0x7f5c7c067b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.786+0000 7f5cb3f2c640 1 -- 172.21.15.89:0/823941791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cac06ff00 msgr2=0x7f5cac15e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:18.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.786+0000 7f5cb3f2c640 1 --2- 172.21.15.89:0/823941791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cac06ff00 0x7f5cac15e000 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5cac06a830 tx=0x7f5c9c01c440 comp rx=0 tx=0).stop 2023-10-18T10:46:18.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.786+0000 7f5cb3f2c640 1 -- 172.21.15.89:0/823941791 shutdown_connections 2023-10-18T10:46:18.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.786+0000 7f5cb3f2c640 1 --2- 172.21.15.89:0/823941791 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cac15b680 0x7f5cac0c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.786+0000 7f5cb3f2c640 1 --2- 172.21.15.89:0/823941791 >> v2:172.21.15.89:3300/0 conn(0x7f5cac0ca340 0x7f5cac15b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.786+0000 7f5cb3f2c640 1 --2- 172.21.15.89:0/823941791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cac06ff00 0x7f5cac15e000 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.786+0000 7f5cb3f2c640 1 -- 172.21.15.89:0/823941791 >> 172.21.15.89:0/823941791 conn(0x7f5cac069f00 msgr2=0x7f5cac0cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:18.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.786+0000 7f96d6ffd640 1 --2- 172.21.15.89:0/3336568431 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f96c8065750 0x7f96c8067c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.790+0000 7f5c9df2b640 1 --2- 172.21.15.89:0/1159290063 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5c7c0656d0 0x7f5c7c067b90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5c880ae000 tx=0x7f5c88004740 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:18.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.790+0000 7f5cb3f2c640 1 -- 172.21.15.89:0/823941791 shutdown_connections 2023-10-18T10:46:18.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.790+0000 7f96e659c640 1 --2- 172.21.15.89:0/3336568431 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f96c8065750 0x7f96c8067c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.790+0000 7f96d6ffd640 1 -- 172.21.15.89:0/3336568431 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f96e011e650 con 0x7f96e815c310 2023-10-18T10:46:18.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.790+0000 7f5cb3f2c640 1 -- 172.21.15.89:0/823941791 wait complete. 2023-10-18T10:46:18.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.790+0000 7f6f9effd640 1 -- 172.21.15.89:0/905788006 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f6fa4009070 con 0x7f6fa8047bd0 2023-10-18T10:46:18.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.790+0000 7f96e659c640 1 --2- 172.21.15.89:0/3336568431 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f96c8065750 0x7f96c8067c10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f96d800b810 tx=0x7f96d8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:18.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.790+0000 7f7200f56640 1 --2- 172.21.15.89:0/3592660081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71fc15c2f0 0x7f71fc0b18e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f71fc0c96b0 tx=0x7f71f801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40617fc6009289df server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:18.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.790+0000 7f96d6ffd640 1 -- 172.21.15.89:0/3336568431 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f96e00ebe90 con 0x7f96e815c310 2023-10-18T10:46:18.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.790+0000 7f5cb3f2c640 1 Processor -- start 2023-10-18T10:46:18.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.790+0000 7f6f9effd640 1 --2- 172.21.15.89:0/905788006 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6f90065780 0x7f6f90067c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.790+0000 7f6f9effd640 1 -- 172.21.15.89:0/905788006 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6fa40850c0 con 0x7f6fa8047bd0 2023-10-18T10:46:18.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.790+0000 7f5cb3f2c640 1 -- start start 2023-10-18T10:46:18.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.790+0000 7f6fae59c640 1 --2- 172.21.15.89:0/905788006 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6f90065780 0x7f6f90067c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.790+0000 7f254ffff640 1 -- 172.21.15.89:0/3900946924 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f255800b7c0 con 0x7f25680b02d0 2023-10-18T10:46:18.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.794+0000 7f5cb3f2c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cac06ff00 0x7f5cac0ae730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.794+0000 7f5cb3f2c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cac0ca340 0x7f5cac0c1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.794+0000 7f5cb3f2c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cac15d740 0x7f5cac15dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.794+0000 7f6fae59c640 1 --2- 172.21.15.89:0/905788006 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6f90065780 0x7f6f90067c40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6fa00ad000 tx=0x7f6fa0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:18.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.794+0000 7f5cb3f2c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac15d2b0 con 0x7f5cac06ff00 2023-10-18T10:46:18.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.794+0000 7f5cb3f2c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac14d920 con 0x7f5cac15d740 2023-10-18T10:46:18.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.794+0000 7f5cb3f2c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac0aec70 con 0x7f5cac0ca340 2023-10-18T10:46:18.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.794+0000 7f254ffff640 1 --2- 172.21.15.89:0/3900946924 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2554065680 0x7f2554067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.794+0000 7f5cb14c7640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cac0ca340 0x7f5cac0c1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.794+0000 7f5cb14c7640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cac0ca340 0x7f5cac0c1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:37490/0 (socket says 172.21.15.89:37490) 2023-10-18T10:46:18.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.794+0000 7f5cb14c7640 1 -- 172.21.15.89:0/2034305004 learned_addr learned my addr 172.21.15.89:0/2034305004 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:18.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.794+0000 7f5cb1cc8640 1 --2- 172.21.15.89:0/2034305004 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cac06ff00 0x7f5cac0ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.794+0000 7f2566ffd640 1 --2- 172.21.15.89:0/3900946924 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2554065680 0x7f2554067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.794+0000 7f5cb24c9640 1 --2- 172.21.15.89:0/2034305004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cac15d740 0x7f5cac15dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.794+0000 7f5cb14c7640 1 -- 172.21.15.89:0/2034305004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cac15d740 msgr2=0x7f5cac15dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:18.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.794+0000 7f5cb14c7640 1 --2- 172.21.15.89:0/2034305004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cac15d740 0x7f5cac15dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.794+0000 7f5cb14c7640 1 -- 172.21.15.89:0/2034305004 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cac06ff00 msgr2=0x7f5cac0ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:18.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.794+0000 7f5cb14c7640 1 --2- 172.21.15.89:0/2034305004 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cac06ff00 0x7f5cac0ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.794+0000 7f5cb14c7640 1 -- 172.21.15.89:0/2034305004 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9c0094d0 con 0x7f5cac0ca340 2023-10-18T10:46:18.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.794+0000 7f2566ffd640 1 --2- 172.21.15.89:0/3900946924 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2554065680 0x7f2554067b40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f25680c88c0 tx=0x7f255c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:18.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.794+0000 7f5cb14c7640 1 --2- 172.21.15.89:0/2034305004 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cac0ca340 0x7f5cac0c1590 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5ca800c7c0 tx=0x7f5ca800cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:18.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.794+0000 7f5ca27fc640 1 -- 172.21.15.89:0/2034305004 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8007a30 con 0x7f5cac0ca340 2023-10-18T10:46:18.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.794+0000 7f254ffff640 1 -- 172.21.15.89:0/3900946924 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f25580f24a0 con 0x7f25680b02d0 2023-10-18T10:46:18.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.794+0000 7f5cb3f2c640 1 -- 172.21.15.89:0/2034305004 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cac15c6a0 con 0x7f5cac0ca340 2023-10-18T10:46:18.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.794+0000 7f5cb3f2c640 1 -- 172.21.15.89:0/2034305004 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cac15cc60 con 0x7f5cac0ca340 2023-10-18T10:46:18.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.794+0000 7f5c8e7fc640 1 -- 172.21.15.89:0/1159290063 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f5c94050d30 con 0x7f5c98160bb0 2023-10-18T10:46:18.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.798+0000 7f5cb3f2c640 1 -- 172.21.15.89:0/2034305004 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cac04dbe0 con 0x7f5cac0ca340 2023-10-18T10:46:18.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.798+0000 7f71f37fe640 1 -- 172.21.15.89:0/3592660081 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f80ac600 con 0x7f71fc15c2f0 2023-10-18T10:46:18.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.798+0000 7f71f37fe640 1 -- 172.21.15.89:0/3592660081 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f80a8050 con 0x7f71fc15c2f0 2023-10-18T10:46:18.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.798+0000 7f71f37fe640 1 -- 172.21.15.89:0/3592660081 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f80ac980 con 0x7f71fc15c2f0 2023-10-18T10:46:18.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.798+0000 7f6f9effd640 1 -- 172.21.15.89:0/905788006 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f6fa40563b0 con 0x7f6fa8047bd0 2023-10-18T10:46:18.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.798+0000 7f5ca27fc640 1 -- 172.21.15.89:0/2034305004 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ca800ce70 con 0x7f5cac0ca340 2023-10-18T10:46:18.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.798+0000 7f5ca27fc640 1 -- 172.21.15.89:0/2034305004 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8007d90 con 0x7f5cac0ca340 2023-10-18T10:46:18.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.802+0000 7f72031ba640 1 -- 172.21.15.89:0/3592660081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71fc15c2f0 msgr2=0x7f71fc0b18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:18.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.802+0000 7f72031ba640 1 --2- 172.21.15.89:0/3592660081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71fc15c2f0 0x7f71fc0b18e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f71fc0c96b0 tx=0x7f71f801c490 comp rx=0 tx=0).stop 2023-10-18T10:46:18.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.802+0000 7f254ffff640 1 -- 172.21.15.89:0/3900946924 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f25580eaad0 con 0x7f25680b02d0 2023-10-18T10:46:18.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.802+0000 7f72031ba640 1 -- 172.21.15.89:0/3592660081 shutdown_connections 2023-10-18T10:46:18.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.802+0000 7f72031ba640 1 --2- 172.21.15.89:0/3592660081 >> v2:172.21.15.154:3300/0 conn(0x7f71fc0d0270 0x7f71fc0d0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.802+0000 7f72031ba640 1 --2- 172.21.15.89:0/3592660081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71fc15c2f0 0x7f71fc0b18e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.802+0000 7f72031ba640 1 -- 172.21.15.89:0/3592660081 >> 172.21.15.89:0/3592660081 conn(0x7f71fc069ad0 msgr2=0x7f71fc069eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:18.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.802+0000 7f5ca27fc640 1 -- 172.21.15.89:0/2034305004 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f5ca8024480 con 0x7f5cac0ca340 2023-10-18T10:46:18.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.802+0000 7f5ca27fc640 1 --2- 172.21.15.89:0/2034305004 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5c90065680 0x7f5c90067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.802+0000 7f5ca27fc640 1 -- 172.21.15.89:0/2034305004 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5ca8083760 con 0x7f5cac0ca340 2023-10-18T10:46:18.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.806+0000 7f5cb1cc8640 1 --2- 172.21.15.89:0/2034305004 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5c90065680 0x7f5c90067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.806+0000 7f72031ba640 1 -- 172.21.15.89:0/3592660081 shutdown_connections 2023-10-18T10:46:18.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.806+0000 7f72031ba640 1 -- 172.21.15.89:0/3592660081 wait complete. 2023-10-18T10:46:18.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.806+0000 7f5ca27fc640 1 -- 172.21.15.89:0/2034305004 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f5ca8050e40 con 0x7f5cac0ca340 2023-10-18T10:46:18.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.806+0000 7f72031ba640 1 Processor -- start 2023-10-18T10:46:18.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.806+0000 7f72031ba640 1 -- start start 2023-10-18T10:46:18.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.810+0000 7f72031ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71fc0d0270 0x7f71fc0cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.810+0000 7f72031ba640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71fc15c2f0 0x7f71fc15ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.810+0000 7f72031ba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f71fc15c6d0 0x7f71fc15f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.806+0000 7f5cb1cc8640 1 --2- 172.21.15.89:0/2034305004 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5c90065680 0x7f5c90067b40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c0ae000 tx=0x7f5c9c0a6070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:18.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.810+0000 7f72031ba640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc06e450 con 0x7f71fc15c2f0 2023-10-18T10:46:18.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.810+0000 7f72031ba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc0cb5b0 con 0x7f71fc0d0270 2023-10-18T10:46:18.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.810+0000 7f72031ba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc06d840 con 0x7f71fc15c6d0 2023-10-18T10:46:18.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.810+0000 7f7201757640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f71fc15c6d0 0x7f71fc15f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.810+0000 7f7201757640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f71fc15c6d0 0x7f71fc15f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:37494/0 (socket says 172.21.15.89:37494) 2023-10-18T10:46:18.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.810+0000 7f7201757640 1 -- 172.21.15.89:0/3714246130 learned_addr learned my addr 172.21.15.89:0/3714246130 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:18.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.810+0000 7f7200f56640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71fc0d0270 0x7f71fc0cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.814+0000 7f7201757640 1 -- 172.21.15.89:0/3714246130 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71fc0d0270 msgr2=0x7f71fc0cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:18.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.814+0000 7f7201757640 1 --2- 172.21.15.89:0/3714246130 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71fc0d0270 0x7f71fc0cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.814+0000 7f7201757640 1 -- 172.21.15.89:0/3714246130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71fc15c2f0 msgr2=0x7f71fc15ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:18.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.814+0000 7f7201757640 1 --2- 172.21.15.89:0/3714246130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71fc15c2f0 0x7f71fc15ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:18.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.814+0000 7f7201757640 1 -- 172.21.15.89:0/3714246130 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f80094d0 con 0x7f71fc15c6d0 2023-10-18T10:46:18.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.814+0000 7f7201757640 1 --2- 172.21.15.89:0/3714246130 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f71fc15c6d0 0x7f71fc15f120 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f71f400c780 tx=0x7f71f400cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:18.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.814+0000 7f7200f56640 1 --2- 172.21.15.89:0/3714246130 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71fc0d0270 0x7f71fc0cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:18.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.814+0000 7f71f17fa640 1 -- 172.21.15.89:0/3714246130 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f4003d90 con 0x7f71fc15c6d0 2023-10-18T10:46:18.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.814+0000 7f71f17fa640 1 -- 172.21.15.89:0/3714246130 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f400ce90 con 0x7f71fc15c6d0 2023-10-18T10:46:18.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.814+0000 7f71f17fa640 1 -- 172.21.15.89:0/3714246130 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f40052b0 con 0x7f71fc15c6d0 2023-10-18T10:46:18.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.814+0000 7f72031ba640 1 -- 172.21.15.89:0/3714246130 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71fc06db20 con 0x7f71fc15c6d0 2023-10-18T10:46:18.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.814+0000 7f72031ba640 1 -- 172.21.15.89:0/3714246130 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71fc06e0e0 con 0x7f71fc15c6d0 2023-10-18T10:46:18.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.818+0000 7f71f17fa640 1 -- 172.21.15.89:0/3714246130 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f71f4005410 con 0x7f71fc15c6d0 2023-10-18T10:46:18.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.818+0000 7f71f17fa640 1 --2- 172.21.15.89:0/3714246130 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f71e80656d0 0x7f71e8067b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:18.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.818+0000 7f71f17fa640 1 -- 172.21.15.89:0/3714246130 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f71f4082870 con 0x7f71fc15c6d0 2023-10-18T10:46:18.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.822+0000 7f72031ba640 1 -- 172.21.15.89:0/3714246130 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71fc066ca0 con 0x7f71fc15c6d0 2023-10-18T10:46:18.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.822+0000 7f7200f56640 1 --2- 172.21.15.89:0/3714246130 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f71e80656d0 0x7f71e8067b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:18.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.822+0000 7f7200f56640 1 --2- 172.21.15.89:0/3714246130 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f71e80656d0 0x7f71e8067b90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f71f80ae000 tx=0x7f71f8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:18.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:18.826+0000 7f71f17fa640 1 -- 172.21.15.89:0/3714246130 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f71f4053c40 con 0x7f71fc15c6d0 2023-10-18T10:46:19.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.038+0000 7ff75cb2d640 1 -- 172.21.15.89:0/4211970223 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff7580c8190 con 0x7ff7580701e0 2023-10-18T10:46:19.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.038+0000 7ff73ffff640 1 -- 172.21.15.89:0/4211970223 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff7480b2090 con 0x7ff7580701e0 2023-10-18T10:46:19.039 INFO:teuthology.orchestra.run.smithi089.stdout:55834574857 2023-10-18T10:46:19.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.038+0000 7ff75cb2d640 1 -- 172.21.15.89:0/4211970223 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff7440656d0 msgr2=0x7ff744067b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:19.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.038+0000 7ff75cb2d640 1 --2- 172.21.15.89:0/4211970223 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff7440656d0 0x7ff744067b90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff7580c1590 tx=0x7ff74c008040 comp rx=0 tx=0).stop 2023-10-18T10:46:19.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.038+0000 7ff75cb2d640 1 -- 172.21.15.89:0/4211970223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7580701e0 msgr2=0x7ff758148070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:19.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.038+0000 7ff75cb2d640 1 --2- 172.21.15.89:0/4211970223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7580701e0 0x7ff758148070 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff748019090 tx=0x7ff748019220 comp rx=0 tx=0).stop 2023-10-18T10:46:19.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.042+0000 7ff75cb2d640 1 -- 172.21.15.89:0/4211970223 shutdown_connections 2023-10-18T10:46:19.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.042+0000 7ff75cb2d640 1 --2- 172.21.15.89:0/4211970223 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff7440656d0 0x7ff744067b90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.042+0000 7ff75cb2d640 1 --2- 172.21.15.89:0/4211970223 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7581587e0 0x7ff75814fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.042+0000 7ff75cb2d640 1 --2- 172.21.15.89:0/4211970223 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7580cd680 0x7ff75814f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.042+0000 7ff75cb2d640 1 --2- 172.21.15.89:0/4211970223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7580701e0 0x7ff758148070 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.042+0000 7ff75cb2d640 1 -- 172.21.15.89:0/4211970223 >> 172.21.15.89:0/4211970223 conn(0x7ff7580cab10 msgr2=0x7ff75806f580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:19.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.042+0000 7ff75cb2d640 1 -- 172.21.15.89:0/4211970223 shutdown_connections 2023-10-18T10:46:19.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.042+0000 7ff75cb2d640 1 -- 172.21.15.89:0/4211970223 wait complete. 2023-10-18T10:46:19.075 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.1 2023-10-18T10:46:19.076 DEBUG:teuthology.parallel:result is None 2023-10-18T10:46:19.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.106+0000 7f72031ba640 1 -- 172.21.15.89:0/3714246130 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f71fc0c9090 con 0x7f71fc15c6d0 2023-10-18T10:46:19.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.110+0000 7f71f17fa640 1 -- 172.21.15.89:0/3714246130 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f71f4017020 con 0x7f71fc15c6d0 2023-10-18T10:46:19.111 INFO:teuthology.orchestra.run.smithi089.stdout:55834574857 2023-10-18T10:46:19.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.110+0000 7f5ca018f640 1 -- 172.21.15.89:0/1159290063 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5c980c36c0 con 0x7f5c98160bb0 2023-10-18T10:46:19.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.110+0000 7f72031ba640 1 -- 172.21.15.89:0/3714246130 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f71e80656d0 msgr2=0x7f71e8067b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:19.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.110+0000 7f72031ba640 1 --2- 172.21.15.89:0/3714246130 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f71e80656d0 0x7f71e8067b90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f71f80ae000 tx=0x7f71f8008040 comp rx=0 tx=0).stop 2023-10-18T10:46:19.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.110+0000 7f72031ba640 1 -- 172.21.15.89:0/3714246130 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f71fc15c6d0 msgr2=0x7f71fc15f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:19.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.110+0000 7f72031ba640 1 --2- 172.21.15.89:0/3714246130 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f71fc15c6d0 0x7f71fc15f120 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f71f400c780 tx=0x7f71f400cc40 comp rx=0 tx=0).stop 2023-10-18T10:46:19.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.110+0000 7f72031ba640 1 -- 172.21.15.89:0/3714246130 shutdown_connections 2023-10-18T10:46:19.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.110+0000 7f72031ba640 1 --2- 172.21.15.89:0/3714246130 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f71e80656d0 0x7f71e8067b90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.110+0000 7f72031ba640 1 --2- 172.21.15.89:0/3714246130 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f71fc15c6d0 0x7f71fc15f120 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.110+0000 7f72031ba640 1 --2- 172.21.15.89:0/3714246130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71fc15c2f0 0x7f71fc15ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.110+0000 7f72031ba640 1 --2- 172.21.15.89:0/3714246130 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71fc0d0270 0x7f71fc0cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.110+0000 7f72031ba640 1 -- 172.21.15.89:0/3714246130 >> 172.21.15.89:0/3714246130 conn(0x7f71fc069ad0 msgr2=0x7f71fc15dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:19.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.110+0000 7f5c8e7fc640 1 -- 172.21.15.89:0/1159290063 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5c980c36c0 con 0x7f5c98160bb0 2023-10-18T10:46:19.116 INFO:teuthology.orchestra.run.smithi089.stdout:55834574858 2023-10-18T10:46:19.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.110+0000 7f72031ba640 1 -- 172.21.15.89:0/3714246130 shutdown_connections 2023-10-18T10:46:19.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.110+0000 7f72031ba640 1 -- 172.21.15.89:0/3714246130 wait complete. 2023-10-18T10:46:19.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.114+0000 7f5ca018f640 1 -- 172.21.15.89:0/1159290063 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5c7c0656d0 msgr2=0x7f5c7c067b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:19.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.114+0000 7f5ca018f640 1 --2- 172.21.15.89:0/1159290063 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5c7c0656d0 0x7f5c7c067b90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5c880ae000 tx=0x7f5c88004740 comp rx=0 tx=0).stop 2023-10-18T10:46:19.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.114+0000 7f5ca018f640 1 -- 172.21.15.89:0/1159290063 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c98160bb0 msgr2=0x7f5c980c5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:19.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.114+0000 7f5ca018f640 1 --2- 172.21.15.89:0/1159290063 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c98160bb0 0x7f5c980c5ce0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5c9400e7f0 tx=0x7f5c9400ecb0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.114+0000 7f5ca018f640 1 -- 172.21.15.89:0/1159290063 shutdown_connections 2023-10-18T10:46:19.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.114+0000 7f5ca018f640 1 --2- 172.21.15.89:0/1159290063 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5c7c0656d0 0x7f5c7c067b90 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.114+0000 7f5ca018f640 1 --2- 172.21.15.89:0/1159290063 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c98160bb0 0x7f5c980c5ce0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.114+0000 7f5ca018f640 1 --2- 172.21.15.89:0/1159290063 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c980c8bc0 0x7f5c980c2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.114+0000 7f5ca018f640 1 --2- 172.21.15.89:0/1159290063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c9806df60 0x7f5c980c0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.114+0000 7f5ca018f640 1 -- 172.21.15.89:0/1159290063 >> 172.21.15.89:0/1159290063 conn(0x7f5c980afe00 msgr2=0x7f5c980ac720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:19.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.114+0000 7fe1b4d34640 1 -- 172.21.15.89:0/40217697 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe1b006c720 con 0x7fe1b015a4e0 2023-10-18T10:46:19.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.118+0000 7f5ca018f640 1 -- 172.21.15.89:0/1159290063 shutdown_connections 2023-10-18T10:46:19.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.118+0000 7f5ca018f640 1 -- 172.21.15.89:0/1159290063 wait complete. 2023-10-18T10:46:19.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.118+0000 7fe1acff9640 1 -- 172.21.15.89:0/40217697 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe1a8053130 con 0x7fe1b015a4e0 2023-10-18T10:46:19.121 INFO:teuthology.orchestra.run.smithi089.stdout:55834574857 2023-10-18T10:46:19.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.118+0000 7fe1b4d34640 1 -- 172.21.15.89:0/40217697 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe190065680 msgr2=0x7fe190067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:19.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.118+0000 7fe1b4d34640 1 --2- 172.21.15.89:0/40217697 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe190065680 0x7fe190067b40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe1a000c420 tx=0x7fe1a0009570 comp rx=0 tx=0).stop 2023-10-18T10:46:19.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.118+0000 7fe1b4d34640 1 -- 172.21.15.89:0/40217697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1b015a4e0 msgr2=0x7fe1b02017d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:19.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.118+0000 7fe1b4d34640 1 --2- 172.21.15.89:0/40217697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1b015a4e0 0x7fe1b02017d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe1a800c390 tx=0x7fe1a800c850 comp rx=0 tx=0).stop 2023-10-18T10:46:19.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.118+0000 7fe1b4d34640 1 -- 172.21.15.89:0/40217697 shutdown_connections 2023-10-18T10:46:19.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.118+0000 7fe1b4d34640 1 --2- 172.21.15.89:0/40217697 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe190065680 0x7fe190067b40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.118+0000 7fe1b4d34640 1 --2- 172.21.15.89:0/40217697 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1b00cb660 0x7fe1b01fb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.118+0000 7fe1b4d34640 1 --2- 172.21.15.89:0/40217697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1b015a4e0 0x7fe1b02017d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.118+0000 7fe1b4d34640 1 --2- 172.21.15.89:0/40217697 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1b006cbf0 0x7fe1b0157ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.118+0000 7fe1b4d34640 1 -- 172.21.15.89:0/40217697 >> 172.21.15.89:0/40217697 conn(0x7fe1b00cab40 msgr2=0x7fe1b015e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:19.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.122+0000 7fe1b4d34640 1 -- 172.21.15.89:0/40217697 shutdown_connections 2023-10-18T10:46:19.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.122+0000 7fe1b4d34640 1 -- 172.21.15.89:0/40217697 wait complete. 2023-10-18T10:46:19.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.122+0000 7f96e759e640 1 -- 172.21.15.89:0/3336568431 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f96e8159c00 con 0x7f96e815c310 2023-10-18T10:46:19.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.122+0000 7f96d6ffd640 1 -- 172.21.15.89:0/3336568431 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f96e8159c00 con 0x7f96e815c310 2023-10-18T10:46:19.125 INFO:teuthology.orchestra.run.smithi089.stdout:55834574856 2023-10-18T10:46:19.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.126+0000 7f5cb3f2c640 1 -- 172.21.15.89:0/2034305004 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f5cac15d3f0 con 0x7f5cac0ca340 2023-10-18T10:46:19.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.126+0000 7f96e759e640 1 -- 172.21.15.89:0/3336568431 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f96c8065750 msgr2=0x7f96c8067c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:19.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.126+0000 7f96e759e640 1 --2- 172.21.15.89:0/3336568431 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f96c8065750 0x7f96c8067c10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f96d800b810 tx=0x7f96d8008040 comp rx=0 tx=0).stop 2023-10-18T10:46:19.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.126+0000 7f96e759e640 1 -- 172.21.15.89:0/3336568431 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96e815c310 msgr2=0x7f96e80cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:19.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.126+0000 7f96e759e640 1 --2- 172.21.15.89:0/3336568431 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96e815c310 0x7f96e80cb8a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f96e001af60 tx=0x7f96e0019650 comp rx=0 tx=0).stop 2023-10-18T10:46:19.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.126+0000 7f96e759e640 1 -- 172.21.15.89:0/3336568431 shutdown_connections 2023-10-18T10:46:19.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.126+0000 7f96e759e640 1 --2- 172.21.15.89:0/3336568431 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f96c8065750 0x7f96c8067c10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.126+0000 7f96e759e640 1 --2- 172.21.15.89:0/3336568431 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96e815c310 0x7f96e80cb8a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.126+0000 7f96e759e640 1 --2- 172.21.15.89:0/3336568431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96e80cf950 0x7f96e814f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.126+0000 7f96e759e640 1 --2- 172.21.15.89:0/3336568431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96e80cd4e0 0x7f96e814fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.126+0000 7f96e759e640 1 -- 172.21.15.89:0/3336568431 >> 172.21.15.89:0/3336568431 conn(0x7f96e8070120 msgr2=0x7f96e8158170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:19.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.126+0000 7f96e759e640 1 -- 172.21.15.89:0/3336568431 shutdown_connections 2023-10-18T10:46:19.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.126+0000 7f96e759e640 1 -- 172.21.15.89:0/3336568431 wait complete. 2023-10-18T10:46:19.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.126+0000 7f5ca27fc640 1 -- 172.21.15.89:0/2034305004 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5ca801f090 con 0x7f5cac0ca340 2023-10-18T10:46:19.132 INFO:teuthology.orchestra.run.smithi089.stdout:55834574857 2023-10-18T10:46:19.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.130+0000 7f6faf59e640 1 -- 172.21.15.89:0/905788006 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f6f94005510 con 0x7f6fa8047bd0 2023-10-18T10:46:19.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.130+0000 7f6f9effd640 1 -- 172.21.15.89:0/905788006 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6fa4055f40 con 0x7f6fa8047bd0 2023-10-18T10:46:19.133 INFO:teuthology.orchestra.run.smithi089.stdout:55834574857 2023-10-18T10:46:19.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.134+0000 7f256df37640 1 -- 172.21.15.89:0/3900946924 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f256804dbe0 con 0x7f25680b02d0 2023-10-18T10:46:19.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.134+0000 7f5cb3f2c640 1 -- 172.21.15.89:0/2034305004 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5c90065680 msgr2=0x7f5c90067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:19.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.134+0000 7f5cb3f2c640 1 --2- 172.21.15.89:0/2034305004 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5c90065680 0x7f5c90067b40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c0ae000 tx=0x7f5c9c0a6070 comp rx=0 tx=0).stop 2023-10-18T10:46:19.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.134+0000 7f5cb3f2c640 1 -- 172.21.15.89:0/2034305004 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cac0ca340 msgr2=0x7f5cac0c1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:19.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.134+0000 7f5cb3f2c640 1 --2- 172.21.15.89:0/2034305004 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cac0ca340 0x7f5cac0c1590 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5ca800c7c0 tx=0x7f5ca800cc80 comp rx=0 tx=0).stop 2023-10-18T10:46:19.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.134+0000 7f6faf59e640 1 -- 172.21.15.89:0/905788006 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6f90065780 msgr2=0x7f6f90067c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:19.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.134+0000 7f6faf59e640 1 --2- 172.21.15.89:0/905788006 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6f90065780 0x7f6f90067c40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6fa00ad000 tx=0x7f6fa0008040 comp rx=0 tx=0).stop 2023-10-18T10:46:19.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.134+0000 7f6faf59e640 1 -- 172.21.15.89:0/905788006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fa8047bd0 msgr2=0x7f6fa80e5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:19.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.134+0000 7f6faf59e640 1 --2- 172.21.15.89:0/905788006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fa8047bd0 0x7f6fa80e5b70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4009f30 tx=0x7f6fa400c4f0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.134+0000 7f6faf59e640 1 -- 172.21.15.89:0/905788006 shutdown_connections 2023-10-18T10:46:19.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.134+0000 7f6faf59e640 1 --2- 172.21.15.89:0/905788006 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6f90065780 0x7f6f90067c40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.134+0000 7f6faf59e640 1 --2- 172.21.15.89:0/905788006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6fa8048200 0x7f6fa80e60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.134+0000 7f6faf59e640 1 --2- 172.21.15.89:0/905788006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fa8047bd0 0x7f6fa80e5b70 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.134+0000 7f6faf59e640 1 --2- 172.21.15.89:0/905788006 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6fa8033d10 0x7f6fa80ea490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.134+0000 7f6faf59e640 1 -- 172.21.15.89:0/905788006 >> 172.21.15.89:0/905788006 conn(0x7f6fa8054b10 msgr2=0x7f6fa8007250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:19.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.134+0000 7f254ffff640 1 -- 172.21.15.89:0/3900946924 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f25580ad090 con 0x7f25680b02d0 2023-10-18T10:46:19.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.134+0000 7f6faf59e640 1 -- 172.21.15.89:0/905788006 shutdown_connections 2023-10-18T10:46:19.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.134+0000 7f6faf59e640 1 -- 172.21.15.89:0/905788006 wait complete. 2023-10-18T10:46:19.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.134+0000 7f5cb3f2c640 1 -- 172.21.15.89:0/2034305004 shutdown_connections 2023-10-18T10:46:19.139 INFO:teuthology.orchestra.run.smithi089.stdout:55834574857 2023-10-18T10:46:19.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.134+0000 7f5cb3f2c640 1 --2- 172.21.15.89:0/2034305004 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5c90065680 0x7f5c90067b40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.134+0000 7f5cb3f2c640 1 --2- 172.21.15.89:0/2034305004 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5cac0ca340 0x7f5cac0c1590 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.134+0000 7f5cb3f2c640 1 --2- 172.21.15.89:0/2034305004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cac15d740 0x7f5cac15dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.134+0000 7f5cb3f2c640 1 --2- 172.21.15.89:0/2034305004 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cac06ff00 0x7f5cac0ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.134+0000 7f5cb3f2c640 1 -- 172.21.15.89:0/2034305004 >> 172.21.15.89:0/2034305004 conn(0x7f5cac069f00 msgr2=0x7f5cac06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:19.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.134+0000 7f5cb3f2c640 1 -- 172.21.15.89:0/2034305004 shutdown_connections 2023-10-18T10:46:19.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.134+0000 7f5cb3f2c640 1 -- 172.21.15.89:0/2034305004 wait complete. 2023-10-18T10:46:19.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.138+0000 7f256df37640 1 -- 172.21.15.89:0/3900946924 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2554065680 msgr2=0x7f2554067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:19.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.138+0000 7f256df37640 1 --2- 172.21.15.89:0/3900946924 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2554065680 0x7f2554067b40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f25680c88c0 tx=0x7f255c009040 comp rx=0 tx=0).stop 2023-10-18T10:46:19.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.138+0000 7f256df37640 1 -- 172.21.15.89:0/3900946924 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25680b02d0 msgr2=0x7f25680c4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:19.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.138+0000 7f256df37640 1 --2- 172.21.15.89:0/3900946924 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25680b02d0 0x7f25680c4810 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f25580b3040 tx=0x7f255801d100 comp rx=0 tx=0).stop 2023-10-18T10:46:19.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.138+0000 7f256df37640 1 -- 172.21.15.89:0/3900946924 shutdown_connections 2023-10-18T10:46:19.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.138+0000 7f256df37640 1 --2- 172.21.15.89:0/3900946924 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2554065680 0x7f2554067b40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.138+0000 7f256df37640 1 --2- 172.21.15.89:0/3900946924 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2568159530 0x7f25680c7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.138+0000 7f256df37640 1 --2- 172.21.15.89:0/3900946924 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25680c94b0 0x7f25680c7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.138+0000 7f256df37640 1 --2- 172.21.15.89:0/3900946924 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25680b02d0 0x7f25680c4810 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.138+0000 7f256df37640 1 -- 172.21.15.89:0/3900946924 >> 172.21.15.89:0/3900946924 conn(0x7f2568069f00 msgr2=0x7f2568152770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:19.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.138+0000 7f256df37640 1 -- 172.21.15.89:0/3900946924 shutdown_connections 2023-10-18T10:46:19.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.138+0000 7f256df37640 1 -- 172.21.15.89:0/3900946924 wait complete. 2023-10-18T10:46:19.145 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.6 2023-10-18T10:46:19.145 DEBUG:teuthology.parallel:result is None 2023-10-18T10:46:19.162 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.2 2023-10-18T10:46:19.163 DEBUG:teuthology.parallel:result is None 2023-10-18T10:46:19.163 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.0 2023-10-18T10:46:19.164 DEBUG:teuthology.parallel:result is None 2023-10-18T10:46:19.167 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.3 2023-10-18T10:46:19.167 DEBUG:teuthology.parallel:result is None 2023-10-18T10:46:19.171 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.5 2023-10-18T10:46:19.171 DEBUG:teuthology.parallel:result is None 2023-10-18T10:46:19.176 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.4 2023-10-18T10:46:19.176 DEBUG:teuthology.parallel:result is None 2023-10-18T10:46:19.177 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.7 2023-10-18T10:46:19.177 DEBUG:teuthology.parallel:result is None 2023-10-18T10:46:19.178 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-18T10:46:19.178 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:46:19.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.354+0000 7f39767ba640 1 Processor -- start 2023-10-18T10:46:19.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.354+0000 7f39767ba640 1 -- start start 2023-10-18T10:46:19.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.354+0000 7f39767ba640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f39700cd4f0 0x7f3970143470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:19.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.354+0000 7f39767ba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3970145b70 0x7f3970160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:19.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.354+0000 7f39767ba640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39700cd8d0 con 0x7f39700ce070 2023-10-18T10:46:19.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.354+0000 7f39767ba640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3970069900 con 0x7f39700cd4f0 2023-10-18T10:46:19.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.354+0000 7f39767ba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f397015cbf0 con 0x7f3970145b70 2023-10-18T10:46:19.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.354+0000 7f396ffff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f39700cd4f0 0x7f3970143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:19.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.354+0000 7f396ffff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f39700cd4f0 0x7f3970143470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45514/0 (socket says 172.21.15.89:45514) 2023-10-18T10:46:19.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.354+0000 7f396ffff640 1 -- 172.21.15.89:0/1815197236 learned_addr learned my addr 172.21.15.89:0/1815197236 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:19.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.358+0000 7f396ffff640 1 -- 172.21.15.89:0/1815197236 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3970145b70 msgr2=0x7f3970160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:19.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.358+0000 7f396ffff640 1 --2- 172.21.15.89:0/1815197236 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3970145b70 0x7f3970160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.358+0000 7f396ffff640 1 -- 172.21.15.89:0/1815197236 >> v1:172.21.15.89:6789/0 conn(0x7f39700ce070 legacy=0x7f397006abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:19.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.358+0000 7f396ffff640 1 -- 172.21.15.89:0/1815197236 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3970161230 con 0x7f39700cd4f0 2023-10-18T10:46:19.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.358+0000 7f396ffff640 1 --2- 172.21.15.89:0/1815197236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39700cd4f0 0x7f3970143470 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3960009870 tx=0x7f396001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b456dd7f94770b87 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:19.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.358+0000 7f396effd640 1 -- 172.21.15.89:0/1815197236 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39600aa670 con 0x7f39700cd4f0 2023-10-18T10:46:19.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.358+0000 7f396effd640 1 -- 172.21.15.89:0/1815197236 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39600aa7d0 con 0x7f39700cd4f0 2023-10-18T10:46:19.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.358+0000 7f396effd640 1 -- 172.21.15.89:0/1815197236 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39600aab60 con 0x7f39700cd4f0 2023-10-18T10:46:19.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.362+0000 7f39767ba640 1 -- 172.21.15.89:0/1815197236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39700cd4f0 msgr2=0x7f3970143470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:19.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.362+0000 7f39767ba640 1 --2- 172.21.15.89:0/1815197236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39700cd4f0 0x7f3970143470 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3960009870 tx=0x7f396001c510 comp rx=0 tx=0).stop 2023-10-18T10:46:19.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.362+0000 7f39767ba640 1 -- 172.21.15.89:0/1815197236 shutdown_connections 2023-10-18T10:46:19.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.362+0000 7f39767ba640 1 --2- 172.21.15.89:0/1815197236 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3970145b70 0x7f3970160cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.362+0000 7f39767ba640 1 --2- 172.21.15.89:0/1815197236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39700cd4f0 0x7f3970143470 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.362+0000 7f39767ba640 1 -- 172.21.15.89:0/1815197236 >> 172.21.15.89:0/1815197236 conn(0x7f397006dad0 msgr2=0x7f397006dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:19.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.362+0000 7f39767ba640 1 -- 172.21.15.89:0/1815197236 shutdown_connections 2023-10-18T10:46:19.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.362+0000 7f39767ba640 1 -- 172.21.15.89:0/1815197236 wait complete. 2023-10-18T10:46:19.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.362+0000 7f39767ba640 1 Processor -- start 2023-10-18T10:46:19.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.362+0000 7f39767ba640 1 -- start start 2023-10-18T10:46:19.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.362+0000 7f39767ba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39700cd4f0 0x7f3970155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:19.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.362+0000 7f39767ba640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39700ce070 0x7f39700b1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:19.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.362+0000 7f39767ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3970145b70 0x7f3970152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:19.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.362+0000 7f39767ba640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39700c8fd0 con 0x7f39700ce070 2023-10-18T10:46:19.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.362+0000 7f39767ba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3970155ee0 con 0x7f3970145b70 2023-10-18T10:46:19.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.362+0000 7f39767ba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39700c9220 con 0x7f39700cd4f0 2023-10-18T10:46:19.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.362+0000 7f396f7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39700ce070 0x7f39700b1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:19.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.362+0000 7f396f7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39700ce070 0x7f39700b1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45518/0 (socket says 172.21.15.89:45518) 2023-10-18T10:46:19.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.362+0000 7f396f7fe640 1 -- 172.21.15.89:0/3923650680 learned_addr learned my addr 172.21.15.89:0/3923650680 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:19.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.362+0000 7f396ffff640 1 --2- 172.21.15.89:0/3923650680 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39700cd4f0 0x7f3970155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:19.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.362+0000 7f396f7fe640 1 -- 172.21.15.89:0/3923650680 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39700cd4f0 msgr2=0x7f3970155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:19.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.362+0000 7f3974d57640 1 --2- 172.21.15.89:0/3923650680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3970145b70 0x7f3970152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:19.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.362+0000 7f396f7fe640 1 --2- 172.21.15.89:0/3923650680 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39700cd4f0 0x7f3970155820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.362+0000 7f396f7fe640 1 -- 172.21.15.89:0/3923650680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3970145b70 msgr2=0x7f3970152740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:19.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.362+0000 7f396f7fe640 1 --2- 172.21.15.89:0/3923650680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3970145b70 0x7f3970152740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.362+0000 7f396f7fe640 1 -- 172.21.15.89:0/3923650680 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39600094d0 con 0x7f39700ce070 2023-10-18T10:46:19.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.362+0000 7f396ffff640 1 --2- 172.21.15.89:0/3923650680 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39700cd4f0 0x7f3970155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:19.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.362+0000 7f3974d57640 1 --2- 172.21.15.89:0/3923650680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3970145b70 0x7f3970152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:19.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.362+0000 7f396f7fe640 1 --2- 172.21.15.89:0/3923650680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39700ce070 0x7f39700b1730 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f396400b550 tx=0x7f396400ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:19.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.366+0000 7f396cff9640 1 -- 172.21.15.89:0/3923650680 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3964010870 con 0x7f39700ce070 2023-10-18T10:46:19.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.366+0000 7f396cff9640 1 -- 172.21.15.89:0/3923650680 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39640109f0 con 0x7f39700ce070 2023-10-18T10:46:19.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.366+0000 7f39767ba640 1 -- 172.21.15.89:0/3923650680 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39700c9500 con 0x7f39700ce070 2023-10-18T10:46:19.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.366+0000 7f396cff9640 1 -- 172.21.15.89:0/3923650680 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3964010da0 con 0x7f39700ce070 2023-10-18T10:46:19.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.366+0000 7f39767ba640 1 -- 172.21.15.89:0/3923650680 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39700c9ac0 con 0x7f39700ce070 2023-10-18T10:46:19.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.366+0000 7f39767ba640 1 -- 172.21.15.89:0/3923650680 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f397004dbe0 con 0x7f39700ce070 2023-10-18T10:46:19.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.366+0000 7f396cff9640 1 -- 172.21.15.89:0/3923650680 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f396400f3c0 con 0x7f39700ce070 2023-10-18T10:46:19.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.366+0000 7f396cff9640 1 --2- 172.21.15.89:0/3923650680 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f395c065680 0x7f395c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:19.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.366+0000 7f396cff9640 1 -- 172.21.15.89:0/3923650680 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f39640831c0 con 0x7f39700ce070 2023-10-18T10:46:19.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.366+0000 7f396ffff640 1 --2- 172.21.15.89:0/3923650680 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f395c065680 0x7f395c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:19.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.370+0000 7f396ffff640 1 --2- 172.21.15.89:0/3923650680 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f395c065680 0x7f395c067b40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f39600ae000 tx=0x7f3960008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:19.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.370+0000 7f396cff9640 1 -- 172.21.15.89:0/3923650680 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f39640508f0 con 0x7f39700ce070 2023-10-18T10:46:19.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.518+0000 7f39767ba640 1 -- 172.21.15.89:0/3923650680 --> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f397006a3b0 con 0x7f395c065680 2023-10-18T10:46:19.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.530+0000 7f396cff9640 1 -- 172.21.15.89:0/3923650680 <== mgr.4099 v2:172.21.15.154:6832/1086608968 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367299 (secure 0 0 0) 0x7f39600ba090 con 0x7f395c065680 2023-10-18T10:46:19.533 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:19.536 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-18T10:46:19.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.530+0000 7f39767ba640 1 -- 172.21.15.89:0/3923650680 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f395c065680 msgr2=0x7f395c067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:19.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.530+0000 7f39767ba640 1 --2- 172.21.15.89:0/3923650680 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f395c065680 0x7f395c067b40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f39600ae000 tx=0x7f3960008040 comp rx=0 tx=0).stop 2023-10-18T10:46:19.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.530+0000 7f39767ba640 1 -- 172.21.15.89:0/3923650680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39700ce070 msgr2=0x7f39700b1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:19.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.530+0000 7f39767ba640 1 --2- 172.21.15.89:0/3923650680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39700ce070 0x7f39700b1730 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f396400b550 tx=0x7f396400ba10 comp rx=0 tx=0).stop 2023-10-18T10:46:19.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.530+0000 7f39767ba640 1 -- 172.21.15.89:0/3923650680 shutdown_connections 2023-10-18T10:46:19.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.534+0000 7f39767ba640 1 --2- 172.21.15.89:0/3923650680 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f395c065680 0x7f395c067b40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.534+0000 7f39767ba640 1 --2- 172.21.15.89:0/3923650680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3970145b70 0x7f3970152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.534+0000 7f39767ba640 1 --2- 172.21.15.89:0/3923650680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39700ce070 0x7f39700b1730 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.534+0000 7f39767ba640 1 --2- 172.21.15.89:0/3923650680 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39700cd4f0 0x7f3970155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.534+0000 7f39767ba640 1 -- 172.21.15.89:0/3923650680 >> 172.21.15.89:0/3923650680 conn(0x7f397006dad0 msgr2=0x7f39700d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:19.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.534+0000 7f39767ba640 1 -- 172.21.15.89:0/3923650680 shutdown_connections 2023-10-18T10:46:19.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.534+0000 7f39767ba640 1 -- 172.21.15.89:0/3923650680 wait complete. 2023-10-18T10:46:19.553 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2023-10-18T10:46:19.058633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219660,"kb_used_data":3496,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754755060,"statfs":{"total":773094113280,"available":772869181440,"internally_reserved":0,"allocated":3579904,"data_stored":2149222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2229,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003605"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084532+0000","last_change":"2023-10-18T10:45:59.809013+0000","last_active":"2023-10-18T10:46:01.084532+0000","last_peered":"2023-10-18T10:46:01.084532+0000","last_clean":"2023-10-18T10:46:01.084532+0000","last_became_active":"2023-10-18T10:45:59.808636+0000","last_became_peered":"2023-10-18T10:45:59.808636+0000","last_unstale":"2023-10-18T10:46:01.084532+0000","last_undegraded":"2023-10-18T10:46:01.084532+0000","last_fullsized":"2023-10-18T10:46:01.084532+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:30:08.797534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085907+0000","last_change":"2023-10-18T10:45:55.297617+0000","last_active":"2023-10-18T10:46:01.085907+0000","last_peered":"2023-10-18T10:46:01.085907+0000","last_clean":"2023-10-18T10:46:01.085907+0000","last_became_active":"2023-10-18T10:45:55.296996+0000","last_became_peered":"2023-10-18T10:45:55.296996+0000","last_unstale":"2023-10-18T10:46:01.085907+0000","last_undegraded":"2023-10-18T10:46:01.085907+0000","last_fullsized":"2023-10-18T10:46:01.085907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:04:27.847910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768182+0000","last_change":"2023-10-18T10:45:55.770859+0000","last_active":"2023-10-18T10:46:00.768182+0000","last_peered":"2023-10-18T10:46:00.768182+0000","last_clean":"2023-10-18T10:46:00.768182+0000","last_became_active":"2023-10-18T10:45:55.769578+0000","last_became_peered":"2023-10-18T10:45:55.769578+0000","last_unstale":"2023-10-18T10:46:00.768182+0000","last_undegraded":"2023-10-18T10:46:00.768182+0000","last_fullsized":"2023-10-18T10:46:00.768182+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:08:17.812888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.767991+0000","last_change":"2023-10-18T10:45:59.790837+0000","last_active":"2023-10-18T10:46:00.767991+0000","last_peered":"2023-10-18T10:46:00.767991+0000","last_clean":"2023-10-18T10:46:00.767991+0000","last_became_active":"2023-10-18T10:45:59.790488+0000","last_became_peered":"2023-10-18T10:45:59.790488+0000","last_unstale":"2023-10-18T10:46:00.767991+0000","last_undegraded":"2023-10-18T10:46:00.767991+0000","last_fullsized":"2023-10-18T10:46:00.767991+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:43:23.609561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902666+0000","last_change":"2023-10-18T10:45:55.295114+0000","last_active":"2023-10-18T10:46:00.902666+0000","last_peered":"2023-10-18T10:46:00.902666+0000","last_clean":"2023-10-18T10:46:00.902666+0000","last_became_active":"2023-10-18T10:45:55.294672+0000","last_became_peered":"2023-10-18T10:45:55.294672+0000","last_unstale":"2023-10-18T10:46:00.902666+0000","last_undegraded":"2023-10-18T10:46:00.902666+0000","last_fullsized":"2023-10-18T10:46:00.902666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:06:43.162381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902162+0000","last_change":"2023-10-18T10:45:55.756885+0000","last_active":"2023-10-18T10:46:00.902162+0000","last_peered":"2023-10-18T10:46:00.902162+0000","last_clean":"2023-10-18T10:46:00.902162+0000","last_became_active":"2023-10-18T10:45:55.756184+0000","last_became_peered":"2023-10-18T10:45:55.756184+0000","last_unstale":"2023-10-18T10:46:00.902162+0000","last_undegraded":"2023-10-18T10:46:00.902162+0000","last_fullsized":"2023-10-18T10:46:00.902162+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:20:46.641913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085021+0000","last_change":"2023-10-18T10:45:59.796520+0000","last_active":"2023-10-18T10:46:01.085021+0000","last_peered":"2023-10-18T10:46:01.085021+0000","last_clean":"2023-10-18T10:46:01.085021+0000","last_became_active":"2023-10-18T10:45:59.796056+0000","last_became_peered":"2023-10-18T10:45:59.796056+0000","last_unstale":"2023-10-18T10:46:01.085021+0000","last_undegraded":"2023-10-18T10:46:01.085021+0000","last_fullsized":"2023-10-18T10:46:01.085021+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:55:12.299657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085752+0000","last_change":"2023-10-18T10:45:55.302985+0000","last_active":"2023-10-18T10:46:01.085752+0000","last_peered":"2023-10-18T10:46:01.085752+0000","last_clean":"2023-10-18T10:46:01.085752+0000","last_became_active":"2023-10-18T10:45:55.302657+0000","last_became_peered":"2023-10-18T10:45:55.302657+0000","last_unstale":"2023-10-18T10:46:01.085752+0000","last_undegraded":"2023-10-18T10:46:01.085752+0000","last_fullsized":"2023-10-18T10:46:01.085752+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:55:03.412092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768360+0000","last_change":"2023-10-18T10:45:55.776143+0000","last_active":"2023-10-18T10:46:00.768360+0000","last_peered":"2023-10-18T10:46:00.768360+0000","last_clean":"2023-10-18T10:46:00.768360+0000","last_became_active":"2023-10-18T10:45:55.770969+0000","last_became_peered":"2023-10-18T10:45:55.770969+0000","last_unstale":"2023-10-18T10:46:00.768360+0000","last_undegraded":"2023-10-18T10:46:00.768360+0000","last_fullsized":"2023-10-18T10:46:00.768360+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:53:59.036438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085289+0000","last_change":"2023-10-18T10:45:59.810699+0000","last_active":"2023-10-18T10:46:01.085289+0000","last_peered":"2023-10-18T10:46:01.085289+0000","last_clean":"2023-10-18T10:46:01.085289+0000","last_became_active":"2023-10-18T10:45:59.810219+0000","last_became_peered":"2023-10-18T10:45:59.810219+0000","last_unstale":"2023-10-18T10:46:01.085289+0000","last_undegraded":"2023-10-18T10:46:01.085289+0000","last_fullsized":"2023-10-18T10:46:01.085289+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:06:30.723321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085485+0000","last_change":"2023-10-18T10:45:55.300287+0000","last_active":"2023-10-18T10:46:01.085485+0000","last_peered":"2023-10-18T10:46:01.085485+0000","last_clean":"2023-10-18T10:46:01.085485+0000","last_became_active":"2023-10-18T10:45:55.300014+0000","last_became_peered":"2023-10-18T10:45:55.300014+0000","last_unstale":"2023-10-18T10:46:01.085485+0000","last_undegraded":"2023-10-18T10:46:01.085485+0000","last_fullsized":"2023-10-18T10:46:01.085485+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:40:14.773073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768476+0000","last_change":"2023-10-18T10:45:55.772481+0000","last_active":"2023-10-18T10:46:00.768476+0000","last_peered":"2023-10-18T10:46:00.768476+0000","last_clean":"2023-10-18T10:46:00.768476+0000","last_became_active":"2023-10-18T10:45:55.770365+0000","last_became_peered":"2023-10-18T10:45:55.770365+0000","last_unstale":"2023-10-18T10:46:00.768476+0000","last_undegraded":"2023-10-18T10:46:00.768476+0000","last_fullsized":"2023-10-18T10:46:00.768476+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:21:41.323647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085226+0000","last_change":"2023-10-18T10:45:59.796654+0000","last_active":"2023-10-18T10:46:01.085226+0000","last_peered":"2023-10-18T10:46:01.085226+0000","last_clean":"2023-10-18T10:46:01.085226+0000","last_became_active":"2023-10-18T10:45:59.796115+0000","last_became_peered":"2023-10-18T10:45:59.796115+0000","last_unstale":"2023-10-18T10:46:01.085226+0000","last_undegraded":"2023-10-18T10:46:01.085226+0000","last_fullsized":"2023-10-18T10:46:01.085226+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:28:08.869152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086358+0000","last_change":"2023-10-18T10:45:55.302642+0000","last_active":"2023-10-18T10:46:01.086358+0000","last_peered":"2023-10-18T10:46:01.086358+0000","last_clean":"2023-10-18T10:46:01.086358+0000","last_became_active":"2023-10-18T10:45:55.302307+0000","last_became_peered":"2023-10-18T10:45:55.302307+0000","last_unstale":"2023-10-18T10:46:01.086358+0000","last_undegraded":"2023-10-18T10:46:01.086358+0000","last_fullsized":"2023-10-18T10:46:01.086358+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:40:52.185453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.767207+0000","last_change":"2023-10-18T10:45:55.765319+0000","last_active":"2023-10-18T10:46:00.767207+0000","last_peered":"2023-10-18T10:46:00.767207+0000","last_clean":"2023-10-18T10:46:00.767207+0000","last_became_active":"2023-10-18T10:45:55.764958+0000","last_became_peered":"2023-10-18T10:45:55.764958+0000","last_unstale":"2023-10-18T10:46:00.767207+0000","last_undegraded":"2023-10-18T10:46:00.767207+0000","last_fullsized":"2023-10-18T10:46:00.767207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:30:49.094396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768337+0000","last_change":"2023-10-18T10:45:59.791333+0000","last_active":"2023-10-18T10:46:00.768337+0000","last_peered":"2023-10-18T10:46:00.768337+0000","last_clean":"2023-10-18T10:46:00.768337+0000","last_became_active":"2023-10-18T10:45:59.790903+0000","last_became_peered":"2023-10-18T10:45:59.790903+0000","last_unstale":"2023-10-18T10:46:00.768337+0000","last_undegraded":"2023-10-18T10:46:00.768337+0000","last_fullsized":"2023-10-18T10:46:00.768337+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:52:13.857250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768597+0000","last_change":"2023-10-18T10:45:55.302330+0000","last_active":"2023-10-18T10:46:00.768597+0000","last_peered":"2023-10-18T10:46:00.768597+0000","last_clean":"2023-10-18T10:46:00.768597+0000","last_became_active":"2023-10-18T10:45:55.301709+0000","last_became_peered":"2023-10-18T10:45:55.301709+0000","last_unstale":"2023-10-18T10:46:00.768597+0000","last_undegraded":"2023-10-18T10:46:00.768597+0000","last_fullsized":"2023-10-18T10:46:00.768597+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:40:49.128175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902373+0000","last_change":"2023-10-18T10:45:55.767859+0000","last_active":"2023-10-18T10:46:00.902373+0000","last_peered":"2023-10-18T10:46:00.902373+0000","last_clean":"2023-10-18T10:46:00.902373+0000","last_became_active":"2023-10-18T10:45:55.767505+0000","last_became_peered":"2023-10-18T10:45:55.767505+0000","last_unstale":"2023-10-18T10:46:00.902373+0000","last_undegraded":"2023-10-18T10:46:00.902373+0000","last_fullsized":"2023-10-18T10:46:00.902373+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:12:23.654089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.770164+0000","last_change":"2023-10-18T10:45:59.794892+0000","last_active":"2023-10-18T10:46:00.770164+0000","last_peered":"2023-10-18T10:46:00.770164+0000","last_clean":"2023-10-18T10:46:00.770164+0000","last_became_active":"2023-10-18T10:45:59.794477+0000","last_became_peered":"2023-10-18T10:45:59.794477+0000","last_unstale":"2023-10-18T10:46:00.770164+0000","last_undegraded":"2023-10-18T10:46:00.770164+0000","last_fullsized":"2023-10-18T10:46:00.770164+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:04:07.040979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084932+0000","last_change":"2023-10-18T10:45:55.290491+0000","last_active":"2023-10-18T10:46:01.084932+0000","last_peered":"2023-10-18T10:46:01.084932+0000","last_clean":"2023-10-18T10:46:01.084932+0000","last_became_active":"2023-10-18T10:45:55.290134+0000","last_became_peered":"2023-10-18T10:45:55.290134+0000","last_unstale":"2023-10-18T10:46:01.084932+0000","last_undegraded":"2023-10-18T10:46:01.084932+0000","last_fullsized":"2023-10-18T10:46:01.084932+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:39.977541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086354+0000","last_change":"2023-10-18T10:45:55.765415+0000","last_active":"2023-10-18T10:46:01.086354+0000","last_peered":"2023-10-18T10:46:01.086354+0000","last_clean":"2023-10-18T10:46:01.086354+0000","last_became_active":"2023-10-18T10:45:55.764573+0000","last_became_peered":"2023-10-18T10:45:55.764573+0000","last_unstale":"2023-10-18T10:46:01.086354+0000","last_undegraded":"2023-10-18T10:46:01.086354+0000","last_fullsized":"2023-10-18T10:46:01.086354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:33:49.751994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086225+0000","last_change":"2023-10-18T10:45:59.801842+0000","last_active":"2023-10-18T10:46:01.086225+0000","last_peered":"2023-10-18T10:46:01.086225+0000","last_clean":"2023-10-18T10:46:01.086225+0000","last_became_active":"2023-10-18T10:45:59.801367+0000","last_became_peered":"2023-10-18T10:45:59.801367+0000","last_unstale":"2023-10-18T10:46:01.086225+0000","last_undegraded":"2023-10-18T10:46:01.086225+0000","last_fullsized":"2023-10-18T10:46:01.086225+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:56:46.630947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902154+0000","last_change":"2023-10-18T10:45:55.298611+0000","last_active":"2023-10-18T10:46:00.902154+0000","last_peered":"2023-10-18T10:46:00.902154+0000","last_clean":"2023-10-18T10:46:00.902154+0000","last_became_active":"2023-10-18T10:45:55.297980+0000","last_became_peered":"2023-10-18T10:45:55.297980+0000","last_unstale":"2023-10-18T10:46:00.902154+0000","last_undegraded":"2023-10-18T10:46:00.902154+0000","last_fullsized":"2023-10-18T10:46:00.902154+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:05:56.466612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.770285+0000","last_change":"2023-10-18T10:45:55.775435+0000","last_active":"2023-10-18T10:46:00.770285+0000","last_peered":"2023-10-18T10:46:00.770285+0000","last_clean":"2023-10-18T10:46:00.770285+0000","last_became_active":"2023-10-18T10:45:55.774841+0000","last_became_peered":"2023-10-18T10:45:55.774841+0000","last_unstale":"2023-10-18T10:46:00.770285+0000","last_undegraded":"2023-10-18T10:46:00.770285+0000","last_fullsized":"2023-10-18T10:46:00.770285+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:27:03.226674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.901875+0000","last_change":"2023-10-18T10:45:55.768221+0000","last_active":"2023-10-18T10:46:00.901875+0000","last_peered":"2023-10-18T10:46:00.901875+0000","last_clean":"2023-10-18T10:46:00.901875+0000","last_became_active":"2023-10-18T10:45:55.767975+0000","last_became_peered":"2023-10-18T10:45:55.767975+0000","last_unstale":"2023-10-18T10:46:00.901875+0000","last_undegraded":"2023-10-18T10:46:00.901875+0000","last_fullsized":"2023-10-18T10:46:00.901875+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:14:00.849685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086003+0000","last_change":"2023-10-18T10:45:55.291798+0000","last_active":"2023-10-18T10:46:01.086003+0000","last_peered":"2023-10-18T10:46:01.086003+0000","last_clean":"2023-10-18T10:46:01.086003+0000","last_became_active":"2023-10-18T10:45:55.291380+0000","last_became_peered":"2023-10-18T10:45:55.291380+0000","last_unstale":"2023-10-18T10:46:01.086003+0000","last_undegraded":"2023-10-18T10:46:01.086003+0000","last_fullsized":"2023-10-18T10:46:01.086003+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:49:45.673807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.769758+0000","last_change":"2023-10-18T10:45:59.794170+0000","last_active":"2023-10-18T10:46:00.769758+0000","last_peered":"2023-10-18T10:46:00.769758+0000","last_clean":"2023-10-18T10:46:00.769758+0000","last_became_active":"2023-10-18T10:45:59.793675+0000","last_became_peered":"2023-10-18T10:45:59.793675+0000","last_unstale":"2023-10-18T10:46:00.769758+0000","last_undegraded":"2023-10-18T10:46:00.769758+0000","last_fullsized":"2023-10-18T10:46:00.769758+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:08:00.737560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.901966+0000","last_change":"2023-10-18T10:45:55.765758+0000","last_active":"2023-10-18T10:46:00.901966+0000","last_peered":"2023-10-18T10:46:00.901966+0000","last_clean":"2023-10-18T10:46:00.901966+0000","last_became_active":"2023-10-18T10:45:55.765436+0000","last_became_peered":"2023-10-18T10:45:55.765436+0000","last_unstale":"2023-10-18T10:46:00.901966+0000","last_undegraded":"2023-10-18T10:46:00.901966+0000","last_fullsized":"2023-10-18T10:46:00.901966+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:40:53.192771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086132+0000","last_change":"2023-10-18T10:45:55.295720+0000","last_active":"2023-10-18T10:46:01.086132+0000","last_peered":"2023-10-18T10:46:01.086132+0000","last_clean":"2023-10-18T10:46:01.086132+0000","last_became_active":"2023-10-18T10:45:55.295401+0000","last_became_peered":"2023-10-18T10:45:55.295401+0000","last_unstale":"2023-10-18T10:46:01.086132+0000","last_undegraded":"2023-10-18T10:46:01.086132+0000","last_fullsized":"2023-10-18T10:46:01.086132+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:49:45.673807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084670+0000","last_change":"2023-10-18T10:45:59.798312+0000","last_active":"2023-10-18T10:46:01.084670+0000","last_peered":"2023-10-18T10:46:01.084670+0000","last_clean":"2023-10-18T10:46:01.084670+0000","last_became_active":"2023-10-18T10:45:59.797944+0000","last_became_peered":"2023-10-18T10:45:59.797944+0000","last_unstale":"2023-10-18T10:46:01.084670+0000","last_undegraded":"2023-10-18T10:46:01.084670+0000","last_fullsized":"2023-10-18T10:46:01.084670+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:51:40.857910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085288+0000","last_change":"2023-10-18T10:45:55.307908+0000","last_active":"2023-10-18T10:46:01.085288+0000","last_peered":"2023-10-18T10:46:01.085288+0000","last_clean":"2023-10-18T10:46:01.085288+0000","last_became_active":"2023-10-18T10:45:55.307593+0000","last_became_peered":"2023-10-18T10:45:55.307593+0000","last_unstale":"2023-10-18T10:46:01.085288+0000","last_undegraded":"2023-10-18T10:46:01.085288+0000","last_fullsized":"2023-10-18T10:46:01.085288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:20:45.634595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084888+0000","last_change":"2023-10-18T10:45:55.778356+0000","last_active":"2023-10-18T10:46:01.084888+0000","last_peered":"2023-10-18T10:46:01.084888+0000","last_clean":"2023-10-18T10:46:01.084888+0000","last_became_active":"2023-10-18T10:45:55.776465+0000","last_became_peered":"2023-10-18T10:45:55.776465+0000","last_unstale":"2023-10-18T10:46:01.084888+0000","last_undegraded":"2023-10-18T10:46:01.084888+0000","last_fullsized":"2023-10-18T10:46:01.084888+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:10:59.545961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902026+0000","last_change":"2023-10-18T10:45:59.788972+0000","last_active":"2023-10-18T10:46:00.902026+0000","last_peered":"2023-10-18T10:46:00.902026+0000","last_clean":"2023-10-18T10:46:00.902026+0000","last_became_active":"2023-10-18T10:45:59.788613+0000","last_became_peered":"2023-10-18T10:45:59.788613+0000","last_unstale":"2023-10-18T10:46:00.902026+0000","last_undegraded":"2023-10-18T10:46:00.902026+0000","last_fullsized":"2023-10-18T10:46:00.902026+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:33:04.407310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"27'9","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:06.342753+0000","last_change":"2023-10-18T10:45:55.298794+0000","last_active":"2023-10-18T10:46:06.342753+0000","last_peered":"2023-10-18T10:46:06.342753+0000","last_clean":"2023-10-18T10:46:06.342753+0000","last_became_active":"2023-10-18T10:45:55.297181+0000","last_became_peered":"2023-10-18T10:45:55.297181+0000","last_unstale":"2023-10-18T10:46:06.342753+0000","last_undegraded":"2023-10-18T10:46:06.342753+0000","last_fullsized":"2023-10-18T10:46:06.342753+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:16:43.562003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768593+0000","last_change":"2023-10-18T10:45:55.777329+0000","last_active":"2023-10-18T10:46:00.768593+0000","last_peered":"2023-10-18T10:46:00.768593+0000","last_clean":"2023-10-18T10:46:00.768593+0000","last_became_active":"2023-10-18T10:45:55.777047+0000","last_became_peered":"2023-10-18T10:45:55.777047+0000","last_unstale":"2023-10-18T10:46:00.768593+0000","last_undegraded":"2023-10-18T10:46:00.768593+0000","last_fullsized":"2023-10-18T10:46:00.768593+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:09:20.566242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084662+0000","last_change":"2023-10-18T10:45:59.804498+0000","last_active":"2023-10-18T10:46:01.084662+0000","last_peered":"2023-10-18T10:46:01.084662+0000","last_clean":"2023-10-18T10:46:01.084662+0000","last_became_active":"2023-10-18T10:45:59.804083+0000","last_became_peered":"2023-10-18T10:45:59.804083+0000","last_unstale":"2023-10-18T10:46:01.084662+0000","last_undegraded":"2023-10-18T10:46:01.084662+0000","last_fullsized":"2023-10-18T10:46:01.084662+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:04:58.822427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.769276+0000","last_change":"2023-10-18T10:45:55.774541+0000","last_active":"2023-10-18T10:46:00.769276+0000","last_peered":"2023-10-18T10:46:00.769276+0000","last_clean":"2023-10-18T10:46:00.769276+0000","last_became_active":"2023-10-18T10:45:55.774258+0000","last_became_peered":"2023-10-18T10:45:55.774258+0000","last_unstale":"2023-10-18T10:46:00.769276+0000","last_undegraded":"2023-10-18T10:46:00.769276+0000","last_fullsized":"2023-10-18T10:46:00.769276+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:11:42.497290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.770380+0000","last_change":"2023-10-18T10:45:50.252021+0000","last_active":"2023-10-18T10:46:00.770380+0000","last_peered":"2023-10-18T10:46:00.770380+0000","last_clean":"2023-10-18T10:46:00.770380+0000","last_became_active":"2023-10-18T10:45:50.251427+0000","last_became_peered":"2023-10-18T10:45:50.251427+0000","last_unstale":"2023-10-18T10:46:00.770380+0000","last_undegraded":"2023-10-18T10:46:00.770380+0000","last_fullsized":"2023-10-18T10:46:00.770380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:43.575003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"26'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086531+0000","last_change":"2023-10-18T10:45:55.296927+0000","last_active":"2023-10-18T10:46:01.086531+0000","last_peered":"2023-10-18T10:46:01.086531+0000","last_clean":"2023-10-18T10:46:01.086531+0000","last_became_active":"2023-10-18T10:45:55.296614+0000","last_became_peered":"2023-10-18T10:45:55.296614+0000","last_unstale":"2023-10-18T10:46:01.086531+0000","last_undegraded":"2023-10-18T10:46:01.086531+0000","last_fullsized":"2023-10-18T10:46:01.086531+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:05:56.466612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902246+0000","last_change":"2023-10-18T10:45:59.786937+0000","last_active":"2023-10-18T10:46:00.902246+0000","last_peered":"2023-10-18T10:46:00.902246+0000","last_clean":"2023-10-18T10:46:00.902246+0000","last_became_active":"2023-10-18T10:45:59.786436+0000","last_became_peered":"2023-10-18T10:45:59.786436+0000","last_unstale":"2023-10-18T10:46:00.902246+0000","last_undegraded":"2023-10-18T10:46:00.902246+0000","last_fullsized":"2023-10-18T10:46:00.902246+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:37:40.801018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902006+0000","last_change":"2023-10-18T10:45:55.766947+0000","last_active":"2023-10-18T10:46:00.902006+0000","last_peered":"2023-10-18T10:46:00.902006+0000","last_clean":"2023-10-18T10:46:00.902006+0000","last_became_active":"2023-10-18T10:45:55.765951+0000","last_became_peered":"2023-10-18T10:45:55.765951+0000","last_unstale":"2023-10-18T10:46:00.902006+0000","last_undegraded":"2023-10-18T10:46:00.902006+0000","last_fullsized":"2023-10-18T10:46:00.902006+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:06:27.237867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085276+0000","last_change":"2023-10-18T10:45:50.252876+0000","last_active":"2023-10-18T10:46:01.085276+0000","last_peered":"2023-10-18T10:46:01.085276+0000","last_clean":"2023-10-18T10:46:01.085276+0000","last_became_active":"2023-10-18T10:45:50.252559+0000","last_became_peered":"2023-10-18T10:45:50.252559+0000","last_unstale":"2023-10-18T10:46:01.085276+0000","last_undegraded":"2023-10-18T10:46:01.085276+0000","last_fullsized":"2023-10-18T10:46:01.085276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:55:15.913186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902858+0000","last_change":"2023-10-18T10:45:55.300421+0000","last_active":"2023-10-18T10:46:00.902858+0000","last_peered":"2023-10-18T10:46:00.902858+0000","last_clean":"2023-10-18T10:46:00.902858+0000","last_became_active":"2023-10-18T10:45:55.300170+0000","last_became_peered":"2023-10-18T10:45:55.300170+0000","last_unstale":"2023-10-18T10:46:00.902858+0000","last_undegraded":"2023-10-18T10:46:00.902858+0000","last_fullsized":"2023-10-18T10:46:00.902858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:52:09.364478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768261+0000","last_change":"2023-10-18T10:45:59.799805+0000","last_active":"2023-10-18T10:46:00.768261+0000","last_peered":"2023-10-18T10:46:00.768261+0000","last_clean":"2023-10-18T10:46:00.768261+0000","last_became_active":"2023-10-18T10:45:59.799112+0000","last_became_peered":"2023-10-18T10:45:59.799112+0000","last_unstale":"2023-10-18T10:46:00.768261+0000","last_undegraded":"2023-10-18T10:46:00.768261+0000","last_fullsized":"2023-10-18T10:46:00.768261+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:49:10.356115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086095+0000","last_change":"2023-10-18T10:45:55.774299+0000","last_active":"2023-10-18T10:46:01.086095+0000","last_peered":"2023-10-18T10:46:01.086095+0000","last_clean":"2023-10-18T10:46:01.086095+0000","last_became_active":"2023-10-18T10:45:55.773409+0000","last_became_peered":"2023-10-18T10:45:55.773409+0000","last_unstale":"2023-10-18T10:46:01.086095+0000","last_undegraded":"2023-10-18T10:46:01.086095+0000","last_fullsized":"2023-10-18T10:46:01.086095+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:37:37.315564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085581+0000","last_change":"2023-10-18T10:45:50.585224+0000","last_active":"2023-10-18T10:46:01.085581+0000","last_peered":"2023-10-18T10:46:01.085581+0000","last_clean":"2023-10-18T10:46:01.085581+0000","last_became_active":"2023-10-18T10:45:50.584872+0000","last_became_peered":"2023-10-18T10:45:50.584872+0000","last_unstale":"2023-10-18T10:46:01.085581+0000","last_undegraded":"2023-10-18T10:46:01.085581+0000","last_fullsized":"2023-10-18T10:46:01.085581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:07:41.932960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085445+0000","last_change":"2023-10-18T10:45:55.298601+0000","last_active":"2023-10-18T10:46:01.085445+0000","last_peered":"2023-10-18T10:46:01.085445+0000","last_clean":"2023-10-18T10:46:01.085445+0000","last_became_active":"2023-10-18T10:45:55.295209+0000","last_became_peered":"2023-10-18T10:45:55.295209+0000","last_unstale":"2023-10-18T10:46:01.085445+0000","last_undegraded":"2023-10-18T10:46:01.085445+0000","last_fullsized":"2023-10-18T10:46:01.085445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:55:07.806885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.769905+0000","last_change":"2023-10-18T10:45:59.787239+0000","last_active":"2023-10-18T10:46:00.769905+0000","last_peered":"2023-10-18T10:46:00.769905+0000","last_clean":"2023-10-18T10:46:00.769905+0000","last_became_active":"2023-10-18T10:45:59.786770+0000","last_became_peered":"2023-10-18T10:45:59.786770+0000","last_unstale":"2023-10-18T10:46:00.769905+0000","last_undegraded":"2023-10-18T10:46:00.769905+0000","last_fullsized":"2023-10-18T10:46:00.769905+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:09:26.681314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084957+0000","last_change":"2023-10-18T10:45:55.770838+0000","last_active":"2023-10-18T10:46:01.084957+0000","last_peered":"2023-10-18T10:46:01.084957+0000","last_clean":"2023-10-18T10:46:01.084957+0000","last_became_active":"2023-10-18T10:45:55.770308+0000","last_became_peered":"2023-10-18T10:45:55.770308+0000","last_unstale":"2023-10-18T10:46:01.084957+0000","last_undegraded":"2023-10-18T10:46:01.084957+0000","last_fullsized":"2023-10-18T10:46:01.084957+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:42:54.758946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085651+0000","last_change":"2023-10-18T10:45:50.584946+0000","last_active":"2023-10-18T10:46:01.085651+0000","last_peered":"2023-10-18T10:46:01.085651+0000","last_clean":"2023-10-18T10:46:01.085651+0000","last_became_active":"2023-10-18T10:45:50.584638+0000","last_became_peered":"2023-10-18T10:45:50.584638+0000","last_unstale":"2023-10-18T10:46:01.085651+0000","last_undegraded":"2023-10-18T10:46:01.085651+0000","last_fullsized":"2023-10-18T10:46:01.085651+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:40:09.742761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085828+0000","last_change":"2023-10-18T10:45:55.299483+0000","last_active":"2023-10-18T10:46:01.085828+0000","last_peered":"2023-10-18T10:46:01.085828+0000","last_clean":"2023-10-18T10:46:01.085828+0000","last_became_active":"2023-10-18T10:45:55.299195+0000","last_became_peered":"2023-10-18T10:45:55.299195+0000","last_unstale":"2023-10-18T10:46:01.085828+0000","last_undegraded":"2023-10-18T10:46:01.085828+0000","last_fullsized":"2023-10-18T10:46:01.085828+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:25:17.957984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085926+0000","last_change":"2023-10-18T10:45:59.801291+0000","last_active":"2023-10-18T10:46:01.085926+0000","last_peered":"2023-10-18T10:46:01.085926+0000","last_clean":"2023-10-18T10:46:01.085926+0000","last_became_active":"2023-10-18T10:45:59.800819+0000","last_became_peered":"2023-10-18T10:45:59.800819+0000","last_unstale":"2023-10-18T10:46:01.085926+0000","last_undegraded":"2023-10-18T10:46:01.085926+0000","last_fullsized":"2023-10-18T10:46:01.085926+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:06:30.412609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086211+0000","last_change":"2023-10-18T10:45:55.773950+0000","last_active":"2023-10-18T10:46:01.086211+0000","last_peered":"2023-10-18T10:46:01.086211+0000","last_clean":"2023-10-18T10:46:01.086211+0000","last_became_active":"2023-10-18T10:45:55.769581+0000","last_became_peered":"2023-10-18T10:45:55.769581+0000","last_unstale":"2023-10-18T10:46:01.086211+0000","last_undegraded":"2023-10-18T10:46:01.086211+0000","last_fullsized":"2023-10-18T10:46:01.086211+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:31:36.488372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768705+0000","last_change":"2023-10-18T10:45:55.297879+0000","last_active":"2023-10-18T10:46:00.768705+0000","last_peered":"2023-10-18T10:46:00.768705+0000","last_clean":"2023-10-18T10:46:00.768705+0000","last_became_active":"2023-10-18T10:45:55.297097+0000","last_became_peered":"2023-10-18T10:45:55.297097+0000","last_unstale":"2023-10-18T10:46:00.768705+0000","last_undegraded":"2023-10-18T10:46:00.768705+0000","last_fullsized":"2023-10-18T10:46:00.768705+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:27:53.149351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086078+0000","last_change":"2023-10-18T10:45:50.251505+0000","last_active":"2023-10-18T10:46:01.086078+0000","last_peered":"2023-10-18T10:46:01.086078+0000","last_clean":"2023-10-18T10:46:01.086078+0000","last_became_active":"2023-10-18T10:45:50.251111+0000","last_became_peered":"2023-10-18T10:45:50.251111+0000","last_unstale":"2023-10-18T10:46:01.086078+0000","last_undegraded":"2023-10-18T10:46:01.086078+0000","last_fullsized":"2023-10-18T10:46:01.086078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:04:22.817598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085546+0000","last_change":"2023-10-18T10:45:59.800297+0000","last_active":"2023-10-18T10:46:01.085546+0000","last_peered":"2023-10-18T10:46:01.085546+0000","last_clean":"2023-10-18T10:46:01.085546+0000","last_became_active":"2023-10-18T10:45:59.799434+0000","last_became_peered":"2023-10-18T10:45:59.799434+0000","last_unstale":"2023-10-18T10:46:01.085546+0000","last_undegraded":"2023-10-18T10:46:01.085546+0000","last_fullsized":"2023-10-18T10:46:01.085546+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:55:07.904865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086122+0000","last_change":"2023-10-18T10:45:55.774137+0000","last_active":"2023-10-18T10:46:01.086122+0000","last_peered":"2023-10-18T10:46:01.086122+0000","last_clean":"2023-10-18T10:46:01.086122+0000","last_became_active":"2023-10-18T10:45:55.773832+0000","last_became_peered":"2023-10-18T10:45:55.773832+0000","last_unstale":"2023-10-18T10:46:01.086122+0000","last_undegraded":"2023-10-18T10:46:01.086122+0000","last_fullsized":"2023-10-18T10:46:01.086122+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:31:21.466919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084731+0000","last_change":"2023-10-18T10:45:55.294225+0000","last_active":"2023-10-18T10:46:01.084731+0000","last_peered":"2023-10-18T10:46:01.084731+0000","last_clean":"2023-10-18T10:46:01.084731+0000","last_became_active":"2023-10-18T10:45:55.292869+0000","last_became_peered":"2023-10-18T10:45:55.292869+0000","last_unstale":"2023-10-18T10:46:01.084731+0000","last_undegraded":"2023-10-18T10:46:01.084731+0000","last_fullsized":"2023-10-18T10:46:01.084731+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:18:31.743377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.766604+0000","last_change":"2023-10-18T10:45:50.259019+0000","last_active":"2023-10-18T10:46:00.766604+0000","last_peered":"2023-10-18T10:46:00.766604+0000","last_clean":"2023-10-18T10:46:00.766604+0000","last_became_active":"2023-10-18T10:45:50.258607+0000","last_became_peered":"2023-10-18T10:45:50.258607+0000","last_unstale":"2023-10-18T10:46:00.766604+0000","last_undegraded":"2023-10-18T10:46:00.766604+0000","last_fullsized":"2023-10-18T10:46:00.766604+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:14:34.875058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.767128+0000","last_change":"2023-10-18T10:45:59.803747+0000","last_active":"2023-10-18T10:46:00.767128+0000","last_peered":"2023-10-18T10:46:00.767128+0000","last_clean":"2023-10-18T10:46:00.767128+0000","last_became_active":"2023-10-18T10:45:59.795250+0000","last_became_peered":"2023-10-18T10:45:59.795250+0000","last_unstale":"2023-10-18T10:46:00.767128+0000","last_undegraded":"2023-10-18T10:46:00.767128+0000","last_fullsized":"2023-10-18T10:46:00.767128+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:25.942126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086446+0000","last_change":"2023-10-18T10:45:55.763100+0000","last_active":"2023-10-18T10:46:01.086446+0000","last_peered":"2023-10-18T10:46:01.086446+0000","last_clean":"2023-10-18T10:46:01.086446+0000","last_became_active":"2023-10-18T10:45:55.762545+0000","last_became_peered":"2023-10-18T10:45:55.762545+0000","last_unstale":"2023-10-18T10:46:01.086446+0000","last_undegraded":"2023-10-18T10:46:01.086446+0000","last_fullsized":"2023-10-18T10:46:01.086446+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:33:00.921856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085280+0000","last_change":"2023-10-18T10:45:55.305049+0000","last_active":"2023-10-18T10:46:01.085280+0000","last_peered":"2023-10-18T10:46:01.085280+0000","last_clean":"2023-10-18T10:46:01.085280+0000","last_became_active":"2023-10-18T10:45:55.304746+0000","last_became_peered":"2023-10-18T10:45:55.304746+0000","last_unstale":"2023-10-18T10:46:01.085280+0000","last_undegraded":"2023-10-18T10:46:01.085280+0000","last_fullsized":"2023-10-18T10:46:01.085280+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:37:42.159482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084766+0000","last_change":"2023-10-18T10:45:50.251375+0000","last_active":"2023-10-18T10:46:01.084766+0000","last_peered":"2023-10-18T10:46:01.084766+0000","last_clean":"2023-10-18T10:46:01.084766+0000","last_became_active":"2023-10-18T10:45:50.251055+0000","last_became_peered":"2023-10-18T10:45:50.251055+0000","last_unstale":"2023-10-18T10:46:01.084766+0000","last_undegraded":"2023-10-18T10:46:01.084766+0000","last_fullsized":"2023-10-18T10:46:01.084766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:49:40.643495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085615+0000","last_change":"2023-10-18T10:45:59.791676+0000","last_active":"2023-10-18T10:46:01.085615+0000","last_peered":"2023-10-18T10:46:01.085615+0000","last_clean":"2023-10-18T10:46:01.085615+0000","last_became_active":"2023-10-18T10:45:59.791341+0000","last_became_peered":"2023-10-18T10:45:59.791341+0000","last_unstale":"2023-10-18T10:46:01.085615+0000","last_undegraded":"2023-10-18T10:46:01.085615+0000","last_fullsized":"2023-10-18T10:46:01.085615+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:15:44.664969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.770245+0000","last_change":"2023-10-18T10:45:55.777240+0000","last_active":"2023-10-18T10:46:00.770245+0000","last_peered":"2023-10-18T10:46:00.770245+0000","last_clean":"2023-10-18T10:46:00.770245+0000","last_became_active":"2023-10-18T10:45:55.775286+0000","last_became_peered":"2023-10-18T10:45:55.775286+0000","last_unstale":"2023-10-18T10:46:00.770245+0000","last_undegraded":"2023-10-18T10:46:00.770245+0000","last_fullsized":"2023-10-18T10:46:00.770245+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:52.500160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902463+0000","last_change":"2023-10-18T10:45:55.296440+0000","last_active":"2023-10-18T10:46:00.902463+0000","last_peered":"2023-10-18T10:46:00.902463+0000","last_clean":"2023-10-18T10:46:00.902463+0000","last_became_active":"2023-10-18T10:45:55.296124+0000","last_became_peered":"2023-10-18T10:45:55.296124+0000","last_unstale":"2023-10-18T10:46:00.902463+0000","last_undegraded":"2023-10-18T10:46:00.902463+0000","last_fullsized":"2023-10-18T10:46:00.902463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:43:19.116789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085874+0000","last_change":"2023-10-18T10:45:50.251310+0000","last_active":"2023-10-18T10:46:01.085874+0000","last_peered":"2023-10-18T10:46:01.085874+0000","last_clean":"2023-10-18T10:46:01.085874+0000","last_became_active":"2023-10-18T10:45:50.250823+0000","last_became_peered":"2023-10-18T10:45:50.250823+0000","last_unstale":"2023-10-18T10:46:01.085874+0000","last_undegraded":"2023-10-18T10:46:01.085874+0000","last_fullsized":"2023-10-18T10:46:01.085874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:33:28.469766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":183,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084881+0000","last_change":"2023-10-18T10:45:44.563531+0000","last_active":"2023-10-18T10:46:01.084881+0000","last_peered":"2023-10-18T10:46:01.084881+0000","last_clean":"2023-10-18T10:46:01.084881+0000","last_became_active":"2023-10-18T10:45:44.562504+0000","last_became_peered":"2023-10-18T10:45:44.562504+0000","last_unstale":"2023-10-18T10:46:01.084881+0000","last_undegraded":"2023-10-18T10:46:01.084881+0000","last_fullsized":"2023-10-18T10:46:01.084881+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:43.181461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:43.181461+0000","last_clean_scrub_stamp":"2023-10-18T10:45:43.181461+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:37:25.232224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084962+0000","last_change":"2023-10-18T10:45:59.807765+0000","last_active":"2023-10-18T10:46:01.084962+0000","last_peered":"2023-10-18T10:46:01.084962+0000","last_clean":"2023-10-18T10:46:01.084962+0000","last_became_active":"2023-10-18T10:45:59.807369+0000","last_became_peered":"2023-10-18T10:45:59.807369+0000","last_unstale":"2023-10-18T10:46:01.084962+0000","last_undegraded":"2023-10-18T10:46:01.084962+0000","last_fullsized":"2023-10-18T10:46:01.084962+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:14:04.335139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086027+0000","last_change":"2023-10-18T10:45:55.763363+0000","last_active":"2023-10-18T10:46:01.086027+0000","last_peered":"2023-10-18T10:46:01.086027+0000","last_clean":"2023-10-18T10:46:01.086027+0000","last_became_active":"2023-10-18T10:45:55.763155+0000","last_became_peered":"2023-10-18T10:45:55.763155+0000","last_unstale":"2023-10-18T10:46:01.086027+0000","last_undegraded":"2023-10-18T10:46:01.086027+0000","last_fullsized":"2023-10-18T10:46:01.086027+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:15:41.179515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902288+0000","last_change":"2023-10-18T10:45:55.304658+0000","last_active":"2023-10-18T10:46:00.902288+0000","last_peered":"2023-10-18T10:46:00.902288+0000","last_clean":"2023-10-18T10:46:00.902288+0000","last_became_active":"2023-10-18T10:45:55.303900+0000","last_became_peered":"2023-10-18T10:45:55.303900+0000","last_unstale":"2023-10-18T10:46:00.902288+0000","last_undegraded":"2023-10-18T10:46:00.902288+0000","last_fullsized":"2023-10-18T10:46:00.902288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:34:05.567387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768906+0000","last_change":"2023-10-18T10:45:59.792438+0000","last_active":"2023-10-18T10:46:00.768906+0000","last_peered":"2023-10-18T10:46:00.768906+0000","last_clean":"2023-10-18T10:46:00.768906+0000","last_became_active":"2023-10-18T10:45:59.791975+0000","last_became_peered":"2023-10-18T10:45:59.791975+0000","last_unstale":"2023-10-18T10:46:00.768906+0000","last_undegraded":"2023-10-18T10:46:00.768906+0000","last_fullsized":"2023-10-18T10:46:00.768906+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:19:39.692866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768991+0000","last_change":"2023-10-18T10:45:55.773190+0000","last_active":"2023-10-18T10:46:00.768991+0000","last_peered":"2023-10-18T10:46:00.768991+0000","last_clean":"2023-10-18T10:46:00.768991+0000","last_became_active":"2023-10-18T10:45:55.769619+0000","last_became_peered":"2023-10-18T10:45:55.769619+0000","last_unstale":"2023-10-18T10:46:00.768991+0000","last_undegraded":"2023-10-18T10:46:00.768991+0000","last_fullsized":"2023-10-18T10:46:00.768991+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:12:47.136700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.769371+0000","last_change":"2023-10-18T10:45:55.299320+0000","last_active":"2023-10-18T10:46:00.769371+0000","last_peered":"2023-10-18T10:46:00.769371+0000","last_clean":"2023-10-18T10:46:00.769371+0000","last_became_active":"2023-10-18T10:45:55.299008+0000","last_became_peered":"2023-10-18T10:45:55.299008+0000","last_unstale":"2023-10-18T10:46:00.769371+0000","last_undegraded":"2023-10-18T10:46:00.769371+0000","last_fullsized":"2023-10-18T10:46:00.769371+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:51.786437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085465+0000","last_change":"2023-10-18T10:45:59.801383+0000","last_active":"2023-10-18T10:46:01.085465+0000","last_peered":"2023-10-18T10:46:01.085465+0000","last_clean":"2023-10-18T10:46:01.085465+0000","last_became_active":"2023-10-18T10:45:59.801052+0000","last_became_peered":"2023-10-18T10:45:59.801052+0000","last_unstale":"2023-10-18T10:46:01.085465+0000","last_undegraded":"2023-10-18T10:46:01.085465+0000","last_fullsized":"2023-10-18T10:46:01.085465+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:57:31.735612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768713+0000","last_change":"2023-10-18T10:45:55.762399+0000","last_active":"2023-10-18T10:46:00.768713+0000","last_peered":"2023-10-18T10:46:00.768713+0000","last_clean":"2023-10-18T10:46:00.768713+0000","last_became_active":"2023-10-18T10:45:55.762074+0000","last_became_peered":"2023-10-18T10:45:55.762074+0000","last_unstale":"2023-10-18T10:46:00.768713+0000","last_undegraded":"2023-10-18T10:46:00.768713+0000","last_fullsized":"2023-10-18T10:46:00.768713+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:44:41.551817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.770066+0000","last_change":"2023-10-18T10:45:55.301204+0000","last_active":"2023-10-18T10:46:00.770066+0000","last_peered":"2023-10-18T10:46:00.770066+0000","last_clean":"2023-10-18T10:46:00.770066+0000","last_became_active":"2023-10-18T10:45:55.300895+0000","last_became_peered":"2023-10-18T10:45:55.300895+0000","last_unstale":"2023-10-18T10:46:00.770066+0000","last_undegraded":"2023-10-18T10:46:00.770066+0000","last_fullsized":"2023-10-18T10:46:00.770066+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:33:30.055966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084972+0000","last_change":"2023-10-18T10:45:59.794810+0000","last_active":"2023-10-18T10:46:01.084972+0000","last_peered":"2023-10-18T10:46:01.084972+0000","last_clean":"2023-10-18T10:46:01.084972+0000","last_became_active":"2023-10-18T10:45:59.794392+0000","last_became_peered":"2023-10-18T10:45:59.794392+0000","last_unstale":"2023-10-18T10:46:01.084972+0000","last_undegraded":"2023-10-18T10:46:01.084972+0000","last_fullsized":"2023-10-18T10:46:01.084972+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:05:09.364081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768556+0000","last_change":"2023-10-18T10:45:55.775591+0000","last_active":"2023-10-18T10:46:00.768556+0000","last_peered":"2023-10-18T10:46:00.768556+0000","last_clean":"2023-10-18T10:46:00.768556+0000","last_became_active":"2023-10-18T10:45:55.770394+0000","last_became_peered":"2023-10-18T10:45:55.770394+0000","last_unstale":"2023-10-18T10:46:00.768556+0000","last_undegraded":"2023-10-18T10:46:00.768556+0000","last_fullsized":"2023-10-18T10:46:00.768556+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:03:08.191047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902332+0000","last_change":"2023-10-18T10:45:55.299012+0000","last_active":"2023-10-18T10:46:00.902332+0000","last_peered":"2023-10-18T10:46:00.902332+0000","last_clean":"2023-10-18T10:46:00.902332+0000","last_became_active":"2023-10-18T10:45:55.296941+0000","last_became_peered":"2023-10-18T10:45:55.296941+0000","last_unstale":"2023-10-18T10:46:00.902332+0000","last_undegraded":"2023-10-18T10:46:00.902332+0000","last_fullsized":"2023-10-18T10:46:00.902332+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:07:46.963272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.766976+0000","last_change":"2023-10-18T10:45:59.797124+0000","last_active":"2023-10-18T10:46:00.766976+0000","last_peered":"2023-10-18T10:46:00.766976+0000","last_clean":"2023-10-18T10:46:00.766976+0000","last_became_active":"2023-10-18T10:45:59.796721+0000","last_became_peered":"2023-10-18T10:45:59.796721+0000","last_unstale":"2023-10-18T10:46:00.766976+0000","last_undegraded":"2023-10-18T10:46:00.766976+0000","last_fullsized":"2023-10-18T10:46:00.766976+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:11:32.963010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085998+0000","last_change":"2023-10-18T10:45:55.767007+0000","last_active":"2023-10-18T10:46:01.085998+0000","last_peered":"2023-10-18T10:46:01.085998+0000","last_clean":"2023-10-18T10:46:01.085998+0000","last_became_active":"2023-10-18T10:45:55.766727+0000","last_became_peered":"2023-10-18T10:45:55.766727+0000","last_unstale":"2023-10-18T10:46:01.085998+0000","last_undegraded":"2023-10-18T10:46:01.085998+0000","last_fullsized":"2023-10-18T10:46:01.085998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:51:37.372456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085606+0000","last_change":"2023-10-18T10:45:55.293762+0000","last_active":"2023-10-18T10:46:01.085606+0000","last_peered":"2023-10-18T10:46:01.085606+0000","last_clean":"2023-10-18T10:46:01.085606+0000","last_became_active":"2023-10-18T10:45:55.293349+0000","last_became_peered":"2023-10-18T10:45:55.293349+0000","last_unstale":"2023-10-18T10:46:01.085606+0000","last_undegraded":"2023-10-18T10:46:01.085606+0000","last_fullsized":"2023-10-18T10:46:01.085606+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:31:20.459601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.766635+0000","last_change":"2023-10-18T10:45:59.790292+0000","last_active":"2023-10-18T10:46:00.766635+0000","last_peered":"2023-10-18T10:46:00.766635+0000","last_clean":"2023-10-18T10:46:00.766635+0000","last_became_active":"2023-10-18T10:45:59.789901+0000","last_became_peered":"2023-10-18T10:45:59.789901+0000","last_unstale":"2023-10-18T10:46:00.766635+0000","last_undegraded":"2023-10-18T10:46:00.766635+0000","last_fullsized":"2023-10-18T10:46:00.766635+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:26:52.984157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.769853+0000","last_change":"2023-10-18T10:45:55.768064+0000","last_active":"2023-10-18T10:46:00.769853+0000","last_peered":"2023-10-18T10:46:00.769853+0000","last_clean":"2023-10-18T10:46:00.769853+0000","last_became_active":"2023-10-18T10:45:55.767773+0000","last_became_peered":"2023-10-18T10:45:55.767773+0000","last_unstale":"2023-10-18T10:46:00.769853+0000","last_undegraded":"2023-10-18T10:46:00.769853+0000","last_fullsized":"2023-10-18T10:46:00.769853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:30:00.014556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085414+0000","last_change":"2023-10-18T10:45:55.303880+0000","last_active":"2023-10-18T10:46:01.085414+0000","last_peered":"2023-10-18T10:46:01.085414+0000","last_clean":"2023-10-18T10:46:01.085414+0000","last_became_active":"2023-10-18T10:45:55.303577+0000","last_became_peered":"2023-10-18T10:45:55.303577+0000","last_unstale":"2023-10-18T10:46:01.085414+0000","last_undegraded":"2023-10-18T10:46:01.085414+0000","last_fullsized":"2023-10-18T10:46:01.085414+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:02:55.960777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.766839+0000","last_change":"2023-10-18T10:45:59.789006+0000","last_active":"2023-10-18T10:46:00.766839+0000","last_peered":"2023-10-18T10:46:00.766839+0000","last_clean":"2023-10-18T10:46:00.766839+0000","last_became_active":"2023-10-18T10:45:59.788544+0000","last_became_peered":"2023-10-18T10:45:59.788544+0000","last_unstale":"2023-10-18T10:46:00.766839+0000","last_undegraded":"2023-10-18T10:46:00.766839+0000","last_fullsized":"2023-10-18T10:46:00.766839+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:32:31.878208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.769128+0000","last_change":"2023-10-18T10:45:55.775207+0000","last_active":"2023-10-18T10:46:00.769128+0000","last_peered":"2023-10-18T10:46:00.769128+0000","last_clean":"2023-10-18T10:46:00.769128+0000","last_became_active":"2023-10-18T10:45:55.774909+0000","last_became_peered":"2023-10-18T10:45:55.774909+0000","last_unstale":"2023-10-18T10:46:00.769128+0000","last_undegraded":"2023-10-18T10:46:00.769128+0000","last_fullsized":"2023-10-18T10:46:00.769128+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:07:30.694604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085978+0000","last_change":"2023-10-18T10:45:55.302100+0000","last_active":"2023-10-18T10:46:01.085978+0000","last_peered":"2023-10-18T10:46:01.085978+0000","last_clean":"2023-10-18T10:46:01.085978+0000","last_became_active":"2023-10-18T10:45:55.301652+0000","last_became_peered":"2023-10-18T10:45:55.301652+0000","last_unstale":"2023-10-18T10:46:01.085978+0000","last_undegraded":"2023-10-18T10:46:01.085978+0000","last_fullsized":"2023-10-18T10:46:01.085978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:12:22.646771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.766536+0000","last_change":"2023-10-18T10:45:59.794997+0000","last_active":"2023-10-18T10:46:00.766536+0000","last_peered":"2023-10-18T10:46:00.766536+0000","last_clean":"2023-10-18T10:46:00.766536+0000","last_became_active":"2023-10-18T10:45:59.794587+0000","last_became_peered":"2023-10-18T10:45:59.794587+0000","last_unstale":"2023-10-18T10:46:00.766536+0000","last_undegraded":"2023-10-18T10:46:00.766536+0000","last_fullsized":"2023-10-18T10:46:00.766536+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:22:23.799739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084896+0000","last_change":"2023-10-18T10:45:55.772538+0000","last_active":"2023-10-18T10:46:01.084896+0000","last_peered":"2023-10-18T10:46:01.084896+0000","last_clean":"2023-10-18T10:46:01.084896+0000","last_became_active":"2023-10-18T10:45:55.772207+0000","last_became_peered":"2023-10-18T10:45:55.772207+0000","last_unstale":"2023-10-18T10:46:01.084896+0000","last_undegraded":"2023-10-18T10:46:01.084896+0000","last_fullsized":"2023-10-18T10:46:01.084896+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:55:21.950816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086120+0000","last_change":"2023-10-18T10:45:55.305858+0000","last_active":"2023-10-18T10:46:01.086120+0000","last_peered":"2023-10-18T10:46:01.086120+0000","last_clean":"2023-10-18T10:46:01.086120+0000","last_became_active":"2023-10-18T10:45:55.305591+0000","last_became_peered":"2023-10-18T10:45:55.305591+0000","last_unstale":"2023-10-18T10:46:01.086120+0000","last_undegraded":"2023-10-18T10:46:01.086120+0000","last_fullsized":"2023-10-18T10:46:01.086120+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:43:19.116789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084960+0000","last_change":"2023-10-18T10:45:59.799125+0000","last_active":"2023-10-18T10:46:01.084960+0000","last_peered":"2023-10-18T10:46:01.084960+0000","last_clean":"2023-10-18T10:46:01.084960+0000","last_became_active":"2023-10-18T10:45:59.798734+0000","last_became_peered":"2023-10-18T10:45:59.798734+0000","last_unstale":"2023-10-18T10:46:01.084960+0000","last_undegraded":"2023-10-18T10:46:01.084960+0000","last_fullsized":"2023-10-18T10:46:01.084960+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:56:46.630947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085751+0000","last_change":"2023-10-18T10:45:59.794340+0000","last_active":"2023-10-18T10:46:01.085751+0000","last_peered":"2023-10-18T10:46:01.085751+0000","last_clean":"2023-10-18T10:46:01.085751+0000","last_became_active":"2023-10-18T10:45:59.793965+0000","last_became_peered":"2023-10-18T10:45:59.793965+0000","last_unstale":"2023-10-18T10:46:01.085751+0000","last_undegraded":"2023-10-18T10:46:01.085751+0000","last_fullsized":"2023-10-18T10:46:01.085751+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:29.702107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.767968+0000","last_change":"2023-10-18T10:45:55.292258+0000","last_active":"2023-10-18T10:46:00.767968+0000","last_peered":"2023-10-18T10:46:00.767968+0000","last_clean":"2023-10-18T10:46:00.767968+0000","last_became_active":"2023-10-18T10:45:55.291007+0000","last_became_peered":"2023-10-18T10:45:55.291007+0000","last_unstale":"2023-10-18T10:46:00.767968+0000","last_undegraded":"2023-10-18T10:46:00.767968+0000","last_fullsized":"2023-10-18T10:46:00.767968+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:06:37.792273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902848+0000","last_change":"2023-10-18T10:45:55.765261+0000","last_active":"2023-10-18T10:46:00.902848+0000","last_peered":"2023-10-18T10:46:00.902848+0000","last_clean":"2023-10-18T10:46:00.902848+0000","last_became_active":"2023-10-18T10:45:55.765025+0000","last_became_peered":"2023-10-18T10:45:55.765025+0000","last_unstale":"2023-10-18T10:46:00.902848+0000","last_undegraded":"2023-10-18T10:46:00.902848+0000","last_fullsized":"2023-10-18T10:46:00.902848+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:49:06.870660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902759+0000","last_change":"2023-10-18T10:45:59.795015+0000","last_active":"2023-10-18T10:46:00.902759+0000","last_peered":"2023-10-18T10:46:00.902759+0000","last_clean":"2023-10-18T10:46:00.902759+0000","last_became_active":"2023-10-18T10:45:59.794646+0000","last_became_peered":"2023-10-18T10:45:59.794646+0000","last_unstale":"2023-10-18T10:46:00.902759+0000","last_undegraded":"2023-10-18T10:46:00.902759+0000","last_fullsized":"2023-10-18T10:46:00.902759+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:08:21.298342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085153+0000","last_change":"2023-10-18T10:45:55.305369+0000","last_active":"2023-10-18T10:46:01.085153+0000","last_peered":"2023-10-18T10:46:01.085153+0000","last_clean":"2023-10-18T10:46:01.085153+0000","last_became_active":"2023-10-18T10:45:55.304821+0000","last_became_peered":"2023-10-18T10:45:55.304821+0000","last_unstale":"2023-10-18T10:46:01.085153+0000","last_undegraded":"2023-10-18T10:46:01.085153+0000","last_fullsized":"2023-10-18T10:46:01.085153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:32:59.914538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086514+0000","last_change":"2023-10-18T10:45:55.763301+0000","last_active":"2023-10-18T10:46:01.086514+0000","last_peered":"2023-10-18T10:46:01.086514+0000","last_clean":"2023-10-18T10:46:01.086514+0000","last_became_active":"2023-10-18T10:45:55.762910+0000","last_became_peered":"2023-10-18T10:45:55.762910+0000","last_unstale":"2023-10-18T10:46:01.086514+0000","last_undegraded":"2023-10-18T10:46:01.086514+0000","last_fullsized":"2023-10-18T10:46:01.086514+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:06:27.237867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085615+0000","last_change":"2023-10-18T10:45:55.302395+0000","last_active":"2023-10-18T10:46:01.085615+0000","last_peered":"2023-10-18T10:46:01.085615+0000","last_clean":"2023-10-18T10:46:01.085615+0000","last_became_active":"2023-10-18T10:45:55.302090+0000","last_became_peered":"2023-10-18T10:45:55.302090+0000","last_unstale":"2023-10-18T10:46:01.085615+0000","last_undegraded":"2023-10-18T10:46:01.085615+0000","last_fullsized":"2023-10-18T10:46:01.085615+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:25:17.957984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085682+0000","last_change":"2023-10-18T10:45:55.776022+0000","last_active":"2023-10-18T10:46:01.085682+0000","last_peered":"2023-10-18T10:46:01.085682+0000","last_clean":"2023-10-18T10:46:01.085682+0000","last_became_active":"2023-10-18T10:45:55.775711+0000","last_became_peered":"2023-10-18T10:45:55.775711+0000","last_unstale":"2023-10-18T10:46:01.085682+0000","last_undegraded":"2023-10-18T10:46:01.085682+0000","last_fullsized":"2023-10-18T10:46:01.085682+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:31:36.488372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902232+0000","last_change":"2023-10-18T10:45:59.798164+0000","last_active":"2023-10-18T10:46:00.902232+0000","last_peered":"2023-10-18T10:46:00.902232+0000","last_clean":"2023-10-18T10:46:00.902232+0000","last_became_active":"2023-10-18T10:45:59.797723+0000","last_became_peered":"2023-10-18T10:45:59.797723+0000","last_unstale":"2023-10-18T10:46:00.902232+0000","last_undegraded":"2023-10-18T10:46:00.902232+0000","last_fullsized":"2023-10-18T10:46:00.902232+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:03:00.453549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085393+0000","last_change":"2023-10-18T10:45:55.306329+0000","last_active":"2023-10-18T10:46:01.085393+0000","last_peered":"2023-10-18T10:46:01.085393+0000","last_clean":"2023-10-18T10:46:01.085393+0000","last_became_active":"2023-10-18T10:45:55.306033+0000","last_became_peered":"2023-10-18T10:45:55.306033+0000","last_unstale":"2023-10-18T10:46:01.085393+0000","last_undegraded":"2023-10-18T10:46:01.085393+0000","last_fullsized":"2023-10-18T10:46:01.085393+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:25:36.844395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902485+0000","last_change":"2023-10-18T10:45:55.768454+0000","last_active":"2023-10-18T10:46:00.902485+0000","last_peered":"2023-10-18T10:46:00.902485+0000","last_clean":"2023-10-18T10:46:00.902485+0000","last_became_active":"2023-10-18T10:45:55.767547+0000","last_became_peered":"2023-10-18T10:45:55.767547+0000","last_unstale":"2023-10-18T10:46:00.902485+0000","last_undegraded":"2023-10-18T10:46:00.902485+0000","last_fullsized":"2023-10-18T10:46:00.902485+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:19:20.955967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085524+0000","last_change":"2023-10-18T10:45:59.790629+0000","last_active":"2023-10-18T10:46:01.085524+0000","last_peered":"2023-10-18T10:46:01.085524+0000","last_clean":"2023-10-18T10:46:01.085524+0000","last_became_active":"2023-10-18T10:45:59.790163+0000","last_became_peered":"2023-10-18T10:45:59.790163+0000","last_unstale":"2023-10-18T10:46:01.085524+0000","last_undegraded":"2023-10-18T10:46:01.085524+0000","last_fullsized":"2023-10-18T10:46:01.085524+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:10:56.439036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.767840+0000","last_change":"2023-10-18T10:45:55.759248+0000","last_active":"2023-10-18T10:46:00.767840+0000","last_peered":"2023-10-18T10:46:00.767840+0000","last_clean":"2023-10-18T10:46:00.767840+0000","last_became_active":"2023-10-18T10:45:55.758926+0000","last_became_peered":"2023-10-18T10:45:55.758926+0000","last_unstale":"2023-10-18T10:46:00.767840+0000","last_undegraded":"2023-10-18T10:46:00.767840+0000","last_fullsized":"2023-10-18T10:46:00.767840+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:37:00.951309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.901903+0000","last_change":"2023-10-18T10:45:55.777594+0000","last_active":"2023-10-18T10:46:00.901903+0000","last_peered":"2023-10-18T10:46:00.901903+0000","last_clean":"2023-10-18T10:46:00.901903+0000","last_became_active":"2023-10-18T10:45:55.777126+0000","last_became_peered":"2023-10-18T10:45:55.777126+0000","last_unstale":"2023-10-18T10:46:00.901903+0000","last_undegraded":"2023-10-18T10:46:00.901903+0000","last_fullsized":"2023-10-18T10:46:00.901903+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:33:49.751994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768469+0000","last_change":"2023-10-18T10:45:55.770662+0000","last_active":"2023-10-18T10:46:00.768469+0000","last_peered":"2023-10-18T10:46:00.768469+0000","last_clean":"2023-10-18T10:46:00.768469+0000","last_became_active":"2023-10-18T10:45:55.769816+0000","last_became_peered":"2023-10-18T10:45:55.769816+0000","last_unstale":"2023-10-18T10:46:00.768469+0000","last_undegraded":"2023-10-18T10:46:00.768469+0000","last_fullsized":"2023-10-18T10:46:00.768469+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:40:54.772659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902183+0000","last_change":"2023-10-18T10:45:55.764434+0000","last_active":"2023-10-18T10:46:00.902183+0000","last_peered":"2023-10-18T10:46:00.902183+0000","last_clean":"2023-10-18T10:46:00.902183+0000","last_became_active":"2023-10-18T10:45:55.764164+0000","last_became_peered":"2023-10-18T10:45:55.764164+0000","last_unstale":"2023-10-18T10:46:00.902183+0000","last_undegraded":"2023-10-18T10:46:00.902183+0000","last_fullsized":"2023-10-18T10:46:00.902183+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:25:37.851713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085549+0000","last_change":"2023-10-18T10:45:55.773793+0000","last_active":"2023-10-18T10:46:01.085549+0000","last_peered":"2023-10-18T10:46:01.085549+0000","last_clean":"2023-10-18T10:46:01.085549+0000","last_became_active":"2023-10-18T10:45:55.769530+0000","last_became_peered":"2023-10-18T10:45:55.769530+0000","last_unstale":"2023-10-18T10:46:01.085549+0000","last_undegraded":"2023-10-18T10:46:01.085549+0000","last_fullsized":"2023-10-18T10:46:01.085549+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:57:28.250158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085697+0000","last_change":"2023-10-18T10:45:55.780769+0000","last_active":"2023-10-18T10:46:01.085697+0000","last_peered":"2023-10-18T10:46:01.085697+0000","last_clean":"2023-10-18T10:46:01.085697+0000","last_became_active":"2023-10-18T10:45:55.780434+0000","last_became_peered":"2023-10-18T10:45:55.780434+0000","last_unstale":"2023-10-18T10:46:01.085697+0000","last_undegraded":"2023-10-18T10:46:01.085697+0000","last_fullsized":"2023-10-18T10:46:01.085697+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:37:43.166800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085181+0000","last_change":"2023-10-18T10:45:55.770696+0000","last_active":"2023-10-18T10:46:01.085181+0000","last_peered":"2023-10-18T10:46:01.085181+0000","last_clean":"2023-10-18T10:46:01.085181+0000","last_became_active":"2023-10-18T10:45:55.769723+0000","last_became_peered":"2023-10-18T10:45:55.769723+0000","last_unstale":"2023-10-18T10:46:01.085181+0000","last_undegraded":"2023-10-18T10:46:01.085181+0000","last_fullsized":"2023-10-18T10:46:01.085181+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:59:15.197613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085919+0000","last_change":"2023-10-18T10:45:55.766491+0000","last_active":"2023-10-18T10:46:01.085919+0000","last_peered":"2023-10-18T10:46:01.085919+0000","last_clean":"2023-10-18T10:46:01.085919+0000","last_became_active":"2023-10-18T10:45:55.766215+0000","last_became_peered":"2023-10-18T10:45:55.766215+0000","last_unstale":"2023-10-18T10:46:01.085919+0000","last_undegraded":"2023-10-18T10:46:01.085919+0000","last_fullsized":"2023-10-18T10:46:01.085919+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:10:52.953582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085494+0000","last_change":"2023-10-18T10:45:55.767712+0000","last_active":"2023-10-18T10:46:01.085494+0000","last_peered":"2023-10-18T10:46:01.085494+0000","last_clean":"2023-10-18T10:46:01.085494+0000","last_became_active":"2023-10-18T10:45:55.765004+0000","last_became_peered":"2023-10-18T10:45:55.765004+0000","last_unstale":"2023-10-18T10:46:01.085494+0000","last_undegraded":"2023-10-18T10:46:01.085494+0000","last_fullsized":"2023-10-18T10:46:01.085494+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:34:06.574705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.901824+0000","last_change":"2023-10-18T10:45:55.757810+0000","last_active":"2023-10-18T10:46:00.901824+0000","last_peered":"2023-10-18T10:46:00.901824+0000","last_clean":"2023-10-18T10:46:00.901824+0000","last_became_active":"2023-10-18T10:45:55.756492+0000","last_became_peered":"2023-10-18T10:45:55.756492+0000","last_unstale":"2023-10-18T10:46:00.901824+0000","last_undegraded":"2023-10-18T10:46:00.901824+0000","last_fullsized":"2023-10-18T10:46:00.901824+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:56:43.145493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085491+0000","last_change":"2023-10-18T10:45:55.767803+0000","last_active":"2023-10-18T10:46:01.085491+0000","last_peered":"2023-10-18T10:46:01.085491+0000","last_clean":"2023-10-18T10:46:01.085491+0000","last_became_active":"2023-10-18T10:45:55.767525+0000","last_became_peered":"2023-10-18T10:45:55.767525+0000","last_unstale":"2023-10-18T10:46:01.085491+0000","last_undegraded":"2023-10-18T10:46:01.085491+0000","last_fullsized":"2023-10-18T10:46:01.085491+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:26.216653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085266+0000","last_change":"2023-10-18T10:45:55.769366+0000","last_active":"2023-10-18T10:46:01.085266+0000","last_peered":"2023-10-18T10:46:01.085266+0000","last_clean":"2023-10-18T10:46:01.085266+0000","last_became_active":"2023-10-18T10:45:55.769083+0000","last_became_peered":"2023-10-18T10:45:55.769083+0000","last_unstale":"2023-10-18T10:46:01.085266+0000","last_undegraded":"2023-10-18T10:46:01.085266+0000","last_fullsized":"2023-10-18T10:46:01.085266+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:05:57.473930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768585+0000","last_change":"2023-10-18T10:45:55.770711+0000","last_active":"2023-10-18T10:46:00.768585+0000","last_peered":"2023-10-18T10:46:00.768585+0000","last_clean":"2023-10-18T10:46:00.768585+0000","last_became_active":"2023-10-18T10:45:55.770147+0000","last_became_peered":"2023-10-18T10:45:55.770147+0000","last_unstale":"2023-10-18T10:46:00.768585+0000","last_undegraded":"2023-10-18T10:46:00.768585+0000","last_fullsized":"2023-10-18T10:46:00.768585+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:20:18.473140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.767102+0000","last_change":"2023-10-18T10:45:55.758011+0000","last_active":"2023-10-18T10:46:00.767102+0000","last_peered":"2023-10-18T10:46:00.767102+0000","last_clean":"2023-10-18T10:46:00.767102+0000","last_became_active":"2023-10-18T10:45:55.757668+0000","last_became_peered":"2023-10-18T10:45:55.757668+0000","last_unstale":"2023-10-18T10:46:00.767102+0000","last_undegraded":"2023-10-18T10:46:00.767102+0000","last_fullsized":"2023-10-18T10:46:00.767102+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:41:48.881411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.769446+0000","last_change":"2023-10-18T10:45:55.763084+0000","last_active":"2023-10-18T10:46:00.769446+0000","last_peered":"2023-10-18T10:46:00.769446+0000","last_clean":"2023-10-18T10:46:00.769446+0000","last_became_active":"2023-10-18T10:45:55.762776+0000","last_became_peered":"2023-10-18T10:45:55.762776+0000","last_unstale":"2023-10-18T10:46:00.769446+0000","last_undegraded":"2023-10-18T10:46:00.769446+0000","last_fullsized":"2023-10-18T10:46:00.769446+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:28:02.667670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085807+0000","last_change":"2023-10-18T10:45:55.762254+0000","last_active":"2023-10-18T10:46:01.085807+0000","last_peered":"2023-10-18T10:46:01.085807+0000","last_clean":"2023-10-18T10:46:01.085807+0000","last_became_active":"2023-10-18T10:45:55.761870+0000","last_became_peered":"2023-10-18T10:45:55.761870+0000","last_unstale":"2023-10-18T10:46:01.085807+0000","last_undegraded":"2023-10-18T10:46:01.085807+0000","last_fullsized":"2023-10-18T10:46:01.085807+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:43:01.654769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768118+0000","last_change":"2023-10-18T10:45:55.765743+0000","last_active":"2023-10-18T10:46:00.768118+0000","last_peered":"2023-10-18T10:46:00.768118+0000","last_clean":"2023-10-18T10:46:00.768118+0000","last_became_active":"2023-10-18T10:45:55.765470+0000","last_became_peered":"2023-10-18T10:45:55.765470+0000","last_unstale":"2023-10-18T10:46:00.768118+0000","last_undegraded":"2023-10-18T10:46:00.768118+0000","last_fullsized":"2023-10-18T10:46:00.768118+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:19:20.955967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085124+0000","last_change":"2023-10-18T10:45:55.781643+0000","last_active":"2023-10-18T10:46:01.085124+0000","last_peered":"2023-10-18T10:46:01.085124+0000","last_clean":"2023-10-18T10:46:01.085124+0000","last_became_active":"2023-10-18T10:45:55.781252+0000","last_became_peered":"2023-10-18T10:45:55.781252+0000","last_unstale":"2023-10-18T10:46:01.085124+0000","last_undegraded":"2023-10-18T10:46:01.085124+0000","last_fullsized":"2023-10-18T10:46:01.085124+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:40:53.192771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085958+0000","last_change":"2023-10-18T10:45:55.774683+0000","last_active":"2023-10-18T10:46:01.085958+0000","last_peered":"2023-10-18T10:46:01.085958+0000","last_clean":"2023-10-18T10:46:01.085958+0000","last_became_active":"2023-10-18T10:45:55.773664+0000","last_became_peered":"2023-10-18T10:45:55.773664+0000","last_unstale":"2023-10-18T10:46:01.085958+0000","last_undegraded":"2023-10-18T10:46:01.085958+0000","last_fullsized":"2023-10-18T10:46:01.085958+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:06:26.927155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.766819+0000","last_change":"2023-10-18T10:45:55.755077+0000","last_active":"2023-10-18T10:46:00.766819+0000","last_peered":"2023-10-18T10:46:00.766819+0000","last_clean":"2023-10-18T10:46:00.766819+0000","last_became_active":"2023-10-18T10:45:55.754723+0000","last_became_peered":"2023-10-18T10:45:55.754723+0000","last_unstale":"2023-10-18T10:46:00.766819+0000","last_undegraded":"2023-10-18T10:46:00.766819+0000","last_fullsized":"2023-10-18T10:46:00.766819+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:17:50.477786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.767162+0000","last_change":"2023-10-18T10:45:55.771484+0000","last_active":"2023-10-18T10:46:00.767162+0000","last_peered":"2023-10-18T10:46:00.767162+0000","last_clean":"2023-10-18T10:46:00.767162+0000","last_became_active":"2023-10-18T10:45:55.771108+0000","last_became_peered":"2023-10-18T10:45:55.771108+0000","last_unstale":"2023-10-18T10:46:00.767162+0000","last_undegraded":"2023-10-18T10:46:00.767162+0000","last_fullsized":"2023-10-18T10:46:00.767162+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:18.131261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768708+0000","last_change":"2023-10-18T10:45:55.778320+0000","last_active":"2023-10-18T10:46:00.768708+0000","last_peered":"2023-10-18T10:46:00.768708+0000","last_clean":"2023-10-18T10:46:00.768708+0000","last_became_active":"2023-10-18T10:45:55.777767+0000","last_became_peered":"2023-10-18T10:45:55.777767+0000","last_unstale":"2023-10-18T10:46:00.768708+0000","last_undegraded":"2023-10-18T10:46:00.768708+0000","last_fullsized":"2023-10-18T10:46:00.768708+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:05:56.688630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086126+0000","last_change":"2023-10-18T10:45:55.771747+0000","last_active":"2023-10-18T10:46:01.086126+0000","last_peered":"2023-10-18T10:46:01.086126+0000","last_clean":"2023-10-18T10:46:01.086126+0000","last_became_active":"2023-10-18T10:45:55.771297+0000","last_became_peered":"2023-10-18T10:45:55.771297+0000","last_unstale":"2023-10-18T10:46:01.086126+0000","last_undegraded":"2023-10-18T10:46:01.086126+0000","last_fullsized":"2023-10-18T10:46:01.086126+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:51:37.372456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085611+0000","last_change":"2023-10-18T10:45:55.756546+0000","last_active":"2023-10-18T10:46:01.085611+0000","last_peered":"2023-10-18T10:46:01.085611+0000","last_clean":"2023-10-18T10:46:01.085611+0000","last_became_active":"2023-10-18T10:45:55.756277+0000","last_became_peered":"2023-10-18T10:45:55.756277+0000","last_unstale":"2023-10-18T10:46:01.085611+0000","last_undegraded":"2023-10-18T10:46:01.085611+0000","last_fullsized":"2023-10-18T10:46:01.085611+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:33:34.507396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084785+0000","last_change":"2023-10-18T10:45:55.778442+0000","last_active":"2023-10-18T10:46:01.084785+0000","last_peered":"2023-10-18T10:46:01.084785+0000","last_clean":"2023-10-18T10:46:01.084785+0000","last_became_active":"2023-10-18T10:45:55.778198+0000","last_became_peered":"2023-10-18T10:45:55.778198+0000","last_unstale":"2023-10-18T10:46:01.084785+0000","last_undegraded":"2023-10-18T10:46:01.084785+0000","last_fullsized":"2023-10-18T10:46:01.084785+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:28:05.383698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085042+0000","last_change":"2023-10-18T10:45:55.779285+0000","last_active":"2023-10-18T10:46:01.085042+0000","last_peered":"2023-10-18T10:46:01.085042+0000","last_clean":"2023-10-18T10:46:01.085042+0000","last_became_active":"2023-10-18T10:45:55.778975+0000","last_became_peered":"2023-10-18T10:45:55.778975+0000","last_unstale":"2023-10-18T10:46:01.085042+0000","last_undegraded":"2023-10-18T10:46:01.085042+0000","last_fullsized":"2023-10-18T10:46:01.085042+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:33:49.751994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768170+0000","last_change":"2023-10-18T10:45:55.770892+0000","last_active":"2023-10-18T10:46:00.768170+0000","last_peered":"2023-10-18T10:46:00.768170+0000","last_clean":"2023-10-18T10:46:00.768170+0000","last_became_active":"2023-10-18T10:45:55.770181+0000","last_became_peered":"2023-10-18T10:45:55.770181+0000","last_unstale":"2023-10-18T10:46:00.768170+0000","last_undegraded":"2023-10-18T10:46:00.768170+0000","last_fullsized":"2023-10-18T10:46:00.768170+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:06:44.169699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086251+0000","last_change":"2023-10-18T10:45:55.759114+0000","last_active":"2023-10-18T10:46:01.086251+0000","last_peered":"2023-10-18T10:46:01.086251+0000","last_clean":"2023-10-18T10:46:01.086251+0000","last_became_active":"2023-10-18T10:45:55.758825+0000","last_became_peered":"2023-10-18T10:45:55.758825+0000","last_unstale":"2023-10-18T10:46:01.086251+0000","last_undegraded":"2023-10-18T10:46:01.086251+0000","last_fullsized":"2023-10-18T10:46:01.086251+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:14:00.849685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.770488+0000","last_change":"2023-10-18T10:45:55.763407+0000","last_active":"2023-10-18T10:46:00.770488+0000","last_peered":"2023-10-18T10:46:00.770488+0000","last_clean":"2023-10-18T10:46:00.770488+0000","last_became_active":"2023-10-18T10:45:55.763083+0000","last_became_peered":"2023-10-18T10:45:55.763083+0000","last_unstale":"2023-10-18T10:46:00.770488+0000","last_undegraded":"2023-10-18T10:46:00.770488+0000","last_fullsized":"2023-10-18T10:46:00.770488+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:30.982488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768596+0000","last_change":"2023-10-18T10:45:55.770469+0000","last_active":"2023-10-18T10:46:00.768596+0000","last_peered":"2023-10-18T10:46:00.768596+0000","last_clean":"2023-10-18T10:46:00.768596+0000","last_became_active":"2023-10-18T10:45:55.769998+0000","last_became_peered":"2023-10-18T10:45:55.769998+0000","last_unstale":"2023-10-18T10:46:00.768596+0000","last_undegraded":"2023-10-18T10:46:00.768596+0000","last_fullsized":"2023-10-18T10:46:00.768596+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:00:14.580995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085929+0000","last_change":"2023-10-18T10:45:55.781899+0000","last_active":"2023-10-18T10:46:01.085929+0000","last_peered":"2023-10-18T10:46:01.085929+0000","last_clean":"2023-10-18T10:46:01.085929+0000","last_became_active":"2023-10-18T10:45:55.781624+0000","last_became_peered":"2023-10-18T10:45:55.781624+0000","last_unstale":"2023-10-18T10:46:01.085929+0000","last_undegraded":"2023-10-18T10:46:01.085929+0000","last_fullsized":"2023-10-18T10:46:01.085929+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:43:01.654769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.767009+0000","last_change":"2023-10-18T10:45:55.757924+0000","last_active":"2023-10-18T10:46:00.767009+0000","last_peered":"2023-10-18T10:46:00.767009+0000","last_clean":"2023-10-18T10:46:00.767009+0000","last_became_active":"2023-10-18T10:45:55.757644+0000","last_became_peered":"2023-10-18T10:45:55.757644+0000","last_unstale":"2023-10-18T10:46:00.767009+0000","last_undegraded":"2023-10-18T10:46:00.767009+0000","last_fullsized":"2023-10-18T10:46:00.767009+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:08:54.769872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343932,"statfs":{"total":96636764160,"available":96608186368,"internally_reserved":0,"allocated":913408,"data_stored":714326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9219999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":120553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":311296,"data_stored":123798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":119991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":888832,"data_stored":710851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.0410000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.3150000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.9700000000000006}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.625}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.6790000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.6559999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.782}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:46:19.561 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:46:19.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.730+0000 7f772e4d8640 1 Processor -- start 2023-10-18T10:46:19.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.730+0000 7f772e4d8640 1 -- start start 2023-10-18T10:46:19.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.734+0000 7f772e4d8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77280b0220 0x7f7728070990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:19.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.734+0000 7f772e4d8640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f772806fe10 con 0x7f7728158d20 2023-10-18T10:46:19.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.734+0000 7f772e4d8640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77280b06b0 con 0x7f77280afd90 2023-10-18T10:46:19.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.734+0000 7f772e4d8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77280c9220 con 0x7f77280b0220 2023-10-18T10:46:19.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.734+0000 7f7727fff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7728158d20 0x7f772815d110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35694/0 (socket says 172.21.15.89:35694) 2023-10-18T10:46:19.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.734+0000 7f7727fff640 1 -- 172.21.15.89:0/476134038 learned_addr learned my addr 172.21.15.89:0/476134038 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:19.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.734+0000 7f7726ffd640 1 -- 172.21.15.89:0/476134038 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2583478470 0 0) 0x7f772806fe10 con 0x7f7728158d20 2023-10-18T10:46:19.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.734+0000 7f772ca75640 1 --2- 172.21.15.89:0/476134038 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77280b0220 0x7f7728070990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:19.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.734+0000 7f7726ffd640 1 -- 172.21.15.89:0/476134038 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7710003450 con 0x7f7728158d20 2023-10-18T10:46:19.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.734+0000 7f7726ffd640 1 -- 172.21.15.89:0/476134038 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 265234830 0 0) 0x7f7710003450 con 0x7f7728158d20 2023-10-18T10:46:19.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.734+0000 7f7726ffd640 1 -- 172.21.15.89:0/476134038 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77280b0220 msgr2=0x7f7728070990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:19.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.734+0000 7f7726ffd640 1 --2- 172.21.15.89:0/476134038 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77280b0220 0x7f7728070990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.734+0000 7f7726ffd640 1 -- 172.21.15.89:0/476134038 >> v1:172.21.15.154:6789/0 conn(0x7f77280afd90 legacy=0x7f7728070280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:19.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.734+0000 7f7726ffd640 1 -- 172.21.15.89:0/476134038 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77280c9500 con 0x7f7728158d20 2023-10-18T10:46:19.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.734+0000 7f7726ffd640 1 -- 172.21.15.89:0/476134038 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7f7718001680 con 0x7f7728158d20 2023-10-18T10:46:19.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.734+0000 7f772ca75640 1 --2- 172.21.15.89:0/476134038 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77280b0220 0x7f7728070990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:19.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.734+0000 7f7726ffd640 1 -- 172.21.15.89:0/476134038 >> v1:172.21.15.89:6789/0 conn(0x7f7728158d20 legacy=0x7f772815d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:19.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.734+0000 7f7726ffd640 1 --2- 172.21.15.89:0/476134038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7710003aa0 0x7f7710003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:19.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.734+0000 7f7726ffd640 1 --2- 172.21.15.89:0/476134038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77100088b0 0x7f771000ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:19.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.734+0000 7f7726ffd640 1 --2- 172.21.15.89:0/476134038 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77280b0220 0x7f771000d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:19.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.734+0000 7f7726ffd640 1 -- 172.21.15.89:0/476134038 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77280c9220 con 0x7f77100088b0 2023-10-18T10:46:19.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.734+0000 7f7726ffd640 1 -- 172.21.15.89:0/476134038 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771000dc90 con 0x7f7710003aa0 2023-10-18T10:46:19.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.734+0000 7f7726ffd640 1 -- 172.21.15.89:0/476134038 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f771000de50 con 0x7f77280b0220 2023-10-18T10:46:19.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.734+0000 7f77277fe640 1 --2- 172.21.15.89:0/476134038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77100088b0 0x7f771000ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:19.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.734+0000 7f772ca75640 1 --2- 172.21.15.89:0/476134038 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77280b0220 0x7f771000d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:19.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.734+0000 7f7727fff640 1 --2- 172.21.15.89:0/476134038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7710003aa0 0x7f7710003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:19.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.734+0000 7f77277fe640 1 -- 172.21.15.89:0/476134038 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77280b0220 msgr2=0x7f771000d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:19.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.734+0000 7f77277fe640 1 --2- 172.21.15.89:0/476134038 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77280b0220 0x7f771000d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.734+0000 7f77277fe640 1 -- 172.21.15.89:0/476134038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7710003aa0 msgr2=0x7f7710003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:19.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.734+0000 7f77277fe640 1 --2- 172.21.15.89:0/476134038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7710003aa0 0x7f7710003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.734+0000 7f77277fe640 1 -- 172.21.15.89:0/476134038 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7710003450 con 0x7f77100088b0 2023-10-18T10:46:19.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.738+0000 7f77277fe640 1 --2- 172.21.15.89:0/476134038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77100088b0 0x7f771000ad20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f771c001780 tx=0x7f771c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=990a5dae0af186d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:19.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.738+0000 7f7726ffd640 1 -- 172.21.15.89:0/476134038 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771c0052d0 con 0x7f77100088b0 2023-10-18T10:46:19.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.738+0000 7f7726ffd640 1 -- 172.21.15.89:0/476134038 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f771c0a7050 con 0x7f77100088b0 2023-10-18T10:46:19.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.738+0000 7f7726ffd640 1 -- 172.21.15.89:0/476134038 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771c003600 con 0x7f77100088b0 2023-10-18T10:46:19.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.738+0000 7f772e4d8640 1 -- 172.21.15.89:0/476134038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77100088b0 msgr2=0x7f771000ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:19.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.738+0000 7f772e4d8640 1 --2- 172.21.15.89:0/476134038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77100088b0 0x7f771000ad20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f771c001780 tx=0x7f771c01c400 comp rx=0 tx=0).stop 2023-10-18T10:46:19.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.738+0000 7f772ca75640 1 -- 172.21.15.89:0/476134038 reap_dead start 2023-10-18T10:46:19.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.738+0000 7f772e4d8640 1 -- 172.21.15.89:0/476134038 shutdown_connections 2023-10-18T10:46:19.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.738+0000 7f772e4d8640 1 -- 172.21.15.89:0/476134038 >> 172.21.15.89:0/476134038 conn(0x7f7728069f00 msgr2=0x7f77280cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:19.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.738+0000 7f772e4d8640 1 -- 172.21.15.89:0/476134038 shutdown_connections 2023-10-18T10:46:19.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.738+0000 7f772e4d8640 1 -- 172.21.15.89:0/476134038 wait complete. 2023-10-18T10:46:19.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.738+0000 7f772e4d8640 1 Processor -- start 2023-10-18T10:46:19.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.738+0000 7f772e4d8640 1 -- start start 2023-10-18T10:46:19.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.742+0000 7f772e4d8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77100088b0 0x7f77281f5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:19.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.742+0000 7f772e4d8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77280b2500 0x7f7728145dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:19.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.742+0000 7f772e4d8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f772806b7d0 0x7f772806bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:19.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.742+0000 7f772e4d8640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77281f2980 con 0x7f772806b7d0 2023-10-18T10:46:19.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.742+0000 7f772e4d8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772806c190 con 0x7f77280b2500 2023-10-18T10:46:19.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.742+0000 7f772e4d8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77281f1e00 con 0x7f77100088b0 2023-10-18T10:46:19.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.742+0000 7f772ca75640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f772806b7d0 0x7f772806bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:19.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.742+0000 7f7727fff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77100088b0 0x7f77281f5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:19.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.742+0000 7f772ca75640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f772806b7d0 0x7f772806bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45534/0 (socket says 172.21.15.89:45534) 2023-10-18T10:46:19.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.742+0000 7f772ca75640 1 -- 172.21.15.89:0/886879791 learned_addr learned my addr 172.21.15.89:0/886879791 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:19.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.742+0000 7f77277fe640 1 --2- 172.21.15.89:0/886879791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77280b2500 0x7f7728145dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:19.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.742+0000 7f772ca75640 1 -- 172.21.15.89:0/886879791 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77100088b0 msgr2=0x7f77281f5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:19.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.742+0000 7f772ca75640 1 --2- 172.21.15.89:0/886879791 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77100088b0 0x7f77281f5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.742+0000 7f772ca75640 1 -- 172.21.15.89:0/886879791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77280b2500 msgr2=0x7f7728145dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:19.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.742+0000 7f772ca75640 1 --2- 172.21.15.89:0/886879791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77280b2500 0x7f7728145dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.742+0000 7f772ca75640 1 -- 172.21.15.89:0/886879791 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77281f20e0 con 0x7f772806b7d0 2023-10-18T10:46:19.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.742+0000 7f772ca75640 1 --2- 172.21.15.89:0/886879791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f772806b7d0 0x7f772806bc50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f772815c690 tx=0x7f7720010530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:19.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.742+0000 7f7724ff9640 1 -- 172.21.15.89:0/886879791 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772001b070 con 0x7f772806b7d0 2023-10-18T10:46:19.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.742+0000 7f7724ff9640 1 -- 172.21.15.89:0/886879791 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7720010cd0 con 0x7f772806b7d0 2023-10-18T10:46:19.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.742+0000 7f772e4d8640 1 -- 172.21.15.89:0/886879791 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77281f23c0 con 0x7f772806b7d0 2023-10-18T10:46:19.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.742+0000 7f7724ff9640 1 -- 172.21.15.89:0/886879791 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7720010e80 con 0x7f772806b7d0 2023-10-18T10:46:19.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.742+0000 7f772e4d8640 1 -- 172.21.15.89:0/886879791 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7728146310 con 0x7f772806b7d0 2023-10-18T10:46:19.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.746+0000 7f772e4d8640 1 -- 172.21.15.89:0/886879791 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7728148800 con 0x7f772806b7d0 2023-10-18T10:46:19.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.746+0000 7f7724ff9640 1 -- 172.21.15.89:0/886879791 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f7720017430 con 0x7f772806b7d0 2023-10-18T10:46:19.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.746+0000 7f7724ff9640 1 --2- 172.21.15.89:0/886879791 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7714065680 0x7f7714067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:19.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.746+0000 7f7724ff9640 1 -- 172.21.15.89:0/886879791 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f77200834f0 con 0x7f772806b7d0 2023-10-18T10:46:19.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.746+0000 7f7727fff640 1 --2- 172.21.15.89:0/886879791 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7714065680 0x7f7714067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:19.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.746+0000 7f7727fff640 1 --2- 172.21.15.89:0/886879791 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7714065680 0x7f7714067b40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7718001060 tx=0x7f7718001c20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:19.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.750+0000 7f7724ff9640 1 -- 172.21.15.89:0/886879791 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f7728148800 con 0x7f772806b7d0 2023-10-18T10:46:19.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.902+0000 7f772e4d8640 1 -- 172.21.15.89:0/886879791 --> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f77280c9ac0 con 0x7f7714065680 2023-10-18T10:46:19.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.914+0000 7f7724ff9640 1 -- 172.21.15.89:0/886879791 <== mgr.4099 v2:172.21.15.154:6832/1086608968 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367299 (secure 0 0 0) 0x7f7718001560 con 0x7f7714065680 2023-10-18T10:46:19.919 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:19.921 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-18T10:46:19.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.918+0000 7f772e4d8640 1 -- 172.21.15.89:0/886879791 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7714065680 msgr2=0x7f7714067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:19.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.918+0000 7f772e4d8640 1 --2- 172.21.15.89:0/886879791 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7714065680 0x7f7714067b40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7718001060 tx=0x7f7718001c20 comp rx=0 tx=0).stop 2023-10-18T10:46:19.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.918+0000 7f772e4d8640 1 -- 172.21.15.89:0/886879791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f772806b7d0 msgr2=0x7f772806bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:19.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.918+0000 7f772e4d8640 1 --2- 172.21.15.89:0/886879791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f772806b7d0 0x7f772806bc50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f772815c690 tx=0x7f7720010530 comp rx=0 tx=0).stop 2023-10-18T10:46:19.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.918+0000 7f772e4d8640 1 -- 172.21.15.89:0/886879791 shutdown_connections 2023-10-18T10:46:19.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.918+0000 7f772e4d8640 1 --2- 172.21.15.89:0/886879791 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7714065680 0x7f7714067b40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.918+0000 7f772e4d8640 1 --2- 172.21.15.89:0/886879791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f772806b7d0 0x7f772806bc50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.918+0000 7f772e4d8640 1 --2- 172.21.15.89:0/886879791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77280b2500 0x7f7728145dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.918+0000 7f772e4d8640 1 --2- 172.21.15.89:0/886879791 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77100088b0 0x7f77281f5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:19.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.918+0000 7f772e4d8640 1 -- 172.21.15.89:0/886879791 >> 172.21.15.89:0/886879791 conn(0x7f7728069f00 msgr2=0x7f7728154ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:19.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.918+0000 7f772e4d8640 1 -- 172.21.15.89:0/886879791 shutdown_connections 2023-10-18T10:46:19.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:19.918+0000 7f772e4d8640 1 -- 172.21.15.89:0/886879791 wait complete. 2023-10-18T10:46:19.938 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2023-10-18T10:46:19.058633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219660,"kb_used_data":3496,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754755060,"statfs":{"total":773094113280,"available":772869181440,"internally_reserved":0,"allocated":3579904,"data_stored":2149222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2229,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003605"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084532+0000","last_change":"2023-10-18T10:45:59.809013+0000","last_active":"2023-10-18T10:46:01.084532+0000","last_peered":"2023-10-18T10:46:01.084532+0000","last_clean":"2023-10-18T10:46:01.084532+0000","last_became_active":"2023-10-18T10:45:59.808636+0000","last_became_peered":"2023-10-18T10:45:59.808636+0000","last_unstale":"2023-10-18T10:46:01.084532+0000","last_undegraded":"2023-10-18T10:46:01.084532+0000","last_fullsized":"2023-10-18T10:46:01.084532+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:30:08.797534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085907+0000","last_change":"2023-10-18T10:45:55.297617+0000","last_active":"2023-10-18T10:46:01.085907+0000","last_peered":"2023-10-18T10:46:01.085907+0000","last_clean":"2023-10-18T10:46:01.085907+0000","last_became_active":"2023-10-18T10:45:55.296996+0000","last_became_peered":"2023-10-18T10:45:55.296996+0000","last_unstale":"2023-10-18T10:46:01.085907+0000","last_undegraded":"2023-10-18T10:46:01.085907+0000","last_fullsized":"2023-10-18T10:46:01.085907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:04:27.847910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768182+0000","last_change":"2023-10-18T10:45:55.770859+0000","last_active":"2023-10-18T10:46:00.768182+0000","last_peered":"2023-10-18T10:46:00.768182+0000","last_clean":"2023-10-18T10:46:00.768182+0000","last_became_active":"2023-10-18T10:45:55.769578+0000","last_became_peered":"2023-10-18T10:45:55.769578+0000","last_unstale":"2023-10-18T10:46:00.768182+0000","last_undegraded":"2023-10-18T10:46:00.768182+0000","last_fullsized":"2023-10-18T10:46:00.768182+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:08:17.812888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.767991+0000","last_change":"2023-10-18T10:45:59.790837+0000","last_active":"2023-10-18T10:46:00.767991+0000","last_peered":"2023-10-18T10:46:00.767991+0000","last_clean":"2023-10-18T10:46:00.767991+0000","last_became_active":"2023-10-18T10:45:59.790488+0000","last_became_peered":"2023-10-18T10:45:59.790488+0000","last_unstale":"2023-10-18T10:46:00.767991+0000","last_undegraded":"2023-10-18T10:46:00.767991+0000","last_fullsized":"2023-10-18T10:46:00.767991+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:43:23.609561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902666+0000","last_change":"2023-10-18T10:45:55.295114+0000","last_active":"2023-10-18T10:46:00.902666+0000","last_peered":"2023-10-18T10:46:00.902666+0000","last_clean":"2023-10-18T10:46:00.902666+0000","last_became_active":"2023-10-18T10:45:55.294672+0000","last_became_peered":"2023-10-18T10:45:55.294672+0000","last_unstale":"2023-10-18T10:46:00.902666+0000","last_undegraded":"2023-10-18T10:46:00.902666+0000","last_fullsized":"2023-10-18T10:46:00.902666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:06:43.162381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902162+0000","last_change":"2023-10-18T10:45:55.756885+0000","last_active":"2023-10-18T10:46:00.902162+0000","last_peered":"2023-10-18T10:46:00.902162+0000","last_clean":"2023-10-18T10:46:00.902162+0000","last_became_active":"2023-10-18T10:45:55.756184+0000","last_became_peered":"2023-10-18T10:45:55.756184+0000","last_unstale":"2023-10-18T10:46:00.902162+0000","last_undegraded":"2023-10-18T10:46:00.902162+0000","last_fullsized":"2023-10-18T10:46:00.902162+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:20:46.641913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085021+0000","last_change":"2023-10-18T10:45:59.796520+0000","last_active":"2023-10-18T10:46:01.085021+0000","last_peered":"2023-10-18T10:46:01.085021+0000","last_clean":"2023-10-18T10:46:01.085021+0000","last_became_active":"2023-10-18T10:45:59.796056+0000","last_became_peered":"2023-10-18T10:45:59.796056+0000","last_unstale":"2023-10-18T10:46:01.085021+0000","last_undegraded":"2023-10-18T10:46:01.085021+0000","last_fullsized":"2023-10-18T10:46:01.085021+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:55:12.299657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085752+0000","last_change":"2023-10-18T10:45:55.302985+0000","last_active":"2023-10-18T10:46:01.085752+0000","last_peered":"2023-10-18T10:46:01.085752+0000","last_clean":"2023-10-18T10:46:01.085752+0000","last_became_active":"2023-10-18T10:45:55.302657+0000","last_became_peered":"2023-10-18T10:45:55.302657+0000","last_unstale":"2023-10-18T10:46:01.085752+0000","last_undegraded":"2023-10-18T10:46:01.085752+0000","last_fullsized":"2023-10-18T10:46:01.085752+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:55:03.412092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768360+0000","last_change":"2023-10-18T10:45:55.776143+0000","last_active":"2023-10-18T10:46:00.768360+0000","last_peered":"2023-10-18T10:46:00.768360+0000","last_clean":"2023-10-18T10:46:00.768360+0000","last_became_active":"2023-10-18T10:45:55.770969+0000","last_became_peered":"2023-10-18T10:45:55.770969+0000","last_unstale":"2023-10-18T10:46:00.768360+0000","last_undegraded":"2023-10-18T10:46:00.768360+0000","last_fullsized":"2023-10-18T10:46:00.768360+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:53:59.036438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085289+0000","last_change":"2023-10-18T10:45:59.810699+0000","last_active":"2023-10-18T10:46:01.085289+0000","last_peered":"2023-10-18T10:46:01.085289+0000","last_clean":"2023-10-18T10:46:01.085289+0000","last_became_active":"2023-10-18T10:45:59.810219+0000","last_became_peered":"2023-10-18T10:45:59.810219+0000","last_unstale":"2023-10-18T10:46:01.085289+0000","last_undegraded":"2023-10-18T10:46:01.085289+0000","last_fullsized":"2023-10-18T10:46:01.085289+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:06:30.723321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085485+0000","last_change":"2023-10-18T10:45:55.300287+0000","last_active":"2023-10-18T10:46:01.085485+0000","last_peered":"2023-10-18T10:46:01.085485+0000","last_clean":"2023-10-18T10:46:01.085485+0000","last_became_active":"2023-10-18T10:45:55.300014+0000","last_became_peered":"2023-10-18T10:45:55.300014+0000","last_unstale":"2023-10-18T10:46:01.085485+0000","last_undegraded":"2023-10-18T10:46:01.085485+0000","last_fullsized":"2023-10-18T10:46:01.085485+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:40:14.773073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768476+0000","last_change":"2023-10-18T10:45:55.772481+0000","last_active":"2023-10-18T10:46:00.768476+0000","last_peered":"2023-10-18T10:46:00.768476+0000","last_clean":"2023-10-18T10:46:00.768476+0000","last_became_active":"2023-10-18T10:45:55.770365+0000","last_became_peered":"2023-10-18T10:45:55.770365+0000","last_unstale":"2023-10-18T10:46:00.768476+0000","last_undegraded":"2023-10-18T10:46:00.768476+0000","last_fullsized":"2023-10-18T10:46:00.768476+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:21:41.323647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085226+0000","last_change":"2023-10-18T10:45:59.796654+0000","last_active":"2023-10-18T10:46:01.085226+0000","last_peered":"2023-10-18T10:46:01.085226+0000","last_clean":"2023-10-18T10:46:01.085226+0000","last_became_active":"2023-10-18T10:45:59.796115+0000","last_became_peered":"2023-10-18T10:45:59.796115+0000","last_unstale":"2023-10-18T10:46:01.085226+0000","last_undegraded":"2023-10-18T10:46:01.085226+0000","last_fullsized":"2023-10-18T10:46:01.085226+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:28:08.869152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086358+0000","last_change":"2023-10-18T10:45:55.302642+0000","last_active":"2023-10-18T10:46:01.086358+0000","last_peered":"2023-10-18T10:46:01.086358+0000","last_clean":"2023-10-18T10:46:01.086358+0000","last_became_active":"2023-10-18T10:45:55.302307+0000","last_became_peered":"2023-10-18T10:45:55.302307+0000","last_unstale":"2023-10-18T10:46:01.086358+0000","last_undegraded":"2023-10-18T10:46:01.086358+0000","last_fullsized":"2023-10-18T10:46:01.086358+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:40:52.185453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.767207+0000","last_change":"2023-10-18T10:45:55.765319+0000","last_active":"2023-10-18T10:46:00.767207+0000","last_peered":"2023-10-18T10:46:00.767207+0000","last_clean":"2023-10-18T10:46:00.767207+0000","last_became_active":"2023-10-18T10:45:55.764958+0000","last_became_peered":"2023-10-18T10:45:55.764958+0000","last_unstale":"2023-10-18T10:46:00.767207+0000","last_undegraded":"2023-10-18T10:46:00.767207+0000","last_fullsized":"2023-10-18T10:46:00.767207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:30:49.094396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768337+0000","last_change":"2023-10-18T10:45:59.791333+0000","last_active":"2023-10-18T10:46:00.768337+0000","last_peered":"2023-10-18T10:46:00.768337+0000","last_clean":"2023-10-18T10:46:00.768337+0000","last_became_active":"2023-10-18T10:45:59.790903+0000","last_became_peered":"2023-10-18T10:45:59.790903+0000","last_unstale":"2023-10-18T10:46:00.768337+0000","last_undegraded":"2023-10-18T10:46:00.768337+0000","last_fullsized":"2023-10-18T10:46:00.768337+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:52:13.857250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768597+0000","last_change":"2023-10-18T10:45:55.302330+0000","last_active":"2023-10-18T10:46:00.768597+0000","last_peered":"2023-10-18T10:46:00.768597+0000","last_clean":"2023-10-18T10:46:00.768597+0000","last_became_active":"2023-10-18T10:45:55.301709+0000","last_became_peered":"2023-10-18T10:45:55.301709+0000","last_unstale":"2023-10-18T10:46:00.768597+0000","last_undegraded":"2023-10-18T10:46:00.768597+0000","last_fullsized":"2023-10-18T10:46:00.768597+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:40:49.128175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902373+0000","last_change":"2023-10-18T10:45:55.767859+0000","last_active":"2023-10-18T10:46:00.902373+0000","last_peered":"2023-10-18T10:46:00.902373+0000","last_clean":"2023-10-18T10:46:00.902373+0000","last_became_active":"2023-10-18T10:45:55.767505+0000","last_became_peered":"2023-10-18T10:45:55.767505+0000","last_unstale":"2023-10-18T10:46:00.902373+0000","last_undegraded":"2023-10-18T10:46:00.902373+0000","last_fullsized":"2023-10-18T10:46:00.902373+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:12:23.654089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.770164+0000","last_change":"2023-10-18T10:45:59.794892+0000","last_active":"2023-10-18T10:46:00.770164+0000","last_peered":"2023-10-18T10:46:00.770164+0000","last_clean":"2023-10-18T10:46:00.770164+0000","last_became_active":"2023-10-18T10:45:59.794477+0000","last_became_peered":"2023-10-18T10:45:59.794477+0000","last_unstale":"2023-10-18T10:46:00.770164+0000","last_undegraded":"2023-10-18T10:46:00.770164+0000","last_fullsized":"2023-10-18T10:46:00.770164+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:04:07.040979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084932+0000","last_change":"2023-10-18T10:45:55.290491+0000","last_active":"2023-10-18T10:46:01.084932+0000","last_peered":"2023-10-18T10:46:01.084932+0000","last_clean":"2023-10-18T10:46:01.084932+0000","last_became_active":"2023-10-18T10:45:55.290134+0000","last_became_peered":"2023-10-18T10:45:55.290134+0000","last_unstale":"2023-10-18T10:46:01.084932+0000","last_undegraded":"2023-10-18T10:46:01.084932+0000","last_fullsized":"2023-10-18T10:46:01.084932+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:39.977541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086354+0000","last_change":"2023-10-18T10:45:55.765415+0000","last_active":"2023-10-18T10:46:01.086354+0000","last_peered":"2023-10-18T10:46:01.086354+0000","last_clean":"2023-10-18T10:46:01.086354+0000","last_became_active":"2023-10-18T10:45:55.764573+0000","last_became_peered":"2023-10-18T10:45:55.764573+0000","last_unstale":"2023-10-18T10:46:01.086354+0000","last_undegraded":"2023-10-18T10:46:01.086354+0000","last_fullsized":"2023-10-18T10:46:01.086354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:33:49.751994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086225+0000","last_change":"2023-10-18T10:45:59.801842+0000","last_active":"2023-10-18T10:46:01.086225+0000","last_peered":"2023-10-18T10:46:01.086225+0000","last_clean":"2023-10-18T10:46:01.086225+0000","last_became_active":"2023-10-18T10:45:59.801367+0000","last_became_peered":"2023-10-18T10:45:59.801367+0000","last_unstale":"2023-10-18T10:46:01.086225+0000","last_undegraded":"2023-10-18T10:46:01.086225+0000","last_fullsized":"2023-10-18T10:46:01.086225+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:56:46.630947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902154+0000","last_change":"2023-10-18T10:45:55.298611+0000","last_active":"2023-10-18T10:46:00.902154+0000","last_peered":"2023-10-18T10:46:00.902154+0000","last_clean":"2023-10-18T10:46:00.902154+0000","last_became_active":"2023-10-18T10:45:55.297980+0000","last_became_peered":"2023-10-18T10:45:55.297980+0000","last_unstale":"2023-10-18T10:46:00.902154+0000","last_undegraded":"2023-10-18T10:46:00.902154+0000","last_fullsized":"2023-10-18T10:46:00.902154+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:05:56.466612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.770285+0000","last_change":"2023-10-18T10:45:55.775435+0000","last_active":"2023-10-18T10:46:00.770285+0000","last_peered":"2023-10-18T10:46:00.770285+0000","last_clean":"2023-10-18T10:46:00.770285+0000","last_became_active":"2023-10-18T10:45:55.774841+0000","last_became_peered":"2023-10-18T10:45:55.774841+0000","last_unstale":"2023-10-18T10:46:00.770285+0000","last_undegraded":"2023-10-18T10:46:00.770285+0000","last_fullsized":"2023-10-18T10:46:00.770285+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:27:03.226674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.901875+0000","last_change":"2023-10-18T10:45:55.768221+0000","last_active":"2023-10-18T10:46:00.901875+0000","last_peered":"2023-10-18T10:46:00.901875+0000","last_clean":"2023-10-18T10:46:00.901875+0000","last_became_active":"2023-10-18T10:45:55.767975+0000","last_became_peered":"2023-10-18T10:45:55.767975+0000","last_unstale":"2023-10-18T10:46:00.901875+0000","last_undegraded":"2023-10-18T10:46:00.901875+0000","last_fullsized":"2023-10-18T10:46:00.901875+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:14:00.849685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086003+0000","last_change":"2023-10-18T10:45:55.291798+0000","last_active":"2023-10-18T10:46:01.086003+0000","last_peered":"2023-10-18T10:46:01.086003+0000","last_clean":"2023-10-18T10:46:01.086003+0000","last_became_active":"2023-10-18T10:45:55.291380+0000","last_became_peered":"2023-10-18T10:45:55.291380+0000","last_unstale":"2023-10-18T10:46:01.086003+0000","last_undegraded":"2023-10-18T10:46:01.086003+0000","last_fullsized":"2023-10-18T10:46:01.086003+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:49:45.673807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.769758+0000","last_change":"2023-10-18T10:45:59.794170+0000","last_active":"2023-10-18T10:46:00.769758+0000","last_peered":"2023-10-18T10:46:00.769758+0000","last_clean":"2023-10-18T10:46:00.769758+0000","last_became_active":"2023-10-18T10:45:59.793675+0000","last_became_peered":"2023-10-18T10:45:59.793675+0000","last_unstale":"2023-10-18T10:46:00.769758+0000","last_undegraded":"2023-10-18T10:46:00.769758+0000","last_fullsized":"2023-10-18T10:46:00.769758+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:08:00.737560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.901966+0000","last_change":"2023-10-18T10:45:55.765758+0000","last_active":"2023-10-18T10:46:00.901966+0000","last_peered":"2023-10-18T10:46:00.901966+0000","last_clean":"2023-10-18T10:46:00.901966+0000","last_became_active":"2023-10-18T10:45:55.765436+0000","last_became_peered":"2023-10-18T10:45:55.765436+0000","last_unstale":"2023-10-18T10:46:00.901966+0000","last_undegraded":"2023-10-18T10:46:00.901966+0000","last_fullsized":"2023-10-18T10:46:00.901966+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:40:53.192771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086132+0000","last_change":"2023-10-18T10:45:55.295720+0000","last_active":"2023-10-18T10:46:01.086132+0000","last_peered":"2023-10-18T10:46:01.086132+0000","last_clean":"2023-10-18T10:46:01.086132+0000","last_became_active":"2023-10-18T10:45:55.295401+0000","last_became_peered":"2023-10-18T10:45:55.295401+0000","last_unstale":"2023-10-18T10:46:01.086132+0000","last_undegraded":"2023-10-18T10:46:01.086132+0000","last_fullsized":"2023-10-18T10:46:01.086132+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:49:45.673807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084670+0000","last_change":"2023-10-18T10:45:59.798312+0000","last_active":"2023-10-18T10:46:01.084670+0000","last_peered":"2023-10-18T10:46:01.084670+0000","last_clean":"2023-10-18T10:46:01.084670+0000","last_became_active":"2023-10-18T10:45:59.797944+0000","last_became_peered":"2023-10-18T10:45:59.797944+0000","last_unstale":"2023-10-18T10:46:01.084670+0000","last_undegraded":"2023-10-18T10:46:01.084670+0000","last_fullsized":"2023-10-18T10:46:01.084670+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:51:40.857910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085288+0000","last_change":"2023-10-18T10:45:55.307908+0000","last_active":"2023-10-18T10:46:01.085288+0000","last_peered":"2023-10-18T10:46:01.085288+0000","last_clean":"2023-10-18T10:46:01.085288+0000","last_became_active":"2023-10-18T10:45:55.307593+0000","last_became_peered":"2023-10-18T10:45:55.307593+0000","last_unstale":"2023-10-18T10:46:01.085288+0000","last_undegraded":"2023-10-18T10:46:01.085288+0000","last_fullsized":"2023-10-18T10:46:01.085288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:20:45.634595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084888+0000","last_change":"2023-10-18T10:45:55.778356+0000","last_active":"2023-10-18T10:46:01.084888+0000","last_peered":"2023-10-18T10:46:01.084888+0000","last_clean":"2023-10-18T10:46:01.084888+0000","last_became_active":"2023-10-18T10:45:55.776465+0000","last_became_peered":"2023-10-18T10:45:55.776465+0000","last_unstale":"2023-10-18T10:46:01.084888+0000","last_undegraded":"2023-10-18T10:46:01.084888+0000","last_fullsized":"2023-10-18T10:46:01.084888+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:10:59.545961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902026+0000","last_change":"2023-10-18T10:45:59.788972+0000","last_active":"2023-10-18T10:46:00.902026+0000","last_peered":"2023-10-18T10:46:00.902026+0000","last_clean":"2023-10-18T10:46:00.902026+0000","last_became_active":"2023-10-18T10:45:59.788613+0000","last_became_peered":"2023-10-18T10:45:59.788613+0000","last_unstale":"2023-10-18T10:46:00.902026+0000","last_undegraded":"2023-10-18T10:46:00.902026+0000","last_fullsized":"2023-10-18T10:46:00.902026+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:33:04.407310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"27'9","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:06.342753+0000","last_change":"2023-10-18T10:45:55.298794+0000","last_active":"2023-10-18T10:46:06.342753+0000","last_peered":"2023-10-18T10:46:06.342753+0000","last_clean":"2023-10-18T10:46:06.342753+0000","last_became_active":"2023-10-18T10:45:55.297181+0000","last_became_peered":"2023-10-18T10:45:55.297181+0000","last_unstale":"2023-10-18T10:46:06.342753+0000","last_undegraded":"2023-10-18T10:46:06.342753+0000","last_fullsized":"2023-10-18T10:46:06.342753+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:16:43.562003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768593+0000","last_change":"2023-10-18T10:45:55.777329+0000","last_active":"2023-10-18T10:46:00.768593+0000","last_peered":"2023-10-18T10:46:00.768593+0000","last_clean":"2023-10-18T10:46:00.768593+0000","last_became_active":"2023-10-18T10:45:55.777047+0000","last_became_peered":"2023-10-18T10:45:55.777047+0000","last_unstale":"2023-10-18T10:46:00.768593+0000","last_undegraded":"2023-10-18T10:46:00.768593+0000","last_fullsized":"2023-10-18T10:46:00.768593+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:09:20.566242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084662+0000","last_change":"2023-10-18T10:45:59.804498+0000","last_active":"2023-10-18T10:46:01.084662+0000","last_peered":"2023-10-18T10:46:01.084662+0000","last_clean":"2023-10-18T10:46:01.084662+0000","last_became_active":"2023-10-18T10:45:59.804083+0000","last_became_peered":"2023-10-18T10:45:59.804083+0000","last_unstale":"2023-10-18T10:46:01.084662+0000","last_undegraded":"2023-10-18T10:46:01.084662+0000","last_fullsized":"2023-10-18T10:46:01.084662+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:04:58.822427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.769276+0000","last_change":"2023-10-18T10:45:55.774541+0000","last_active":"2023-10-18T10:46:00.769276+0000","last_peered":"2023-10-18T10:46:00.769276+0000","last_clean":"2023-10-18T10:46:00.769276+0000","last_became_active":"2023-10-18T10:45:55.774258+0000","last_became_peered":"2023-10-18T10:45:55.774258+0000","last_unstale":"2023-10-18T10:46:00.769276+0000","last_undegraded":"2023-10-18T10:46:00.769276+0000","last_fullsized":"2023-10-18T10:46:00.769276+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:11:42.497290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.770380+0000","last_change":"2023-10-18T10:45:50.252021+0000","last_active":"2023-10-18T10:46:00.770380+0000","last_peered":"2023-10-18T10:46:00.770380+0000","last_clean":"2023-10-18T10:46:00.770380+0000","last_became_active":"2023-10-18T10:45:50.251427+0000","last_became_peered":"2023-10-18T10:45:50.251427+0000","last_unstale":"2023-10-18T10:46:00.770380+0000","last_undegraded":"2023-10-18T10:46:00.770380+0000","last_fullsized":"2023-10-18T10:46:00.770380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:43.575003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"26'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086531+0000","last_change":"2023-10-18T10:45:55.296927+0000","last_active":"2023-10-18T10:46:01.086531+0000","last_peered":"2023-10-18T10:46:01.086531+0000","last_clean":"2023-10-18T10:46:01.086531+0000","last_became_active":"2023-10-18T10:45:55.296614+0000","last_became_peered":"2023-10-18T10:45:55.296614+0000","last_unstale":"2023-10-18T10:46:01.086531+0000","last_undegraded":"2023-10-18T10:46:01.086531+0000","last_fullsized":"2023-10-18T10:46:01.086531+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:05:56.466612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902246+0000","last_change":"2023-10-18T10:45:59.786937+0000","last_active":"2023-10-18T10:46:00.902246+0000","last_peered":"2023-10-18T10:46:00.902246+0000","last_clean":"2023-10-18T10:46:00.902246+0000","last_became_active":"2023-10-18T10:45:59.786436+0000","last_became_peered":"2023-10-18T10:45:59.786436+0000","last_unstale":"2023-10-18T10:46:00.902246+0000","last_undegraded":"2023-10-18T10:46:00.902246+0000","last_fullsized":"2023-10-18T10:46:00.902246+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:37:40.801018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902006+0000","last_change":"2023-10-18T10:45:55.766947+0000","last_active":"2023-10-18T10:46:00.902006+0000","last_peered":"2023-10-18T10:46:00.902006+0000","last_clean":"2023-10-18T10:46:00.902006+0000","last_became_active":"2023-10-18T10:45:55.765951+0000","last_became_peered":"2023-10-18T10:45:55.765951+0000","last_unstale":"2023-10-18T10:46:00.902006+0000","last_undegraded":"2023-10-18T10:46:00.902006+0000","last_fullsized":"2023-10-18T10:46:00.902006+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:06:27.237867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085276+0000","last_change":"2023-10-18T10:45:50.252876+0000","last_active":"2023-10-18T10:46:01.085276+0000","last_peered":"2023-10-18T10:46:01.085276+0000","last_clean":"2023-10-18T10:46:01.085276+0000","last_became_active":"2023-10-18T10:45:50.252559+0000","last_became_peered":"2023-10-18T10:45:50.252559+0000","last_unstale":"2023-10-18T10:46:01.085276+0000","last_undegraded":"2023-10-18T10:46:01.085276+0000","last_fullsized":"2023-10-18T10:46:01.085276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:55:15.913186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902858+0000","last_change":"2023-10-18T10:45:55.300421+0000","last_active":"2023-10-18T10:46:00.902858+0000","last_peered":"2023-10-18T10:46:00.902858+0000","last_clean":"2023-10-18T10:46:00.902858+0000","last_became_active":"2023-10-18T10:45:55.300170+0000","last_became_peered":"2023-10-18T10:45:55.300170+0000","last_unstale":"2023-10-18T10:46:00.902858+0000","last_undegraded":"2023-10-18T10:46:00.902858+0000","last_fullsized":"2023-10-18T10:46:00.902858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:52:09.364478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768261+0000","last_change":"2023-10-18T10:45:59.799805+0000","last_active":"2023-10-18T10:46:00.768261+0000","last_peered":"2023-10-18T10:46:00.768261+0000","last_clean":"2023-10-18T10:46:00.768261+0000","last_became_active":"2023-10-18T10:45:59.799112+0000","last_became_peered":"2023-10-18T10:45:59.799112+0000","last_unstale":"2023-10-18T10:46:00.768261+0000","last_undegraded":"2023-10-18T10:46:00.768261+0000","last_fullsized":"2023-10-18T10:46:00.768261+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:49:10.356115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086095+0000","last_change":"2023-10-18T10:45:55.774299+0000","last_active":"2023-10-18T10:46:01.086095+0000","last_peered":"2023-10-18T10:46:01.086095+0000","last_clean":"2023-10-18T10:46:01.086095+0000","last_became_active":"2023-10-18T10:45:55.773409+0000","last_became_peered":"2023-10-18T10:45:55.773409+0000","last_unstale":"2023-10-18T10:46:01.086095+0000","last_undegraded":"2023-10-18T10:46:01.086095+0000","last_fullsized":"2023-10-18T10:46:01.086095+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:37:37.315564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085581+0000","last_change":"2023-10-18T10:45:50.585224+0000","last_active":"2023-10-18T10:46:01.085581+0000","last_peered":"2023-10-18T10:46:01.085581+0000","last_clean":"2023-10-18T10:46:01.085581+0000","last_became_active":"2023-10-18T10:45:50.584872+0000","last_became_peered":"2023-10-18T10:45:50.584872+0000","last_unstale":"2023-10-18T10:46:01.085581+0000","last_undegraded":"2023-10-18T10:46:01.085581+0000","last_fullsized":"2023-10-18T10:46:01.085581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:07:41.932960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085445+0000","last_change":"2023-10-18T10:45:55.298601+0000","last_active":"2023-10-18T10:46:01.085445+0000","last_peered":"2023-10-18T10:46:01.085445+0000","last_clean":"2023-10-18T10:46:01.085445+0000","last_became_active":"2023-10-18T10:45:55.295209+0000","last_became_peered":"2023-10-18T10:45:55.295209+0000","last_unstale":"2023-10-18T10:46:01.085445+0000","last_undegraded":"2023-10-18T10:46:01.085445+0000","last_fullsized":"2023-10-18T10:46:01.085445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:55:07.806885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.769905+0000","last_change":"2023-10-18T10:45:59.787239+0000","last_active":"2023-10-18T10:46:00.769905+0000","last_peered":"2023-10-18T10:46:00.769905+0000","last_clean":"2023-10-18T10:46:00.769905+0000","last_became_active":"2023-10-18T10:45:59.786770+0000","last_became_peered":"2023-10-18T10:45:59.786770+0000","last_unstale":"2023-10-18T10:46:00.769905+0000","last_undegraded":"2023-10-18T10:46:00.769905+0000","last_fullsized":"2023-10-18T10:46:00.769905+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:09:26.681314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084957+0000","last_change":"2023-10-18T10:45:55.770838+0000","last_active":"2023-10-18T10:46:01.084957+0000","last_peered":"2023-10-18T10:46:01.084957+0000","last_clean":"2023-10-18T10:46:01.084957+0000","last_became_active":"2023-10-18T10:45:55.770308+0000","last_became_peered":"2023-10-18T10:45:55.770308+0000","last_unstale":"2023-10-18T10:46:01.084957+0000","last_undegraded":"2023-10-18T10:46:01.084957+0000","last_fullsized":"2023-10-18T10:46:01.084957+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:42:54.758946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085651+0000","last_change":"2023-10-18T10:45:50.584946+0000","last_active":"2023-10-18T10:46:01.085651+0000","last_peered":"2023-10-18T10:46:01.085651+0000","last_clean":"2023-10-18T10:46:01.085651+0000","last_became_active":"2023-10-18T10:45:50.584638+0000","last_became_peered":"2023-10-18T10:45:50.584638+0000","last_unstale":"2023-10-18T10:46:01.085651+0000","last_undegraded":"2023-10-18T10:46:01.085651+0000","last_fullsized":"2023-10-18T10:46:01.085651+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:40:09.742761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085828+0000","last_change":"2023-10-18T10:45:55.299483+0000","last_active":"2023-10-18T10:46:01.085828+0000","last_peered":"2023-10-18T10:46:01.085828+0000","last_clean":"2023-10-18T10:46:01.085828+0000","last_became_active":"2023-10-18T10:45:55.299195+0000","last_became_peered":"2023-10-18T10:45:55.299195+0000","last_unstale":"2023-10-18T10:46:01.085828+0000","last_undegraded":"2023-10-18T10:46:01.085828+0000","last_fullsized":"2023-10-18T10:46:01.085828+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:25:17.957984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085926+0000","last_change":"2023-10-18T10:45:59.801291+0000","last_active":"2023-10-18T10:46:01.085926+0000","last_peered":"2023-10-18T10:46:01.085926+0000","last_clean":"2023-10-18T10:46:01.085926+0000","last_became_active":"2023-10-18T10:45:59.800819+0000","last_became_peered":"2023-10-18T10:45:59.800819+0000","last_unstale":"2023-10-18T10:46:01.085926+0000","last_undegraded":"2023-10-18T10:46:01.085926+0000","last_fullsized":"2023-10-18T10:46:01.085926+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:06:30.412609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086211+0000","last_change":"2023-10-18T10:45:55.773950+0000","last_active":"2023-10-18T10:46:01.086211+0000","last_peered":"2023-10-18T10:46:01.086211+0000","last_clean":"2023-10-18T10:46:01.086211+0000","last_became_active":"2023-10-18T10:45:55.769581+0000","last_became_peered":"2023-10-18T10:45:55.769581+0000","last_unstale":"2023-10-18T10:46:01.086211+0000","last_undegraded":"2023-10-18T10:46:01.086211+0000","last_fullsized":"2023-10-18T10:46:01.086211+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:31:36.488372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768705+0000","last_change":"2023-10-18T10:45:55.297879+0000","last_active":"2023-10-18T10:46:00.768705+0000","last_peered":"2023-10-18T10:46:00.768705+0000","last_clean":"2023-10-18T10:46:00.768705+0000","last_became_active":"2023-10-18T10:45:55.297097+0000","last_became_peered":"2023-10-18T10:45:55.297097+0000","last_unstale":"2023-10-18T10:46:00.768705+0000","last_undegraded":"2023-10-18T10:46:00.768705+0000","last_fullsized":"2023-10-18T10:46:00.768705+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:27:53.149351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086078+0000","last_change":"2023-10-18T10:45:50.251505+0000","last_active":"2023-10-18T10:46:01.086078+0000","last_peered":"2023-10-18T10:46:01.086078+0000","last_clean":"2023-10-18T10:46:01.086078+0000","last_became_active":"2023-10-18T10:45:50.251111+0000","last_became_peered":"2023-10-18T10:45:50.251111+0000","last_unstale":"2023-10-18T10:46:01.086078+0000","last_undegraded":"2023-10-18T10:46:01.086078+0000","last_fullsized":"2023-10-18T10:46:01.086078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:04:22.817598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085546+0000","last_change":"2023-10-18T10:45:59.800297+0000","last_active":"2023-10-18T10:46:01.085546+0000","last_peered":"2023-10-18T10:46:01.085546+0000","last_clean":"2023-10-18T10:46:01.085546+0000","last_became_active":"2023-10-18T10:45:59.799434+0000","last_became_peered":"2023-10-18T10:45:59.799434+0000","last_unstale":"2023-10-18T10:46:01.085546+0000","last_undegraded":"2023-10-18T10:46:01.085546+0000","last_fullsized":"2023-10-18T10:46:01.085546+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:55:07.904865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086122+0000","last_change":"2023-10-18T10:45:55.774137+0000","last_active":"2023-10-18T10:46:01.086122+0000","last_peered":"2023-10-18T10:46:01.086122+0000","last_clean":"2023-10-18T10:46:01.086122+0000","last_became_active":"2023-10-18T10:45:55.773832+0000","last_became_peered":"2023-10-18T10:45:55.773832+0000","last_unstale":"2023-10-18T10:46:01.086122+0000","last_undegraded":"2023-10-18T10:46:01.086122+0000","last_fullsized":"2023-10-18T10:46:01.086122+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:31:21.466919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084731+0000","last_change":"2023-10-18T10:45:55.294225+0000","last_active":"2023-10-18T10:46:01.084731+0000","last_peered":"2023-10-18T10:46:01.084731+0000","last_clean":"2023-10-18T10:46:01.084731+0000","last_became_active":"2023-10-18T10:45:55.292869+0000","last_became_peered":"2023-10-18T10:45:55.292869+0000","last_unstale":"2023-10-18T10:46:01.084731+0000","last_undegraded":"2023-10-18T10:46:01.084731+0000","last_fullsized":"2023-10-18T10:46:01.084731+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:18:31.743377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.766604+0000","last_change":"2023-10-18T10:45:50.259019+0000","last_active":"2023-10-18T10:46:00.766604+0000","last_peered":"2023-10-18T10:46:00.766604+0000","last_clean":"2023-10-18T10:46:00.766604+0000","last_became_active":"2023-10-18T10:45:50.258607+0000","last_became_peered":"2023-10-18T10:45:50.258607+0000","last_unstale":"2023-10-18T10:46:00.766604+0000","last_undegraded":"2023-10-18T10:46:00.766604+0000","last_fullsized":"2023-10-18T10:46:00.766604+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:14:34.875058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.767128+0000","last_change":"2023-10-18T10:45:59.803747+0000","last_active":"2023-10-18T10:46:00.767128+0000","last_peered":"2023-10-18T10:46:00.767128+0000","last_clean":"2023-10-18T10:46:00.767128+0000","last_became_active":"2023-10-18T10:45:59.795250+0000","last_became_peered":"2023-10-18T10:45:59.795250+0000","last_unstale":"2023-10-18T10:46:00.767128+0000","last_undegraded":"2023-10-18T10:46:00.767128+0000","last_fullsized":"2023-10-18T10:46:00.767128+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:25.942126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086446+0000","last_change":"2023-10-18T10:45:55.763100+0000","last_active":"2023-10-18T10:46:01.086446+0000","last_peered":"2023-10-18T10:46:01.086446+0000","last_clean":"2023-10-18T10:46:01.086446+0000","last_became_active":"2023-10-18T10:45:55.762545+0000","last_became_peered":"2023-10-18T10:45:55.762545+0000","last_unstale":"2023-10-18T10:46:01.086446+0000","last_undegraded":"2023-10-18T10:46:01.086446+0000","last_fullsized":"2023-10-18T10:46:01.086446+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:33:00.921856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085280+0000","last_change":"2023-10-18T10:45:55.305049+0000","last_active":"2023-10-18T10:46:01.085280+0000","last_peered":"2023-10-18T10:46:01.085280+0000","last_clean":"2023-10-18T10:46:01.085280+0000","last_became_active":"2023-10-18T10:45:55.304746+0000","last_became_peered":"2023-10-18T10:45:55.304746+0000","last_unstale":"2023-10-18T10:46:01.085280+0000","last_undegraded":"2023-10-18T10:46:01.085280+0000","last_fullsized":"2023-10-18T10:46:01.085280+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:37:42.159482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084766+0000","last_change":"2023-10-18T10:45:50.251375+0000","last_active":"2023-10-18T10:46:01.084766+0000","last_peered":"2023-10-18T10:46:01.084766+0000","last_clean":"2023-10-18T10:46:01.084766+0000","last_became_active":"2023-10-18T10:45:50.251055+0000","last_became_peered":"2023-10-18T10:45:50.251055+0000","last_unstale":"2023-10-18T10:46:01.084766+0000","last_undegraded":"2023-10-18T10:46:01.084766+0000","last_fullsized":"2023-10-18T10:46:01.084766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:49:40.643495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085615+0000","last_change":"2023-10-18T10:45:59.791676+0000","last_active":"2023-10-18T10:46:01.085615+0000","last_peered":"2023-10-18T10:46:01.085615+0000","last_clean":"2023-10-18T10:46:01.085615+0000","last_became_active":"2023-10-18T10:45:59.791341+0000","last_became_peered":"2023-10-18T10:45:59.791341+0000","last_unstale":"2023-10-18T10:46:01.085615+0000","last_undegraded":"2023-10-18T10:46:01.085615+0000","last_fullsized":"2023-10-18T10:46:01.085615+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:15:44.664969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.770245+0000","last_change":"2023-10-18T10:45:55.777240+0000","last_active":"2023-10-18T10:46:00.770245+0000","last_peered":"2023-10-18T10:46:00.770245+0000","last_clean":"2023-10-18T10:46:00.770245+0000","last_became_active":"2023-10-18T10:45:55.775286+0000","last_became_peered":"2023-10-18T10:45:55.775286+0000","last_unstale":"2023-10-18T10:46:00.770245+0000","last_undegraded":"2023-10-18T10:46:00.770245+0000","last_fullsized":"2023-10-18T10:46:00.770245+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:52.500160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902463+0000","last_change":"2023-10-18T10:45:55.296440+0000","last_active":"2023-10-18T10:46:00.902463+0000","last_peered":"2023-10-18T10:46:00.902463+0000","last_clean":"2023-10-18T10:46:00.902463+0000","last_became_active":"2023-10-18T10:45:55.296124+0000","last_became_peered":"2023-10-18T10:45:55.296124+0000","last_unstale":"2023-10-18T10:46:00.902463+0000","last_undegraded":"2023-10-18T10:46:00.902463+0000","last_fullsized":"2023-10-18T10:46:00.902463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:43:19.116789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085874+0000","last_change":"2023-10-18T10:45:50.251310+0000","last_active":"2023-10-18T10:46:01.085874+0000","last_peered":"2023-10-18T10:46:01.085874+0000","last_clean":"2023-10-18T10:46:01.085874+0000","last_became_active":"2023-10-18T10:45:50.250823+0000","last_became_peered":"2023-10-18T10:45:50.250823+0000","last_unstale":"2023-10-18T10:46:01.085874+0000","last_undegraded":"2023-10-18T10:46:01.085874+0000","last_fullsized":"2023-10-18T10:46:01.085874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:33:28.469766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":183,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084881+0000","last_change":"2023-10-18T10:45:44.563531+0000","last_active":"2023-10-18T10:46:01.084881+0000","last_peered":"2023-10-18T10:46:01.084881+0000","last_clean":"2023-10-18T10:46:01.084881+0000","last_became_active":"2023-10-18T10:45:44.562504+0000","last_became_peered":"2023-10-18T10:45:44.562504+0000","last_unstale":"2023-10-18T10:46:01.084881+0000","last_undegraded":"2023-10-18T10:46:01.084881+0000","last_fullsized":"2023-10-18T10:46:01.084881+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:43.181461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:43.181461+0000","last_clean_scrub_stamp":"2023-10-18T10:45:43.181461+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:37:25.232224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084962+0000","last_change":"2023-10-18T10:45:59.807765+0000","last_active":"2023-10-18T10:46:01.084962+0000","last_peered":"2023-10-18T10:46:01.084962+0000","last_clean":"2023-10-18T10:46:01.084962+0000","last_became_active":"2023-10-18T10:45:59.807369+0000","last_became_peered":"2023-10-18T10:45:59.807369+0000","last_unstale":"2023-10-18T10:46:01.084962+0000","last_undegraded":"2023-10-18T10:46:01.084962+0000","last_fullsized":"2023-10-18T10:46:01.084962+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:14:04.335139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086027+0000","last_change":"2023-10-18T10:45:55.763363+0000","last_active":"2023-10-18T10:46:01.086027+0000","last_peered":"2023-10-18T10:46:01.086027+0000","last_clean":"2023-10-18T10:46:01.086027+0000","last_became_active":"2023-10-18T10:45:55.763155+0000","last_became_peered":"2023-10-18T10:45:55.763155+0000","last_unstale":"2023-10-18T10:46:01.086027+0000","last_undegraded":"2023-10-18T10:46:01.086027+0000","last_fullsized":"2023-10-18T10:46:01.086027+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:15:41.179515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902288+0000","last_change":"2023-10-18T10:45:55.304658+0000","last_active":"2023-10-18T10:46:00.902288+0000","last_peered":"2023-10-18T10:46:00.902288+0000","last_clean":"2023-10-18T10:46:00.902288+0000","last_became_active":"2023-10-18T10:45:55.303900+0000","last_became_peered":"2023-10-18T10:45:55.303900+0000","last_unstale":"2023-10-18T10:46:00.902288+0000","last_undegraded":"2023-10-18T10:46:00.902288+0000","last_fullsized":"2023-10-18T10:46:00.902288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:34:05.567387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768906+0000","last_change":"2023-10-18T10:45:59.792438+0000","last_active":"2023-10-18T10:46:00.768906+0000","last_peered":"2023-10-18T10:46:00.768906+0000","last_clean":"2023-10-18T10:46:00.768906+0000","last_became_active":"2023-10-18T10:45:59.791975+0000","last_became_peered":"2023-10-18T10:45:59.791975+0000","last_unstale":"2023-10-18T10:46:00.768906+0000","last_undegraded":"2023-10-18T10:46:00.768906+0000","last_fullsized":"2023-10-18T10:46:00.768906+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:19:39.692866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768991+0000","last_change":"2023-10-18T10:45:55.773190+0000","last_active":"2023-10-18T10:46:00.768991+0000","last_peered":"2023-10-18T10:46:00.768991+0000","last_clean":"2023-10-18T10:46:00.768991+0000","last_became_active":"2023-10-18T10:45:55.769619+0000","last_became_peered":"2023-10-18T10:45:55.769619+0000","last_unstale":"2023-10-18T10:46:00.768991+0000","last_undegraded":"2023-10-18T10:46:00.768991+0000","last_fullsized":"2023-10-18T10:46:00.768991+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:12:47.136700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.769371+0000","last_change":"2023-10-18T10:45:55.299320+0000","last_active":"2023-10-18T10:46:00.769371+0000","last_peered":"2023-10-18T10:46:00.769371+0000","last_clean":"2023-10-18T10:46:00.769371+0000","last_became_active":"2023-10-18T10:45:55.299008+0000","last_became_peered":"2023-10-18T10:45:55.299008+0000","last_unstale":"2023-10-18T10:46:00.769371+0000","last_undegraded":"2023-10-18T10:46:00.769371+0000","last_fullsized":"2023-10-18T10:46:00.769371+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:51.786437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085465+0000","last_change":"2023-10-18T10:45:59.801383+0000","last_active":"2023-10-18T10:46:01.085465+0000","last_peered":"2023-10-18T10:46:01.085465+0000","last_clean":"2023-10-18T10:46:01.085465+0000","last_became_active":"2023-10-18T10:45:59.801052+0000","last_became_peered":"2023-10-18T10:45:59.801052+0000","last_unstale":"2023-10-18T10:46:01.085465+0000","last_undegraded":"2023-10-18T10:46:01.085465+0000","last_fullsized":"2023-10-18T10:46:01.085465+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:57:31.735612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768713+0000","last_change":"2023-10-18T10:45:55.762399+0000","last_active":"2023-10-18T10:46:00.768713+0000","last_peered":"2023-10-18T10:46:00.768713+0000","last_clean":"2023-10-18T10:46:00.768713+0000","last_became_active":"2023-10-18T10:45:55.762074+0000","last_became_peered":"2023-10-18T10:45:55.762074+0000","last_unstale":"2023-10-18T10:46:00.768713+0000","last_undegraded":"2023-10-18T10:46:00.768713+0000","last_fullsized":"2023-10-18T10:46:00.768713+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:44:41.551817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.770066+0000","last_change":"2023-10-18T10:45:55.301204+0000","last_active":"2023-10-18T10:46:00.770066+0000","last_peered":"2023-10-18T10:46:00.770066+0000","last_clean":"2023-10-18T10:46:00.770066+0000","last_became_active":"2023-10-18T10:45:55.300895+0000","last_became_peered":"2023-10-18T10:45:55.300895+0000","last_unstale":"2023-10-18T10:46:00.770066+0000","last_undegraded":"2023-10-18T10:46:00.770066+0000","last_fullsized":"2023-10-18T10:46:00.770066+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:33:30.055966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084972+0000","last_change":"2023-10-18T10:45:59.794810+0000","last_active":"2023-10-18T10:46:01.084972+0000","last_peered":"2023-10-18T10:46:01.084972+0000","last_clean":"2023-10-18T10:46:01.084972+0000","last_became_active":"2023-10-18T10:45:59.794392+0000","last_became_peered":"2023-10-18T10:45:59.794392+0000","last_unstale":"2023-10-18T10:46:01.084972+0000","last_undegraded":"2023-10-18T10:46:01.084972+0000","last_fullsized":"2023-10-18T10:46:01.084972+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:05:09.364081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768556+0000","last_change":"2023-10-18T10:45:55.775591+0000","last_active":"2023-10-18T10:46:00.768556+0000","last_peered":"2023-10-18T10:46:00.768556+0000","last_clean":"2023-10-18T10:46:00.768556+0000","last_became_active":"2023-10-18T10:45:55.770394+0000","last_became_peered":"2023-10-18T10:45:55.770394+0000","last_unstale":"2023-10-18T10:46:00.768556+0000","last_undegraded":"2023-10-18T10:46:00.768556+0000","last_fullsized":"2023-10-18T10:46:00.768556+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:03:08.191047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902332+0000","last_change":"2023-10-18T10:45:55.299012+0000","last_active":"2023-10-18T10:46:00.902332+0000","last_peered":"2023-10-18T10:46:00.902332+0000","last_clean":"2023-10-18T10:46:00.902332+0000","last_became_active":"2023-10-18T10:45:55.296941+0000","last_became_peered":"2023-10-18T10:45:55.296941+0000","last_unstale":"2023-10-18T10:46:00.902332+0000","last_undegraded":"2023-10-18T10:46:00.902332+0000","last_fullsized":"2023-10-18T10:46:00.902332+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:07:46.963272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.766976+0000","last_change":"2023-10-18T10:45:59.797124+0000","last_active":"2023-10-18T10:46:00.766976+0000","last_peered":"2023-10-18T10:46:00.766976+0000","last_clean":"2023-10-18T10:46:00.766976+0000","last_became_active":"2023-10-18T10:45:59.796721+0000","last_became_peered":"2023-10-18T10:45:59.796721+0000","last_unstale":"2023-10-18T10:46:00.766976+0000","last_undegraded":"2023-10-18T10:46:00.766976+0000","last_fullsized":"2023-10-18T10:46:00.766976+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:11:32.963010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085998+0000","last_change":"2023-10-18T10:45:55.767007+0000","last_active":"2023-10-18T10:46:01.085998+0000","last_peered":"2023-10-18T10:46:01.085998+0000","last_clean":"2023-10-18T10:46:01.085998+0000","last_became_active":"2023-10-18T10:45:55.766727+0000","last_became_peered":"2023-10-18T10:45:55.766727+0000","last_unstale":"2023-10-18T10:46:01.085998+0000","last_undegraded":"2023-10-18T10:46:01.085998+0000","last_fullsized":"2023-10-18T10:46:01.085998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:51:37.372456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085606+0000","last_change":"2023-10-18T10:45:55.293762+0000","last_active":"2023-10-18T10:46:01.085606+0000","last_peered":"2023-10-18T10:46:01.085606+0000","last_clean":"2023-10-18T10:46:01.085606+0000","last_became_active":"2023-10-18T10:45:55.293349+0000","last_became_peered":"2023-10-18T10:45:55.293349+0000","last_unstale":"2023-10-18T10:46:01.085606+0000","last_undegraded":"2023-10-18T10:46:01.085606+0000","last_fullsized":"2023-10-18T10:46:01.085606+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:31:20.459601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.766635+0000","last_change":"2023-10-18T10:45:59.790292+0000","last_active":"2023-10-18T10:46:00.766635+0000","last_peered":"2023-10-18T10:46:00.766635+0000","last_clean":"2023-10-18T10:46:00.766635+0000","last_became_active":"2023-10-18T10:45:59.789901+0000","last_became_peered":"2023-10-18T10:45:59.789901+0000","last_unstale":"2023-10-18T10:46:00.766635+0000","last_undegraded":"2023-10-18T10:46:00.766635+0000","last_fullsized":"2023-10-18T10:46:00.766635+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:26:52.984157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.769853+0000","last_change":"2023-10-18T10:45:55.768064+0000","last_active":"2023-10-18T10:46:00.769853+0000","last_peered":"2023-10-18T10:46:00.769853+0000","last_clean":"2023-10-18T10:46:00.769853+0000","last_became_active":"2023-10-18T10:45:55.767773+0000","last_became_peered":"2023-10-18T10:45:55.767773+0000","last_unstale":"2023-10-18T10:46:00.769853+0000","last_undegraded":"2023-10-18T10:46:00.769853+0000","last_fullsized":"2023-10-18T10:46:00.769853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:30:00.014556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085414+0000","last_change":"2023-10-18T10:45:55.303880+0000","last_active":"2023-10-18T10:46:01.085414+0000","last_peered":"2023-10-18T10:46:01.085414+0000","last_clean":"2023-10-18T10:46:01.085414+0000","last_became_active":"2023-10-18T10:45:55.303577+0000","last_became_peered":"2023-10-18T10:45:55.303577+0000","last_unstale":"2023-10-18T10:46:01.085414+0000","last_undegraded":"2023-10-18T10:46:01.085414+0000","last_fullsized":"2023-10-18T10:46:01.085414+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:02:55.960777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.766839+0000","last_change":"2023-10-18T10:45:59.789006+0000","last_active":"2023-10-18T10:46:00.766839+0000","last_peered":"2023-10-18T10:46:00.766839+0000","last_clean":"2023-10-18T10:46:00.766839+0000","last_became_active":"2023-10-18T10:45:59.788544+0000","last_became_peered":"2023-10-18T10:45:59.788544+0000","last_unstale":"2023-10-18T10:46:00.766839+0000","last_undegraded":"2023-10-18T10:46:00.766839+0000","last_fullsized":"2023-10-18T10:46:00.766839+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:32:31.878208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.769128+0000","last_change":"2023-10-18T10:45:55.775207+0000","last_active":"2023-10-18T10:46:00.769128+0000","last_peered":"2023-10-18T10:46:00.769128+0000","last_clean":"2023-10-18T10:46:00.769128+0000","last_became_active":"2023-10-18T10:45:55.774909+0000","last_became_peered":"2023-10-18T10:45:55.774909+0000","last_unstale":"2023-10-18T10:46:00.769128+0000","last_undegraded":"2023-10-18T10:46:00.769128+0000","last_fullsized":"2023-10-18T10:46:00.769128+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:07:30.694604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085978+0000","last_change":"2023-10-18T10:45:55.302100+0000","last_active":"2023-10-18T10:46:01.085978+0000","last_peered":"2023-10-18T10:46:01.085978+0000","last_clean":"2023-10-18T10:46:01.085978+0000","last_became_active":"2023-10-18T10:45:55.301652+0000","last_became_peered":"2023-10-18T10:45:55.301652+0000","last_unstale":"2023-10-18T10:46:01.085978+0000","last_undegraded":"2023-10-18T10:46:01.085978+0000","last_fullsized":"2023-10-18T10:46:01.085978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:12:22.646771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.766536+0000","last_change":"2023-10-18T10:45:59.794997+0000","last_active":"2023-10-18T10:46:00.766536+0000","last_peered":"2023-10-18T10:46:00.766536+0000","last_clean":"2023-10-18T10:46:00.766536+0000","last_became_active":"2023-10-18T10:45:59.794587+0000","last_became_peered":"2023-10-18T10:45:59.794587+0000","last_unstale":"2023-10-18T10:46:00.766536+0000","last_undegraded":"2023-10-18T10:46:00.766536+0000","last_fullsized":"2023-10-18T10:46:00.766536+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:22:23.799739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084896+0000","last_change":"2023-10-18T10:45:55.772538+0000","last_active":"2023-10-18T10:46:01.084896+0000","last_peered":"2023-10-18T10:46:01.084896+0000","last_clean":"2023-10-18T10:46:01.084896+0000","last_became_active":"2023-10-18T10:45:55.772207+0000","last_became_peered":"2023-10-18T10:45:55.772207+0000","last_unstale":"2023-10-18T10:46:01.084896+0000","last_undegraded":"2023-10-18T10:46:01.084896+0000","last_fullsized":"2023-10-18T10:46:01.084896+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:55:21.950816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086120+0000","last_change":"2023-10-18T10:45:55.305858+0000","last_active":"2023-10-18T10:46:01.086120+0000","last_peered":"2023-10-18T10:46:01.086120+0000","last_clean":"2023-10-18T10:46:01.086120+0000","last_became_active":"2023-10-18T10:45:55.305591+0000","last_became_peered":"2023-10-18T10:45:55.305591+0000","last_unstale":"2023-10-18T10:46:01.086120+0000","last_undegraded":"2023-10-18T10:46:01.086120+0000","last_fullsized":"2023-10-18T10:46:01.086120+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:43:19.116789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084960+0000","last_change":"2023-10-18T10:45:59.799125+0000","last_active":"2023-10-18T10:46:01.084960+0000","last_peered":"2023-10-18T10:46:01.084960+0000","last_clean":"2023-10-18T10:46:01.084960+0000","last_became_active":"2023-10-18T10:45:59.798734+0000","last_became_peered":"2023-10-18T10:45:59.798734+0000","last_unstale":"2023-10-18T10:46:01.084960+0000","last_undegraded":"2023-10-18T10:46:01.084960+0000","last_fullsized":"2023-10-18T10:46:01.084960+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:56:46.630947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085751+0000","last_change":"2023-10-18T10:45:59.794340+0000","last_active":"2023-10-18T10:46:01.085751+0000","last_peered":"2023-10-18T10:46:01.085751+0000","last_clean":"2023-10-18T10:46:01.085751+0000","last_became_active":"2023-10-18T10:45:59.793965+0000","last_became_peered":"2023-10-18T10:45:59.793965+0000","last_unstale":"2023-10-18T10:46:01.085751+0000","last_undegraded":"2023-10-18T10:46:01.085751+0000","last_fullsized":"2023-10-18T10:46:01.085751+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:29.702107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.767968+0000","last_change":"2023-10-18T10:45:55.292258+0000","last_active":"2023-10-18T10:46:00.767968+0000","last_peered":"2023-10-18T10:46:00.767968+0000","last_clean":"2023-10-18T10:46:00.767968+0000","last_became_active":"2023-10-18T10:45:55.291007+0000","last_became_peered":"2023-10-18T10:45:55.291007+0000","last_unstale":"2023-10-18T10:46:00.767968+0000","last_undegraded":"2023-10-18T10:46:00.767968+0000","last_fullsized":"2023-10-18T10:46:00.767968+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:06:37.792273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902848+0000","last_change":"2023-10-18T10:45:55.765261+0000","last_active":"2023-10-18T10:46:00.902848+0000","last_peered":"2023-10-18T10:46:00.902848+0000","last_clean":"2023-10-18T10:46:00.902848+0000","last_became_active":"2023-10-18T10:45:55.765025+0000","last_became_peered":"2023-10-18T10:45:55.765025+0000","last_unstale":"2023-10-18T10:46:00.902848+0000","last_undegraded":"2023-10-18T10:46:00.902848+0000","last_fullsized":"2023-10-18T10:46:00.902848+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:49:06.870660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902759+0000","last_change":"2023-10-18T10:45:59.795015+0000","last_active":"2023-10-18T10:46:00.902759+0000","last_peered":"2023-10-18T10:46:00.902759+0000","last_clean":"2023-10-18T10:46:00.902759+0000","last_became_active":"2023-10-18T10:45:59.794646+0000","last_became_peered":"2023-10-18T10:45:59.794646+0000","last_unstale":"2023-10-18T10:46:00.902759+0000","last_undegraded":"2023-10-18T10:46:00.902759+0000","last_fullsized":"2023-10-18T10:46:00.902759+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:08:21.298342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085153+0000","last_change":"2023-10-18T10:45:55.305369+0000","last_active":"2023-10-18T10:46:01.085153+0000","last_peered":"2023-10-18T10:46:01.085153+0000","last_clean":"2023-10-18T10:46:01.085153+0000","last_became_active":"2023-10-18T10:45:55.304821+0000","last_became_peered":"2023-10-18T10:45:55.304821+0000","last_unstale":"2023-10-18T10:46:01.085153+0000","last_undegraded":"2023-10-18T10:46:01.085153+0000","last_fullsized":"2023-10-18T10:46:01.085153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:32:59.914538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086514+0000","last_change":"2023-10-18T10:45:55.763301+0000","last_active":"2023-10-18T10:46:01.086514+0000","last_peered":"2023-10-18T10:46:01.086514+0000","last_clean":"2023-10-18T10:46:01.086514+0000","last_became_active":"2023-10-18T10:45:55.762910+0000","last_became_peered":"2023-10-18T10:45:55.762910+0000","last_unstale":"2023-10-18T10:46:01.086514+0000","last_undegraded":"2023-10-18T10:46:01.086514+0000","last_fullsized":"2023-10-18T10:46:01.086514+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:06:27.237867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085615+0000","last_change":"2023-10-18T10:45:55.302395+0000","last_active":"2023-10-18T10:46:01.085615+0000","last_peered":"2023-10-18T10:46:01.085615+0000","last_clean":"2023-10-18T10:46:01.085615+0000","last_became_active":"2023-10-18T10:45:55.302090+0000","last_became_peered":"2023-10-18T10:45:55.302090+0000","last_unstale":"2023-10-18T10:46:01.085615+0000","last_undegraded":"2023-10-18T10:46:01.085615+0000","last_fullsized":"2023-10-18T10:46:01.085615+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:25:17.957984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085682+0000","last_change":"2023-10-18T10:45:55.776022+0000","last_active":"2023-10-18T10:46:01.085682+0000","last_peered":"2023-10-18T10:46:01.085682+0000","last_clean":"2023-10-18T10:46:01.085682+0000","last_became_active":"2023-10-18T10:45:55.775711+0000","last_became_peered":"2023-10-18T10:45:55.775711+0000","last_unstale":"2023-10-18T10:46:01.085682+0000","last_undegraded":"2023-10-18T10:46:01.085682+0000","last_fullsized":"2023-10-18T10:46:01.085682+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:31:36.488372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902232+0000","last_change":"2023-10-18T10:45:59.798164+0000","last_active":"2023-10-18T10:46:00.902232+0000","last_peered":"2023-10-18T10:46:00.902232+0000","last_clean":"2023-10-18T10:46:00.902232+0000","last_became_active":"2023-10-18T10:45:59.797723+0000","last_became_peered":"2023-10-18T10:45:59.797723+0000","last_unstale":"2023-10-18T10:46:00.902232+0000","last_undegraded":"2023-10-18T10:46:00.902232+0000","last_fullsized":"2023-10-18T10:46:00.902232+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:03:00.453549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085393+0000","last_change":"2023-10-18T10:45:55.306329+0000","last_active":"2023-10-18T10:46:01.085393+0000","last_peered":"2023-10-18T10:46:01.085393+0000","last_clean":"2023-10-18T10:46:01.085393+0000","last_became_active":"2023-10-18T10:45:55.306033+0000","last_became_peered":"2023-10-18T10:45:55.306033+0000","last_unstale":"2023-10-18T10:46:01.085393+0000","last_undegraded":"2023-10-18T10:46:01.085393+0000","last_fullsized":"2023-10-18T10:46:01.085393+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:54.257483+0000","last_clean_scrub_stamp":"2023-10-18T10:45:54.257483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:25:36.844395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902485+0000","last_change":"2023-10-18T10:45:55.768454+0000","last_active":"2023-10-18T10:46:00.902485+0000","last_peered":"2023-10-18T10:46:00.902485+0000","last_clean":"2023-10-18T10:46:00.902485+0000","last_became_active":"2023-10-18T10:45:55.767547+0000","last_became_peered":"2023-10-18T10:45:55.767547+0000","last_unstale":"2023-10-18T10:46:00.902485+0000","last_undegraded":"2023-10-18T10:46:00.902485+0000","last_fullsized":"2023-10-18T10:46:00.902485+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:19:20.955967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085524+0000","last_change":"2023-10-18T10:45:59.790629+0000","last_active":"2023-10-18T10:46:01.085524+0000","last_peered":"2023-10-18T10:46:01.085524+0000","last_clean":"2023-10-18T10:46:01.085524+0000","last_became_active":"2023-10-18T10:45:59.790163+0000","last_became_peered":"2023-10-18T10:45:59.790163+0000","last_unstale":"2023-10-18T10:46:01.085524+0000","last_undegraded":"2023-10-18T10:46:01.085524+0000","last_fullsized":"2023-10-18T10:46:01.085524+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:58.750256+0000","last_clean_scrub_stamp":"2023-10-18T10:45:58.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:10:56.439036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.767840+0000","last_change":"2023-10-18T10:45:55.759248+0000","last_active":"2023-10-18T10:46:00.767840+0000","last_peered":"2023-10-18T10:46:00.767840+0000","last_clean":"2023-10-18T10:46:00.767840+0000","last_became_active":"2023-10-18T10:45:55.758926+0000","last_became_peered":"2023-10-18T10:45:55.758926+0000","last_unstale":"2023-10-18T10:46:00.767840+0000","last_undegraded":"2023-10-18T10:46:00.767840+0000","last_fullsized":"2023-10-18T10:46:00.767840+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:37:00.951309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.901903+0000","last_change":"2023-10-18T10:45:55.777594+0000","last_active":"2023-10-18T10:46:00.901903+0000","last_peered":"2023-10-18T10:46:00.901903+0000","last_clean":"2023-10-18T10:46:00.901903+0000","last_became_active":"2023-10-18T10:45:55.777126+0000","last_became_peered":"2023-10-18T10:45:55.777126+0000","last_unstale":"2023-10-18T10:46:00.901903+0000","last_undegraded":"2023-10-18T10:46:00.901903+0000","last_fullsized":"2023-10-18T10:46:00.901903+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:33:49.751994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768469+0000","last_change":"2023-10-18T10:45:55.770662+0000","last_active":"2023-10-18T10:46:00.768469+0000","last_peered":"2023-10-18T10:46:00.768469+0000","last_clean":"2023-10-18T10:46:00.768469+0000","last_became_active":"2023-10-18T10:45:55.769816+0000","last_became_peered":"2023-10-18T10:45:55.769816+0000","last_unstale":"2023-10-18T10:46:00.768469+0000","last_undegraded":"2023-10-18T10:46:00.768469+0000","last_fullsized":"2023-10-18T10:46:00.768469+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:40:54.772659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.902183+0000","last_change":"2023-10-18T10:45:55.764434+0000","last_active":"2023-10-18T10:46:00.902183+0000","last_peered":"2023-10-18T10:46:00.902183+0000","last_clean":"2023-10-18T10:46:00.902183+0000","last_became_active":"2023-10-18T10:45:55.764164+0000","last_became_peered":"2023-10-18T10:45:55.764164+0000","last_unstale":"2023-10-18T10:46:00.902183+0000","last_undegraded":"2023-10-18T10:46:00.902183+0000","last_fullsized":"2023-10-18T10:46:00.902183+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:25:37.851713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085549+0000","last_change":"2023-10-18T10:45:55.773793+0000","last_active":"2023-10-18T10:46:01.085549+0000","last_peered":"2023-10-18T10:46:01.085549+0000","last_clean":"2023-10-18T10:46:01.085549+0000","last_became_active":"2023-10-18T10:45:55.769530+0000","last_became_peered":"2023-10-18T10:45:55.769530+0000","last_unstale":"2023-10-18T10:46:01.085549+0000","last_undegraded":"2023-10-18T10:46:01.085549+0000","last_fullsized":"2023-10-18T10:46:01.085549+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:57:28.250158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085697+0000","last_change":"2023-10-18T10:45:55.780769+0000","last_active":"2023-10-18T10:46:01.085697+0000","last_peered":"2023-10-18T10:46:01.085697+0000","last_clean":"2023-10-18T10:46:01.085697+0000","last_became_active":"2023-10-18T10:45:55.780434+0000","last_became_peered":"2023-10-18T10:45:55.780434+0000","last_unstale":"2023-10-18T10:46:01.085697+0000","last_undegraded":"2023-10-18T10:46:01.085697+0000","last_fullsized":"2023-10-18T10:46:01.085697+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:37:43.166800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085181+0000","last_change":"2023-10-18T10:45:55.770696+0000","last_active":"2023-10-18T10:46:01.085181+0000","last_peered":"2023-10-18T10:46:01.085181+0000","last_clean":"2023-10-18T10:46:01.085181+0000","last_became_active":"2023-10-18T10:45:55.769723+0000","last_became_peered":"2023-10-18T10:45:55.769723+0000","last_unstale":"2023-10-18T10:46:01.085181+0000","last_undegraded":"2023-10-18T10:46:01.085181+0000","last_fullsized":"2023-10-18T10:46:01.085181+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:59:15.197613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085919+0000","last_change":"2023-10-18T10:45:55.766491+0000","last_active":"2023-10-18T10:46:01.085919+0000","last_peered":"2023-10-18T10:46:01.085919+0000","last_clean":"2023-10-18T10:46:01.085919+0000","last_became_active":"2023-10-18T10:45:55.766215+0000","last_became_peered":"2023-10-18T10:45:55.766215+0000","last_unstale":"2023-10-18T10:46:01.085919+0000","last_undegraded":"2023-10-18T10:46:01.085919+0000","last_fullsized":"2023-10-18T10:46:01.085919+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:10:52.953582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085494+0000","last_change":"2023-10-18T10:45:55.767712+0000","last_active":"2023-10-18T10:46:01.085494+0000","last_peered":"2023-10-18T10:46:01.085494+0000","last_clean":"2023-10-18T10:46:01.085494+0000","last_became_active":"2023-10-18T10:45:55.765004+0000","last_became_peered":"2023-10-18T10:45:55.765004+0000","last_unstale":"2023-10-18T10:46:01.085494+0000","last_undegraded":"2023-10-18T10:46:01.085494+0000","last_fullsized":"2023-10-18T10:46:01.085494+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:34:06.574705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.901824+0000","last_change":"2023-10-18T10:45:55.757810+0000","last_active":"2023-10-18T10:46:00.901824+0000","last_peered":"2023-10-18T10:46:00.901824+0000","last_clean":"2023-10-18T10:46:00.901824+0000","last_became_active":"2023-10-18T10:45:55.756492+0000","last_became_peered":"2023-10-18T10:45:55.756492+0000","last_unstale":"2023-10-18T10:46:00.901824+0000","last_undegraded":"2023-10-18T10:46:00.901824+0000","last_fullsized":"2023-10-18T10:46:00.901824+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:56:43.145493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085491+0000","last_change":"2023-10-18T10:45:55.767803+0000","last_active":"2023-10-18T10:46:01.085491+0000","last_peered":"2023-10-18T10:46:01.085491+0000","last_clean":"2023-10-18T10:46:01.085491+0000","last_became_active":"2023-10-18T10:45:55.767525+0000","last_became_peered":"2023-10-18T10:45:55.767525+0000","last_unstale":"2023-10-18T10:46:01.085491+0000","last_undegraded":"2023-10-18T10:46:01.085491+0000","last_fullsized":"2023-10-18T10:46:01.085491+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:26.216653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085266+0000","last_change":"2023-10-18T10:45:55.769366+0000","last_active":"2023-10-18T10:46:01.085266+0000","last_peered":"2023-10-18T10:46:01.085266+0000","last_clean":"2023-10-18T10:46:01.085266+0000","last_became_active":"2023-10-18T10:45:55.769083+0000","last_became_peered":"2023-10-18T10:45:55.769083+0000","last_unstale":"2023-10-18T10:46:01.085266+0000","last_undegraded":"2023-10-18T10:46:01.085266+0000","last_fullsized":"2023-10-18T10:46:01.085266+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:05:57.473930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768585+0000","last_change":"2023-10-18T10:45:55.770711+0000","last_active":"2023-10-18T10:46:00.768585+0000","last_peered":"2023-10-18T10:46:00.768585+0000","last_clean":"2023-10-18T10:46:00.768585+0000","last_became_active":"2023-10-18T10:45:55.770147+0000","last_became_peered":"2023-10-18T10:45:55.770147+0000","last_unstale":"2023-10-18T10:46:00.768585+0000","last_undegraded":"2023-10-18T10:46:00.768585+0000","last_fullsized":"2023-10-18T10:46:00.768585+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:20:18.473140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.767102+0000","last_change":"2023-10-18T10:45:55.758011+0000","last_active":"2023-10-18T10:46:00.767102+0000","last_peered":"2023-10-18T10:46:00.767102+0000","last_clean":"2023-10-18T10:46:00.767102+0000","last_became_active":"2023-10-18T10:45:55.757668+0000","last_became_peered":"2023-10-18T10:45:55.757668+0000","last_unstale":"2023-10-18T10:46:00.767102+0000","last_undegraded":"2023-10-18T10:46:00.767102+0000","last_fullsized":"2023-10-18T10:46:00.767102+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:41:48.881411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.769446+0000","last_change":"2023-10-18T10:45:55.763084+0000","last_active":"2023-10-18T10:46:00.769446+0000","last_peered":"2023-10-18T10:46:00.769446+0000","last_clean":"2023-10-18T10:46:00.769446+0000","last_became_active":"2023-10-18T10:45:55.762776+0000","last_became_peered":"2023-10-18T10:45:55.762776+0000","last_unstale":"2023-10-18T10:46:00.769446+0000","last_undegraded":"2023-10-18T10:46:00.769446+0000","last_fullsized":"2023-10-18T10:46:00.769446+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:28:02.667670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085807+0000","last_change":"2023-10-18T10:45:55.762254+0000","last_active":"2023-10-18T10:46:01.085807+0000","last_peered":"2023-10-18T10:46:01.085807+0000","last_clean":"2023-10-18T10:46:01.085807+0000","last_became_active":"2023-10-18T10:45:55.761870+0000","last_became_peered":"2023-10-18T10:45:55.761870+0000","last_unstale":"2023-10-18T10:46:01.085807+0000","last_undegraded":"2023-10-18T10:46:01.085807+0000","last_fullsized":"2023-10-18T10:46:01.085807+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:43:01.654769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768118+0000","last_change":"2023-10-18T10:45:55.765743+0000","last_active":"2023-10-18T10:46:00.768118+0000","last_peered":"2023-10-18T10:46:00.768118+0000","last_clean":"2023-10-18T10:46:00.768118+0000","last_became_active":"2023-10-18T10:45:55.765470+0000","last_became_peered":"2023-10-18T10:45:55.765470+0000","last_unstale":"2023-10-18T10:46:00.768118+0000","last_undegraded":"2023-10-18T10:46:00.768118+0000","last_fullsized":"2023-10-18T10:46:00.768118+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:19:20.955967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085124+0000","last_change":"2023-10-18T10:45:55.781643+0000","last_active":"2023-10-18T10:46:01.085124+0000","last_peered":"2023-10-18T10:46:01.085124+0000","last_clean":"2023-10-18T10:46:01.085124+0000","last_became_active":"2023-10-18T10:45:55.781252+0000","last_became_peered":"2023-10-18T10:45:55.781252+0000","last_unstale":"2023-10-18T10:46:01.085124+0000","last_undegraded":"2023-10-18T10:46:01.085124+0000","last_fullsized":"2023-10-18T10:46:01.085124+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:40:53.192771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085958+0000","last_change":"2023-10-18T10:45:55.774683+0000","last_active":"2023-10-18T10:46:01.085958+0000","last_peered":"2023-10-18T10:46:01.085958+0000","last_clean":"2023-10-18T10:46:01.085958+0000","last_became_active":"2023-10-18T10:45:55.773664+0000","last_became_peered":"2023-10-18T10:45:55.773664+0000","last_unstale":"2023-10-18T10:46:01.085958+0000","last_undegraded":"2023-10-18T10:46:01.085958+0000","last_fullsized":"2023-10-18T10:46:01.085958+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:06:26.927155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.766819+0000","last_change":"2023-10-18T10:45:55.755077+0000","last_active":"2023-10-18T10:46:00.766819+0000","last_peered":"2023-10-18T10:46:00.766819+0000","last_clean":"2023-10-18T10:46:00.766819+0000","last_became_active":"2023-10-18T10:45:55.754723+0000","last_became_peered":"2023-10-18T10:45:55.754723+0000","last_unstale":"2023-10-18T10:46:00.766819+0000","last_undegraded":"2023-10-18T10:46:00.766819+0000","last_fullsized":"2023-10-18T10:46:00.766819+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:17:50.477786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.767162+0000","last_change":"2023-10-18T10:45:55.771484+0000","last_active":"2023-10-18T10:46:00.767162+0000","last_peered":"2023-10-18T10:46:00.767162+0000","last_clean":"2023-10-18T10:46:00.767162+0000","last_became_active":"2023-10-18T10:45:55.771108+0000","last_became_peered":"2023-10-18T10:45:55.771108+0000","last_unstale":"2023-10-18T10:46:00.767162+0000","last_undegraded":"2023-10-18T10:46:00.767162+0000","last_fullsized":"2023-10-18T10:46:00.767162+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:18.131261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768708+0000","last_change":"2023-10-18T10:45:55.778320+0000","last_active":"2023-10-18T10:46:00.768708+0000","last_peered":"2023-10-18T10:46:00.768708+0000","last_clean":"2023-10-18T10:46:00.768708+0000","last_became_active":"2023-10-18T10:45:55.777767+0000","last_became_peered":"2023-10-18T10:45:55.777767+0000","last_unstale":"2023-10-18T10:46:00.768708+0000","last_undegraded":"2023-10-18T10:46:00.768708+0000","last_fullsized":"2023-10-18T10:46:00.768708+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:05:56.688630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086126+0000","last_change":"2023-10-18T10:45:55.771747+0000","last_active":"2023-10-18T10:46:01.086126+0000","last_peered":"2023-10-18T10:46:01.086126+0000","last_clean":"2023-10-18T10:46:01.086126+0000","last_became_active":"2023-10-18T10:45:55.771297+0000","last_became_peered":"2023-10-18T10:45:55.771297+0000","last_unstale":"2023-10-18T10:46:01.086126+0000","last_undegraded":"2023-10-18T10:46:01.086126+0000","last_fullsized":"2023-10-18T10:46:01.086126+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:51:37.372456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085611+0000","last_change":"2023-10-18T10:45:55.756546+0000","last_active":"2023-10-18T10:46:01.085611+0000","last_peered":"2023-10-18T10:46:01.085611+0000","last_clean":"2023-10-18T10:46:01.085611+0000","last_became_active":"2023-10-18T10:45:55.756277+0000","last_became_peered":"2023-10-18T10:45:55.756277+0000","last_unstale":"2023-10-18T10:46:01.085611+0000","last_undegraded":"2023-10-18T10:46:01.085611+0000","last_fullsized":"2023-10-18T10:46:01.085611+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:33:34.507396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.084785+0000","last_change":"2023-10-18T10:45:55.778442+0000","last_active":"2023-10-18T10:46:01.084785+0000","last_peered":"2023-10-18T10:46:01.084785+0000","last_clean":"2023-10-18T10:46:01.084785+0000","last_became_active":"2023-10-18T10:45:55.778198+0000","last_became_peered":"2023-10-18T10:45:55.778198+0000","last_unstale":"2023-10-18T10:46:01.084785+0000","last_undegraded":"2023-10-18T10:46:01.084785+0000","last_fullsized":"2023-10-18T10:46:01.084785+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:28:05.383698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085042+0000","last_change":"2023-10-18T10:45:55.779285+0000","last_active":"2023-10-18T10:46:01.085042+0000","last_peered":"2023-10-18T10:46:01.085042+0000","last_clean":"2023-10-18T10:46:01.085042+0000","last_became_active":"2023-10-18T10:45:55.778975+0000","last_became_peered":"2023-10-18T10:45:55.778975+0000","last_unstale":"2023-10-18T10:46:01.085042+0000","last_undegraded":"2023-10-18T10:46:01.085042+0000","last_fullsized":"2023-10-18T10:46:01.085042+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:33:49.751994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768170+0000","last_change":"2023-10-18T10:45:55.770892+0000","last_active":"2023-10-18T10:46:00.768170+0000","last_peered":"2023-10-18T10:46:00.768170+0000","last_clean":"2023-10-18T10:46:00.768170+0000","last_became_active":"2023-10-18T10:45:55.770181+0000","last_became_peered":"2023-10-18T10:45:55.770181+0000","last_unstale":"2023-10-18T10:46:00.768170+0000","last_undegraded":"2023-10-18T10:46:00.768170+0000","last_fullsized":"2023-10-18T10:46:00.768170+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:06:44.169699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.086251+0000","last_change":"2023-10-18T10:45:55.759114+0000","last_active":"2023-10-18T10:46:01.086251+0000","last_peered":"2023-10-18T10:46:01.086251+0000","last_clean":"2023-10-18T10:46:01.086251+0000","last_became_active":"2023-10-18T10:45:55.758825+0000","last_became_peered":"2023-10-18T10:45:55.758825+0000","last_unstale":"2023-10-18T10:46:01.086251+0000","last_undegraded":"2023-10-18T10:46:01.086251+0000","last_fullsized":"2023-10-18T10:46:01.086251+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:14:00.849685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.770488+0000","last_change":"2023-10-18T10:45:55.763407+0000","last_active":"2023-10-18T10:46:00.770488+0000","last_peered":"2023-10-18T10:46:00.770488+0000","last_clean":"2023-10-18T10:46:00.770488+0000","last_became_active":"2023-10-18T10:45:55.763083+0000","last_became_peered":"2023-10-18T10:45:55.763083+0000","last_unstale":"2023-10-18T10:46:00.770488+0000","last_undegraded":"2023-10-18T10:46:00.770488+0000","last_fullsized":"2023-10-18T10:46:00.770488+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:30.982488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.768596+0000","last_change":"2023-10-18T10:45:55.770469+0000","last_active":"2023-10-18T10:46:00.768596+0000","last_peered":"2023-10-18T10:46:00.768596+0000","last_clean":"2023-10-18T10:46:00.768596+0000","last_became_active":"2023-10-18T10:45:55.769998+0000","last_became_peered":"2023-10-18T10:45:55.769998+0000","last_unstale":"2023-10-18T10:46:00.768596+0000","last_undegraded":"2023-10-18T10:46:00.768596+0000","last_fullsized":"2023-10-18T10:46:00.768596+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:00:14.580995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:01.085929+0000","last_change":"2023-10-18T10:45:55.781899+0000","last_active":"2023-10-18T10:46:01.085929+0000","last_peered":"2023-10-18T10:46:01.085929+0000","last_clean":"2023-10-18T10:46:01.085929+0000","last_became_active":"2023-10-18T10:45:55.781624+0000","last_became_peered":"2023-10-18T10:45:55.781624+0000","last_unstale":"2023-10-18T10:46:01.085929+0000","last_undegraded":"2023-10-18T10:46:01.085929+0000","last_fullsized":"2023-10-18T10:46:01.085929+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:43:01.654769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-10-18T10:46:00.767009+0000","last_change":"2023-10-18T10:45:55.757924+0000","last_active":"2023-10-18T10:46:00.767009+0000","last_peered":"2023-10-18T10:46:00.767009+0000","last_clean":"2023-10-18T10:46:00.767009+0000","last_became_active":"2023-10-18T10:45:55.757644+0000","last_became_peered":"2023-10-18T10:45:55.757644+0000","last_unstale":"2023-10-18T10:46:00.767009+0000","last_undegraded":"2023-10-18T10:46:00.767009+0000","last_fullsized":"2023-10-18T10:46:00.767009+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:55.264801+0000","last_clean_scrub_stamp":"2023-10-18T10:45:55.264801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:08:54.769872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343932,"statfs":{"total":96636764160,"available":96608186368,"internally_reserved":0,"allocated":913408,"data_stored":714326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9219999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":120553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":311296,"data_stored":123798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":119991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":888832,"data_stored":710851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.0410000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.3150000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.9700000000000006}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.625}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.6790000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.6559999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.782}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:46:19.945 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-18T10:46:19.945 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-18T10:46:19.946 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-18T10:46:19.946 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T10:46:20.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.114+0000 7fa728dd2640 1 Processor -- start 2023-10-18T10:46:20.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.114+0000 7fa728dd2640 1 -- start start 2023-10-18T10:46:20.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.118+0000 7fa728dd2640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa7240d2040 0x7fa7241670b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:20.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.118+0000 7fa728dd2640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7240b4a60 con 0x7fa7240d0050 2023-10-18T10:46:20.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.118+0000 7fa728dd2640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa72415c940 con 0x7fa72415c560 2023-10-18T10:46:20.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.118+0000 7fa728dd2640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7240d0550 con 0x7fa7240d2040 2023-10-18T10:46:20.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.118+0000 7fa723fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa7240d2040 0x7fa7241670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:20.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.118+0000 7fa723fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa7240d2040 0x7fa7241670b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45550/0 (socket says 172.21.15.89:45550) 2023-10-18T10:46:20.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.118+0000 7fa723fff640 1 -- 172.21.15.89:0/180159244 learned_addr learned my addr 172.21.15.89:0/180159244 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:20.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.118+0000 7fa723fff640 1 -- 172.21.15.89:0/180159244 >> v1:172.21.15.154:6789/0 conn(0x7fa72415c560 legacy=0x7fa7240b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:20.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.118+0000 7fa723fff640 1 -- 172.21.15.89:0/180159244 >> v1:172.21.15.89:6789/0 conn(0x7fa7240d0050 legacy=0x7fa7240d1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:20.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.118+0000 7fa723fff640 1 -- 172.21.15.89:0/180159244 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7240d0830 con 0x7fa7240d2040 2023-10-18T10:46:20.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.118+0000 7fa723fff640 1 --2- 172.21.15.89:0/180159244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7240d2040 0x7fa7241670b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa71c009e50 tx=0x7fa71c01ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53217e902b73a17e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:20.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.122+0000 7fa7227fc640 1 -- 172.21.15.89:0/180159244 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c0a57d0 con 0x7fa7240d2040 2023-10-18T10:46:20.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.122+0000 7fa7227fc640 1 -- 172.21.15.89:0/180159244 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa71c0a5930 con 0x7fa7240d2040 2023-10-18T10:46:20.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.122+0000 7fa7227fc640 1 -- 172.21.15.89:0/180159244 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c0a5cc0 con 0x7fa7240d2040 2023-10-18T10:46:20.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.122+0000 7fa728dd2640 1 -- 172.21.15.89:0/180159244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7240d2040 msgr2=0x7fa7241670b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:20.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.122+0000 7fa728dd2640 1 --2- 172.21.15.89:0/180159244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7240d2040 0x7fa7241670b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa71c009e50 tx=0x7fa71c01ca10 comp rx=0 tx=0).stop 2023-10-18T10:46:20.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.122+0000 7fa728dd2640 1 -- 172.21.15.89:0/180159244 shutdown_connections 2023-10-18T10:46:20.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.122+0000 7fa728dd2640 1 --2- 172.21.15.89:0/180159244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7240d2040 0x7fa7241670b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:20.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.122+0000 7fa728dd2640 1 -- 172.21.15.89:0/180159244 >> 172.21.15.89:0/180159244 conn(0x7fa7240d2510 msgr2=0x7fa7240d2920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:20.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.122+0000 7fa728dd2640 1 -- 172.21.15.89:0/180159244 shutdown_connections 2023-10-18T10:46:20.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.122+0000 7fa728dd2640 1 -- 172.21.15.89:0/180159244 wait complete. 2023-10-18T10:46:20.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.122+0000 7fa728dd2640 1 Processor -- start 2023-10-18T10:46:20.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.122+0000 7fa728dd2640 1 -- start start 2023-10-18T10:46:20.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.122+0000 7fa728dd2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7240d0050 0x7fa724153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:20.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.122+0000 7fa728dd2640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7240d2040 0x7fa72414fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:20.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.122+0000 7fa728dd2640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa72415c560 0x7fa724150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:20.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.122+0000 7fa728dd2640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa724153070 con 0x7fa7240d2040 2023-10-18T10:46:20.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.122+0000 7fa728dd2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7240b7780 con 0x7fa7240d0050 2023-10-18T10:46:20.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.122+0000 7fa728dd2640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7240b7a20 con 0x7fa72415c560 2023-10-18T10:46:20.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.126+0000 7fa722ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7240d2040 0x7fa72414fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:20.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.126+0000 7fa722ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7240d2040 0x7fa72414fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45562/0 (socket says 172.21.15.89:45562) 2023-10-18T10:46:20.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.126+0000 7fa722ffd640 1 -- 172.21.15.89:0/3980206948 learned_addr learned my addr 172.21.15.89:0/3980206948 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:20.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.126+0000 7fa7237fe640 1 --2- 172.21.15.89:0/3980206948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7240d0050 0x7fa724153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:20.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.126+0000 7fa723fff640 1 --2- 172.21.15.89:0/3980206948 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa72415c560 0x7fa724150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:20.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.126+0000 7fa722ffd640 1 -- 172.21.15.89:0/3980206948 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa72415c560 msgr2=0x7fa724150010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:20.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.126+0000 7fa722ffd640 1 --2- 172.21.15.89:0/3980206948 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa72415c560 0x7fa724150010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:20.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.126+0000 7fa722ffd640 1 -- 172.21.15.89:0/3980206948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7240d0050 msgr2=0x7fa724153500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:20.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.126+0000 7fa722ffd640 1 --2- 172.21.15.89:0/3980206948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7240d0050 0x7fa724153500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:20.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.126+0000 7fa722ffd640 1 -- 172.21.15.89:0/3980206948 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa71c009b30 con 0x7fa7240d2040 2023-10-18T10:46:20.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.126+0000 7fa722ffd640 1 --2- 172.21.15.89:0/3980206948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7240d2040 0x7fa72414fad0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa71800c4d0 tx=0x7fa71800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:20.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.126+0000 7fa70bfff640 1 -- 172.21.15.89:0/3980206948 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7180052d0 con 0x7fa7240d2040 2023-10-18T10:46:20.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.126+0000 7fa70bfff640 1 -- 172.21.15.89:0/3980206948 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa718005430 con 0x7fa7240d2040 2023-10-18T10:46:20.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.126+0000 7fa728dd2640 1 -- 172.21.15.89:0/3980206948 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa724152a20 con 0x7fa7240d2040 2023-10-18T10:46:20.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.126+0000 7fa70bfff640 1 -- 172.21.15.89:0/3980206948 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa718012830 con 0x7fa7240d2040 2023-10-18T10:46:20.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.126+0000 7fa728dd2640 1 -- 172.21.15.89:0/3980206948 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa72414b010 con 0x7fa7240d2040 2023-10-18T10:46:20.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.126+0000 7fa728dd2640 1 -- 172.21.15.89:0/3980206948 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa724073b50 con 0x7fa7240d2040 2023-10-18T10:46:20.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.126+0000 7fa70bfff640 1 -- 172.21.15.89:0/3980206948 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fa718012990 con 0x7fa7240d2040 2023-10-18T10:46:20.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.130+0000 7fa70bfff640 1 --2- 172.21.15.89:0/3980206948 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa710065680 0x7fa710067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:20.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.130+0000 7fa70bfff640 1 -- 172.21.15.89:0/3980206948 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa718082bf0 con 0x7fa7240d2040 2023-10-18T10:46:20.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.130+0000 7fa7237fe640 1 --2- 172.21.15.89:0/3980206948 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa710065680 0x7fa710067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:20.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.130+0000 7fa7237fe640 1 --2- 172.21.15.89:0/3980206948 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa710065680 0x7fa710067b40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa714000fc0 tx=0x7fa714009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:20.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.134+0000 7fa70bfff640 1 -- 172.21.15.89:0/3980206948 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fa718050350 con 0x7fa7240d2040 2023-10-18T10:46:20.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.314+0000 7fa728dd2640 1 -- 172.21.15.89:0/3980206948 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa7241531b0 con 0x7fa7240d2040 2023-10-18T10:46:20.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.314+0000 7fa70bfff640 1 -- 172.21.15.89:0/3980206948 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fa718053970 con 0x7fa7240d2040 2023-10-18T10:46:20.315 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:20.315 INFO:teuthology.orchestra.run.smithi089.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-18T10:46:20.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.314+0000 7fa728dd2640 1 -- 172.21.15.89:0/3980206948 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa710065680 msgr2=0x7fa710067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:20.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.314+0000 7fa728dd2640 1 --2- 172.21.15.89:0/3980206948 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa710065680 0x7fa710067b40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa714000fc0 tx=0x7fa714009040 comp rx=0 tx=0).stop 2023-10-18T10:46:20.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.314+0000 7fa728dd2640 1 -- 172.21.15.89:0/3980206948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7240d2040 msgr2=0x7fa72414fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:20.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.314+0000 7fa728dd2640 1 --2- 172.21.15.89:0/3980206948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7240d2040 0x7fa72414fad0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa71800c4d0 tx=0x7fa71800c990 comp rx=0 tx=0).stop 2023-10-18T10:46:20.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.314+0000 7fa728dd2640 1 -- 172.21.15.89:0/3980206948 shutdown_connections 2023-10-18T10:46:20.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.314+0000 7fa728dd2640 1 --2- 172.21.15.89:0/3980206948 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa710065680 0x7fa710067b40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:20.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.314+0000 7fa728dd2640 1 --2- 172.21.15.89:0/3980206948 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa72415c560 0x7fa724150010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:20.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.314+0000 7fa728dd2640 1 --2- 172.21.15.89:0/3980206948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7240d2040 0x7fa72414fad0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:20.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.314+0000 7fa728dd2640 1 --2- 172.21.15.89:0/3980206948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7240d0050 0x7fa724153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:20.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.314+0000 7fa728dd2640 1 -- 172.21.15.89:0/3980206948 >> 172.21.15.89:0/3980206948 conn(0x7fa7240d2510 msgr2=0x7fa7240b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:20.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.314+0000 7fa728dd2640 1 -- 172.21.15.89:0/3980206948 shutdown_connections 2023-10-18T10:46:20.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.314+0000 7fa728dd2640 1 -- 172.21.15.89:0/3980206948 wait complete. 2023-10-18T10:46:20.335 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-18T10:46:20.336 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:46:20.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.470+0000 7f8f4112d640 1 Processor -- start 2023-10-18T10:46:20.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.470+0000 7f8f4112d640 1 -- start start 2023-10-18T10:46:20.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.474+0000 7f8f4112d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8f3c15c550 0x7f8f3c0d0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:20.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.474+0000 7f8f4112d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8f3c0d1a40 0x7f8f3c0cee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:20.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.474+0000 7f8f4112d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f3c0cf370 0x7f8f3c069900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:20.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.474+0000 7f8f4112d640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f3c0cf780 con 0x7f8f3c15c550 2023-10-18T10:46:20.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.474+0000 7f8f4112d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f3c0d0e40 con 0x7f8f3c0d1a40 2023-10-18T10:46:20.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.474+0000 7f8f4112d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c0d1120 con 0x7f8f3c0cf370 2023-10-18T10:46:20.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.474+0000 7f8f3bfff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8f3c15c550 0x7f8f3c0d0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:20.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.474+0000 7f8f3bfff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8f3c15c550 0x7f8f3c0d0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45574/0 (socket says 172.21.15.89:45574) 2023-10-18T10:46:20.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.474+0000 7f8f3bfff640 1 -- 172.21.15.89:0/3687109004 learned_addr learned my addr 172.21.15.89:0/3687109004 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:20.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.474+0000 7f8f3b7fe640 1 --2- 172.21.15.89:0/3687109004 >> v2:172.21.15.154:3300/0 conn(0x7f8f3c0d1a40 0x7f8f3c0cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:20.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.474+0000 7f8f3bfff640 1 -- 172.21.15.89:0/3687109004 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f3c0cf370 msgr2=0x7f8f3c069900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:20.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.474+0000 7f8f4092c640 1 --2- 172.21.15.89:0/3687109004 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f3c0cf370 0x7f8f3c069900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:20.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.474+0000 7f8f3bfff640 1 --2- 172.21.15.89:0/3687109004 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f3c0cf370 0x7f8f3c069900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:20.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.474+0000 7f8f3bfff640 1 -- 172.21.15.89:0/3687109004 >> v2:172.21.15.154:3300/0 conn(0x7f8f3c0d1a40 msgr2=0x7f8f3c0cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:20.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.474+0000 7f8f3bfff640 1 --2- 172.21.15.89:0/3687109004 >> v2:172.21.15.154:3300/0 conn(0x7f8f3c0d1a40 0x7f8f3c0cee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:20.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.474+0000 7f8f3bfff640 1 -- 172.21.15.89:0/3687109004 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f3c0d1400 con 0x7f8f3c15c550 2023-10-18T10:46:20.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.474+0000 7f8f3b7fe640 1 --2- 172.21.15.89:0/3687109004 >> v2:172.21.15.154:3300/0 conn(0x7f8f3c0d1a40 0x7f8f3c0cee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:20.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.474+0000 7f8f4092c640 1 --2- 172.21.15.89:0/3687109004 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f3c0cf370 0x7f8f3c069900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:20.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.474+0000 7f8f3bfff640 1 --2- 172.21.15.89:0/3687109004 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f3c15c550 0x7f8f3c0d0540 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0b5350 tx=0x7f8f2c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d4dba313ee473f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:20.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.474+0000 7f8f3affd640 1 -- 172.21.15.89:0/3687109004 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c0aa610 con 0x7f8f3c15c550 2023-10-18T10:46:20.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.478+0000 7f8f3affd640 1 -- 172.21.15.89:0/3687109004 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f2c0a6040 con 0x7f8f3c15c550 2023-10-18T10:46:20.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.478+0000 7f8f3affd640 1 -- 172.21.15.89:0/3687109004 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c0aa990 con 0x7f8f3c15c550 2023-10-18T10:46:20.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.478+0000 7f8f4112d640 1 -- 172.21.15.89:0/3687109004 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f3c15c550 msgr2=0x7f8f3c0d0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:20.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.478+0000 7f8f4112d640 1 --2- 172.21.15.89:0/3687109004 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f3c15c550 0x7f8f3c0d0540 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0b5350 tx=0x7f8f2c01c4d0 comp rx=0 tx=0).stop 2023-10-18T10:46:20.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.478+0000 7f8f4112d640 1 -- 172.21.15.89:0/3687109004 shutdown_connections 2023-10-18T10:46:20.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.478+0000 7f8f4112d640 1 --2- 172.21.15.89:0/3687109004 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f3c0cf370 0x7f8f3c069900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:20.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.478+0000 7f8f4112d640 1 --2- 172.21.15.89:0/3687109004 >> v2:172.21.15.154:3300/0 conn(0x7f8f3c0d1a40 0x7f8f3c0cee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:20.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.478+0000 7f8f4112d640 1 --2- 172.21.15.89:0/3687109004 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f3c15c550 0x7f8f3c0d0540 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:20.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.478+0000 7f8f4112d640 1 -- 172.21.15.89:0/3687109004 >> 172.21.15.89:0/3687109004 conn(0x7f8f3c070320 msgr2=0x7f8f3c070730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:20.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.478+0000 7f8f4112d640 1 -- 172.21.15.89:0/3687109004 shutdown_connections 2023-10-18T10:46:20.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.478+0000 7f8f4112d640 1 -- 172.21.15.89:0/3687109004 wait complete. 2023-10-18T10:46:20.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.478+0000 7f8f4112d640 1 Processor -- start 2023-10-18T10:46:20.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.478+0000 7f8f4112d640 1 -- start start 2023-10-18T10:46:20.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.478+0000 7f8f4112d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f3c0cf370 0x7f8f3c155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:20.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.478+0000 7f8f4112d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f3c0d1a40 0x7f8f3c158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:20.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.478+0000 7f8f4112d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f3c15c550 0x7f8f3c15dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:20.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.478+0000 7f8f4112d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c1590b0 con 0x7f8f3c15c550 2023-10-18T10:46:20.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.478+0000 7f8f4112d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c159390 con 0x7f8f3c0d1a40 2023-10-18T10:46:20.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.478+0000 7f8f4112d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c159670 con 0x7f8f3c0cf370 2023-10-18T10:46:20.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.478+0000 7f8f4092c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f3c15c550 0x7f8f3c15dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:20.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.478+0000 7f8f4092c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f3c15c550 0x7f8f3c15dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45578/0 (socket says 172.21.15.89:45578) 2023-10-18T10:46:20.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.478+0000 7f8f4092c640 1 -- 172.21.15.89:0/1436263028 learned_addr learned my addr 172.21.15.89:0/1436263028 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:20.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.478+0000 7f8f3bfff640 1 --2- 172.21.15.89:0/1436263028 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f3c0cf370 0x7f8f3c155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:20.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.478+0000 7f8f3b7fe640 1 --2- 172.21.15.89:0/1436263028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f3c0d1a40 0x7f8f3c158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:20.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.482+0000 7f8f4092c640 1 -- 172.21.15.89:0/1436263028 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f3c0cf370 msgr2=0x7f8f3c155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:20.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.482+0000 7f8f4092c640 1 --2- 172.21.15.89:0/1436263028 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f3c0cf370 0x7f8f3c155d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:20.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.482+0000 7f8f4092c640 1 -- 172.21.15.89:0/1436263028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f3c0d1a40 msgr2=0x7f8f3c158660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:20.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.482+0000 7f8f4092c640 1 --2- 172.21.15.89:0/1436263028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f3c0d1a40 0x7f8f3c158660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:20.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.482+0000 7f8f4092c640 1 -- 172.21.15.89:0/1436263028 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f2c0094d0 con 0x7f8f3c15c550 2023-10-18T10:46:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.482+0000 7f8f4092c640 1 --2- 172.21.15.89:0/1436263028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f3c15c550 0x7f8f3c15dd40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400cd10 tx=0x7f8f3400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.482+0000 7f8f38ff9640 1 -- 172.21.15.89:0/1436263028 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3400f680 con 0x7f8f3c15c550 2023-10-18T10:46:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.482+0000 7f8f38ff9640 1 -- 172.21.15.89:0/1436263028 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3400f800 con 0x7f8f3c15c550 2023-10-18T10:46:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.482+0000 7f8f38ff9640 1 -- 172.21.15.89:0/1436263028 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3400fbb0 con 0x7f8f3c15c550 2023-10-18T10:46:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.482+0000 7f8f4112d640 1 -- 172.21.15.89:0/1436263028 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f3c15cab0 con 0x7f8f3c15c550 2023-10-18T10:46:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.482+0000 7f8f4112d640 1 -- 172.21.15.89:0/1436263028 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f3c0c9820 con 0x7f8f3c15c550 2023-10-18T10:46:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.482+0000 7f8f4112d640 1 -- 172.21.15.89:0/1436263028 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f3c073b50 con 0x7f8f3c15c550 2023-10-18T10:46:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.482+0000 7f8f38ff9640 1 -- 172.21.15.89:0/1436263028 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f8f3400b040 con 0x7f8f3c15c550 2023-10-18T10:46:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.482+0000 7f8f38ff9640 1 --2- 172.21.15.89:0/1436263028 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8f28065680 0x7f8f28067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:20.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.482+0000 7f8f38ff9640 1 -- 172.21.15.89:0/1436263028 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8f34083980 con 0x7f8f3c15c550 2023-10-18T10:46:20.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.482+0000 7f8f3bfff640 1 --2- 172.21.15.89:0/1436263028 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8f28065680 0x7f8f28067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:20.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.486+0000 7f8f3bfff640 1 --2- 172.21.15.89:0/1436263028 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8f28065680 0x7f8f28067b40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c0ae000 tx=0x7f8f2c0b5040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:20.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.486+0000 7f8f38ff9640 1 -- 172.21.15.89:0/1436263028 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f8f34051200 con 0x7f8f3c15c550 2023-10-18T10:46:20.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.638+0000 7f8f4112d640 1 -- 172.21.15.89:0/1436263028 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f3c15f440 con 0x7f8f3c15c550 2023-10-18T10:46:20.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.638+0000 7f8f38ff9640 1 -- 172.21.15.89:0/1436263028 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20096 (secure 0 0 0) 0x7f8f34054820 con 0x7f8f3c15c550 2023-10-18T10:46:20.640 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:20.641 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":27,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:46:00.166702+0000","last_up_change":"2023-10-18T10:45:42.164688+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:45:53.309122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-18T10:45:54.659763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:45:57.796514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6824","nonce":773078364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6827","nonce":773078364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6831","nonce":773078364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6829","nonce":773078364}]},"public_addr":"172.21.15.154:6824/773078364","cluster_addr":"172.21.15.154:6827/773078364","heartbeat_back_addr":"172.21.15.154:6831/773078364","heartbeat_front_addr":"172.21.15.154:6829/773078364","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:46:20.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.642+0000 7f8f4112d640 1 -- 172.21.15.89:0/1436263028 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8f28065680 msgr2=0x7f8f28067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:20.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.642+0000 7f8f4112d640 1 --2- 172.21.15.89:0/1436263028 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8f28065680 0x7f8f28067b40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c0ae000 tx=0x7f8f2c0b5040 comp rx=0 tx=0).stop 2023-10-18T10:46:20.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.642+0000 7f8f4112d640 1 -- 172.21.15.89:0/1436263028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f3c15c550 msgr2=0x7f8f3c15dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:20.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.642+0000 7f8f4112d640 1 --2- 172.21.15.89:0/1436263028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f3c15c550 0x7f8f3c15dd40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400cd10 tx=0x7f8f3400a650 comp rx=0 tx=0).stop 2023-10-18T10:46:20.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.642+0000 7f8f4112d640 1 -- 172.21.15.89:0/1436263028 shutdown_connections 2023-10-18T10:46:20.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.642+0000 7f8f4112d640 1 --2- 172.21.15.89:0/1436263028 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8f28065680 0x7f8f28067b40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:20.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.642+0000 7f8f4112d640 1 --2- 172.21.15.89:0/1436263028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f3c15c550 0x7f8f3c15dd40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:20.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.642+0000 7f8f4112d640 1 --2- 172.21.15.89:0/1436263028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f3c0d1a40 0x7f8f3c158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:20.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.642+0000 7f8f4112d640 1 --2- 172.21.15.89:0/1436263028 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f3c0cf370 0x7f8f3c155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:20.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.642+0000 7f8f4112d640 1 -- 172.21.15.89:0/1436263028 >> 172.21.15.89:0/1436263028 conn(0x7f8f3c070320 msgr2=0x7f8f3c0b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:20.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.642+0000 7f8f4112d640 1 -- 172.21.15.89:0/1436263028 shutdown_connections 2023-10-18T10:46:20.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.642+0000 7f8f4112d640 1 -- 172.21.15.89:0/1436263028 wait complete. 2023-10-18T10:46:20.662 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:45:43.114184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-18T10:45:48.730968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:45:53.309122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:45:54.659763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T10:45:57.796514+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T10:46:20.663 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:46:20.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.802+0000 7f54a625f640 1 Processor -- start 2023-10-18T10:46:20.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.802+0000 7f54a625f640 1 -- start start 2023-10-18T10:46:20.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.802+0000 7f54a625f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f54a015c940 0x7f54a00d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:20.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.802+0000 7f54a625f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f54a00d2f40 0x7f54a00d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:20.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.802+0000 7f54a625f640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54a00d10f0 con 0x7f54a00d0ca0 2023-10-18T10:46:20.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.802+0000 7f54a625f640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54a015da50 con 0x7f54a00d2f40 2023-10-18T10:46:20.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.802+0000 7f54a625f640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54a015ce40 con 0x7f54a015c940 2023-10-18T10:46:20.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.802+0000 7f5496ffd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f54a00d2f40 0x7f54a00d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:20.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.802+0000 7f5496ffd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f54a00d2f40 0x7f54a00d0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45584/0 (socket says 172.21.15.89:45584) 2023-10-18T10:46:20.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.802+0000 7f5496ffd640 1 -- 172.21.15.89:0/1865634989 learned_addr learned my addr 172.21.15.89:0/1865634989 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:20.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.802+0000 7f549f7fe640 1 --2- 172.21.15.89:0/1865634989 >> v2:172.21.15.154:3300/0 conn(0x7f54a015c940 0x7f54a00d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:20.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.802+0000 7f5496ffd640 1 -- 172.21.15.89:0/1865634989 >> v2:172.21.15.154:3300/0 conn(0x7f54a015c940 msgr2=0x7f54a00d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:20.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.802+0000 7f5496ffd640 1 --2- 172.21.15.89:0/1865634989 >> v2:172.21.15.154:3300/0 conn(0x7f54a015c940 0x7f54a00d2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:20.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.802+0000 7f5496ffd640 1 -- 172.21.15.89:0/1865634989 >> v1:172.21.15.154:6789/0 conn(0x7f54a00d0ca0 legacy=0x7f54a00cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:20.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.802+0000 7f5496ffd640 1 -- 172.21.15.89:0/1865634989 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a015d120 con 0x7f54a00d2f40 2023-10-18T10:46:20.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.806+0000 7f5496ffd640 1 --2- 172.21.15.89:0/1865634989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54a00d2f40 0x7f54a00d0760 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f54a00725b0 tx=0x7f548801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1330e01d8efd588 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:20.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.806+0000 7f549effd640 1 -- 172.21.15.89:0/1865634989 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54880ac640 con 0x7f54a00d2f40 2023-10-18T10:46:20.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.806+0000 7f549effd640 1 -- 172.21.15.89:0/1865634989 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54880ac7a0 con 0x7f54a00d2f40 2023-10-18T10:46:20.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.806+0000 7f549effd640 1 -- 172.21.15.89:0/1865634989 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54880acb30 con 0x7f54a00d2f40 2023-10-18T10:46:20.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.806+0000 7f54a625f640 1 -- 172.21.15.89:0/1865634989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54a00d2f40 msgr2=0x7f54a00d0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:20.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.806+0000 7f54a625f640 1 --2- 172.21.15.89:0/1865634989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54a00d2f40 0x7f54a00d0760 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f54a00725b0 tx=0x7f548801c4d0 comp rx=0 tx=0).stop 2023-10-18T10:46:20.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.806+0000 7f54a625f640 1 -- 172.21.15.89:0/1865634989 shutdown_connections 2023-10-18T10:46:20.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.806+0000 7f54a625f640 1 --2- 172.21.15.89:0/1865634989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54a00d2f40 0x7f54a00d0760 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:20.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.806+0000 7f54a625f640 1 --2- 172.21.15.89:0/1865634989 >> v2:172.21.15.154:3300/0 conn(0x7f54a015c940 0x7f54a00d2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:20.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.806+0000 7f54a625f640 1 -- 172.21.15.89:0/1865634989 >> 172.21.15.89:0/1865634989 conn(0x7f54a0089fd0 msgr2=0x7f54a009b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:20.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.806+0000 7f54a625f640 1 -- 172.21.15.89:0/1865634989 shutdown_connections 2023-10-18T10:46:20.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.806+0000 7f54a625f640 1 -- 172.21.15.89:0/1865634989 wait complete. 2023-10-18T10:46:20.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.810+0000 7f54a625f640 1 Processor -- start 2023-10-18T10:46:20.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.810+0000 7f54a625f640 1 -- start start 2023-10-18T10:46:20.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.810+0000 7f54a625f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f54a00d0ca0 0x7f54a014ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:20.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.810+0000 7f54a625f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54a0155730 0x7f54a0150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:20.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.810+0000 7f54a625f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54a01508d0 0x7f54a015aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:20.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.810+0000 7f54a625f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a0150d50 con 0x7f54a01508d0 2023-10-18T10:46:20.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.810+0000 7f54a625f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a0155f80 con 0x7f54a0155730 2023-10-18T10:46:20.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.810+0000 7f54a625f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54a014f280 con 0x7f54a00d0ca0 2023-10-18T10:46:20.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.810+0000 7f549ffff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54a01508d0 0x7f54a015aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:20.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.810+0000 7f549ffff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54a01508d0 0x7f54a015aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45594/0 (socket says 172.21.15.89:45594) 2023-10-18T10:46:20.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.810+0000 7f549ffff640 1 -- 172.21.15.89:0/3543377499 learned_addr learned my addr 172.21.15.89:0/3543377499 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:20.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.810+0000 7f5496ffd640 1 --2- 172.21.15.89:0/3543377499 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54a0155730 0x7f54a0150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:20.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.810+0000 7f549f7fe640 1 --2- 172.21.15.89:0/3543377499 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f54a00d0ca0 0x7f54a014ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:20.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.810+0000 7f549ffff640 1 -- 172.21.15.89:0/3543377499 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f54a00d0ca0 msgr2=0x7f54a014ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:20.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.810+0000 7f549ffff640 1 --2- 172.21.15.89:0/3543377499 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f54a00d0ca0 0x7f54a014ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:20.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.810+0000 7f549ffff640 1 -- 172.21.15.89:0/3543377499 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54a0155730 msgr2=0x7f54a0150390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:20.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.810+0000 7f549ffff640 1 --2- 172.21.15.89:0/3543377499 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54a0155730 0x7f54a0150390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:20.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.810+0000 7f549ffff640 1 -- 172.21.15.89:0/3543377499 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54880094d0 con 0x7f54a01508d0 2023-10-18T10:46:20.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.810+0000 7f549ffff640 1 --2- 172.21.15.89:0/3543377499 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54a01508d0 0x7f54a015aa80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f549800c7d0 tx=0x7f549800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:20.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.810+0000 7f549cff9640 1 -- 172.21.15.89:0/3543377499 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5498003b80 con 0x7f54a01508d0 2023-10-18T10:46:20.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.810+0000 7f549cff9640 1 -- 172.21.15.89:0/3543377499 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5498003ce0 con 0x7f54a01508d0 2023-10-18T10:46:20.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.810+0000 7f549cff9640 1 -- 172.21.15.89:0/3543377499 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549800aa90 con 0x7f54a01508d0 2023-10-18T10:46:20.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.810+0000 7f54a625f640 1 -- 172.21.15.89:0/3543377499 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54a014f610 con 0x7f54a01508d0 2023-10-18T10:46:20.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.810+0000 7f54a625f640 1 -- 172.21.15.89:0/3543377499 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54a014fb90 con 0x7f54a01508d0 2023-10-18T10:46:20.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.814+0000 7f54a625f640 1 -- 172.21.15.89:0/3543377499 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54a004dbe0 con 0x7f54a01508d0 2023-10-18T10:46:20.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.814+0000 7f549cff9640 1 -- 172.21.15.89:0/3543377499 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f549800abf0 con 0x7f54a01508d0 2023-10-18T10:46:20.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.814+0000 7f549cff9640 1 --2- 172.21.15.89:0/3543377499 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5480065680 0x7f5480067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:20.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.814+0000 7f549cff9640 1 -- 172.21.15.89:0/3543377499 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5498082d80 con 0x7f54a01508d0 2023-10-18T10:46:20.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.814+0000 7f549f7fe640 1 --2- 172.21.15.89:0/3543377499 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5480065680 0x7f5480067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:20.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.814+0000 7f549f7fe640 1 --2- 172.21.15.89:0/3543377499 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5480065680 0x7f5480067b40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f54a00c5f50 tx=0x7f5490008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:20.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.818+0000 7f549cff9640 1 -- 172.21.15.89:0/3543377499 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f54980504b0 con 0x7f54a01508d0 2023-10-18T10:46:20.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.974+0000 7f54a625f640 1 -- 172.21.15.89:0/3543377499 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f54a00d10f0 con 0x7f54a01508d0 2023-10-18T10:46:20.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.974+0000 7f549cff9640 1 -- 172.21.15.89:0/3543377499 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f54a00d10f0 con 0x7f54a01508d0 2023-10-18T10:46:20.975 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-18T10:46:20.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.974+0000 7f54a625f640 1 -- 172.21.15.89:0/3543377499 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5480065680 msgr2=0x7f5480067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:20.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.974+0000 7f54a625f640 1 --2- 172.21.15.89:0/3543377499 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5480065680 0x7f5480067b40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f54a00c5f50 tx=0x7f5490008040 comp rx=0 tx=0).stop 2023-10-18T10:46:20.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.974+0000 7f54a625f640 1 -- 172.21.15.89:0/3543377499 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54a01508d0 msgr2=0x7f54a015aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:20.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.974+0000 7f54a625f640 1 --2- 172.21.15.89:0/3543377499 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54a01508d0 0x7f54a015aa80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f549800c7d0 tx=0x7f549800cc90 comp rx=0 tx=0).stop 2023-10-18T10:46:20.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.974+0000 7f54a625f640 1 -- 172.21.15.89:0/3543377499 shutdown_connections 2023-10-18T10:46:20.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.974+0000 7f54a625f640 1 --2- 172.21.15.89:0/3543377499 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5480065680 0x7f5480067b40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:20.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.974+0000 7f54a625f640 1 --2- 172.21.15.89:0/3543377499 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54a01508d0 0x7f54a015aa80 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:20.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.974+0000 7f54a625f640 1 --2- 172.21.15.89:0/3543377499 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54a0155730 0x7f54a0150390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:20.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.974+0000 7f54a625f640 1 --2- 172.21.15.89:0/3543377499 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f54a00d0ca0 0x7f54a014ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:20.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.974+0000 7f54a625f640 1 -- 172.21.15.89:0/3543377499 >> 172.21.15.89:0/3543377499 conn(0x7f54a0089fd0 msgr2=0x7f54a00cf040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:20.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.974+0000 7f54a625f640 1 -- 172.21.15.89:0/3543377499 shutdown_connections 2023-10-18T10:46:20.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:20.974+0000 7f54a625f640 1 -- 172.21.15.89:0/3543377499 wait complete. 2023-10-18T10:46:20.997 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-18T10:46:21.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.134+0000 7f49d22bd640 1 Processor -- start 2023-10-18T10:46:21.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.134+0000 7f49d22bd640 1 -- start start 2023-10-18T10:46:21.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.138+0000 7f49d22bd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f49cc06a010 0x7f49cc0d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:21.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.138+0000 7f49d22bd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f49cc0b1950 0x7f49cc0b0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:21.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.138+0000 7f49d22bd640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49cc0b1430 con 0x7f49cc15ff30 2023-10-18T10:46:21.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.138+0000 7f49d22bd640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49cc0d3690 con 0x7f49cc0b1950 2023-10-18T10:46:21.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.138+0000 7f49d22bd640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49cc0d3080 con 0x7f49cc06a010 2023-10-18T10:46:21.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.138+0000 7f49cbfff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f49cc15ff30 0x7f49cc069900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35714/0 (socket says 172.21.15.89:35714) 2023-10-18T10:46:21.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.138+0000 7f49cbfff640 1 -- 172.21.15.89:0/3299646764 learned_addr learned my addr 172.21.15.89:0/3299646764 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:21.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.138+0000 7f49d085a640 1 --2- 172.21.15.89:0/3299646764 >> v2:172.21.15.89:3300/0 conn(0x7f49cc0b1950 0x7f49cc0b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:21.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.138+0000 7f49cb7fe640 1 --2- 172.21.15.89:0/3299646764 >> v2:172.21.15.154:3300/0 conn(0x7f49cc06a010 0x7f49cc0d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:21.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.138+0000 7f49d085a640 1 -- 172.21.15.89:0/3299646764 >> v2:172.21.15.154:3300/0 conn(0x7f49cc06a010 msgr2=0x7f49cc0d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:21.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.138+0000 7f49d085a640 1 --2- 172.21.15.89:0/3299646764 >> v2:172.21.15.154:3300/0 conn(0x7f49cc06a010 0x7f49cc0d0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:21.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.138+0000 7f49d085a640 1 -- 172.21.15.89:0/3299646764 >> v1:172.21.15.89:6789/0 conn(0x7f49cc15ff30 legacy=0x7f49cc069900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:21.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.138+0000 7f49d085a640 1 -- 172.21.15.89:0/3299646764 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49cc0d3320 con 0x7f49cc0b1950 2023-10-18T10:46:21.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.142+0000 7f49d085a640 1 --2- 172.21.15.89:0/3299646764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49cc0b1950 0x7f49cc0b0260 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f49cc09c100 tx=0x7f49c401c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9807c87dee062c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:21.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.142+0000 7f49caffd640 1 -- 172.21.15.89:0/3299646764 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c40a58d0 con 0x7f49cc0b1950 2023-10-18T10:46:21.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.142+0000 7f49caffd640 1 -- 172.21.15.89:0/3299646764 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c40a5a30 con 0x7f49cc0b1950 2023-10-18T10:46:21.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.142+0000 7f49caffd640 1 -- 172.21.15.89:0/3299646764 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c40a5dc0 con 0x7f49cc0b1950 2023-10-18T10:46:21.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.142+0000 7f49d22bd640 1 -- 172.21.15.89:0/3299646764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49cc0b1950 msgr2=0x7f49cc0b0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:21.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.142+0000 7f49d22bd640 1 --2- 172.21.15.89:0/3299646764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49cc0b1950 0x7f49cc0b0260 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f49cc09c100 tx=0x7f49c401c7f0 comp rx=0 tx=0).stop 2023-10-18T10:46:21.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.142+0000 7f49d22bd640 1 -- 172.21.15.89:0/3299646764 shutdown_connections 2023-10-18T10:46:21.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.142+0000 7f49d22bd640 1 --2- 172.21.15.89:0/3299646764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49cc0b1950 0x7f49cc0b0260 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:21.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.142+0000 7f49d22bd640 1 --2- 172.21.15.89:0/3299646764 >> v2:172.21.15.154:3300/0 conn(0x7f49cc06a010 0x7f49cc0d0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:21.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.142+0000 7f49d22bd640 1 -- 172.21.15.89:0/3299646764 >> 172.21.15.89:0/3299646764 conn(0x7f49cc09b9d0 msgr2=0x7f49cc0d26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:21.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.142+0000 7f49d22bd640 1 -- 172.21.15.89:0/3299646764 shutdown_connections 2023-10-18T10:46:21.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.142+0000 7f49d22bd640 1 -- 172.21.15.89:0/3299646764 wait complete. 2023-10-18T10:46:21.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.142+0000 7f49d22bd640 1 Processor -- start 2023-10-18T10:46:21.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.142+0000 7f49d22bd640 1 -- start start 2023-10-18T10:46:21.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.146+0000 7f49d22bd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49cc06a010 0x7f49cc0b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:21.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.146+0000 7f49d22bd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49cc0b1950 0x7f49cc0b7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:21.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.146+0000 7f49d22bd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49cc15ff30 0x7f49cc157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:21.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.146+0000 7f49d22bd640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc159700 con 0x7f49cc0b1950 2023-10-18T10:46:21.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.146+0000 7f49d22bd640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc159420 con 0x7f49cc06a010 2023-10-18T10:46:21.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.146+0000 7f49d22bd640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc0c7ef0 con 0x7f49cc15ff30 2023-10-18T10:46:21.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.146+0000 7f49cb7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49cc0b1950 0x7f49cc0b7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:21.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.146+0000 7f49cb7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49cc0b1950 0x7f49cc0b7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45624/0 (socket says 172.21.15.89:45624) 2023-10-18T10:46:21.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.146+0000 7f49cb7fe640 1 -- 172.21.15.89:0/2094232065 learned_addr learned my addr 172.21.15.89:0/2094232065 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:21.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.146+0000 7f49cb7fe640 1 -- 172.21.15.89:0/2094232065 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49cc15ff30 msgr2=0x7f49cc157580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:21.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.146+0000 7f49cb7fe640 1 --2- 172.21.15.89:0/2094232065 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49cc15ff30 0x7f49cc157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:21.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.146+0000 7f49cb7fe640 1 -- 172.21.15.89:0/2094232065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49cc06a010 msgr2=0x7f49cc0b79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:21.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.146+0000 7f49cb7fe640 1 --2- 172.21.15.89:0/2094232065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49cc06a010 0x7f49cc0b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:21.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.146+0000 7f49cb7fe640 1 -- 172.21.15.89:0/2094232065 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c4009a70 con 0x7f49cc0b1950 2023-10-18T10:46:21.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.146+0000 7f49cb7fe640 1 --2- 172.21.15.89:0/2094232065 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49cc0b1950 0x7f49cc0b7ee0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f49c000e930 tx=0x7f49c000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:21.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.146+0000 7f49c8ff9640 1 -- 172.21.15.89:0/2094232065 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c0009690 con 0x7f49cc0b1950 2023-10-18T10:46:21.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.146+0000 7f49c8ff9640 1 -- 172.21.15.89:0/2094232065 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c000ca00 con 0x7f49cc0b1950 2023-10-18T10:46:21.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.146+0000 7f49c8ff9640 1 -- 172.21.15.89:0/2094232065 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c000cd70 con 0x7f49cc0b1950 2023-10-18T10:46:21.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.146+0000 7f49d22bd640 1 -- 172.21.15.89:0/2094232065 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49cc0c81d0 con 0x7f49cc0b1950 2023-10-18T10:46:21.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.146+0000 7f49d22bd640 1 -- 172.21.15.89:0/2094232065 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49cc158370 con 0x7f49cc0b1950 2023-10-18T10:46:21.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.146+0000 7f49d22bd640 1 -- 172.21.15.89:0/2094232065 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49cc04dbe0 con 0x7f49cc0b1950 2023-10-18T10:46:21.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.146+0000 7f49c8ff9640 1 -- 172.21.15.89:0/2094232065 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f49c0013470 con 0x7f49cc0b1950 2023-10-18T10:46:21.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.150+0000 7f49c8ff9640 1 --2- 172.21.15.89:0/2094232065 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f49b8065680 0x7f49b8067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:21.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.150+0000 7f49c8ff9640 1 -- 172.21.15.89:0/2094232065 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f49c0083360 con 0x7f49cc0b1950 2023-10-18T10:46:21.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.150+0000 7f49cbfff640 1 --2- 172.21.15.89:0/2094232065 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f49b8065680 0x7f49b8067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:21.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.150+0000 7f49cbfff640 1 --2- 172.21.15.89:0/2094232065 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f49b8065680 0x7f49b8067b40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f49bc007fd0 tx=0x7f49bc0078c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:21.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.154+0000 7f49c8ff9640 1 -- 172.21.15.89:0/2094232065 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f49c0050be0 con 0x7f49cc0b1950 2023-10-18T10:46:21.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.302+0000 7f49d22bd640 1 -- 172.21.15.89:0/2094232065 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f49cc0b32c0 con 0x7f49cc0b1950 2023-10-18T10:46:21.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.302+0000 7f49c8ff9640 1 -- 172.21.15.89:0/2094232065 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7f49c0054200 con 0x7f49cc0b1950 2023-10-18T10:46:21.304 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-10-18T10:46:21.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.302+0000 7f49d22bd640 1 -- 172.21.15.89:0/2094232065 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f49b8065680 msgr2=0x7f49b8067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:21.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.302+0000 7f49d22bd640 1 --2- 172.21.15.89:0/2094232065 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f49b8065680 0x7f49b8067b40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f49bc007fd0 tx=0x7f49bc0078c0 comp rx=0 tx=0).stop 2023-10-18T10:46:21.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.302+0000 7f49d22bd640 1 -- 172.21.15.89:0/2094232065 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49cc0b1950 msgr2=0x7f49cc0b7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:21.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.302+0000 7f49d22bd640 1 --2- 172.21.15.89:0/2094232065 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49cc0b1950 0x7f49cc0b7ee0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f49c000e930 tx=0x7f49c000edf0 comp rx=0 tx=0).stop 2023-10-18T10:46:21.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.302+0000 7f49d22bd640 1 -- 172.21.15.89:0/2094232065 shutdown_connections 2023-10-18T10:46:21.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.302+0000 7f49d22bd640 1 --2- 172.21.15.89:0/2094232065 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f49b8065680 0x7f49b8067b40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:21.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.302+0000 7f49d22bd640 1 --2- 172.21.15.89:0/2094232065 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49cc15ff30 0x7f49cc157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:21.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.302+0000 7f49d22bd640 1 --2- 172.21.15.89:0/2094232065 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49cc0b1950 0x7f49cc0b7ee0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:21.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.302+0000 7f49d22bd640 1 --2- 172.21.15.89:0/2094232065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49cc06a010 0x7f49cc0b79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:21.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.302+0000 7f49d22bd640 1 -- 172.21.15.89:0/2094232065 >> 172.21.15.89:0/2094232065 conn(0x7f49cc09b9d0 msgr2=0x7f49cc06a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:21.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.302+0000 7f49d22bd640 1 -- 172.21.15.89:0/2094232065 shutdown_connections 2023-10-18T10:46:21.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.302+0000 7f49d22bd640 1 -- 172.21.15.89:0/2094232065 wait complete. 2023-10-18T10:46:21.325 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:46:21.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.462+0000 7fb65bccb640 1 Processor -- start 2023-10-18T10:46:21.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.462+0000 7fb65bccb640 1 -- start start 2023-10-18T10:46:21.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.462+0000 7fb65bccb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb65415c940 0x7fb6540d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:21.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.462+0000 7fb65bccb640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb6540d2f40 0x7fb6540d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:21.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.462+0000 7fb65bccb640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb65415da50 con 0x7fb6540d0ca0 2023-10-18T10:46:21.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.462+0000 7fb65bccb640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6540d1130 con 0x7fb6540d2f40 2023-10-18T10:46:21.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.462+0000 7fb65bccb640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb65415ce40 con 0x7fb65415c940 2023-10-18T10:46:21.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.466+0000 7fb659a67640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb65415c940 0x7fb6540d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:21.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.466+0000 7fb659266640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb6540d2f40 0x7fb6540d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:21.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.466+0000 7fb659a67640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb65415c940 0x7fb6540d2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:37598/0 (socket says 172.21.15.89:37598) 2023-10-18T10:46:21.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.466+0000 7fb659a67640 1 -- 172.21.15.89:0/3384956229 learned_addr learned my addr 172.21.15.89:0/3384956229 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:21.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.466+0000 7fb659a67640 1 -- 172.21.15.89:0/3384956229 >> v2:172.21.15.154:3300/0 conn(0x7fb6540d2f40 msgr2=0x7fb6540d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:21.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.466+0000 7fb659a67640 1 --2- 172.21.15.89:0/3384956229 >> v2:172.21.15.154:3300/0 conn(0x7fb6540d2f40 0x7fb6540d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:21.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.466+0000 7fb659a67640 1 -- 172.21.15.89:0/3384956229 >> v1:172.21.15.154:6789/0 conn(0x7fb6540d0ca0 legacy=0x7fb6540cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:21.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.466+0000 7fb659a67640 1 -- 172.21.15.89:0/3384956229 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65415d120 con 0x7fb65415c940 2023-10-18T10:46:21.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.466+0000 7fb659a67640 1 --2- 172.21.15.89:0/3384956229 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb65415c940 0x7fb6540d2a00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb650009870 tx=0x7fb65001c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff3d8d1847022a88 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:21.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.470+0000 7fb658a65640 1 -- 172.21.15.89:0/3384956229 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6500ab620 con 0x7fb65415c940 2023-10-18T10:46:21.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.470+0000 7fb658a65640 1 -- 172.21.15.89:0/3384956229 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6500ab780 con 0x7fb65415c940 2023-10-18T10:46:21.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.470+0000 7fb658a65640 1 -- 172.21.15.89:0/3384956229 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6500abb10 con 0x7fb65415c940 2023-10-18T10:46:21.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.470+0000 7fb65bccb640 1 -- 172.21.15.89:0/3384956229 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb65415c940 msgr2=0x7fb6540d2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:21.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.470+0000 7fb65bccb640 1 --2- 172.21.15.89:0/3384956229 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb65415c940 0x7fb6540d2a00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb650009870 tx=0x7fb65001c510 comp rx=0 tx=0).stop 2023-10-18T10:46:21.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.470+0000 7fb65bccb640 1 -- 172.21.15.89:0/3384956229 shutdown_connections 2023-10-18T10:46:21.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.470+0000 7fb65bccb640 1 --2- 172.21.15.89:0/3384956229 >> v2:172.21.15.154:3300/0 conn(0x7fb6540d2f40 0x7fb6540d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:21.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.470+0000 7fb65bccb640 1 --2- 172.21.15.89:0/3384956229 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb65415c940 0x7fb6540d2a00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:21.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.470+0000 7fb65bccb640 1 -- 172.21.15.89:0/3384956229 >> 172.21.15.89:0/3384956229 conn(0x7fb654089fd0 msgr2=0x7fb65409b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:21.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.470+0000 7fb65bccb640 1 -- 172.21.15.89:0/3384956229 shutdown_connections 2023-10-18T10:46:21.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.470+0000 7fb65bccb640 1 -- 172.21.15.89:0/3384956229 wait complete. 2023-10-18T10:46:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.470+0000 7fb65bccb640 1 Processor -- start 2023-10-18T10:46:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.470+0000 7fb65bccb640 1 -- start start 2023-10-18T10:46:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.470+0000 7fb65bccb640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6540d0ca0 0x7fb654154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.470+0000 7fb65bccb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6540cb7e0 0x7fb6540cd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.470+0000 7fb65bccb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb65415b8c0 0x7fb65415bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:21.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.470+0000 7fb65bccb640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65415c290 con 0x7fb6540d0ca0 2023-10-18T10:46:21.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.470+0000 7fb65bccb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6540cc550 con 0x7fb6540cb7e0 2023-10-18T10:46:21.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.470+0000 7fb65bccb640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6540cbcb0 con 0x7fb65415b8c0 2023-10-18T10:46:21.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.470+0000 7fb659a67640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6540d0ca0 0x7fb654154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:21.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.474+0000 7fb659a67640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6540d0ca0 0x7fb654154c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45628/0 (socket says 172.21.15.89:45628) 2023-10-18T10:46:21.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.474+0000 7fb659a67640 1 -- 172.21.15.89:0/1493681922 learned_addr learned my addr 172.21.15.89:0/1493681922 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:21.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.474+0000 7fb65a268640 1 --2- 172.21.15.89:0/1493681922 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb65415b8c0 0x7fb65415bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:21.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.474+0000 7fb659266640 1 --2- 172.21.15.89:0/1493681922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6540cb7e0 0x7fb6540cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:21.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.474+0000 7fb659a67640 1 -- 172.21.15.89:0/1493681922 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb65415b8c0 msgr2=0x7fb65415bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:21.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.474+0000 7fb659a67640 1 --2- 172.21.15.89:0/1493681922 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb65415b8c0 0x7fb65415bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.474+0000 7fb659a67640 1 -- 172.21.15.89:0/1493681922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6540cb7e0 msgr2=0x7fb6540cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.474+0000 7fb659a67640 1 --2- 172.21.15.89:0/1493681922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6540cb7e0 0x7fb6540cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.474+0000 7fb659a67640 1 -- 172.21.15.89:0/1493681922 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6500094d0 con 0x7fb6540d0ca0 2023-10-18T10:46:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.474+0000 7fb659a67640 1 --2- 172.21.15.89:0/1493681922 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6540d0ca0 0x7fb654154c80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb6500ad040 tx=0x7fb65001ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.474+0000 7fb64a7fc640 1 -- 172.21.15.89:0/1493681922 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6500abe40 con 0x7fb6540d0ca0 2023-10-18T10:46:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.474+0000 7fb64a7fc640 1 -- 172.21.15.89:0/1493681922 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6500be3f0 con 0x7fb6540d0ca0 2023-10-18T10:46:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.474+0000 7fb64a7fc640 1 -- 172.21.15.89:0/1493681922 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6500be7a0 con 0x7fb6540d0ca0 2023-10-18T10:46:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.474+0000 7fb65bccb640 1 -- 172.21.15.89:0/1493681922 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6540cbf90 con 0x7fb6540d0ca0 2023-10-18T10:46:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.474+0000 7fb65bccb640 1 -- 172.21.15.89:0/1493681922 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6540b5370 con 0x7fb6540d0ca0 2023-10-18T10:46:21.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.474+0000 7fb65bccb640 1 -- 172.21.15.89:0/1493681922 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6540b5560 con 0x7fb6540d0ca0 2023-10-18T10:46:21.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.474+0000 7fb64a7fc640 1 -- 172.21.15.89:0/1493681922 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fb6500b43f0 con 0x7fb6540d0ca0 2023-10-18T10:46:21.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.474+0000 7fb64a7fc640 1 --2- 172.21.15.89:0/1493681922 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb638065680 0x7fb638067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:21.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.478+0000 7fb64a7fc640 1 -- 172.21.15.89:0/1493681922 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb65011d6c0 con 0x7fb6540d0ca0 2023-10-18T10:46:21.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.478+0000 7fb659266640 1 --2- 172.21.15.89:0/1493681922 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb638065680 0x7fb638067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:21.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.478+0000 7fb659266640 1 --2- 172.21.15.89:0/1493681922 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb638065680 0x7fb638067b40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb6540cbbe0 tx=0x7fb644009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:21.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.478+0000 7fb64a7fc640 1 -- 172.21.15.89:0/1493681922 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fb6500eaf70 con 0x7fb6540d0ca0 2023-10-18T10:46:21.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.626+0000 7fb65bccb640 1 -- 172.21.15.89:0/1493681922 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb65404dbe0 con 0x7fb6540d0ca0 2023-10-18T10:46:21.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.626+0000 7fb64a7fc640 1 -- 172.21.15.89:0/1493681922 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb6500ee590 con 0x7fb6540d0ca0 2023-10-18T10:46:21.629 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-18T10:46:21.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.630+0000 7fb65bccb640 1 -- 172.21.15.89:0/1493681922 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb638065680 msgr2=0x7fb638067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:21.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.630+0000 7fb65bccb640 1 --2- 172.21.15.89:0/1493681922 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb638065680 0x7fb638067b40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb6540cbbe0 tx=0x7fb644009190 comp rx=0 tx=0).stop 2023-10-18T10:46:21.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.630+0000 7fb65bccb640 1 -- 172.21.15.89:0/1493681922 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6540d0ca0 msgr2=0x7fb654154c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:21.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.630+0000 7fb65bccb640 1 --2- 172.21.15.89:0/1493681922 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6540d0ca0 0x7fb654154c80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb6500ad040 tx=0x7fb65001ad50 comp rx=0 tx=0).stop 2023-10-18T10:46:21.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.630+0000 7fb65bccb640 1 -- 172.21.15.89:0/1493681922 shutdown_connections 2023-10-18T10:46:21.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.630+0000 7fb65bccb640 1 --2- 172.21.15.89:0/1493681922 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb638065680 0x7fb638067b40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:21.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.630+0000 7fb65bccb640 1 --2- 172.21.15.89:0/1493681922 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb65415b8c0 0x7fb65415bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:21.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.630+0000 7fb65bccb640 1 --2- 172.21.15.89:0/1493681922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6540cb7e0 0x7fb6540cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:21.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.630+0000 7fb65bccb640 1 --2- 172.21.15.89:0/1493681922 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6540d0ca0 0x7fb654154c80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:21.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.630+0000 7fb65bccb640 1 -- 172.21.15.89:0/1493681922 >> 172.21.15.89:0/1493681922 conn(0x7fb654089fd0 msgr2=0x7fb65415ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:21.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.630+0000 7fb65bccb640 1 -- 172.21.15.89:0/1493681922 shutdown_connections 2023-10-18T10:46:21.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.630+0000 7fb65bccb640 1 -- 172.21.15.89:0/1493681922 wait complete. 2023-10-18T10:46:21.649 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:46:21.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.790+0000 7febdb4cb640 1 Processor -- start 2023-10-18T10:46:21.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.790+0000 7febdb4cb640 1 -- start start 2023-10-18T10:46:21.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.790+0000 7febdb4cb640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7febd415c550 0x7febd40d2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:21.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.790+0000 7febdb4cb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7febd40cfd20 0x7febd40b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:21.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.790+0000 7febdb4cb640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febd40d1e50 con 0x7febd40b1860 2023-10-18T10:46:21.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.790+0000 7febdb4cb640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febd40d2130 con 0x7febd415c550 2023-10-18T10:46:21.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.790+0000 7febdb4cb640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd40d2410 con 0x7febd40cfd20 2023-10-18T10:46:21.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.790+0000 7febd9267640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7febd415c550 0x7febd40d2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:21.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.790+0000 7febd9267640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7febd415c550 0x7febd40d2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45636/0 (socket says 172.21.15.89:45636) 2023-10-18T10:46:21.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.790+0000 7febd9267640 1 -- 172.21.15.89:0/567351350 learned_addr learned my addr 172.21.15.89:0/567351350 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:21.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.790+0000 7febd9267640 1 -- 172.21.15.89:0/567351350 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7febd40cfd20 msgr2=0x7febd40b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:21.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.790+0000 7febd9267640 1 --2- 172.21.15.89:0/567351350 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7febd40cfd20 0x7febd40b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:21.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.790+0000 7febd9267640 1 -- 172.21.15.89:0/567351350 >> v1:172.21.15.89:6789/0 conn(0x7febd40b1860 legacy=0x7febd40cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:21.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.790+0000 7febd9267640 1 -- 172.21.15.89:0/567351350 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd40d26f0 con 0x7febd415c550 2023-10-18T10:46:21.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.794+0000 7febd9267640 1 --2- 172.21.15.89:0/567351350 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febd415c550 0x7febd40d2ef0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7febd0009870 tx=0x7febd001cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d542cfead5beedc server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:21.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.794+0000 7febcbfff640 1 -- 172.21.15.89:0/567351350 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd00aa5f0 con 0x7febd415c550 2023-10-18T10:46:21.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.794+0000 7febcbfff640 1 -- 172.21.15.89:0/567351350 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd00aa750 con 0x7febd415c550 2023-10-18T10:46:21.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.794+0000 7febcbfff640 1 -- 172.21.15.89:0/567351350 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd00aaae0 con 0x7febd415c550 2023-10-18T10:46:21.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.794+0000 7febdb4cb640 1 -- 172.21.15.89:0/567351350 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febd415c550 msgr2=0x7febd40d2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:21.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.794+0000 7febdb4cb640 1 --2- 172.21.15.89:0/567351350 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febd415c550 0x7febd40d2ef0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7febd0009870 tx=0x7febd001cf90 comp rx=0 tx=0).stop 2023-10-18T10:46:21.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.794+0000 7febdb4cb640 1 -- 172.21.15.89:0/567351350 shutdown_connections 2023-10-18T10:46:21.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.794+0000 7febdb4cb640 1 --2- 172.21.15.89:0/567351350 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7febd40cfd20 0x7febd40b55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:21.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.794+0000 7febdb4cb640 1 --2- 172.21.15.89:0/567351350 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febd415c550 0x7febd40d2ef0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:21.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.794+0000 7febdb4cb640 1 -- 172.21.15.89:0/567351350 >> 172.21.15.89:0/567351350 conn(0x7febd4069d00 msgr2=0x7febd406a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:21.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.794+0000 7febdb4cb640 1 -- 172.21.15.89:0/567351350 shutdown_connections 2023-10-18T10:46:21.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.794+0000 7febdb4cb640 1 -- 172.21.15.89:0/567351350 wait complete. 2023-10-18T10:46:21.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.798+0000 7febdb4cb640 1 Processor -- start 2023-10-18T10:46:21.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.798+0000 7febdb4cb640 1 -- start start 2023-10-18T10:46:21.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.798+0000 7febdb4cb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febd40b1860 0x7febd4153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:21.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.798+0000 7febdb4cb640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febd40cfd20 0x7febd40cc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:21.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.798+0000 7febdb4cb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7febd415c550 0x7febd4152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:21.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.798+0000 7febdb4cb640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd41512c0 con 0x7febd40cfd20 2023-10-18T10:46:21.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.798+0000 7febdb4cb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd40cc6a0 con 0x7febd40b1860 2023-10-18T10:46:21.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.798+0000 7febdb4cb640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd4152550 con 0x7febd415c550 2023-10-18T10:46:21.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.798+0000 7febd8a66640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febd40cfd20 0x7febd40cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:21.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.798+0000 7febd9267640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febd40b1860 0x7febd4153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:21.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.798+0000 7febd8a66640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febd40cfd20 0x7febd40cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45650/0 (socket says 172.21.15.89:45650) 2023-10-18T10:46:21.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.798+0000 7febd8a66640 1 -- 172.21.15.89:0/125593105 learned_addr learned my addr 172.21.15.89:0/125593105 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:21.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.798+0000 7febd9a68640 1 --2- 172.21.15.89:0/125593105 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7febd415c550 0x7febd4152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:21.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.798+0000 7febd8a66640 1 -- 172.21.15.89:0/125593105 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7febd415c550 msgr2=0x7febd4152010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:21.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.798+0000 7febd8a66640 1 --2- 172.21.15.89:0/125593105 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7febd415c550 0x7febd4152010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:21.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.798+0000 7febd8a66640 1 -- 172.21.15.89:0/125593105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febd40b1860 msgr2=0x7febd4153c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:21.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.798+0000 7febd8a66640 1 --2- 172.21.15.89:0/125593105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febd40b1860 0x7febd4153c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:21.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.798+0000 7febd8a66640 1 -- 172.21.15.89:0/125593105 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd00094d0 con 0x7febd40cfd20 2023-10-18T10:46:21.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.798+0000 7febd9267640 1 --2- 172.21.15.89:0/125593105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febd40b1860 0x7febd4153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:46:21.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.798+0000 7febd8a66640 1 --2- 172.21.15.89:0/125593105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febd40cfd20 0x7febd40cc160 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7febc400c530 tx=0x7febc400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:21.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.798+0000 7febc9ffb640 1 -- 172.21.15.89:0/125593105 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc4005220 con 0x7febd40cfd20 2023-10-18T10:46:21.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.798+0000 7febc9ffb640 1 -- 172.21.15.89:0/125593105 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc4005380 con 0x7febd40cfd20 2023-10-18T10:46:21.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.798+0000 7febc9ffb640 1 -- 172.21.15.89:0/125593105 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc4007270 con 0x7febd40cfd20 2023-10-18T10:46:21.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.798+0000 7febdb4cb640 1 -- 172.21.15.89:0/125593105 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd4151530 con 0x7febd40cfd20 2023-10-18T10:46:21.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.798+0000 7febdb4cb640 1 -- 172.21.15.89:0/125593105 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd4151af0 con 0x7febd40cfd20 2023-10-18T10:46:21.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.798+0000 7febdb4cb640 1 -- 172.21.15.89:0/125593105 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febd4151d10 con 0x7febd40cfd20 2023-10-18T10:46:21.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.802+0000 7febc9ffb640 1 -- 172.21.15.89:0/125593105 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7febc40073d0 con 0x7febd40cfd20 2023-10-18T10:46:21.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.802+0000 7febc9ffb640 1 --2- 172.21.15.89:0/125593105 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7febb8065680 0x7febb8067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:21.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.802+0000 7febc9ffb640 1 -- 172.21.15.89:0/125593105 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7febc40828c0 con 0x7febd40cfd20 2023-10-18T10:46:21.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.802+0000 7febd9267640 1 --2- 172.21.15.89:0/125593105 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7febb8065680 0x7febb8067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:21.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.802+0000 7febd9267640 1 --2- 172.21.15.89:0/125593105 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7febb8065680 0x7febb8067b40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7febd00ae000 tx=0x7febd00b5040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:21.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.806+0000 7febc9ffb640 1 -- 172.21.15.89:0/125593105 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7febc4050130 con 0x7febd40cfd20 2023-10-18T10:46:21.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.958+0000 7febdb4cb640 1 -- 172.21.15.89:0/125593105 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7febd404dbe0 con 0x7febd40cfd20 2023-10-18T10:46:21.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.958+0000 7febc9ffb640 1 -- 172.21.15.89:0/125593105 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7febc4053750 con 0x7febd40cfd20 2023-10-18T10:46:21.960 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-18T10:46:21.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.958+0000 7febdb4cb640 1 -- 172.21.15.89:0/125593105 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7febb8065680 msgr2=0x7febb8067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:21.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.958+0000 7febdb4cb640 1 --2- 172.21.15.89:0/125593105 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7febb8065680 0x7febb8067b40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7febd00ae000 tx=0x7febd00b5040 comp rx=0 tx=0).stop 2023-10-18T10:46:21.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.958+0000 7febdb4cb640 1 -- 172.21.15.89:0/125593105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febd40cfd20 msgr2=0x7febd40cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:21.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.958+0000 7febdb4cb640 1 --2- 172.21.15.89:0/125593105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febd40cfd20 0x7febd40cc160 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7febc400c530 tx=0x7febc400c9f0 comp rx=0 tx=0).stop 2023-10-18T10:46:21.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.958+0000 7febdb4cb640 1 -- 172.21.15.89:0/125593105 shutdown_connections 2023-10-18T10:46:21.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.958+0000 7febdb4cb640 1 --2- 172.21.15.89:0/125593105 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7febb8065680 0x7febb8067b40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:21.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.958+0000 7febdb4cb640 1 --2- 172.21.15.89:0/125593105 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7febd415c550 0x7febd4152010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:21.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.958+0000 7febdb4cb640 1 --2- 172.21.15.89:0/125593105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febd40cfd20 0x7febd40cc160 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:21.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.958+0000 7febdb4cb640 1 --2- 172.21.15.89:0/125593105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febd40b1860 0x7febd4153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:21.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.958+0000 7febdb4cb640 1 -- 172.21.15.89:0/125593105 >> 172.21.15.89:0/125593105 conn(0x7febd4069d00 msgr2=0x7febd4159550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:21.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.958+0000 7febdb4cb640 1 -- 172.21.15.89:0/125593105 shutdown_connections 2023-10-18T10:46:21.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:21.958+0000 7febdb4cb640 1 -- 172.21.15.89:0/125593105 wait complete. 2023-10-18T10:46:21.980 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T10:46:22.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.118+0000 7fe5dabfa640 1 Processor -- start 2023-10-18T10:46:22.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.122+0000 7fe5dabfa640 1 -- start start 2023-10-18T10:46:22.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.122+0000 7fe5dabfa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5d4070600 0x7fe5d406b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:22.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.122+0000 7fe5dabfa640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe5d406bd10 0x7fe5d40c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:22.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.122+0000 7fe5dabfa640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5d4070b90 con 0x7fe5d4159490 2023-10-18T10:46:22.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.122+0000 7fe5dabfa640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5d406c1a0 con 0x7fe5d406bd10 2023-10-18T10:46:22.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.122+0000 7fe5dabfa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d4070e70 con 0x7fe5d4070600 2023-10-18T10:46:22.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.122+0000 7fe5d9197640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe5d406bd10 0x7fe5d40c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:22.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.122+0000 7fe5d9197640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe5d406bd10 0x7fe5d40c8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45656/0 (socket says 172.21.15.89:45656) 2023-10-18T10:46:22.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.122+0000 7fe5d9197640 1 -- 172.21.15.89:0/2089257720 learned_addr learned my addr 172.21.15.89:0/2089257720 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:22.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.122+0000 7fe5cbfff640 1 --2- 172.21.15.89:0/2089257720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5d4070600 0x7fe5d406b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:22.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.122+0000 7fe5d9197640 1 -- 172.21.15.89:0/2089257720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5d4070600 msgr2=0x7fe5d406b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:22.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.122+0000 7fe5d9197640 1 --2- 172.21.15.89:0/2089257720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5d4070600 0x7fe5d406b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:22.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.122+0000 7fe5d9197640 1 -- 172.21.15.89:0/2089257720 >> v1:172.21.15.89:6789/0 conn(0x7fe5d4159490 legacy=0x7fe5d415d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:22.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.122+0000 7fe5d9197640 1 -- 172.21.15.89:0/2089257720 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d4071150 con 0x7fe5d406bd10 2023-10-18T10:46:22.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.126+0000 7fe5d9197640 1 --2- 172.21.15.89:0/2089257720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5d406bd10 0x7fe5d40c8960 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc009e50 tx=0x7fe5cc01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cda763d01a9c0a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:22.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.126+0000 7fe5cb7fe640 1 -- 172.21.15.89:0/2089257720 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc0a5810 con 0x7fe5d406bd10 2023-10-18T10:46:22.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.126+0000 7fe5cb7fe640 1 -- 172.21.15.89:0/2089257720 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5cc0a5970 con 0x7fe5d406bd10 2023-10-18T10:46:22.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.126+0000 7fe5cb7fe640 1 -- 172.21.15.89:0/2089257720 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc0a5d00 con 0x7fe5d406bd10 2023-10-18T10:46:22.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.126+0000 7fe5dabfa640 1 -- 172.21.15.89:0/2089257720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5d406bd10 msgr2=0x7fe5d40c8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:22.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.126+0000 7fe5dabfa640 1 --2- 172.21.15.89:0/2089257720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5d406bd10 0x7fe5d40c8960 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc009e50 tx=0x7fe5cc01c950 comp rx=0 tx=0).stop 2023-10-18T10:46:22.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.126+0000 7fe5dabfa640 1 -- 172.21.15.89:0/2089257720 shutdown_connections 2023-10-18T10:46:22.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.126+0000 7fe5dabfa640 1 --2- 172.21.15.89:0/2089257720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5d406bd10 0x7fe5d40c8960 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:22.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.126+0000 7fe5dabfa640 1 --2- 172.21.15.89:0/2089257720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5d4070600 0x7fe5d406b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:22.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.126+0000 7fe5dabfa640 1 -- 172.21.15.89:0/2089257720 >> 172.21.15.89:0/2089257720 conn(0x7fe5d4069f00 msgr2=0x7fe5d40cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:22.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.126+0000 7fe5dabfa640 1 -- 172.21.15.89:0/2089257720 shutdown_connections 2023-10-18T10:46:22.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.126+0000 7fe5dabfa640 1 -- 172.21.15.89:0/2089257720 wait complete. 2023-10-18T10:46:22.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.126+0000 7fe5dabfa640 1 Processor -- start 2023-10-18T10:46:22.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.126+0000 7fe5dabfa640 1 -- start start 2023-10-18T10:46:22.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.130+0000 7fe5dabfa640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5d406bd10 0x7fe5d40c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:22.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.130+0000 7fe5dabfa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5d4070600 0x7fe5d40b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:22.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.130+0000 7fe5dabfa640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5d4159490 0x7fe5d4154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:22.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.130+0000 7fe5dabfa640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d40c38c0 con 0x7fe5d406bd10 2023-10-18T10:46:22.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.130+0000 7fe5dabfa640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d40c6b20 con 0x7fe5d4159490 2023-10-18T10:46:22.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.130+0000 7fe5dabfa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d40c82d0 con 0x7fe5d4070600 2023-10-18T10:46:22.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.130+0000 7fe5d8996640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5d406bd10 0x7fe5d40c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:22.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.130+0000 7fe5d8996640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5d406bd10 0x7fe5d40c65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45658/0 (socket says 172.21.15.89:45658) 2023-10-18T10:46:22.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.130+0000 7fe5d8996640 1 -- 172.21.15.89:0/2486463671 learned_addr learned my addr 172.21.15.89:0/2486463671 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:22.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.130+0000 7fe5cbfff640 1 --2- 172.21.15.89:0/2486463671 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5d4070600 0x7fe5d40b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:22.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.130+0000 7fe5d9197640 1 --2- 172.21.15.89:0/2486463671 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5d4159490 0x7fe5d4154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:22.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.130+0000 7fe5d8996640 1 -- 172.21.15.89:0/2486463671 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5d4070600 msgr2=0x7fe5d40b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:22.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.130+0000 7fe5d8996640 1 --2- 172.21.15.89:0/2486463671 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5d4070600 0x7fe5d40b0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:22.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.130+0000 7fe5d8996640 1 -- 172.21.15.89:0/2486463671 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5d4159490 msgr2=0x7fe5d4154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:22.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.130+0000 7fe5d8996640 1 --2- 172.21.15.89:0/2486463671 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5d4159490 0x7fe5d4154680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:22.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.130+0000 7fe5d8996640 1 -- 172.21.15.89:0/2486463671 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5cc009b30 con 0x7fe5d406bd10 2023-10-18T10:46:22.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.130+0000 7fe5cbfff640 1 --2- 172.21.15.89:0/2486463671 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5d4070600 0x7fe5d40b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:22.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.130+0000 7fe5d9197640 1 --2- 172.21.15.89:0/2486463671 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5d4159490 0x7fe5d4154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:22.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.130+0000 7fe5d8996640 1 --2- 172.21.15.89:0/2486463671 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5d406bd10 0x7fe5d40c65e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000c3c0 tx=0x7fe5d000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:22.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.130+0000 7fe5c97fa640 1 -- 172.21.15.89:0/2486463671 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d00052d0 con 0x7fe5d406bd10 2023-10-18T10:46:22.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.130+0000 7fe5c97fa640 1 -- 172.21.15.89:0/2486463671 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d0005430 con 0x7fe5d406bd10 2023-10-18T10:46:22.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.130+0000 7fe5dabfa640 1 -- 172.21.15.89:0/2486463671 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d40c85b0 con 0x7fe5d406bd10 2023-10-18T10:46:22.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.130+0000 7fe5c97fa640 1 -- 172.21.15.89:0/2486463671 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0003840 con 0x7fe5d406bd10 2023-10-18T10:46:22.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.130+0000 7fe5dabfa640 1 -- 172.21.15.89:0/2486463671 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d40c34e0 con 0x7fe5d406bd10 2023-10-18T10:46:22.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.130+0000 7fe5dabfa640 1 -- 172.21.15.89:0/2486463671 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5d40c3a00 con 0x7fe5d406bd10 2023-10-18T10:46:22.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.130+0000 7fe5c97fa640 1 -- 172.21.15.89:0/2486463671 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fe5d00039a0 con 0x7fe5d406bd10 2023-10-18T10:46:22.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.134+0000 7fe5c97fa640 1 --2- 172.21.15.89:0/2486463671 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe5c0065680 0x7fe5c0067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:22.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.134+0000 7fe5c97fa640 1 -- 172.21.15.89:0/2486463671 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe5d0082560 con 0x7fe5d406bd10 2023-10-18T10:46:22.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.134+0000 7fe5cbfff640 1 --2- 172.21.15.89:0/2486463671 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe5c0065680 0x7fe5c0067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:22.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.134+0000 7fe5cbfff640 1 --2- 172.21.15.89:0/2486463671 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe5c0065680 0x7fe5c0067b40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe5d40ad5e0 tx=0x7fe5c4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:22.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.138+0000 7fe5c97fa640 1 -- 172.21.15.89:0/2486463671 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fe5d40c3a00 con 0x7fe5d406bd10 2023-10-18T10:46:22.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.286+0000 7fe5dabfa640 1 -- 172.21.15.89:0/2486463671 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe5d40c3a00 con 0x7fe5d406bd10 2023-10-18T10:46:22.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.286+0000 7fe5c97fa640 1 -- 172.21.15.89:0/2486463671 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe5d40c3a00 con 0x7fe5d406bd10 2023-10-18T10:46:22.289 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-18T10:46:22.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.286+0000 7fe5dabfa640 1 -- 172.21.15.89:0/2486463671 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe5c0065680 msgr2=0x7fe5c0067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:22.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.286+0000 7fe5dabfa640 1 --2- 172.21.15.89:0/2486463671 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe5c0065680 0x7fe5c0067b40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe5d40ad5e0 tx=0x7fe5c4008040 comp rx=0 tx=0).stop 2023-10-18T10:46:22.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.290+0000 7fe5dabfa640 1 -- 172.21.15.89:0/2486463671 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5d406bd10 msgr2=0x7fe5d40c65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:22.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.290+0000 7fe5dabfa640 1 --2- 172.21.15.89:0/2486463671 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5d406bd10 0x7fe5d40c65e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000c3c0 tx=0x7fe5d000c880 comp rx=0 tx=0).stop 2023-10-18T10:46:22.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.290+0000 7fe5dabfa640 1 -- 172.21.15.89:0/2486463671 shutdown_connections 2023-10-18T10:46:22.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.290+0000 7fe5dabfa640 1 --2- 172.21.15.89:0/2486463671 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe5c0065680 0x7fe5c0067b40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:22.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.290+0000 7fe5dabfa640 1 --2- 172.21.15.89:0/2486463671 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5d4070600 0x7fe5d40b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:22.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.290+0000 7fe5dabfa640 1 --2- 172.21.15.89:0/2486463671 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5d4159490 0x7fe5d4154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:22.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.290+0000 7fe5dabfa640 1 --2- 172.21.15.89:0/2486463671 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5d406bd10 0x7fe5d40c65e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:22.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.290+0000 7fe5dabfa640 1 -- 172.21.15.89:0/2486463671 >> 172.21.15.89:0/2486463671 conn(0x7fe5d4069f00 msgr2=0x7fe5d40aec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:22.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.290+0000 7fe5dabfa640 1 -- 172.21.15.89:0/2486463671 shutdown_connections 2023-10-18T10:46:22.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.290+0000 7fe5dabfa640 1 -- 172.21.15.89:0/2486463671 wait complete. 2023-10-18T10:46:22.309 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:46:22.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.446+0000 7f9eb1f2c640 1 Processor -- start 2023-10-18T10:46:22.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.446+0000 7f9eb1f2c640 1 -- start start 2023-10-18T10:46:22.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.446+0000 7f9eb1f2c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9eac159490 0x7f9eac15d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:22.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.446+0000 7f9eb1f2c640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eac15ddf0 con 0x7f9eac070600 2023-10-18T10:46:22.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.446+0000 7f9eb1f2c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eac070b90 con 0x7f9eac06bee0 2023-10-18T10:46:22.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.446+0000 7f9eb1f2c640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eac070e70 con 0x7f9eac159490 2023-10-18T10:46:22.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.446+0000 7f9eab7fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9eac159490 0x7f9eac15d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:22.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.446+0000 7f9eab7fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9eac159490 0x7f9eac15d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59372/0 (socket says 172.21.15.89:59372) 2023-10-18T10:46:22.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.446+0000 7f9eab7fe640 1 -- 172.21.15.89:0/58875026 learned_addr learned my addr 172.21.15.89:0/58875026 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:22.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.446+0000 7f9eab7fe640 1 -- 172.21.15.89:0/58875026 >> v1:172.21.15.154:6789/0 conn(0x7f9eac06bee0 legacy=0x7f9eac0c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:22.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.446+0000 7f9eab7fe640 1 -- 172.21.15.89:0/58875026 >> v1:172.21.15.89:6789/0 conn(0x7f9eac070600 legacy=0x7f9eac06b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:22.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.446+0000 7f9eab7fe640 1 -- 172.21.15.89:0/58875026 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eac071150 con 0x7f9eac159490 2023-10-18T10:46:22.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.450+0000 7f9eab7fe640 1 --2- 172.21.15.89:0/58875026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9eac159490 0x7f9eac15d8b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00d150 tx=0x7f9e9c01eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c6aeb54ed6bb72 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:22.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.450+0000 7f9eaaffd640 1 -- 172.21.15.89:0/58875026 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c00b2d0 con 0x7f9eac159490 2023-10-18T10:46:22.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.450+0000 7f9eaaffd640 1 -- 172.21.15.89:0/58875026 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e9c00b430 con 0x7f9eac159490 2023-10-18T10:46:22.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.450+0000 7f9eaaffd640 1 -- 172.21.15.89:0/58875026 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c0a7c00 con 0x7f9eac159490 2023-10-18T10:46:22.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.450+0000 7f9eb1f2c640 1 -- 172.21.15.89:0/58875026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9eac159490 msgr2=0x7f9eac15d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:22.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.450+0000 7f9eb1f2c640 1 --2- 172.21.15.89:0/58875026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9eac159490 0x7f9eac15d8b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00d150 tx=0x7f9e9c01eb10 comp rx=0 tx=0).stop 2023-10-18T10:46:22.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.450+0000 7f9eb1f2c640 1 -- 172.21.15.89:0/58875026 shutdown_connections 2023-10-18T10:46:22.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.450+0000 7f9eb1f2c640 1 --2- 172.21.15.89:0/58875026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9eac159490 0x7f9eac15d8b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:22.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.450+0000 7f9eb1f2c640 1 -- 172.21.15.89:0/58875026 >> 172.21.15.89:0/58875026 conn(0x7f9eac069f00 msgr2=0x7f9eac0cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:22.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.450+0000 7f9eb1f2c640 1 -- 172.21.15.89:0/58875026 shutdown_connections 2023-10-18T10:46:22.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.450+0000 7f9eb1f2c640 1 -- 172.21.15.89:0/58875026 wait complete. 2023-10-18T10:46:22.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.454+0000 7f9eb1f2c640 1 Processor -- start 2023-10-18T10:46:22.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.454+0000 7f9eb1f2c640 1 -- start start 2023-10-18T10:46:22.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.454+0000 7f9eb1f2c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9eac06bee0 0x7f9eac154b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:22.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.454+0000 7f9eb1f2c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9eac070600 0x7f9eac152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:22.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.454+0000 7f9eb1f2c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9eac159490 0x7f9eac0cc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:22.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.454+0000 7f9eb1f2c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac157f90 con 0x7f9eac06bee0 2023-10-18T10:46:22.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.454+0000 7f9eb1f2c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac1576f0 con 0x7f9eac070600 2023-10-18T10:46:22.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.454+0000 7f9eb1f2c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac1579d0 con 0x7f9eac159490 2023-10-18T10:46:22.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.454+0000 7f9eab7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9eac06bee0 0x7f9eac154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:22.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.454+0000 7f9eab7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9eac06bee0 0x7f9eac154b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59388/0 (socket says 172.21.15.89:59388) 2023-10-18T10:46:22.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.454+0000 7f9eab7fe640 1 -- 172.21.15.89:0/979980604 learned_addr learned my addr 172.21.15.89:0/979980604 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:22.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.454+0000 7f9ea2ffd640 1 --2- 172.21.15.89:0/979980604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9eac070600 0x7f9eac152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:22.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.454+0000 7f9eab7fe640 1 -- 172.21.15.89:0/979980604 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9eac159490 msgr2=0x7f9eac0cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:22.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.454+0000 7f9eabfff640 1 --2- 172.21.15.89:0/979980604 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9eac159490 0x7f9eac0cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:22.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.454+0000 7f9eab7fe640 1 --2- 172.21.15.89:0/979980604 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9eac159490 0x7f9eac0cc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:22.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.454+0000 7f9eab7fe640 1 -- 172.21.15.89:0/979980604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9eac070600 msgr2=0x7f9eac152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:22.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.454+0000 7f9eab7fe640 1 --2- 172.21.15.89:0/979980604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9eac070600 0x7f9eac152870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:22.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.454+0000 7f9eab7fe640 1 -- 172.21.15.89:0/979980604 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9c00bd40 con 0x7f9eac06bee0 2023-10-18T10:46:22.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.454+0000 7f9ea2ffd640 1 --2- 172.21.15.89:0/979980604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9eac070600 0x7f9eac152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:22.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.454+0000 7f9eabfff640 1 --2- 172.21.15.89:0/979980604 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9eac159490 0x7f9eac0cc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:22.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.454+0000 7f9eab7fe640 1 --2- 172.21.15.89:0/979980604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9eac06bee0 0x7f9eac154b20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c0b1040 tx=0x7f9e9c01b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:22.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.454+0000 7f9ea8ff9640 1 -- 172.21.15.89:0/979980604 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c0bd070 con 0x7f9eac06bee0 2023-10-18T10:46:22.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.454+0000 7f9ea8ff9640 1 -- 172.21.15.89:0/979980604 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e9c009600 con 0x7f9eac06bee0 2023-10-18T10:46:22.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.454+0000 7f9ea8ff9640 1 -- 172.21.15.89:0/979980604 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c009990 con 0x7f9eac06bee0 2023-10-18T10:46:22.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.454+0000 7f9eb1f2c640 1 -- 172.21.15.89:0/979980604 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eac157cb0 con 0x7f9eac06bee0 2023-10-18T10:46:22.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.454+0000 7f9eb1f2c640 1 -- 172.21.15.89:0/979980604 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eac155060 con 0x7f9eac06bee0 2023-10-18T10:46:22.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.454+0000 7f9eb1f2c640 1 -- 172.21.15.89:0/979980604 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eac04dbe0 con 0x7f9eac06bee0 2023-10-18T10:46:22.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.458+0000 7f9ea8ff9640 1 -- 172.21.15.89:0/979980604 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f9e9c009af0 con 0x7f9eac06bee0 2023-10-18T10:46:22.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.458+0000 7f9ea8ff9640 1 --2- 172.21.15.89:0/979980604 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f9e8c065680 0x7f9e8c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:22.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.458+0000 7f9ea8ff9640 1 -- 172.21.15.89:0/979980604 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9e9c11e2c0 con 0x7f9eac06bee0 2023-10-18T10:46:22.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.458+0000 7f9ea2ffd640 1 --2- 172.21.15.89:0/979980604 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f9e8c065680 0x7f9e8c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:22.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.458+0000 7f9ea2ffd640 1 --2- 172.21.15.89:0/979980604 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f9e8c065680 0x7f9e8c067b40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0ae880 tx=0x7f9e94009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:22.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.462+0000 7f9ea8ff9640 1 -- 172.21.15.89:0/979980604 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f9e9c0ebb20 con 0x7f9eac06bee0 2023-10-18T10:46:22.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.638+0000 7f9eb1f2c640 1 -- 172.21.15.89:0/979980604 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9eac04dbe0 con 0x7f9eac06bee0 2023-10-18T10:46:22.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.638+0000 7f9ea8ff9640 1 -- 172.21.15.89:0/979980604 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4874 (secure 0 0 0) 0x7f9e9c0ef140 con 0x7f9eac06bee0 2023-10-18T10:46:22.642 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:22.642 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/1247973498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1247973498},{"type":"v1","addr":"172.21.15.154:6836","nonce":1247973498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4284,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/516057316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":516057316},{"type":"v1","addr":"172.21.15.154:6837","nonce":516057316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4299,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6834/3957051407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3957051407},{"type":"v1","addr":"172.21.15.89:6834","nonce":3957051407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:45:55.655016+0000","modified":"2023-10-18T10:46:13.833215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4312":{"gid":4312,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6835/2906113039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6833","nonce":2906113039},{"type":"v1","addr":"172.21.15.89:6835","nonce":2906113039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T10:46:22.643 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 9 2023-10-18T10:46:22.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.642+0000 7f9eb1f2c640 1 -- 172.21.15.89:0/979980604 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f9e8c065680 msgr2=0x7f9e8c067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:22.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.642+0000 7f9eb1f2c640 1 --2- 172.21.15.89:0/979980604 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f9e8c065680 0x7f9e8c067b40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0ae880 tx=0x7f9e94009040 comp rx=0 tx=0).stop 2023-10-18T10:46:22.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.642+0000 7f9eb1f2c640 1 -- 172.21.15.89:0/979980604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9eac06bee0 msgr2=0x7f9eac154b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:22.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.642+0000 7f9eb1f2c640 1 --2- 172.21.15.89:0/979980604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9eac06bee0 0x7f9eac154b20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c0b1040 tx=0x7f9e9c01b8b0 comp rx=0 tx=0).stop 2023-10-18T10:46:22.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.642+0000 7f9eb1f2c640 1 -- 172.21.15.89:0/979980604 shutdown_connections 2023-10-18T10:46:22.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.642+0000 7f9eb1f2c640 1 --2- 172.21.15.89:0/979980604 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f9e8c065680 0x7f9e8c067b40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:22.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.642+0000 7f9eb1f2c640 1 --2- 172.21.15.89:0/979980604 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9eac159490 0x7f9eac0cc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:22.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.642+0000 7f9eb1f2c640 1 --2- 172.21.15.89:0/979980604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9eac070600 0x7f9eac152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:22.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.642+0000 7f9eb1f2c640 1 --2- 172.21.15.89:0/979980604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9eac06bee0 0x7f9eac154b20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:22.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.642+0000 7f9eb1f2c640 1 -- 172.21.15.89:0/979980604 >> 172.21.15.89:0/979980604 conn(0x7f9eac069f00 msgr2=0x7f9eac158860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:22.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.642+0000 7f9eb1f2c640 1 -- 172.21.15.89:0/979980604 shutdown_connections 2023-10-18T10:46:22.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.642+0000 7f9eb1f2c640 1 -- 172.21.15.89:0/979980604 wait complete. 2023-10-18T10:46:22.663 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:46:22.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.798+0000 7fbcf99e1640 1 Processor -- start 2023-10-18T10:46:22.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.798+0000 7fbcf99e1640 1 -- start start 2023-10-18T10:46:22.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.802+0000 7fbcf99e1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbcf40d0760 0x7fbcf40d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:22.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.802+0000 7fbcf99e1640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbcf40cd160 0x7fbcf40cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:22.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.802+0000 7fbcf99e1640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcf40d1100 con 0x7fbcf415c940 2023-10-18T10:46:22.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.802+0000 7fbcf99e1640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcf415da50 con 0x7fbcf40cd160 2023-10-18T10:46:22.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.802+0000 7fbcf99e1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf40d3110 con 0x7fbcf40d0760 2023-10-18T10:46:22.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.802+0000 7fbcf37fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbcf40cd160 0x7fbcf40cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:22.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.802+0000 7fbcf37fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbcf40cd160 0x7fbcf40cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59398/0 (socket says 172.21.15.89:59398) 2023-10-18T10:46:22.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.802+0000 7fbcf37fe640 1 -- 172.21.15.89:0/645123019 learned_addr learned my addr 172.21.15.89:0/645123019 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:22.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.802+0000 7fbcf27fc640 1 --2- 172.21.15.89:0/645123019 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbcf40d0760 0x7fbcf40d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:22.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.802+0000 7fbcf37fe640 1 -- 172.21.15.89:0/645123019 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbcf40d0760 msgr2=0x7fbcf40d0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:22.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.802+0000 7fbcf37fe640 1 --2- 172.21.15.89:0/645123019 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbcf40d0760 0x7fbcf40d0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:22.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.802+0000 7fbcf37fe640 1 -- 172.21.15.89:0/645123019 >> v1:172.21.15.89:6789/0 conn(0x7fbcf415c940 legacy=0x7fbcf40d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:22.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.802+0000 7fbcf37fe640 1 -- 172.21.15.89:0/645123019 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf415ce40 con 0x7fbcf40cd160 2023-10-18T10:46:22.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.802+0000 7fbcf37fe640 1 --2- 172.21.15.89:0/645123019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcf40cd160 0x7fbcf40cd5f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbcec009870 tx=0x7fbcec01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de10b1e5bbdf92c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:22.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.806+0000 7fbcf1ffb640 1 -- 172.21.15.89:0/645123019 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcec0aa670 con 0x7fbcf40cd160 2023-10-18T10:46:22.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.806+0000 7fbcf1ffb640 1 -- 172.21.15.89:0/645123019 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcec0aa7d0 con 0x7fbcf40cd160 2023-10-18T10:46:22.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.806+0000 7fbcf1ffb640 1 -- 172.21.15.89:0/645123019 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcec0aab60 con 0x7fbcf40cd160 2023-10-18T10:46:22.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.806+0000 7fbcf99e1640 1 -- 172.21.15.89:0/645123019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcf40cd160 msgr2=0x7fbcf40cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:22.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.806+0000 7fbcf99e1640 1 --2- 172.21.15.89:0/645123019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcf40cd160 0x7fbcf40cd5f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbcec009870 tx=0x7fbcec01c510 comp rx=0 tx=0).stop 2023-10-18T10:46:22.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.806+0000 7fbcf99e1640 1 -- 172.21.15.89:0/645123019 shutdown_connections 2023-10-18T10:46:22.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.806+0000 7fbcf99e1640 1 --2- 172.21.15.89:0/645123019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcf40cd160 0x7fbcf40cd5f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:22.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.806+0000 7fbcf99e1640 1 --2- 172.21.15.89:0/645123019 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbcf40d0760 0x7fbcf40d0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:22.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.806+0000 7fbcf99e1640 1 -- 172.21.15.89:0/645123019 >> 172.21.15.89:0/645123019 conn(0x7fbcf4089fd0 msgr2=0x7fbcf409b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:22.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.806+0000 7fbcf99e1640 1 -- 172.21.15.89:0/645123019 shutdown_connections 2023-10-18T10:46:22.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.806+0000 7fbcf99e1640 1 -- 172.21.15.89:0/645123019 wait complete. 2023-10-18T10:46:22.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.806+0000 7fbcf99e1640 1 Processor -- start 2023-10-18T10:46:22.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.806+0000 7fbcf99e1640 1 -- start start 2023-10-18T10:46:22.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.806+0000 7fbcf99e1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbcf40cd160 0x7fbcf40b7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:22.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.806+0000 7fbcf99e1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcf40d0760 0x7fbcf40c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:22.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.806+0000 7fbcf99e1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbcf4152010 0x7fbcf40b5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:22.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.806+0000 7fbcf99e1640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf41534b0 con 0x7fbcf40d0760 2023-10-18T10:46:22.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.806+0000 7fbcf99e1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf41524b0 con 0x7fbcf4152010 2023-10-18T10:46:22.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.806+0000 7fbcf99e1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf4152790 con 0x7fbcf40cd160 2023-10-18T10:46:22.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.806+0000 7fbcf27fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcf40d0760 0x7fbcf40c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:22.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.806+0000 7fbcf27fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcf40d0760 0x7fbcf40c87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59400/0 (socket says 172.21.15.89:59400) 2023-10-18T10:46:22.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.806+0000 7fbcf27fc640 1 -- 172.21.15.89:0/2251535742 learned_addr learned my addr 172.21.15.89:0/2251535742 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:22.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.806+0000 7fbcf37fe640 1 --2- 172.21.15.89:0/2251535742 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbcf4152010 0x7fbcf40b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:22.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.806+0000 7fbcf2ffd640 1 --2- 172.21.15.89:0/2251535742 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbcf40cd160 0x7fbcf40b7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:22.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.810+0000 7fbcf27fc640 1 -- 172.21.15.89:0/2251535742 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbcf40cd160 msgr2=0x7fbcf40b7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:22.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.810+0000 7fbcf27fc640 1 --2- 172.21.15.89:0/2251535742 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbcf40cd160 0x7fbcf40b7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:22.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.810+0000 7fbcf27fc640 1 -- 172.21.15.89:0/2251535742 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbcf4152010 msgr2=0x7fbcf40b5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:22.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.810+0000 7fbcf27fc640 1 --2- 172.21.15.89:0/2251535742 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbcf4152010 0x7fbcf40b5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:22.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.810+0000 7fbcf27fc640 1 -- 172.21.15.89:0/2251535742 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcec0094d0 con 0x7fbcf40d0760 2023-10-18T10:46:22.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.810+0000 7fbcf2ffd640 1 --2- 172.21.15.89:0/2251535742 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbcf40cd160 0x7fbcf40b7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:22.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.810+0000 7fbcf27fc640 1 --2- 172.21.15.89:0/2251535742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcf40d0760 0x7fbcf40c87f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbce800e930 tx=0x7fbce800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:22.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.810+0000 7fbcdbfff640 1 -- 172.21.15.89:0/2251535742 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce8009690 con 0x7fbcf40d0760 2023-10-18T10:46:22.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.810+0000 7fbcdbfff640 1 -- 172.21.15.89:0/2251535742 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce800ca00 con 0x7fbcf40d0760 2023-10-18T10:46:22.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.810+0000 7fbcf99e1640 1 -- 172.21.15.89:0/2251535742 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcf41536c0 con 0x7fbcf40d0760 2023-10-18T10:46:22.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.810+0000 7fbcdbfff640 1 -- 172.21.15.89:0/2251535742 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce800cd70 con 0x7fbcf40d0760 2023-10-18T10:46:22.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.810+0000 7fbcf99e1640 1 -- 172.21.15.89:0/2251535742 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf4159fb0 con 0x7fbcf40d0760 2023-10-18T10:46:22.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.810+0000 7fbcf99e1640 1 -- 172.21.15.89:0/2251535742 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcf404dbe0 con 0x7fbcf40d0760 2023-10-18T10:46:22.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.810+0000 7fbcdbfff640 1 -- 172.21.15.89:0/2251535742 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fbce8013470 con 0x7fbcf40d0760 2023-10-18T10:46:22.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.810+0000 7fbcdbfff640 1 --2- 172.21.15.89:0/2251535742 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fbce0065680 0x7fbce0067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:22.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.810+0000 7fbcdbfff640 1 -- 172.21.15.89:0/2251535742 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fbce8083350 con 0x7fbcf40d0760 2023-10-18T10:46:22.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.814+0000 7fbcf2ffd640 1 --2- 172.21.15.89:0/2251535742 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fbce0065680 0x7fbce0067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:22.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.814+0000 7fbcf2ffd640 1 --2- 172.21.15.89:0/2251535742 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fbce0065680 0x7fbce0067b40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbce40012f0 tx=0x7fbce4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:22.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.814+0000 7fbcdbfff640 1 -- 172.21.15.89:0/2251535742 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fbce8050c90 con 0x7fbcf40d0760 2023-10-18T10:46:22.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.966+0000 7fbcf99e1640 1 -- 172.21.15.89:0/2251535742 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbcf40b85d0 con 0x7fbcf40d0760 2023-10-18T10:46:22.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.966+0000 7fbcdbfff640 1 -- 172.21.15.89:0/2251535742 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20096 (secure 0 0 0) 0x7fbce80542b0 con 0x7fbcf40d0760 2023-10-18T10:46:22.969 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:22.969 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":27,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:46:00.166702+0000","last_up_change":"2023-10-18T10:45:42.164688+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:45:53.309122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-18T10:45:54.659763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:45:57.796514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6824","nonce":773078364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6827","nonce":773078364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6831","nonce":773078364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6829","nonce":773078364}]},"public_addr":"172.21.15.154:6824/773078364","cluster_addr":"172.21.15.154:6827/773078364","heartbeat_back_addr":"172.21.15.154:6831/773078364","heartbeat_front_addr":"172.21.15.154:6829/773078364","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:46:22.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.970+0000 7fbcf99e1640 1 -- 172.21.15.89:0/2251535742 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fbce0065680 msgr2=0x7fbce0067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:22.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.970+0000 7fbcf99e1640 1 --2- 172.21.15.89:0/2251535742 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fbce0065680 0x7fbce0067b40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbce40012f0 tx=0x7fbce4009040 comp rx=0 tx=0).stop 2023-10-18T10:46:22.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.970+0000 7fbcf99e1640 1 -- 172.21.15.89:0/2251535742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcf40d0760 msgr2=0x7fbcf40c87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:22.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.970+0000 7fbcf99e1640 1 --2- 172.21.15.89:0/2251535742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcf40d0760 0x7fbcf40c87f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbce800e930 tx=0x7fbce800edf0 comp rx=0 tx=0).stop 2023-10-18T10:46:22.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.970+0000 7fbcf99e1640 1 -- 172.21.15.89:0/2251535742 shutdown_connections 2023-10-18T10:46:22.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.970+0000 7fbcf99e1640 1 --2- 172.21.15.89:0/2251535742 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fbce0065680 0x7fbce0067b40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:22.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.970+0000 7fbcf99e1640 1 --2- 172.21.15.89:0/2251535742 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbcf4152010 0x7fbcf40b5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:22.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.970+0000 7fbcf99e1640 1 --2- 172.21.15.89:0/2251535742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcf40d0760 0x7fbcf40c87f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:22.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.970+0000 7fbcf99e1640 1 --2- 172.21.15.89:0/2251535742 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbcf40cd160 0x7fbcf40b7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:22.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.970+0000 7fbcf99e1640 1 -- 172.21.15.89:0/2251535742 >> 172.21.15.89:0/2251535742 conn(0x7fbcf4089fd0 msgr2=0x7fbcf40cf040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:22.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.970+0000 7fbcf99e1640 1 -- 172.21.15.89:0/2251535742 shutdown_connections 2023-10-18T10:46:22.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:22.970+0000 7fbcf99e1640 1 -- 172.21.15.89:0/2251535742 wait complete. 2023-10-18T10:46:22.990 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:45:43.114184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-18T10:45:48.730968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:45:53.309122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:45:54.659763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T10:45:57.796514+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T10:46:22.991 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:46:23.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.130+0000 7f6863398640 1 Processor -- start 2023-10-18T10:46:23.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.130+0000 7f6863398640 1 -- start start 2023-10-18T10:46:23.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.130+0000 7f6863398640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f685c0b46c0 0x7f685c0d1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:23.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.130+0000 7f6863398640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f685c15c710 0x7f685c0d19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:23.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.130+0000 7f6863398640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f685c0d0070 con 0x7f685c15c330 2023-10-18T10:46:23.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.130+0000 7f6863398640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f685c0d1650 con 0x7f685c15c710 2023-10-18T10:46:23.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.130+0000 7f6863398640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0d0350 con 0x7f685c0b46c0 2023-10-18T10:46:23.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.130+0000 7f6862396640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f685c15c330 0x7f685c0b3fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:45678/0 (socket says 172.21.15.89:45678) 2023-10-18T10:46:23.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.130+0000 7f6862396640 1 -- 172.21.15.89:0/753857678 learned_addr learned my addr 172.21.15.89:0/753857678 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:23.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.130+0000 7f6861b95640 1 --2- 172.21.15.89:0/753857678 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f685c0b46c0 0x7f685c0d1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:23.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.130+0000 7f6862b97640 1 --2- 172.21.15.89:0/753857678 >> v2:172.21.15.154:3300/0 conn(0x7f685c15c710 0x7f685c0d19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:23.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.130+0000 7f6861394640 1 -- 172.21.15.89:0/753857678 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3392694516 0 0) 0x7f685c0d0070 con 0x7f685c15c330 2023-10-18T10:46:23.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.130+0000 7f6861394640 1 -- 172.21.15.89:0/753857678 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6848003530 con 0x7f685c15c330 2023-10-18T10:46:23.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.130+0000 7f6861b95640 1 -- 172.21.15.89:0/753857678 >> v2:172.21.15.154:3300/0 conn(0x7f685c15c710 msgr2=0x7f685c0d19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:23.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.130+0000 7f6861b95640 1 --2- 172.21.15.89:0/753857678 >> v2:172.21.15.154:3300/0 conn(0x7f685c15c710 0x7f685c0d19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:23.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.130+0000 7f6861b95640 1 -- 172.21.15.89:0/753857678 >> v1:172.21.15.89:6789/0 conn(0x7f685c15c330 legacy=0x7f685c0b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:23.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.130+0000 7f6861b95640 1 -- 172.21.15.89:0/753857678 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c0d0630 con 0x7f685c0b46c0 2023-10-18T10:46:23.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.130+0000 7f6862b97640 1 --2- 172.21.15.89:0/753857678 >> v2:172.21.15.154:3300/0 conn(0x7f685c15c710 0x7f685c0d19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:46:23.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.134+0000 7f6861b95640 1 --2- 172.21.15.89:0/753857678 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f685c0b46c0 0x7f685c0d1110 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f684c0097c0 tx=0x7f684c01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31c55a4fe043c17a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:23.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.134+0000 7f6861394640 1 -- 172.21.15.89:0/753857678 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c0ad660 con 0x7f685c0b46c0 2023-10-18T10:46:23.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.134+0000 7f6861394640 1 -- 172.21.15.89:0/753857678 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684c0ad7c0 con 0x7f685c0b46c0 2023-10-18T10:46:23.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.134+0000 7f6861394640 1 -- 172.21.15.89:0/753857678 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c0adb30 con 0x7f685c0b46c0 2023-10-18T10:46:23.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.134+0000 7f6863398640 1 -- 172.21.15.89:0/753857678 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f685c0b46c0 msgr2=0x7f685c0d1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:23.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.134+0000 7f6863398640 1 --2- 172.21.15.89:0/753857678 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f685c0b46c0 0x7f685c0d1110 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f684c0097c0 tx=0x7f684c01c450 comp rx=0 tx=0).stop 2023-10-18T10:46:23.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.138+0000 7f6863398640 1 -- 172.21.15.89:0/753857678 shutdown_connections 2023-10-18T10:46:23.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.138+0000 7f6863398640 1 --2- 172.21.15.89:0/753857678 >> v2:172.21.15.154:3300/0 conn(0x7f685c15c710 0x7f685c0d19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:23.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.138+0000 7f6863398640 1 --2- 172.21.15.89:0/753857678 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f685c0b46c0 0x7f685c0d1110 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:23.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.138+0000 7f6863398640 1 -- 172.21.15.89:0/753857678 >> 172.21.15.89:0/753857678 conn(0x7f685c070320 msgr2=0x7f685c070730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:23.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.138+0000 7f6863398640 1 -- 172.21.15.89:0/753857678 shutdown_connections 2023-10-18T10:46:23.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.138+0000 7f6863398640 1 -- 172.21.15.89:0/753857678 wait complete. 2023-10-18T10:46:23.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.138+0000 7f6863398640 1 Processor -- start 2023-10-18T10:46:23.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.138+0000 7f6863398640 1 -- start start 2023-10-18T10:46:23.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.138+0000 7f6863398640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f685c15c330 0x7f685c155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:23.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.138+0000 7f6863398640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f685c15c710 0x7f685c15d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:23.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.138+0000 7f6863398640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f685c155fc0 0x7f685c15d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:23.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.138+0000 7f6863398640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0c9f40 con 0x7f685c15c330 2023-10-18T10:46:23.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.138+0000 7f6863398640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c156460 con 0x7f685c155fc0 2023-10-18T10:46:23.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.138+0000 7f6863398640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0c9610 con 0x7f685c15c710 2023-10-18T10:46:23.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.138+0000 7f6862396640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f685c15c330 0x7f685c155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:23.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.138+0000 7f6862396640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f685c15c330 0x7f685c155a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59412/0 (socket says 172.21.15.89:59412) 2023-10-18T10:46:23.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.138+0000 7f6862396640 1 -- 172.21.15.89:0/875257744 learned_addr learned my addr 172.21.15.89:0/875257744 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:23.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.138+0000 7f6862396640 1 -- 172.21.15.89:0/875257744 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f685c15c710 msgr2=0x7f685c15d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:23.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.138+0000 7f6862396640 1 --2- 172.21.15.89:0/875257744 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f685c15c710 0x7f685c15d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:23.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.138+0000 7f6862396640 1 -- 172.21.15.89:0/875257744 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f685c155fc0 msgr2=0x7f685c15d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:23.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.138+0000 7f6862396640 1 --2- 172.21.15.89:0/875257744 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f685c155fc0 0x7f685c15d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:23.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.138+0000 7f6862396640 1 -- 172.21.15.89:0/875257744 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6854009420 con 0x7f685c15c330 2023-10-18T10:46:23.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.138+0000 7f6862396640 1 --2- 172.21.15.89:0/875257744 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f685c15c330 0x7f685c155a80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f685800cb40 tx=0x7f685800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:23.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.138+0000 7f6852ffd640 1 -- 172.21.15.89:0/875257744 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685800aca0 con 0x7f685c15c330 2023-10-18T10:46:23.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.138+0000 7f6852ffd640 1 -- 172.21.15.89:0/875257744 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f685800ae00 con 0x7f685c15c330 2023-10-18T10:46:23.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.138+0000 7f6852ffd640 1 -- 172.21.15.89:0/875257744 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6858012620 con 0x7f685c15c330 2023-10-18T10:46:23.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.142+0000 7f6863398640 1 -- 172.21.15.89:0/875257744 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f684c009420 con 0x7f685c15c330 2023-10-18T10:46:23.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.142+0000 7f6863398640 1 -- 172.21.15.89:0/875257744 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f685c0c9bd0 con 0x7f685c15c330 2023-10-18T10:46:23.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.142+0000 7f6863398640 1 -- 172.21.15.89:0/875257744 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f685c073b50 con 0x7f685c15c330 2023-10-18T10:46:23.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.142+0000 7f6852ffd640 1 -- 172.21.15.89:0/875257744 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f6858007070 con 0x7f685c15c330 2023-10-18T10:46:23.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.142+0000 7f6852ffd640 1 --2- 172.21.15.89:0/875257744 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6840065680 0x7f6840067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:23.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.142+0000 7f6852ffd640 1 -- 172.21.15.89:0/875257744 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f68580831f0 con 0x7f685c15c330 2023-10-18T10:46:23.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.142+0000 7f6861b95640 1 --2- 172.21.15.89:0/875257744 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6840065680 0x7f6840067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:23.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.146+0000 7f6861b95640 1 --2- 172.21.15.89:0/875257744 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6840065680 0x7f6840067b40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f684c0af000 tx=0x7f684c004740 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:23.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.146+0000 7f6852ffd640 1 -- 172.21.15.89:0/875257744 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f6858050920 con 0x7f685c15c330 2023-10-18T10:46:23.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.294+0000 7f6863398640 1 -- 172.21.15.89:0/875257744 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f685c0b7dd0 con 0x7f685c15c330 2023-10-18T10:46:23.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.298+0000 7f6852ffd640 1 -- 172.21.15.89:0/875257744 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6858053f40 con 0x7f685c15c330 2023-10-18T10:46:23.299 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-18T10:46:23.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.298+0000 7f6863398640 1 -- 172.21.15.89:0/875257744 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6840065680 msgr2=0x7f6840067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:23.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.298+0000 7f6863398640 1 --2- 172.21.15.89:0/875257744 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6840065680 0x7f6840067b40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f684c0af000 tx=0x7f684c004740 comp rx=0 tx=0).stop 2023-10-18T10:46:23.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.298+0000 7f6863398640 1 -- 172.21.15.89:0/875257744 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f685c15c330 msgr2=0x7f685c155a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:23.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.298+0000 7f6863398640 1 --2- 172.21.15.89:0/875257744 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f685c15c330 0x7f685c155a80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f685800cb40 tx=0x7f685800a060 comp rx=0 tx=0).stop 2023-10-18T10:46:23.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.298+0000 7f6863398640 1 -- 172.21.15.89:0/875257744 shutdown_connections 2023-10-18T10:46:23.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.298+0000 7f6863398640 1 --2- 172.21.15.89:0/875257744 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6840065680 0x7f6840067b40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:23.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.298+0000 7f6863398640 1 --2- 172.21.15.89:0/875257744 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f685c15c710 0x7f685c15d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:23.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.298+0000 7f6863398640 1 --2- 172.21.15.89:0/875257744 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f685c155fc0 0x7f685c15d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:23.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.298+0000 7f6863398640 1 --2- 172.21.15.89:0/875257744 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f685c15c330 0x7f685c155a80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:23.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.298+0000 7f6863398640 1 -- 172.21.15.89:0/875257744 >> 172.21.15.89:0/875257744 conn(0x7f685c070320 msgr2=0x7f685c0b6840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:23.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.298+0000 7f6863398640 1 -- 172.21.15.89:0/875257744 shutdown_connections 2023-10-18T10:46:23.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.298+0000 7f6863398640 1 -- 172.21.15.89:0/875257744 wait complete. 2023-10-18T10:46:23.319 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-18T10:46:23.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.458+0000 7fba7afdb640 1 Processor -- start 2023-10-18T10:46:23.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.458+0000 7fba7afdb640 1 -- start start 2023-10-18T10:46:23.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.458+0000 7fba7afdb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba740d0bc0 0x7fba740cd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:23.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.458+0000 7fba7afdb640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba7415da50 con 0x7fba7415c940 2023-10-18T10:46:23.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.458+0000 7fba7afdb640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba740d3110 con 0x7fba740d0760 2023-10-18T10:46:23.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.458+0000 7fba7afdb640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba7415ce40 con 0x7fba740d0bc0 2023-10-18T10:46:23.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.458+0000 7fba78d77640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fba7415c940 0x7fba740d2a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:45684/0 (socket says 172.21.15.89:45684) 2023-10-18T10:46:23.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.458+0000 7fba78d77640 1 -- 172.21.15.89:0/2610548115 learned_addr learned my addr 172.21.15.89:0/2610548115 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:23.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.458+0000 7fba79578640 1 --2- 172.21.15.89:0/2610548115 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba740d0bc0 0x7fba740cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:23.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.458+0000 7fba737fe640 1 -- 172.21.15.89:0/2610548115 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 816854408 0 0) 0x7fba7415da50 con 0x7fba7415c940 2023-10-18T10:46:23.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.458+0000 7fba737fe640 1 -- 172.21.15.89:0/2610548115 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba60003530 con 0x7fba7415c940 2023-10-18T10:46:23.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.458+0000 7fba737fe640 1 -- 172.21.15.89:0/2610548115 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 213455804 0 0) 0x7fba740d3110 con 0x7fba740d0760 2023-10-18T10:46:23.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.458+0000 7fba737fe640 1 -- 172.21.15.89:0/2610548115 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba7415da50 con 0x7fba740d0760 2023-10-18T10:46:23.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.458+0000 7fba79578640 1 -- 172.21.15.89:0/2610548115 >> v1:172.21.15.154:6789/0 conn(0x7fba740d0760 legacy=0x7fba740cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:23.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.458+0000 7fba79578640 1 -- 172.21.15.89:0/2610548115 >> v1:172.21.15.89:6789/0 conn(0x7fba7415c940 legacy=0x7fba740d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:23.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.458+0000 7fba737fe640 1 -- 172.21.15.89:0/2610548115 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 336692191 0 0) 0x7fba60003530 con 0x7fba7415c940 2023-10-18T10:46:23.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.458+0000 7fba79578640 1 -- 172.21.15.89:0/2610548115 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba7415d120 con 0x7fba740d0bc0 2023-10-18T10:46:23.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.462+0000 7fba79578640 1 --2- 172.21.15.89:0/2610548115 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba740d0bc0 0x7fba740cd870 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fba6c009dc0 tx=0x7fba6c01c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd685e9be898a0b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:23.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.462+0000 7fba737fe640 1 -- 172.21.15.89:0/2610548115 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba64001680 con 0x7fba740d0bc0 2023-10-18T10:46:23.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.462+0000 7fba737fe640 1 -- 172.21.15.89:0/2610548115 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba740ce5c0 con 0x7fba740d0bc0 2023-10-18T10:46:23.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.462+0000 7fba737fe640 1 -- 172.21.15.89:0/2610548115 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c0a58c0 con 0x7fba740d0bc0 2023-10-18T10:46:23.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.462+0000 7fba7afdb640 1 -- 172.21.15.89:0/2610548115 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba740d0bc0 msgr2=0x7fba740cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:23.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.462+0000 7fba7afdb640 1 --2- 172.21.15.89:0/2610548115 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba740d0bc0 0x7fba740cd870 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fba6c009dc0 tx=0x7fba6c01c860 comp rx=0 tx=0).stop 2023-10-18T10:46:23.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.462+0000 7fba7afdb640 1 -- 172.21.15.89:0/2610548115 shutdown_connections 2023-10-18T10:46:23.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.462+0000 7fba7afdb640 1 --2- 172.21.15.89:0/2610548115 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba740d0bc0 0x7fba740cd870 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:23.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.462+0000 7fba7afdb640 1 -- 172.21.15.89:0/2610548115 >> 172.21.15.89:0/2610548115 conn(0x7fba74089fd0 msgr2=0x7fba7409b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:23.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.462+0000 7fba7afdb640 1 -- 172.21.15.89:0/2610548115 shutdown_connections 2023-10-18T10:46:23.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.462+0000 7fba7afdb640 1 -- 172.21.15.89:0/2610548115 wait complete. 2023-10-18T10:46:23.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.466+0000 7fba7afdb640 1 Processor -- start 2023-10-18T10:46:23.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.466+0000 7fba7afdb640 1 -- start start 2023-10-18T10:46:23.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.466+0000 7fba7afdb640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba740d0760 0x7fba740b7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:23.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.466+0000 7fba7afdb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba740d0bc0 0x7fba741555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:23.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.466+0000 7fba7afdb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba740c63e0 0x7fba740c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:23.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.466+0000 7fba7afdb640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba74153a90 con 0x7fba740d0760 2023-10-18T10:46:23.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.466+0000 7fba7afdb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba740c6e50 con 0x7fba740d0bc0 2023-10-18T10:46:23.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.466+0000 7fba7afdb640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba740c7130 con 0x7fba740c63e0 2023-10-18T10:46:23.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.466+0000 7fba78d77640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba740d0760 0x7fba740b7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:23.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.466+0000 7fba78d77640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba740d0760 0x7fba740b7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59422/0 (socket says 172.21.15.89:59422) 2023-10-18T10:46:23.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.466+0000 7fba78d77640 1 -- 172.21.15.89:0/2217112510 learned_addr learned my addr 172.21.15.89:0/2217112510 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:23.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.466+0000 7fba78d77640 1 -- 172.21.15.89:0/2217112510 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba740c63e0 msgr2=0x7fba740c6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:23.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.466+0000 7fba79578640 1 --2- 172.21.15.89:0/2217112510 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba740c63e0 0x7fba740c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:23.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.466+0000 7fba73fff640 1 --2- 172.21.15.89:0/2217112510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba740d0bc0 0x7fba741555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:23.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.466+0000 7fba78d77640 1 --2- 172.21.15.89:0/2217112510 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba740c63e0 0x7fba740c6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:23.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.466+0000 7fba78d77640 1 -- 172.21.15.89:0/2217112510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba740d0bc0 msgr2=0x7fba741555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:23.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.466+0000 7fba78d77640 1 --2- 172.21.15.89:0/2217112510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba740d0bc0 0x7fba741555c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:23.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.466+0000 7fba78d77640 1 -- 172.21.15.89:0/2217112510 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba6c009a70 con 0x7fba740d0760 2023-10-18T10:46:23.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.466+0000 7fba79578640 1 --2- 172.21.15.89:0/2217112510 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba740c63e0 0x7fba740c6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:23.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.466+0000 7fba73fff640 1 --2- 172.21.15.89:0/2217112510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba740d0bc0 0x7fba741555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:23.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.466+0000 7fba78d77640 1 --2- 172.21.15.89:0/2217112510 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba740d0760 0x7fba740b7c80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fba6400c750 tx=0x7fba6400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:23.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.466+0000 7fba717fa640 1 -- 172.21.15.89:0/2217112510 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba64005ae0 con 0x7fba740d0760 2023-10-18T10:46:23.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.466+0000 7fba717fa640 1 -- 172.21.15.89:0/2217112510 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba64005c40 con 0x7fba740d0760 2023-10-18T10:46:23.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.466+0000 7fba7afdb640 1 -- 172.21.15.89:0/2217112510 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba740cb220 con 0x7fba740d0760 2023-10-18T10:46:23.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.470+0000 7fba717fa640 1 -- 172.21.15.89:0/2217112510 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba64007070 con 0x7fba740d0760 2023-10-18T10:46:23.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.470+0000 7fba7afdb640 1 -- 172.21.15.89:0/2217112510 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba74155b00 con 0x7fba740d0760 2023-10-18T10:46:23.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.470+0000 7fba7afdb640 1 -- 172.21.15.89:0/2217112510 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba7404dbe0 con 0x7fba740d0760 2023-10-18T10:46:23.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.470+0000 7fba717fa640 1 -- 172.21.15.89:0/2217112510 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fba640071d0 con 0x7fba740d0760 2023-10-18T10:46:23.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.470+0000 7fba717fa640 1 --2- 172.21.15.89:0/2217112510 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fba54065680 0x7fba54067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:23.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.470+0000 7fba717fa640 1 -- 172.21.15.89:0/2217112510 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fba64082f20 con 0x7fba740d0760 2023-10-18T10:46:23.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.470+0000 7fba73fff640 1 --2- 172.21.15.89:0/2217112510 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fba54065680 0x7fba54067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:23.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.470+0000 7fba73fff640 1 --2- 172.21.15.89:0/2217112510 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fba54065680 0x7fba54067b40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fba740c5d30 tx=0x7fba5c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:23.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.474+0000 7fba717fa640 1 -- 172.21.15.89:0/2217112510 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fba64050660 con 0x7fba740d0760 2023-10-18T10:46:23.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.622+0000 7fba7afdb640 1 -- 172.21.15.89:0/2217112510 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fba740d3330 con 0x7fba740d0760 2023-10-18T10:46:23.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.622+0000 7fba717fa640 1 -- 172.21.15.89:0/2217112510 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7fba64053c80 con 0x7fba740d0760 2023-10-18T10:46:23.626 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-10-18T10:46:23.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.626+0000 7fba7afdb640 1 -- 172.21.15.89:0/2217112510 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fba54065680 msgr2=0x7fba54067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:23.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.626+0000 7fba7afdb640 1 --2- 172.21.15.89:0/2217112510 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fba54065680 0x7fba54067b40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fba740c5d30 tx=0x7fba5c009040 comp rx=0 tx=0).stop 2023-10-18T10:46:23.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.626+0000 7fba7afdb640 1 -- 172.21.15.89:0/2217112510 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba740d0760 msgr2=0x7fba740b7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:23.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.626+0000 7fba7afdb640 1 --2- 172.21.15.89:0/2217112510 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba740d0760 0x7fba740b7c80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fba6400c750 tx=0x7fba6400cc10 comp rx=0 tx=0).stop 2023-10-18T10:46:23.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.626+0000 7fba7afdb640 1 -- 172.21.15.89:0/2217112510 shutdown_connections 2023-10-18T10:46:23.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.626+0000 7fba7afdb640 1 --2- 172.21.15.89:0/2217112510 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fba54065680 0x7fba54067b40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:23.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.626+0000 7fba7afdb640 1 --2- 172.21.15.89:0/2217112510 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba740c63e0 0x7fba740c6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:23.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.626+0000 7fba7afdb640 1 --2- 172.21.15.89:0/2217112510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba740d0bc0 0x7fba741555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:23.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.626+0000 7fba7afdb640 1 --2- 172.21.15.89:0/2217112510 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba740d0760 0x7fba740b7c80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:23.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.626+0000 7fba7afdb640 1 -- 172.21.15.89:0/2217112510 >> 172.21.15.89:0/2217112510 conn(0x7fba74089fd0 msgr2=0x7fba741583a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:23.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.626+0000 7fba7afdb640 1 -- 172.21.15.89:0/2217112510 shutdown_connections 2023-10-18T10:46:23.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.626+0000 7fba7afdb640 1 -- 172.21.15.89:0/2217112510 wait complete. 2023-10-18T10:46:23.647 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:46:23.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.786+0000 7fdf81ad8640 1 Processor -- start 2023-10-18T10:46:23.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.786+0000 7fdf81ad8640 1 -- start start 2023-10-18T10:46:23.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.786+0000 7fdf81ad8640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdf7c0d0760 0x7fdf7c0d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:23.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.786+0000 7fdf81ad8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf7c0cd160 0x7fdf7c0cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:23.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.786+0000 7fdf81ad8640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf7c0d1100 con 0x7fdf7c15c940 2023-10-18T10:46:23.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.786+0000 7fdf81ad8640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf7c15da50 con 0x7fdf7c0d0760 2023-10-18T10:46:23.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.786+0000 7fdf81ad8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c0d3110 con 0x7fdf7c0cd160 2023-10-18T10:46:23.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.790+0000 7fdf7b7fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fdf7c15c940 0x7fdf7c0d2a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:45688/0 (socket says 172.21.15.89:45688) 2023-10-18T10:46:23.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.790+0000 7fdf7b7fe640 1 -- 172.21.15.89:0/810359108 learned_addr learned my addr 172.21.15.89:0/810359108 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:23.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.790+0000 7fdf7affd640 1 --2- 172.21.15.89:0/810359108 >> v2:172.21.15.89:3300/0 conn(0x7fdf7c0d0760 0x7fdf7c0d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:23.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.790+0000 7fdf7bfff640 1 --2- 172.21.15.89:0/810359108 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf7c0cd160 0x7fdf7c0cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:23.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.790+0000 7fdf7affd640 1 -- 172.21.15.89:0/810359108 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf7c0cd160 msgr2=0x7fdf7c0cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:23.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.790+0000 7fdf7affd640 1 --2- 172.21.15.89:0/810359108 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf7c0cd160 0x7fdf7c0cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:23.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.790+0000 7fdf7affd640 1 -- 172.21.15.89:0/810359108 >> v1:172.21.15.89:6789/0 conn(0x7fdf7c15c940 legacy=0x7fdf7c0d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:23.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.790+0000 7fdf7affd640 1 -- 172.21.15.89:0/810359108 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf7c15ce40 con 0x7fdf7c0d0760 2023-10-18T10:46:23.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.790+0000 7fdf7a7fc640 1 -- 172.21.15.89:0/810359108 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4207967144 0 0) 0x7fdf7c0d1100 con 0x7fdf7c15c940 2023-10-18T10:46:23.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.794+0000 7fdf7affd640 1 --2- 172.21.15.89:0/810359108 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf7c0d0760 0x7fdf7c0d0bc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdf700097c0 tx=0x7fdf7001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d07fb2f5a0ccc server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:23.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.794+0000 7fdf7a7fc640 1 -- 172.21.15.89:0/810359108 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf700ab650 con 0x7fdf7c0d0760 2023-10-18T10:46:23.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.794+0000 7fdf7a7fc640 1 -- 172.21.15.89:0/810359108 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf700ab7b0 con 0x7fdf7c0d0760 2023-10-18T10:46:23.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.794+0000 7fdf7a7fc640 1 -- 172.21.15.89:0/810359108 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf700abb40 con 0x7fdf7c0d0760 2023-10-18T10:46:23.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.794+0000 7fdf81ad8640 1 -- 172.21.15.89:0/810359108 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf7c0d0760 msgr2=0x7fdf7c0d0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:23.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.794+0000 7fdf81ad8640 1 --2- 172.21.15.89:0/810359108 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf7c0d0760 0x7fdf7c0d0bc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdf700097c0 tx=0x7fdf7001c450 comp rx=0 tx=0).stop 2023-10-18T10:46:23.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.794+0000 7fdf81ad8640 1 -- 172.21.15.89:0/810359108 shutdown_connections 2023-10-18T10:46:23.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.794+0000 7fdf81ad8640 1 --2- 172.21.15.89:0/810359108 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf7c0cd160 0x7fdf7c0cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:23.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.794+0000 7fdf81ad8640 1 --2- 172.21.15.89:0/810359108 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf7c0d0760 0x7fdf7c0d0bc0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:23.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.794+0000 7fdf81ad8640 1 -- 172.21.15.89:0/810359108 >> 172.21.15.89:0/810359108 conn(0x7fdf7c089fd0 msgr2=0x7fdf7c09b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:23.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.794+0000 7fdf81ad8640 1 -- 172.21.15.89:0/810359108 shutdown_connections 2023-10-18T10:46:23.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.794+0000 7fdf81ad8640 1 -- 172.21.15.89:0/810359108 wait complete. 2023-10-18T10:46:23.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.794+0000 7fdf81ad8640 1 Processor -- start 2023-10-18T10:46:23.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.794+0000 7fdf81ad8640 1 -- start start 2023-10-18T10:46:23.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.798+0000 7fdf81ad8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf7c0cd160 0x7fdf7c0b7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:23.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.798+0000 7fdf81ad8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf7c0d0760 0x7fdf7c0b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:23.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.798+0000 7fdf81ad8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf7c0b4380 0x7fdf7c0b5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:23.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.798+0000 7fdf81ad8640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c0b4810 con 0x7fdf7c0cd160 2023-10-18T10:46:23.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.798+0000 7fdf81ad8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c0b80e0 con 0x7fdf7c0b4380 2023-10-18T10:46:23.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.798+0000 7fdf81ad8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c15a200 con 0x7fdf7c0d0760 2023-10-18T10:46:23.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.798+0000 7fdf7b7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf7c0cd160 0x7fdf7c0b7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:23.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.798+0000 7fdf7b7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf7c0cd160 0x7fdf7c0b7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59428/0 (socket says 172.21.15.89:59428) 2023-10-18T10:46:23.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.798+0000 7fdf7b7fe640 1 -- 172.21.15.89:0/773926902 learned_addr learned my addr 172.21.15.89:0/773926902 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:23.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.798+0000 7fdf7affd640 1 --2- 172.21.15.89:0/773926902 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf7c0d0760 0x7fdf7c0b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:23.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.798+0000 7fdf7bfff640 1 --2- 172.21.15.89:0/773926902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf7c0b4380 0x7fdf7c0b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:23.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.798+0000 7fdf7b7fe640 1 -- 172.21.15.89:0/773926902 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf7c0d0760 msgr2=0x7fdf7c0b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:23.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.798+0000 7fdf7b7fe640 1 --2- 172.21.15.89:0/773926902 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf7c0d0760 0x7fdf7c0b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:23.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.798+0000 7fdf7b7fe640 1 -- 172.21.15.89:0/773926902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf7c0b4380 msgr2=0x7fdf7c0b5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:23.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.798+0000 7fdf7b7fe640 1 --2- 172.21.15.89:0/773926902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf7c0b4380 0x7fdf7c0b5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:23.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.798+0000 7fdf7b7fe640 1 -- 172.21.15.89:0/773926902 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf70009420 con 0x7fdf7c0cd160 2023-10-18T10:46:23.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.798+0000 7fdf7b7fe640 1 --2- 172.21.15.89:0/773926902 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf7c0cd160 0x7fdf7c0b7ba0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c00c480 tx=0x7fdf6c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:23.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.798+0000 7fdf63fff640 1 -- 172.21.15.89:0/773926902 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c0074e0 con 0x7fdf7c0cd160 2023-10-18T10:46:23.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.798+0000 7fdf63fff640 1 -- 172.21.15.89:0/773926902 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf6c007660 con 0x7fdf7c0cd160 2023-10-18T10:46:23.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.798+0000 7fdf63fff640 1 -- 172.21.15.89:0/773926902 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c004a30 con 0x7fdf7c0cd160 2023-10-18T10:46:23.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.798+0000 7fdf81ad8640 1 -- 172.21.15.89:0/773926902 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf7c15a4a0 con 0x7fdf7c0cd160 2023-10-18T10:46:23.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.798+0000 7fdf81ad8640 1 -- 172.21.15.89:0/773926902 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf7c15aa60 con 0x7fdf7c0cd160 2023-10-18T10:46:23.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.798+0000 7fdf81ad8640 1 -- 172.21.15.89:0/773926902 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf7c0c95d0 con 0x7fdf7c0cd160 2023-10-18T10:46:23.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.802+0000 7fdf63fff640 1 -- 172.21.15.89:0/773926902 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fdf6c004b90 con 0x7fdf7c0cd160 2023-10-18T10:46:23.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.802+0000 7fdf63fff640 1 --2- 172.21.15.89:0/773926902 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdf68065680 0x7fdf68067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:23.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.802+0000 7fdf63fff640 1 -- 172.21.15.89:0/773926902 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdf6c082220 con 0x7fdf7c0cd160 2023-10-18T10:46:23.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.802+0000 7fdf7affd640 1 --2- 172.21.15.89:0/773926902 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdf68065680 0x7fdf68067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:23.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.802+0000 7fdf7affd640 1 --2- 172.21.15.89:0/773926902 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdf68065680 0x7fdf68067b40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdf700af000 tx=0x7fdf700a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:23.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.806+0000 7fdf63fff640 1 -- 172.21.15.89:0/773926902 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fdf6c04fa90 con 0x7fdf7c0cd160 2023-10-18T10:46:23.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.954+0000 7fdf81ad8640 1 -- 172.21.15.89:0/773926902 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdf7c04dbe0 con 0x7fdf7c0cd160 2023-10-18T10:46:23.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.954+0000 7fdf63fff640 1 -- 172.21.15.89:0/773926902 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdf6c0530b0 con 0x7fdf7c0cd160 2023-10-18T10:46:23.957 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-18T10:46:23.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.958+0000 7fdf81ad8640 1 -- 172.21.15.89:0/773926902 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdf68065680 msgr2=0x7fdf68067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:23.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.958+0000 7fdf81ad8640 1 --2- 172.21.15.89:0/773926902 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdf68065680 0x7fdf68067b40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdf700af000 tx=0x7fdf700a7000 comp rx=0 tx=0).stop 2023-10-18T10:46:23.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.958+0000 7fdf81ad8640 1 -- 172.21.15.89:0/773926902 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf7c0cd160 msgr2=0x7fdf7c0b7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:23.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.958+0000 7fdf81ad8640 1 --2- 172.21.15.89:0/773926902 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf7c0cd160 0x7fdf7c0b7ba0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c00c480 tx=0x7fdf6c00c940 comp rx=0 tx=0).stop 2023-10-18T10:46:23.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.958+0000 7fdf81ad8640 1 -- 172.21.15.89:0/773926902 shutdown_connections 2023-10-18T10:46:23.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.958+0000 7fdf81ad8640 1 --2- 172.21.15.89:0/773926902 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdf68065680 0x7fdf68067b40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:23.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.958+0000 7fdf81ad8640 1 --2- 172.21.15.89:0/773926902 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf7c0d0760 0x7fdf7c0b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:23.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.958+0000 7fdf81ad8640 1 --2- 172.21.15.89:0/773926902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf7c0b4380 0x7fdf7c0b5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:23.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.958+0000 7fdf81ad8640 1 --2- 172.21.15.89:0/773926902 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf7c0cd160 0x7fdf7c0b7ba0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:23.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.958+0000 7fdf81ad8640 1 -- 172.21.15.89:0/773926902 >> 172.21.15.89:0/773926902 conn(0x7fdf7c089fd0 msgr2=0x7fdf7c15e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:23.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.958+0000 7fdf81ad8640 1 -- 172.21.15.89:0/773926902 shutdown_connections 2023-10-18T10:46:23.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:23.958+0000 7fdf81ad8640 1 -- 172.21.15.89:0/773926902 wait complete. 2023-10-18T10:46:23.978 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:46:24.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.114+0000 7fc9982e5640 1 Processor -- start 2023-10-18T10:46:24.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.118+0000 7fc9982e5640 1 -- start start 2023-10-18T10:46:24.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.118+0000 7fc9982e5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9900ce780 0x7fc990143470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:24.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.118+0000 7fc9982e5640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9900cd8d0 con 0x7fc990145b70 2023-10-18T10:46:24.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.118+0000 7fc9982e5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc99015cbf0 con 0x7fc9900cd4f0 2023-10-18T10:46:24.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.118+0000 7fc9982e5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc990069900 con 0x7fc9900ce780 2023-10-18T10:46:24.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.118+0000 7fc995880640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc990145b70 0x7fc9900ce070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:45694/0 (socket says 172.21.15.89:45694) 2023-10-18T10:46:24.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.118+0000 7fc995880640 1 -- 172.21.15.89:0/3012293343 learned_addr learned my addr 172.21.15.89:0/3012293343 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:24.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.118+0000 7fc996882640 1 --2- 172.21.15.89:0/3012293343 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9900ce780 0x7fc990143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:24.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.118+0000 7fc99507f640 1 -- 172.21.15.89:0/3012293343 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2768587208 0 0) 0x7fc9900cd8d0 con 0x7fc990145b70 2023-10-18T10:46:24.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.118+0000 7fc99507f640 1 -- 172.21.15.89:0/3012293343 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc97c003530 con 0x7fc990145b70 2023-10-18T10:46:24.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.118+0000 7fc99507f640 1 -- 172.21.15.89:0/3012293343 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4169510240 0 0) 0x7fc99015cbf0 con 0x7fc9900cd4f0 2023-10-18T10:46:24.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.118+0000 7fc99507f640 1 -- 172.21.15.89:0/3012293343 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9900cd8d0 con 0x7fc9900cd4f0 2023-10-18T10:46:24.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.118+0000 7fc99507f640 1 -- 172.21.15.89:0/3012293343 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1372690022 0 0) 0x7fc97c003530 con 0x7fc990145b70 2023-10-18T10:46:24.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.118+0000 7fc99507f640 1 -- 172.21.15.89:0/3012293343 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9900ce780 msgr2=0x7fc990143470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:24.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.118+0000 7fc99507f640 1 --2- 172.21.15.89:0/3012293343 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9900ce780 0x7fc990143470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:24.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.118+0000 7fc99507f640 1 -- 172.21.15.89:0/3012293343 >> v1:172.21.15.154:6789/0 conn(0x7fc9900cd4f0 legacy=0x7fc990160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:24.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.118+0000 7fc99507f640 1 -- 172.21.15.89:0/3012293343 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc990069b10 con 0x7fc990145b70 2023-10-18T10:46:24.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.118+0000 7fc99507f640 1 -- 172.21.15.89:0/3012293343 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7fc980001680 con 0x7fc990145b70 2023-10-18T10:46:24.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.118+0000 7fc99507f640 1 -- 172.21.15.89:0/3012293343 >> v1:172.21.15.89:6789/0 conn(0x7fc990145b70 legacy=0x7fc9900ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:24.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.118+0000 7fc99507f640 1 --2- 172.21.15.89:0/3012293343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc97c003b40 0x7fc97c003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:24.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.118+0000 7fc99507f640 1 --2- 172.21.15.89:0/3012293343 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9900ce780 0x7fc97c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:24.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.118+0000 7fc99507f640 1 --2- 172.21.15.89:0/3012293343 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc97c004d10 0x7fc97c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:24.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.118+0000 7fc99507f640 1 -- 172.21.15.89:0/3012293343 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc990069900 con 0x7fc97c003b40 2023-10-18T10:46:24.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.118+0000 7fc99507f640 1 -- 172.21.15.89:0/3012293343 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c00ba40 con 0x7fc97c004d10 2023-10-18T10:46:24.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.118+0000 7fc99507f640 1 -- 172.21.15.89:0/3012293343 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c00bc00 con 0x7fc9900ce780 2023-10-18T10:46:24.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.118+0000 7fc996081640 1 --2- 172.21.15.89:0/3012293343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc97c003b40 0x7fc97c003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:24.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.118+0000 7fc996882640 1 --2- 172.21.15.89:0/3012293343 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc97c004d10 0x7fc97c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:24.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.118+0000 7fc995880640 1 --2- 172.21.15.89:0/3012293343 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9900ce780 0x7fc97c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:24.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.118+0000 7fc996081640 1 -- 172.21.15.89:0/3012293343 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9900ce780 msgr2=0x7fc97c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:24.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.118+0000 7fc996081640 1 --2- 172.21.15.89:0/3012293343 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9900ce780 0x7fc97c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:24.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.118+0000 7fc996081640 1 -- 172.21.15.89:0/3012293343 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc97c004d10 msgr2=0x7fc97c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:24.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.118+0000 7fc996081640 1 --2- 172.21.15.89:0/3012293343 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc97c004d10 0x7fc97c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:24.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.118+0000 7fc996081640 1 -- 172.21.15.89:0/3012293343 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc97c003530 con 0x7fc97c003b40 2023-10-18T10:46:24.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.122+0000 7fc996081640 1 --2- 172.21.15.89:0/3012293343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc97c003b40 0x7fc97c003f40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc9900cda60 tx=0x7fc98c01c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3a614cfeac4de7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:24.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.122+0000 7fc99507f640 1 -- 172.21.15.89:0/3012293343 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c003b80 con 0x7fc97c003b40 2023-10-18T10:46:24.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.122+0000 7fc99507f640 1 -- 172.21.15.89:0/3012293343 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98c003ce0 con 0x7fc97c003b40 2023-10-18T10:46:24.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.122+0000 7fc99507f640 1 -- 172.21.15.89:0/3012293343 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c005280 con 0x7fc97c003b40 2023-10-18T10:46:24.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.122+0000 7fc9982e5640 1 -- 172.21.15.89:0/3012293343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc97c003b40 msgr2=0x7fc97c003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:24.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.122+0000 7fc9982e5640 1 --2- 172.21.15.89:0/3012293343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc97c003b40 0x7fc97c003f40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc9900cda60 tx=0x7fc98c01c890 comp rx=0 tx=0).stop 2023-10-18T10:46:24.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.122+0000 7fc996882640 1 -- 172.21.15.89:0/3012293343 reap_dead start 2023-10-18T10:46:24.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.122+0000 7fc9982e5640 1 -- 172.21.15.89:0/3012293343 shutdown_connections 2023-10-18T10:46:24.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.122+0000 7fc9982e5640 1 -- 172.21.15.89:0/3012293343 >> 172.21.15.89:0/3012293343 conn(0x7fc99006dad0 msgr2=0x7fc99006dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:24.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.122+0000 7fc9982e5640 1 -- 172.21.15.89:0/3012293343 shutdown_connections 2023-10-18T10:46:24.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.122+0000 7fc9982e5640 1 -- 172.21.15.89:0/3012293343 wait complete. 2023-10-18T10:46:24.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.126+0000 7fc9982e5640 1 Processor -- start 2023-10-18T10:46:24.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.126+0000 7fc9982e5640 1 -- start start 2023-10-18T10:46:24.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.126+0000 7fc9982e5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc97c003b40 0x7fc990158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:24.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.126+0000 7fc9982e5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9900c2e80 0x7fc9900cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:24.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.126+0000 7fc9982e5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc990143d30 0x7fc9900cc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:24.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.126+0000 7fc9982e5640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9900c6b90 con 0x7fc990143d30 2023-10-18T10:46:24.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.126+0000 7fc9982e5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9900c6260 con 0x7fc97c003b40 2023-10-18T10:46:24.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.126+0000 7fc9982e5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9900c6540 con 0x7fc9900c2e80 2023-10-18T10:46:24.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.126+0000 7fc996882640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc990143d30 0x7fc9900cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:24.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.126+0000 7fc996882640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc990143d30 0x7fc9900cc190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59440/0 (socket says 172.21.15.89:59440) 2023-10-18T10:46:24.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.126+0000 7fc996882640 1 -- 172.21.15.89:0/1014166548 learned_addr learned my addr 172.21.15.89:0/1014166548 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:24.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.126+0000 7fc996081640 1 --2- 172.21.15.89:0/1014166548 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc97c003b40 0x7fc990158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:24.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.126+0000 7fc995880640 1 --2- 172.21.15.89:0/1014166548 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9900c2e80 0x7fc9900cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:24.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.126+0000 7fc996882640 1 -- 172.21.15.89:0/1014166548 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9900c2e80 msgr2=0x7fc9900cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:24.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.126+0000 7fc996882640 1 --2- 172.21.15.89:0/1014166548 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9900c2e80 0x7fc9900cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:24.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.126+0000 7fc996882640 1 -- 172.21.15.89:0/1014166548 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc97c003b40 msgr2=0x7fc990158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:24.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.126+0000 7fc996882640 1 --2- 172.21.15.89:0/1014166548 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc97c003b40 0x7fc990158490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:24.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.126+0000 7fc996882640 1 -- 172.21.15.89:0/1014166548 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9900c6820 con 0x7fc990143d30 2023-10-18T10:46:24.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.126+0000 7fc996882640 1 --2- 172.21.15.89:0/1014166548 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc990143d30 0x7fc9900cc190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc990143130 tx=0x7fc988011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:24.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.126+0000 7fc986ffd640 1 -- 172.21.15.89:0/1014166548 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc988004240 con 0x7fc990143d30 2023-10-18T10:46:24.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.126+0000 7fc986ffd640 1 -- 172.21.15.89:0/1014166548 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9880043c0 con 0x7fc990143d30 2023-10-18T10:46:24.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.126+0000 7fc986ffd640 1 -- 172.21.15.89:0/1014166548 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc988004530 con 0x7fc990143d30 2023-10-18T10:46:24.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.126+0000 7fc9982e5640 1 -- 172.21.15.89:0/1014166548 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9901589d0 con 0x7fc990143d30 2023-10-18T10:46:24.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.126+0000 7fc9982e5640 1 -- 172.21.15.89:0/1014166548 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc99015c2f0 con 0x7fc990143d30 2023-10-18T10:46:24.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.130+0000 7fc9982e5640 1 -- 172.21.15.89:0/1014166548 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc99004dbe0 con 0x7fc990143d30 2023-10-18T10:46:24.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.130+0000 7fc986ffd640 1 -- 172.21.15.89:0/1014166548 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fc98800f9b0 con 0x7fc990143d30 2023-10-18T10:46:24.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.130+0000 7fc986ffd640 1 --2- 172.21.15.89:0/1014166548 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc9740656b0 0x7fc974067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:24.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.130+0000 7fc986ffd640 1 -- 172.21.15.89:0/1014166548 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc9880831b0 con 0x7fc990143d30 2023-10-18T10:46:24.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.130+0000 7fc996081640 1 --2- 172.21.15.89:0/1014166548 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc9740656b0 0x7fc974067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:24.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.130+0000 7fc996081640 1 --2- 172.21.15.89:0/1014166548 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc9740656b0 0x7fc974067b70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc98c005000 tx=0x7fc98c0016f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:24.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.134+0000 7fc986ffd640 1 -- 172.21.15.89:0/1014166548 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fc988050a30 con 0x7fc990143d30 2023-10-18T10:46:24.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.282+0000 7fc9982e5640 1 -- 172.21.15.89:0/1014166548 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc9900c69b0 con 0x7fc990143d30 2023-10-18T10:46:24.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.282+0000 7fc986ffd640 1 -- 172.21.15.89:0/1014166548 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc988054050 con 0x7fc990143d30 2023-10-18T10:46:24.285 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-18T10:46:24.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.286+0000 7fc9982e5640 1 -- 172.21.15.89:0/1014166548 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc9740656b0 msgr2=0x7fc974067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:24.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.286+0000 7fc9982e5640 1 --2- 172.21.15.89:0/1014166548 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc9740656b0 0x7fc974067b70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc98c005000 tx=0x7fc98c0016f0 comp rx=0 tx=0).stop 2023-10-18T10:46:24.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.286+0000 7fc9982e5640 1 -- 172.21.15.89:0/1014166548 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc990143d30 msgr2=0x7fc9900cc190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:24.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.286+0000 7fc9982e5640 1 --2- 172.21.15.89:0/1014166548 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc990143d30 0x7fc9900cc190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc990143130 tx=0x7fc988011cf0 comp rx=0 tx=0).stop 2023-10-18T10:46:24.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.286+0000 7fc9982e5640 1 -- 172.21.15.89:0/1014166548 shutdown_connections 2023-10-18T10:46:24.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.286+0000 7fc9982e5640 1 --2- 172.21.15.89:0/1014166548 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc9740656b0 0x7fc974067b70 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:24.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.286+0000 7fc9982e5640 1 --2- 172.21.15.89:0/1014166548 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9900c2e80 0x7fc9900cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:24.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.286+0000 7fc9982e5640 1 --2- 172.21.15.89:0/1014166548 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc990143d30 0x7fc9900cc190 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:24.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.286+0000 7fc9982e5640 1 --2- 172.21.15.89:0/1014166548 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc97c003b40 0x7fc990158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:24.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.286+0000 7fc9982e5640 1 -- 172.21.15.89:0/1014166548 >> 172.21.15.89:0/1014166548 conn(0x7fc99006dad0 msgr2=0x7fc9900c7670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:24.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.286+0000 7fc9982e5640 1 -- 172.21.15.89:0/1014166548 shutdown_connections 2023-10-18T10:46:24.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.286+0000 7fc9982e5640 1 -- 172.21.15.89:0/1014166548 wait complete. 2023-10-18T10:46:24.306 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T10:46:24.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.446+0000 7fc51f0cb640 1 Processor -- start 2023-10-18T10:46:24.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.446+0000 7fc51f0cb640 1 -- start start 2023-10-18T10:46:24.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.446+0000 7fc51f0cb640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc518159490 0x7fc51815d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:24.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.446+0000 7fc51f0cb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc518070600 0x7fc51806b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:24.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.446+0000 7fc51f0cb640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc51815ddf0 con 0x7fc51806bd10 2023-10-18T10:46:24.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.446+0000 7fc51f0cb640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc51806c1a0 con 0x7fc518159490 2023-10-18T10:46:24.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.446+0000 7fc51f0cb640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc518070b90 con 0x7fc518070600 2023-10-18T10:46:24.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.446+0000 7fc51ce67640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc518159490 0x7fc51815d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:24.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.446+0000 7fc50ffff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc518070600 0x7fc51806b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:24.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.446+0000 7fc51ce67640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc518159490 0x7fc51815d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:57958/0 (socket says 172.21.15.89:57958) 2023-10-18T10:46:24.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.446+0000 7fc51ce67640 1 -- 172.21.15.89:0/3931189022 learned_addr learned my addr 172.21.15.89:0/3931189022 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:24.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.446+0000 7fc51d668640 1 --1- 172.21.15.89:0/3931189022 >> v1:172.21.15.154:6789/0 conn(0x7fc51806bd10 0x7fc5180cbee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.89:37306/0 (socket says 172.21.15.89:37306) 2023-10-18T10:46:24.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.446+0000 7fc50ffff640 1 -- 172.21.15.89:0/3931189022 >> v2:172.21.15.154:3300/0 conn(0x7fc518159490 msgr2=0x7fc51815d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:24.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.446+0000 7fc50ffff640 1 --2- 172.21.15.89:0/3931189022 >> v2:172.21.15.154:3300/0 conn(0x7fc518159490 0x7fc51815d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:24.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.446+0000 7fc50ffff640 1 -- 172.21.15.89:0/3931189022 >> v1:172.21.15.154:6789/0 conn(0x7fc51806bd10 legacy=0x7fc5180cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:24.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.446+0000 7fc50ffff640 1 -- 172.21.15.89:0/3931189022 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc518070e70 con 0x7fc518070600 2023-10-18T10:46:24.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.450+0000 7fc50ffff640 1 --2- 172.21.15.89:0/3931189022 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc518070600 0x7fc51806b7d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc5080097c0 tx=0x7fc50801c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a126803948c0edf7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:24.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.450+0000 7fc50f7fe640 1 -- 172.21.15.89:0/3931189022 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5080ac620 con 0x7fc518070600 2023-10-18T10:46:24.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.450+0000 7fc50f7fe640 1 -- 172.21.15.89:0/3931189022 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5080ac780 con 0x7fc518070600 2023-10-18T10:46:24.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.450+0000 7fc50f7fe640 1 -- 172.21.15.89:0/3931189022 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5080acb10 con 0x7fc518070600 2023-10-18T10:46:24.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.450+0000 7fc51f0cb640 1 -- 172.21.15.89:0/3931189022 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc518070600 msgr2=0x7fc51806b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:24.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.450+0000 7fc51f0cb640 1 --2- 172.21.15.89:0/3931189022 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc518070600 0x7fc51806b7d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc5080097c0 tx=0x7fc50801c3c0 comp rx=0 tx=0).stop 2023-10-18T10:46:24.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.450+0000 7fc51f0cb640 1 -- 172.21.15.89:0/3931189022 shutdown_connections 2023-10-18T10:46:24.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.450+0000 7fc51f0cb640 1 --2- 172.21.15.89:0/3931189022 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc518070600 0x7fc51806b7d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:24.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.450+0000 7fc51f0cb640 1 --2- 172.21.15.89:0/3931189022 >> v2:172.21.15.154:3300/0 conn(0x7fc518159490 0x7fc51815d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:24.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.450+0000 7fc51f0cb640 1 -- 172.21.15.89:0/3931189022 >> 172.21.15.89:0/3931189022 conn(0x7fc518069f00 msgr2=0x7fc5180cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:24.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.450+0000 7fc51f0cb640 1 -- 172.21.15.89:0/3931189022 shutdown_connections 2023-10-18T10:46:24.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.450+0000 7fc51f0cb640 1 -- 172.21.15.89:0/3931189022 wait complete. 2023-10-18T10:46:24.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.450+0000 7fc51f0cb640 1 Processor -- start 2023-10-18T10:46:24.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.454+0000 7fc51f0cb640 1 -- start start 2023-10-18T10:46:24.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.454+0000 7fc51f0cb640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc51806bd10 0x7fc5181559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:24.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.454+0000 7fc51f0cb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc518070600 0x7fc5181573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:24.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.454+0000 7fc51f0cb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc518159490 0x7fc5181578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:24.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.454+0000 7fc51f0cb640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc518157f00 con 0x7fc51806bd10 2023-10-18T10:46:24.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.454+0000 7fc51f0cb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5181581e0 con 0x7fc518070600 2023-10-18T10:46:24.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.454+0000 7fc51f0cb640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5181584c0 con 0x7fc518159490 2023-10-18T10:46:24.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.454+0000 7fc51ce67640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc51806bd10 0x7fc5181559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.454+0000 7fc51ce67640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc51806bd10 0x7fc5181559a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59444/0 (socket says 172.21.15.89:59444) 2023-10-18T10:46:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.454+0000 7fc51ce67640 1 -- 172.21.15.89:0/523573125 learned_addr learned my addr 172.21.15.89:0/523573125 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.454+0000 7fc50ffff640 1 --2- 172.21.15.89:0/523573125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc518070600 0x7fc5181573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.454+0000 7fc51d668640 1 --2- 172.21.15.89:0/523573125 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc518159490 0x7fc5181578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.454+0000 7fc51ce67640 1 -- 172.21.15.89:0/523573125 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc518159490 msgr2=0x7fc5181578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:24.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.454+0000 7fc51ce67640 1 --2- 172.21.15.89:0/523573125 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc518159490 0x7fc5181578f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:24.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.454+0000 7fc51ce67640 1 -- 172.21.15.89:0/523573125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc518070600 msgr2=0x7fc5181573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:24.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.454+0000 7fc51ce67640 1 --2- 172.21.15.89:0/523573125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc518070600 0x7fc5181573b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:24.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.454+0000 7fc51ce67640 1 -- 172.21.15.89:0/523573125 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc508009420 con 0x7fc51806bd10 2023-10-18T10:46:24.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.454+0000 7fc51ce67640 1 --2- 172.21.15.89:0/523573125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc51806bd10 0x7fc5181559a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc51400e9d0 tx=0x7fc51400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:24.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.454+0000 7fc50d7fa640 1 -- 172.21.15.89:0/523573125 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51400de60 con 0x7fc51806bd10 2023-10-18T10:46:24.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.454+0000 7fc50d7fa640 1 -- 172.21.15.89:0/523573125 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc514010440 con 0x7fc51806bd10 2023-10-18T10:46:24.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.454+0000 7fc51f0cb640 1 -- 172.21.15.89:0/523573125 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5181587a0 con 0x7fc51806bd10 2023-10-18T10:46:24.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.454+0000 7fc50d7fa640 1 -- 172.21.15.89:0/523573125 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc514010810 con 0x7fc51806bd10 2023-10-18T10:46:24.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.454+0000 7fc51f0cb640 1 -- 172.21.15.89:0/523573125 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc518155ee0 con 0x7fc51806bd10 2023-10-18T10:46:24.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.454+0000 7fc51f0cb640 1 -- 172.21.15.89:0/523573125 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc518158cc0 con 0x7fc51806bd10 2023-10-18T10:46:24.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.458+0000 7fc50d7fa640 1 -- 172.21.15.89:0/523573125 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fc514010970 con 0x7fc51806bd10 2023-10-18T10:46:24.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.458+0000 7fc50d7fa640 1 --2- 172.21.15.89:0/523573125 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc504065680 0x7fc504067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:24.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.458+0000 7fc50d7fa640 1 -- 172.21.15.89:0/523573125 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc5140838a0 con 0x7fc51806bd10 2023-10-18T10:46:24.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.458+0000 7fc50ffff640 1 --2- 172.21.15.89:0/523573125 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc504065680 0x7fc504067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:24.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.458+0000 7fc50ffff640 1 --2- 172.21.15.89:0/523573125 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc504065680 0x7fc504067b40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc5080ae000 tx=0x7fc508008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:24.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.462+0000 7fc50d7fa640 1 -- 172.21.15.89:0/523573125 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fc518158cc0 con 0x7fc51806bd10 2023-10-18T10:46:24.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.610+0000 7fc51f0cb640 1 -- 172.21.15.89:0/523573125 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc518158cc0 con 0x7fc51806bd10 2023-10-18T10:46:24.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.610+0000 7fc50d7fa640 1 -- 172.21.15.89:0/523573125 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc514054af0 con 0x7fc51806bd10 2023-10-18T10:46:24.611 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-18T10:46:24.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.610+0000 7fc51f0cb640 1 -- 172.21.15.89:0/523573125 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc504065680 msgr2=0x7fc504067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:24.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.610+0000 7fc51f0cb640 1 --2- 172.21.15.89:0/523573125 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc504065680 0x7fc504067b40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc5080ae000 tx=0x7fc508008040 comp rx=0 tx=0).stop 2023-10-18T10:46:24.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.610+0000 7fc51f0cb640 1 -- 172.21.15.89:0/523573125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc51806bd10 msgr2=0x7fc5181559a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:24.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.610+0000 7fc51f0cb640 1 --2- 172.21.15.89:0/523573125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc51806bd10 0x7fc5181559a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc51400e9d0 tx=0x7fc51400ee90 comp rx=0 tx=0).stop 2023-10-18T10:46:24.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.610+0000 7fc51f0cb640 1 -- 172.21.15.89:0/523573125 shutdown_connections 2023-10-18T10:46:24.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.610+0000 7fc51f0cb640 1 --2- 172.21.15.89:0/523573125 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc504065680 0x7fc504067b40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:24.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.610+0000 7fc51f0cb640 1 --2- 172.21.15.89:0/523573125 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc518159490 0x7fc5181578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:24.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.610+0000 7fc51f0cb640 1 --2- 172.21.15.89:0/523573125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc518070600 0x7fc5181573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:24.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.610+0000 7fc51f0cb640 1 --2- 172.21.15.89:0/523573125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc51806bd10 0x7fc5181559a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:24.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.610+0000 7fc51f0cb640 1 -- 172.21.15.89:0/523573125 >> 172.21.15.89:0/523573125 conn(0x7fc518069f00 msgr2=0x7fc5180ca5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:24.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.610+0000 7fc51f0cb640 1 -- 172.21.15.89:0/523573125 shutdown_connections 2023-10-18T10:46:24.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.610+0000 7fc51f0cb640 1 -- 172.21.15.89:0/523573125 wait complete. 2023-10-18T10:46:24.633 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:46:24.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.770+0000 7f14e449c640 1 Processor -- start 2023-10-18T10:46:24.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.770+0000 7f14e449c640 1 -- start start 2023-10-18T10:46:24.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.770+0000 7f14e449c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f14dc159490 0x7f14dc15d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:24.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.770+0000 7f14e449c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f14dc06bee0 0x7f14dc0c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:24.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.770+0000 7f14e449c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14dc15ddf0 con 0x7f14dc070600 2023-10-18T10:46:24.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.770+0000 7f14e449c640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14dc070b90 con 0x7f14dc06bee0 2023-10-18T10:46:24.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.770+0000 7f14e449c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14dc070e70 con 0x7f14dc159490 2023-10-18T10:46:24.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.770+0000 7f14e2a39640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f14dc06bee0 0x7f14dc0c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:24.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.770+0000 7f14e2a39640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f14dc06bee0 0x7f14dc0c8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59448/0 (socket says 172.21.15.89:59448) 2023-10-18T10:46:24.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.770+0000 7f14e2a39640 1 -- 172.21.15.89:0/3492301731 learned_addr learned my addr 172.21.15.89:0/3492301731 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:24.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.770+0000 7f14e2238640 1 --2- 172.21.15.89:0/3492301731 >> v2:172.21.15.154:3300/0 conn(0x7f14dc159490 0x7f14dc15d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:24.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.770+0000 7f14e2a39640 1 -- 172.21.15.89:0/3492301731 >> v2:172.21.15.154:3300/0 conn(0x7f14dc159490 msgr2=0x7f14dc15d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:24.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.770+0000 7f14e2a39640 1 --2- 172.21.15.89:0/3492301731 >> v2:172.21.15.154:3300/0 conn(0x7f14dc159490 0x7f14dc15d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:24.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.770+0000 7f14e2a39640 1 -- 172.21.15.89:0/3492301731 >> v1:172.21.15.154:6789/0 conn(0x7f14dc070600 legacy=0x7f14dc06b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:24.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.770+0000 7f14e2a39640 1 -- 172.21.15.89:0/3492301731 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14dc071150 con 0x7f14dc06bee0 2023-10-18T10:46:24.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.774+0000 7f14e2a39640 1 --2- 172.21.15.89:0/3492301731 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14dc06bee0 0x7f14dc0c8960 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f14dc06c560 tx=0x7f14d401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9443a99dfb5d454c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:24.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.774+0000 7f14e1236640 1 -- 172.21.15.89:0/3492301731 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d40a57d0 con 0x7f14dc06bee0 2023-10-18T10:46:24.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.774+0000 7f14e1236640 1 -- 172.21.15.89:0/3492301731 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d40a5930 con 0x7f14dc06bee0 2023-10-18T10:46:24.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.774+0000 7f14e1236640 1 -- 172.21.15.89:0/3492301731 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d40a5cc0 con 0x7f14dc06bee0 2023-10-18T10:46:24.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.774+0000 7f14e449c640 1 -- 172.21.15.89:0/3492301731 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14dc06bee0 msgr2=0x7f14dc0c8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:24.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.774+0000 7f14e449c640 1 --2- 172.21.15.89:0/3492301731 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14dc06bee0 0x7f14dc0c8960 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f14dc06c560 tx=0x7f14d401c910 comp rx=0 tx=0).stop 2023-10-18T10:46:24.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.774+0000 7f14e449c640 1 -- 172.21.15.89:0/3492301731 shutdown_connections 2023-10-18T10:46:24.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.774+0000 7f14e449c640 1 --2- 172.21.15.89:0/3492301731 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14dc06bee0 0x7f14dc0c8960 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:24.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.774+0000 7f14e449c640 1 --2- 172.21.15.89:0/3492301731 >> v2:172.21.15.154:3300/0 conn(0x7f14dc159490 0x7f14dc15d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:24.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.774+0000 7f14e449c640 1 -- 172.21.15.89:0/3492301731 >> 172.21.15.89:0/3492301731 conn(0x7f14dc069f00 msgr2=0x7f14dc0cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:24.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.774+0000 7f14e449c640 1 -- 172.21.15.89:0/3492301731 shutdown_connections 2023-10-18T10:46:24.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.774+0000 7f14e449c640 1 -- 172.21.15.89:0/3492301731 wait complete. 2023-10-18T10:46:24.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.778+0000 7f14e449c640 1 Processor -- start 2023-10-18T10:46:24.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.778+0000 7f14e449c640 1 -- start start 2023-10-18T10:46:24.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.778+0000 7f14e449c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14dc06bee0 0x7f14dc157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:24.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.778+0000 7f14e449c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14dc070600 0x7f14dc153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:24.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.778+0000 7f14e449c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f14dc159490 0x7f14dc1f50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:24.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.778+0000 7f14e449c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14dc157600 con 0x7f14dc070600 2023-10-18T10:46:24.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.778+0000 7f14e449c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14dc0c1e40 con 0x7f14dc06bee0 2023-10-18T10:46:24.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.778+0000 7f14e449c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14dc0c2980 con 0x7f14dc159490 2023-10-18T10:46:24.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.778+0000 7f14e1a37640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14dc070600 0x7f14dc153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:24.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.778+0000 7f14e2238640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14dc06bee0 0x7f14dc157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:24.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.778+0000 7f14e1a37640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14dc070600 0x7f14dc153980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59458/0 (socket says 172.21.15.89:59458) 2023-10-18T10:46:24.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.778+0000 7f14e1a37640 1 -- 172.21.15.89:0/951545652 learned_addr learned my addr 172.21.15.89:0/951545652 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:24.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.778+0000 7f14e2a39640 1 --2- 172.21.15.89:0/951545652 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f14dc159490 0x7f14dc1f50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:24.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.778+0000 7f14e1a37640 1 -- 172.21.15.89:0/951545652 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f14dc159490 msgr2=0x7f14dc1f50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:24.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.778+0000 7f14e1a37640 1 --2- 172.21.15.89:0/951545652 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f14dc159490 0x7f14dc1f50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:24.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.778+0000 7f14e1a37640 1 -- 172.21.15.89:0/951545652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14dc06bee0 msgr2=0x7f14dc157020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:24.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.778+0000 7f14e1a37640 1 --2- 172.21.15.89:0/951545652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14dc06bee0 0x7f14dc157020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:24.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.778+0000 7f14e1a37640 1 -- 172.21.15.89:0/951545652 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d4009b30 con 0x7f14dc070600 2023-10-18T10:46:24.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.778+0000 7f14e1a37640 1 --2- 172.21.15.89:0/951545652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14dc070600 0x7f14dc153980 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f14cc00c420 tx=0x7f14cc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:24.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.778+0000 7f14d2ffd640 1 -- 172.21.15.89:0/951545652 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14cc0052d0 con 0x7f14dc070600 2023-10-18T10:46:24.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.778+0000 7f14d2ffd640 1 -- 172.21.15.89:0/951545652 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14cc005430 con 0x7f14dc070600 2023-10-18T10:46:24.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.778+0000 7f14d2ffd640 1 -- 172.21.15.89:0/951545652 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14cc007260 con 0x7f14dc070600 2023-10-18T10:46:24.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.778+0000 7f14e449c640 1 -- 172.21.15.89:0/951545652 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14dc0c2050 con 0x7f14dc070600 2023-10-18T10:46:24.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.778+0000 7f14e449c640 1 -- 172.21.15.89:0/951545652 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14dc0c2610 con 0x7f14dc070600 2023-10-18T10:46:24.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.782+0000 7f14e449c640 1 -- 172.21.15.89:0/951545652 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14dc04dbe0 con 0x7f14dc070600 2023-10-18T10:46:24.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.782+0000 7f14d2ffd640 1 -- 172.21.15.89:0/951545652 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f14cc0073c0 con 0x7f14dc070600 2023-10-18T10:46:24.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.782+0000 7f14d2ffd640 1 --2- 172.21.15.89:0/951545652 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f14c0065680 0x7f14c0067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:24.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.782+0000 7f14d2ffd640 1 -- 172.21.15.89:0/951545652 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f14cc082420 con 0x7f14dc070600 2023-10-18T10:46:24.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.782+0000 7f14e2238640 1 --2- 172.21.15.89:0/951545652 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f14c0065680 0x7f14c0067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:24.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.782+0000 7f14e2238640 1 --2- 172.21.15.89:0/951545652 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f14c0065680 0x7f14c0067b40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f14dc15c870 tx=0x7f14d8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:24.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.786+0000 7f14d2ffd640 1 -- 172.21.15.89:0/951545652 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f14cc04fca0 con 0x7f14dc070600 2023-10-18T10:46:24.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.966+0000 7f14e449c640 1 -- 172.21.15.89:0/951545652 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14dc0cda30 con 0x7f14dc070600 2023-10-18T10:46:24.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.966+0000 7f14d2ffd640 1 -- 172.21.15.89:0/951545652 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4874 (secure 0 0 0) 0x7f14cc0532c0 con 0x7f14dc070600 2023-10-18T10:46:24.969 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:24.969 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/1247973498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1247973498},{"type":"v1","addr":"172.21.15.154:6836","nonce":1247973498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4284,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/516057316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":516057316},{"type":"v1","addr":"172.21.15.154:6837","nonce":516057316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4299,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6834/3957051407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3957051407},{"type":"v1","addr":"172.21.15.89:6834","nonce":3957051407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:45:55.655016+0000","modified":"2023-10-18T10:46:13.833215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4312":{"gid":4312,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6835/2906113039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6833","nonce":2906113039},{"type":"v1","addr":"172.21.15.89:6835","nonce":2906113039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T10:46:24.969 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 9 2023-10-18T10:46:24.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.970+0000 7f14e449c640 1 -- 172.21.15.89:0/951545652 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f14c0065680 msgr2=0x7f14c0067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:24.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.970+0000 7f14e449c640 1 --2- 172.21.15.89:0/951545652 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f14c0065680 0x7f14c0067b40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f14dc15c870 tx=0x7f14d8008040 comp rx=0 tx=0).stop 2023-10-18T10:46:24.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.970+0000 7f14e449c640 1 -- 172.21.15.89:0/951545652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14dc070600 msgr2=0x7f14dc153980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:24.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.970+0000 7f14e449c640 1 --2- 172.21.15.89:0/951545652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14dc070600 0x7f14dc153980 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f14cc00c420 tx=0x7f14cc00c8e0 comp rx=0 tx=0).stop 2023-10-18T10:46:24.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.970+0000 7f14e449c640 1 -- 172.21.15.89:0/951545652 shutdown_connections 2023-10-18T10:46:24.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.970+0000 7f14e449c640 1 --2- 172.21.15.89:0/951545652 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f14c0065680 0x7f14c0067b40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:24.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.970+0000 7f14e449c640 1 --2- 172.21.15.89:0/951545652 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f14dc159490 0x7f14dc1f50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:24.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.970+0000 7f14e449c640 1 --2- 172.21.15.89:0/951545652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14dc070600 0x7f14dc153980 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:24.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.970+0000 7f14e449c640 1 --2- 172.21.15.89:0/951545652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14dc06bee0 0x7f14dc157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:24.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.970+0000 7f14e449c640 1 -- 172.21.15.89:0/951545652 >> 172.21.15.89:0/951545652 conn(0x7f14dc069f00 msgr2=0x7f14dc06b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:24.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.970+0000 7f14e449c640 1 -- 172.21.15.89:0/951545652 shutdown_connections 2023-10-18T10:46:24.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:24.970+0000 7f14e449c640 1 -- 172.21.15.89:0/951545652 wait complete. 2023-10-18T10:46:24.989 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:46:25.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.126+0000 7f282e093640 1 Processor -- start 2023-10-18T10:46:25.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.126+0000 7f282e093640 1 -- start start 2023-10-18T10:46:25.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.126+0000 7f282e093640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f28280d0bc0 0x7f28280cd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:25.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.126+0000 7f282e093640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f282815da50 con 0x7f282815c940 2023-10-18T10:46:25.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.126+0000 7f282e093640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28280d3110 con 0x7f28280d0760 2023-10-18T10:46:25.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.126+0000 7f282e093640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f282815ce40 con 0x7f28280d0bc0 2023-10-18T10:46:25.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.126+0000 7f2827fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f28280d0bc0 0x7f28280cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:25.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.126+0000 7f2827fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f28280d0bc0 0x7f28280cd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59466/0 (socket says 172.21.15.89:59466) 2023-10-18T10:46:25.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.126+0000 7f2827fff640 1 -- 172.21.15.89:0/3382331489 learned_addr learned my addr 172.21.15.89:0/3382331489 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:25.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.126+0000 7f2827fff640 1 -- 172.21.15.89:0/3382331489 >> v1:172.21.15.154:6789/0 conn(0x7f28280d0760 legacy=0x7f28280cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:25.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.126+0000 7f2827fff640 1 -- 172.21.15.89:0/3382331489 >> v1:172.21.15.89:6789/0 conn(0x7f282815c940 legacy=0x7f28280d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:25.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.126+0000 7f2827fff640 1 -- 172.21.15.89:0/3382331489 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f282815d120 con 0x7f28280d0bc0 2023-10-18T10:46:25.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.130+0000 7f2827fff640 1 --2- 172.21.15.89:0/3382331489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28280d0bc0 0x7f28280cd870 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2820009e50 tx=0x7f282001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c088407d6065f11c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:25.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.130+0000 7f2826ffd640 1 -- 172.21.15.89:0/3382331489 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28200a5860 con 0x7f28280d0bc0 2023-10-18T10:46:25.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.130+0000 7f2826ffd640 1 -- 172.21.15.89:0/3382331489 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28280ce5c0 con 0x7f28280d0bc0 2023-10-18T10:46:25.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.130+0000 7f2826ffd640 1 -- 172.21.15.89:0/3382331489 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28200a5be0 con 0x7f28280d0bc0 2023-10-18T10:46:25.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.130+0000 7f282e093640 1 -- 172.21.15.89:0/3382331489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28280d0bc0 msgr2=0x7f28280cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:25.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.130+0000 7f282e093640 1 --2- 172.21.15.89:0/3382331489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28280d0bc0 0x7f28280cd870 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2820009e50 tx=0x7f282001c9a0 comp rx=0 tx=0).stop 2023-10-18T10:46:25.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.130+0000 7f282e093640 1 -- 172.21.15.89:0/3382331489 shutdown_connections 2023-10-18T10:46:25.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.130+0000 7f282e093640 1 --2- 172.21.15.89:0/3382331489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28280d0bc0 0x7f28280cd870 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:25.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.130+0000 7f282e093640 1 -- 172.21.15.89:0/3382331489 >> 172.21.15.89:0/3382331489 conn(0x7f2828089fd0 msgr2=0x7f282809b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:25.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.130+0000 7f282e093640 1 -- 172.21.15.89:0/3382331489 shutdown_connections 2023-10-18T10:46:25.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.130+0000 7f282e093640 1 -- 172.21.15.89:0/3382331489 wait complete. 2023-10-18T10:46:25.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.134+0000 7f282e093640 1 Processor -- start 2023-10-18T10:46:25.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.134+0000 7f282e093640 1 -- start start 2023-10-18T10:46:25.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.134+0000 7f282e093640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28280d0760 0x7f28280cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:25.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.134+0000 7f282e093640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28280d0bc0 0x7f28280cb130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:25.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.134+0000 7f282e093640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2828153560 0x7f28281539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:25.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.134+0000 7f282e093640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28280c8580 con 0x7f28280d0760 2023-10-18T10:46:25.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.134+0000 7f282e093640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28280c7b70 con 0x7f28280d0bc0 2023-10-18T10:46:25.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.134+0000 7f282e093640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28280b59f0 con 0x7f2828153560 2023-10-18T10:46:25.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.134+0000 7f28277fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28280d0760 0x7f28280cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:25.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.134+0000 7f28277fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28280d0760 0x7f28280cabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59472/0 (socket says 172.21.15.89:59472) 2023-10-18T10:46:25.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.134+0000 7f28277fe640 1 -- 172.21.15.89:0/487509349 learned_addr learned my addr 172.21.15.89:0/487509349 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:25.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.134+0000 7f2827fff640 1 --2- 172.21.15.89:0/487509349 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2828153560 0x7f28281539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:25.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.134+0000 7f281ffff640 1 --2- 172.21.15.89:0/487509349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28280d0bc0 0x7f28280cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:25.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.134+0000 7f28277fe640 1 -- 172.21.15.89:0/487509349 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2828153560 msgr2=0x7f28281539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:25.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.134+0000 7f28277fe640 1 --2- 172.21.15.89:0/487509349 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2828153560 0x7f28281539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:25.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.134+0000 7f28277fe640 1 -- 172.21.15.89:0/487509349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28280d0bc0 msgr2=0x7f28280cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:25.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.134+0000 7f28277fe640 1 --2- 172.21.15.89:0/487509349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28280d0bc0 0x7f28280cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:25.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.134+0000 7f28277fe640 1 -- 172.21.15.89:0/487509349 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2820009b30 con 0x7f28280d0760 2023-10-18T10:46:25.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.134+0000 7f281ffff640 1 --2- 172.21.15.89:0/487509349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28280d0bc0 0x7f28280cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:25.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.134+0000 7f28277fe640 1 --2- 172.21.15.89:0/487509349 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28280d0760 0x7f28280cabf0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f281800c3c0 tx=0x7f281800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:25.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.134+0000 7f2824ff9640 1 -- 172.21.15.89:0/487509349 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28180052d0 con 0x7f28280d0760 2023-10-18T10:46:25.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.134+0000 7f2824ff9640 1 -- 172.21.15.89:0/487509349 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2818005430 con 0x7f28280d0760 2023-10-18T10:46:25.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.134+0000 7f2824ff9640 1 -- 172.21.15.89:0/487509349 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2818003840 con 0x7f28280d0760 2023-10-18T10:46:25.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.134+0000 7f282e093640 1 -- 172.21.15.89:0/487509349 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28280cb670 con 0x7f28280d0760 2023-10-18T10:46:25.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.134+0000 7f282e093640 1 -- 172.21.15.89:0/487509349 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28280b53a0 con 0x7f28280d0760 2023-10-18T10:46:25.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.138+0000 7f282e093640 1 -- 172.21.15.89:0/487509349 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f282804dbe0 con 0x7f28280d0760 2023-10-18T10:46:25.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.138+0000 7f2824ff9640 1 -- 172.21.15.89:0/487509349 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f28180039a0 con 0x7f28280d0760 2023-10-18T10:46:25.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.138+0000 7f2824ff9640 1 --2- 172.21.15.89:0/487509349 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2808065680 0x7f2808067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:25.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.138+0000 7f2824ff9640 1 -- 172.21.15.89:0/487509349 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f28180825b0 con 0x7f28280d0760 2023-10-18T10:46:25.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.138+0000 7f281ffff640 1 --2- 172.21.15.89:0/487509349 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2808065680 0x7f2808067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:25.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.138+0000 7f281ffff640 1 --2- 172.21.15.89:0/487509349 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2808065680 0x7f2808067b40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f28280c7d40 tx=0x7f2810009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:25.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.142+0000 7f2824ff9640 1 -- 172.21.15.89:0/487509349 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f281804fe10 con 0x7f28280d0760 2023-10-18T10:46:25.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.290+0000 7f282e093640 1 -- 172.21.15.89:0/487509349 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28280d3330 con 0x7f28280d0760 2023-10-18T10:46:25.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.290+0000 7f2824ff9640 1 -- 172.21.15.89:0/487509349 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20096 (secure 0 0 0) 0x7f281806ae60 con 0x7f28280d0760 2023-10-18T10:46:25.294 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:25.294 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":27,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:46:00.166702+0000","last_up_change":"2023-10-18T10:45:42.164688+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:45:53.309122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-18T10:45:54.659763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:45:57.796514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6824","nonce":773078364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6827","nonce":773078364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6831","nonce":773078364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6829","nonce":773078364}]},"public_addr":"172.21.15.154:6824/773078364","cluster_addr":"172.21.15.154:6827/773078364","heartbeat_back_addr":"172.21.15.154:6831/773078364","heartbeat_front_addr":"172.21.15.154:6829/773078364","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:46:25.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.294+0000 7f282e093640 1 -- 172.21.15.89:0/487509349 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2808065680 msgr2=0x7f2808067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:25.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.294+0000 7f282e093640 1 --2- 172.21.15.89:0/487509349 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2808065680 0x7f2808067b40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f28280c7d40 tx=0x7f2810009040 comp rx=0 tx=0).stop 2023-10-18T10:46:25.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.294+0000 7f282e093640 1 -- 172.21.15.89:0/487509349 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28280d0760 msgr2=0x7f28280cabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:25.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.294+0000 7f282e093640 1 --2- 172.21.15.89:0/487509349 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28280d0760 0x7f28280cabf0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f281800c3c0 tx=0x7f281800c880 comp rx=0 tx=0).stop 2023-10-18T10:46:25.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.294+0000 7f282e093640 1 -- 172.21.15.89:0/487509349 shutdown_connections 2023-10-18T10:46:25.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.294+0000 7f282e093640 1 --2- 172.21.15.89:0/487509349 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2808065680 0x7f2808067b40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:25.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.294+0000 7f282e093640 1 --2- 172.21.15.89:0/487509349 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2828153560 0x7f28281539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:25.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.294+0000 7f282e093640 1 --2- 172.21.15.89:0/487509349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28280d0bc0 0x7f28280cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:25.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.294+0000 7f282e093640 1 --2- 172.21.15.89:0/487509349 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28280d0760 0x7f28280cabf0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:25.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.294+0000 7f282e093640 1 -- 172.21.15.89:0/487509349 >> 172.21.15.89:0/487509349 conn(0x7f2828089fd0 msgr2=0x7f28280cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:25.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.294+0000 7f282e093640 1 -- 172.21.15.89:0/487509349 shutdown_connections 2023-10-18T10:46:25.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.294+0000 7f282e093640 1 -- 172.21.15.89:0/487509349 wait complete. 2023-10-18T10:46:25.315 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:46:25.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.450+0000 7fe3f7893640 1 Processor -- start 2023-10-18T10:46:25.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.450+0000 7fe3f7893640 1 -- start start 2023-10-18T10:46:25.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.450+0000 7fe3f7893640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe3f015e600 0x7fe3f00c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:25.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.450+0000 7fe3f7893640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe3f006c1b0 0x7fe3f0161970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:25.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.450+0000 7fe3f7893640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3f00afd90 0x7fe3f00b01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:25.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.450+0000 7fe3f7893640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f0160f80 con 0x7fe3f006c1b0 2023-10-18T10:46:25.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.450+0000 7fe3f7893640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f00b0720 con 0x7fe3f015e600 2023-10-18T10:46:25.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.450+0000 7fe3f7893640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f01614e0 con 0x7fe3f00afd90 2023-10-18T10:46:25.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.454+0000 7fe3f4e2e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe3f006c1b0 0x7fe3f0161970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:25.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.454+0000 7fe3f4e2e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe3f006c1b0 0x7fe3f0161970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59484/0 (socket says 172.21.15.89:59484) 2023-10-18T10:46:25.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.454+0000 7fe3f4e2e640 1 -- 172.21.15.89:0/77410604 learned_addr learned my addr 172.21.15.89:0/77410604 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:25.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.454+0000 7fe3f5e30640 1 --2- 172.21.15.89:0/77410604 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3f00afd90 0x7fe3f00b01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:25.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.454+0000 7fe3f562f640 1 --2- 172.21.15.89:0/77410604 >> v2:172.21.15.154:3300/0 conn(0x7fe3f015e600 0x7fe3f00c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:25.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.454+0000 7fe3f4e2e640 1 -- 172.21.15.89:0/77410604 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3f00afd90 msgr2=0x7fe3f00b01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:25.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.454+0000 7fe3f4e2e640 1 --2- 172.21.15.89:0/77410604 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3f00afd90 0x7fe3f00b01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:25.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.454+0000 7fe3f4e2e640 1 -- 172.21.15.89:0/77410604 >> v2:172.21.15.154:3300/0 conn(0x7fe3f015e600 msgr2=0x7fe3f00c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:25.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.454+0000 7fe3f4e2e640 1 --2- 172.21.15.89:0/77410604 >> v2:172.21.15.154:3300/0 conn(0x7fe3f015e600 0x7fe3f00c8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:25.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.454+0000 7fe3f4e2e640 1 -- 172.21.15.89:0/77410604 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f006c640 con 0x7fe3f006c1b0 2023-10-18T10:46:25.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.454+0000 7fe3f4e2e640 1 --2- 172.21.15.89:0/77410604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3f006c1b0 0x7fe3f0161970 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe3f006dd60 tx=0x7fe3e001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dde7bff93c3d9ebb server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:25.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.454+0000 7fe3e7fff640 1 -- 172.21.15.89:0/77410604 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e00aa620 con 0x7fe3f006c1b0 2023-10-18T10:46:25.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.454+0000 7fe3e7fff640 1 -- 172.21.15.89:0/77410604 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e00aa780 con 0x7fe3f006c1b0 2023-10-18T10:46:25.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.454+0000 7fe3e7fff640 1 -- 172.21.15.89:0/77410604 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e00aab10 con 0x7fe3f006c1b0 2023-10-18T10:46:25.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.458+0000 7fe3f7893640 1 -- 172.21.15.89:0/77410604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3f006c1b0 msgr2=0x7fe3f0161970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:25.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.458+0000 7fe3f7893640 1 --2- 172.21.15.89:0/77410604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3f006c1b0 0x7fe3f0161970 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe3f006dd60 tx=0x7fe3e001c500 comp rx=0 tx=0).stop 2023-10-18T10:46:25.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.458+0000 7fe3f7893640 1 -- 172.21.15.89:0/77410604 shutdown_connections 2023-10-18T10:46:25.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.458+0000 7fe3f7893640 1 --2- 172.21.15.89:0/77410604 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3f00afd90 0x7fe3f00b01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:25.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.458+0000 7fe3f7893640 1 --2- 172.21.15.89:0/77410604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3f006c1b0 0x7fe3f0161970 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:25.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.458+0000 7fe3f7893640 1 --2- 172.21.15.89:0/77410604 >> v2:172.21.15.154:3300/0 conn(0x7fe3f015e600 0x7fe3f00c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:25.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.458+0000 7fe3f7893640 1 -- 172.21.15.89:0/77410604 >> 172.21.15.89:0/77410604 conn(0x7fe3f0069f00 msgr2=0x7fe3f00cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:25.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.458+0000 7fe3f7893640 1 -- 172.21.15.89:0/77410604 shutdown_connections 2023-10-18T10:46:25.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.458+0000 7fe3f7893640 1 -- 172.21.15.89:0/77410604 wait complete. 2023-10-18T10:46:25.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.458+0000 7fe3f7893640 1 Processor -- start 2023-10-18T10:46:25.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.458+0000 7fe3f7893640 1 -- start start 2023-10-18T10:46:25.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.458+0000 7fe3f7893640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3f006c1b0 0x7fe3f01491f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:25.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.458+0000 7fe3f7893640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3f00afd90 0x7fe3f015d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:25.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.458+0000 7fe3f7893640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3f015e600 0x7fe3f0164530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:25.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.458+0000 7fe3f7893640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f0149730 con 0x7fe3f015e600 2023-10-18T10:46:25.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.458+0000 7fe3f7893640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f00c1cc0 con 0x7fe3f006c1b0 2023-10-18T10:46:25.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.458+0000 7fe3f7893640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f00c1fa0 con 0x7fe3f00afd90 2023-10-18T10:46:25.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.458+0000 7fe3f5e30640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3f015e600 0x7fe3f0164530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:25.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.458+0000 7fe3f5e30640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3f015e600 0x7fe3f0164530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59492/0 (socket says 172.21.15.89:59492) 2023-10-18T10:46:25.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.458+0000 7fe3f5e30640 1 -- 172.21.15.89:0/1858693239 learned_addr learned my addr 172.21.15.89:0/1858693239 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:25.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.458+0000 7fe3f4e2e640 1 --2- 172.21.15.89:0/1858693239 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3f00afd90 0x7fe3f015d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:25.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.458+0000 7fe3f562f640 1 --2- 172.21.15.89:0/1858693239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3f006c1b0 0x7fe3f01491f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:25.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.458+0000 7fe3f5e30640 1 -- 172.21.15.89:0/1858693239 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3f00afd90 msgr2=0x7fe3f015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:25.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.458+0000 7fe3f5e30640 1 --2- 172.21.15.89:0/1858693239 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3f00afd90 0x7fe3f015d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:25.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.458+0000 7fe3f5e30640 1 -- 172.21.15.89:0/1858693239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3f006c1b0 msgr2=0x7fe3f01491f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:25.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.458+0000 7fe3f5e30640 1 --2- 172.21.15.89:0/1858693239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3f006c1b0 0x7fe3f01491f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:25.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.458+0000 7fe3f5e30640 1 -- 172.21.15.89:0/1858693239 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e00094d0 con 0x7fe3f015e600 2023-10-18T10:46:25.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.458+0000 7fe3f562f640 1 --2- 172.21.15.89:0/1858693239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3f006c1b0 0x7fe3f01491f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:25.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.462+0000 7fe3f5e30640 1 --2- 172.21.15.89:0/1858693239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3f015e600 0x7fe3f0164530 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8007d10 tx=0x7fe3e800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:25.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.462+0000 7fe3e5ffb640 1 -- 172.21.15.89:0/1858693239 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8014420 con 0x7fe3f015e600 2023-10-18T10:46:25.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.462+0000 7fe3e5ffb640 1 -- 172.21.15.89:0/1858693239 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e80145a0 con 0x7fe3f015e600 2023-10-18T10:46:25.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.462+0000 7fe3e5ffb640 1 -- 172.21.15.89:0/1858693239 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8014950 con 0x7fe3f015e600 2023-10-18T10:46:25.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.462+0000 7fe3f7893640 1 -- 172.21.15.89:0/1858693239 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f00c2280 con 0x7fe3f015e600 2023-10-18T10:46:25.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.462+0000 7fe3f7893640 1 -- 172.21.15.89:0/1858693239 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f00c2840 con 0x7fe3f015e600 2023-10-18T10:46:25.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.462+0000 7fe3f7893640 1 -- 172.21.15.89:0/1858693239 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f00c2a30 con 0x7fe3f015e600 2023-10-18T10:46:25.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.462+0000 7fe3e5ffb640 1 -- 172.21.15.89:0/1858693239 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fe3e8014ab0 con 0x7fe3f015e600 2023-10-18T10:46:25.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.462+0000 7fe3e5ffb640 1 --2- 172.21.15.89:0/1858693239 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe3d40656b0 0x7fe3d4067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:25.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.462+0000 7fe3e5ffb640 1 -- 172.21.15.89:0/1858693239 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe3e8083910 con 0x7fe3f015e600 2023-10-18T10:46:25.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.462+0000 7fe3f562f640 1 --2- 172.21.15.89:0/1858693239 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe3d40656b0 0x7fe3d4067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:25.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.466+0000 7fe3f562f640 1 --2- 172.21.15.89:0/1858693239 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe3d40656b0 0x7fe3d4067b70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe3f00c4a90 tx=0x7fe3ec008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:25.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.466+0000 7fe3e5ffb640 1 -- 172.21.15.89:0/1858693239 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fe3e80510d0 con 0x7fe3f015e600 2023-10-18T10:46:25.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.646+0000 7fe3f7893640 1 -- 172.21.15.89:0/1858693239 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3f004dbe0 con 0x7fe3f015e600 2023-10-18T10:46:25.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.646+0000 7fe3e5ffb640 1 -- 172.21.15.89:0/1858693239 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4874 (secure 0 0 0) 0x7fe3e80546f0 con 0x7fe3f015e600 2023-10-18T10:46:25.649 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:25.649 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/1247973498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1247973498},{"type":"v1","addr":"172.21.15.154:6836","nonce":1247973498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4284,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/516057316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":516057316},{"type":"v1","addr":"172.21.15.154:6837","nonce":516057316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4299,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6834/3957051407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3957051407},{"type":"v1","addr":"172.21.15.89:6834","nonce":3957051407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:45:55.655016+0000","modified":"2023-10-18T10:46:13.833215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4312":{"gid":4312,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6835/2906113039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6833","nonce":2906113039},{"type":"v1","addr":"172.21.15.89:6835","nonce":2906113039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T10:46:25.649 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 9 2023-10-18T10:46:25.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.650+0000 7fe3f7893640 1 -- 172.21.15.89:0/1858693239 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe3d40656b0 msgr2=0x7fe3d4067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:25.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.650+0000 7fe3f7893640 1 --2- 172.21.15.89:0/1858693239 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe3d40656b0 0x7fe3d4067b70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe3f00c4a90 tx=0x7fe3ec008040 comp rx=0 tx=0).stop 2023-10-18T10:46:25.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.650+0000 7fe3f7893640 1 -- 172.21.15.89:0/1858693239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3f015e600 msgr2=0x7fe3f0164530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:25.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.650+0000 7fe3f7893640 1 --2- 172.21.15.89:0/1858693239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3f015e600 0x7fe3f0164530 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8007d10 tx=0x7fe3e800e4b0 comp rx=0 tx=0).stop 2023-10-18T10:46:25.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.650+0000 7fe3f7893640 1 -- 172.21.15.89:0/1858693239 shutdown_connections 2023-10-18T10:46:25.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.650+0000 7fe3f7893640 1 --2- 172.21.15.89:0/1858693239 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe3d40656b0 0x7fe3d4067b70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:25.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.650+0000 7fe3f7893640 1 --2- 172.21.15.89:0/1858693239 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3f00afd90 0x7fe3f015d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:25.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.650+0000 7fe3f7893640 1 --2- 172.21.15.89:0/1858693239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3f015e600 0x7fe3f0164530 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:25.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.650+0000 7fe3f7893640 1 --2- 172.21.15.89:0/1858693239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3f006c1b0 0x7fe3f01491f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:25.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.650+0000 7fe3f7893640 1 -- 172.21.15.89:0/1858693239 >> 172.21.15.89:0/1858693239 conn(0x7fe3f0069f00 msgr2=0x7fe3f015f350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:25.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.650+0000 7fe3f7893640 1 -- 172.21.15.89:0/1858693239 shutdown_connections 2023-10-18T10:46:25.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:25.650+0000 7fe3f7893640 1 -- 172.21.15.89:0/1858693239 wait complete. 2023-10-18T10:46:26.072 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-18T10:45:55.655016+0000', 'modified': '2023-10-18T10:46:13.833215+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4312}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4312': {'gid': 4312, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.89:6835/2906113039', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6833', 'nonce': 2906113039}, {'type': 'v1', 'addr': '172.21.15.89:6835', 'nonce': 2906113039}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-18T10:46:26.072 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T10:46:26.072 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-18T10:46:26.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.210+0000 7fb3c07a3640 1 Processor -- start 2023-10-18T10:46:26.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.210+0000 7fb3c07a3640 1 -- start start 2023-10-18T10:46:26.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.210+0000 7fb3c07a3640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb3b815c940 0x7fb3b80d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:26.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.210+0000 7fb3c07a3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3b80d2f40 0x7fb3b80d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:26.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.210+0000 7fb3c07a3640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3b815da50 con 0x7fb3b80d0ca0 2023-10-18T10:46:26.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.210+0000 7fb3c07a3640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3b80d1130 con 0x7fb3b815c940 2023-10-18T10:46:26.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.210+0000 7fb3c07a3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b815ce40 con 0x7fb3b80d2f40 2023-10-18T10:46:26.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.210+0000 7fb3be53f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb3b815c940 0x7fb3b80d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:26.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.210+0000 7fb3be53f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb3b815c940 0x7fb3b80d2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59500/0 (socket says 172.21.15.89:59500) 2023-10-18T10:46:26.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.210+0000 7fb3be53f640 1 -- 172.21.15.89:0/3355148243 learned_addr learned my addr 172.21.15.89:0/3355148243 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:26.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.210+0000 7fb3bdd3e640 1 --2- 172.21.15.89:0/3355148243 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3b80d2f40 0x7fb3b80d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:26.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.210+0000 7fb3be53f640 1 -- 172.21.15.89:0/3355148243 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3b80d2f40 msgr2=0x7fb3b80d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:26.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.210+0000 7fb3be53f640 1 --2- 172.21.15.89:0/3355148243 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3b80d2f40 0x7fb3b80d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:26.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.210+0000 7fb3be53f640 1 -- 172.21.15.89:0/3355148243 >> v1:172.21.15.89:6789/0 conn(0x7fb3b80d0ca0 legacy=0x7fb3b80cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:26.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.210+0000 7fb3be53f640 1 -- 172.21.15.89:0/3355148243 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b815d120 con 0x7fb3b815c940 2023-10-18T10:46:26.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.214+0000 7fb3be53f640 1 --2- 172.21.15.89:0/3355148243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3b815c940 0x7fb3b80d2a00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4009870 tx=0x7fb3b401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=640f3c74b858c60e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:26.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.214+0000 7fb3bd53d640 1 -- 172.21.15.89:0/3355148243 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b40ab620 con 0x7fb3b815c940 2023-10-18T10:46:26.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.214+0000 7fb3bd53d640 1 -- 172.21.15.89:0/3355148243 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b40ab780 con 0x7fb3b815c940 2023-10-18T10:46:26.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.214+0000 7fb3bd53d640 1 -- 172.21.15.89:0/3355148243 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b40abb10 con 0x7fb3b815c940 2023-10-18T10:46:26.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.214+0000 7fb3c07a3640 1 -- 172.21.15.89:0/3355148243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3b815c940 msgr2=0x7fb3b80d2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:26.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.214+0000 7fb3c07a3640 1 --2- 172.21.15.89:0/3355148243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3b815c940 0x7fb3b80d2a00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4009870 tx=0x7fb3b401c510 comp rx=0 tx=0).stop 2023-10-18T10:46:26.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.214+0000 7fb3c07a3640 1 -- 172.21.15.89:0/3355148243 shutdown_connections 2023-10-18T10:46:26.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.214+0000 7fb3c07a3640 1 --2- 172.21.15.89:0/3355148243 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3b80d2f40 0x7fb3b80d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:26.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.214+0000 7fb3c07a3640 1 --2- 172.21.15.89:0/3355148243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3b815c940 0x7fb3b80d2a00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:26.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.214+0000 7fb3c07a3640 1 -- 172.21.15.89:0/3355148243 >> 172.21.15.89:0/3355148243 conn(0x7fb3b8089fd0 msgr2=0x7fb3b809b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:26.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.214+0000 7fb3c07a3640 1 -- 172.21.15.89:0/3355148243 shutdown_connections 2023-10-18T10:46:26.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.214+0000 7fb3c07a3640 1 -- 172.21.15.89:0/3355148243 wait complete. 2023-10-18T10:46:26.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.218+0000 7fb3c07a3640 1 Processor -- start 2023-10-18T10:46:26.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.218+0000 7fb3c07a3640 1 -- start start 2023-10-18T10:46:26.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.218+0000 7fb3c07a3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3b80d0ca0 0x7fb3b80b6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:26.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.218+0000 7fb3c07a3640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3b8151190 0x7fb3b8151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:26.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.218+0000 7fb3c07a3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3b81546c0 0x7fb3b8154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:26.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.218+0000 7fb3c07a3640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b8155080 con 0x7fb3b8151190 2023-10-18T10:46:26.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.218+0000 7fb3c07a3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b81521a0 con 0x7fb3b80d0ca0 2023-10-18T10:46:26.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.218+0000 7fb3c07a3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b8152480 con 0x7fb3b81546c0 2023-10-18T10:46:26.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.218+0000 7fb3bdd3e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3b8151190 0x7fb3b8151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:26.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.218+0000 7fb3bdd3e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3b8151190 0x7fb3b8151950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59516/0 (socket says 172.21.15.89:59516) 2023-10-18T10:46:26.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.218+0000 7fb3bdd3e640 1 -- 172.21.15.89:0/2003820688 learned_addr learned my addr 172.21.15.89:0/2003820688 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:26.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.218+0000 7fb3be53f640 1 --2- 172.21.15.89:0/2003820688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3b80d0ca0 0x7fb3b80b6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:26.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.218+0000 7fb3bed40640 1 --2- 172.21.15.89:0/2003820688 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3b81546c0 0x7fb3b8154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:26.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.218+0000 7fb3bdd3e640 1 -- 172.21.15.89:0/2003820688 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3b81546c0 msgr2=0x7fb3b8154b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:26.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.218+0000 7fb3bdd3e640 1 --2- 172.21.15.89:0/2003820688 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3b81546c0 0x7fb3b8154b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:26.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.218+0000 7fb3bdd3e640 1 -- 172.21.15.89:0/2003820688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3b80d0ca0 msgr2=0x7fb3b80b6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:26.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.218+0000 7fb3bdd3e640 1 --2- 172.21.15.89:0/2003820688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3b80d0ca0 0x7fb3b80b6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:26.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.218+0000 7fb3bdd3e640 1 -- 172.21.15.89:0/2003820688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b40094d0 con 0x7fb3b8151190 2023-10-18T10:46:26.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.218+0000 7fb3bed40640 1 --2- 172.21.15.89:0/2003820688 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3b81546c0 0x7fb3b8154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:26.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.218+0000 7fb3bdd3e640 1 --2- 172.21.15.89:0/2003820688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3b8151190 0x7fb3b8151950 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800e930 tx=0x7fb3a800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:26.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.218+0000 7fb3aeffd640 1 -- 172.21.15.89:0/2003820688 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8009690 con 0x7fb3b8151190 2023-10-18T10:46:26.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.218+0000 7fb3aeffd640 1 -- 172.21.15.89:0/2003820688 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a800ca00 con 0x7fb3b8151190 2023-10-18T10:46:26.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.218+0000 7fb3aeffd640 1 -- 172.21.15.89:0/2003820688 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a800cd70 con 0x7fb3b8151190 2023-10-18T10:46:26.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.218+0000 7fb3c07a3640 1 -- 172.21.15.89:0/2003820688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b8152760 con 0x7fb3b8151190 2023-10-18T10:46:26.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.218+0000 7fb3c07a3640 1 -- 172.21.15.89:0/2003820688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b8151e90 con 0x7fb3b8151190 2023-10-18T10:46:26.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.222+0000 7fb3aeffd640 1 -- 172.21.15.89:0/2003820688 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fb3a8013470 con 0x7fb3b8151190 2023-10-18T10:46:26.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.222+0000 7fb3aeffd640 1 --2- 172.21.15.89:0/2003820688 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb39c065680 0x7fb39c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:26.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.222+0000 7fb3aeffd640 1 -- 172.21.15.89:0/2003820688 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb3a8083370 con 0x7fb3b8151190 2023-10-18T10:46:26.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.222+0000 7fb3c3b0a000 1 Processor -- start 2023-10-18T10:46:26.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.222+0000 7fb3be53f640 1 --2- 172.21.15.89:0/2003820688 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb39c065680 0x7fb39c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:26.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.226+0000 7fb3c3b0a000 1 -- start start 2023-10-18T10:46:26.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.226+0000 7fb3be53f640 1 --2- 172.21.15.89:0/2003820688 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb39c065680 0x7fb39c067b40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb3b40af000 tx=0x7fb3b40b6040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:26.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.226+0000 7fb3c3b0a000 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f072036a70 0x55f072036e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:26.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.226+0000 7fb3c3b0a000 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f072037390 0x55f071fec930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:26.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.226+0000 7fb3c3b0a000 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55f071fece70 0x55f071fed250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:26.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.226+0000 7fb3c3b0a000 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f071e565a0 con 0x55f072037390 2023-10-18T10:46:26.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.226+0000 7fb3c3b0a000 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f071e44320 con 0x55f072036a70 2023-10-18T10:46:26.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.226+0000 7fb3c3b0a000 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f071e41850 con 0x55f071fece70 2023-10-18T10:46:26.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.226+0000 7fb3bdd3e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f072037390 0x55f071fec930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:26.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.226+0000 7fb3bdd3e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f072037390 0x55f071fec930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59522/0 (socket says 172.21.15.89:59522) 2023-10-18T10:46:26.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.226+0000 7fb3bed40640 1 --2- 172.21.15.89:0/3285838852 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55f071fece70 0x55f071fed250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:26.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.226+0000 7fb3bdd3e640 1 -- 172.21.15.89:0/3285838852 learned_addr learned my addr 172.21.15.89:0/3285838852 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:26.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.226+0000 7fb3be53f640 1 --2- 172.21.15.89:0/3285838852 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f072036a70 0x55f072036e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:26.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.226+0000 7fb3bdd3e640 1 -- 172.21.15.89:0/3285838852 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55f071fece70 msgr2=0x55f071fed250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:26.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.226+0000 7fb3bdd3e640 1 --2- 172.21.15.89:0/3285838852 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55f071fece70 0x55f071fed250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:26.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.226+0000 7fb3bdd3e640 1 -- 172.21.15.89:0/3285838852 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f072036a70 msgr2=0x55f072036e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:26.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.226+0000 7fb3bdd3e640 1 --2- 172.21.15.89:0/3285838852 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f072036a70 0x55f072036e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:26.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.226+0000 7fb3bdd3e640 1 -- 172.21.15.89:0/3285838852 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f071b37890 con 0x55f072037390 2023-10-18T10:46:26.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.226+0000 7fb3be53f640 1 --2- 172.21.15.89:0/3285838852 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f072036a70 0x55f072036e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:46:26.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.226+0000 7fb3bdd3e640 1 --2- 172.21.15.89:0/3285838852 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f072037390 0x55f071fec930 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8050dd0 tx=0x7fb3a8054360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1bb0bfa77351759 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:26.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.226+0000 7fb39bfff640 1 -- 172.21.15.89:0/3285838852 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8050fe0 con 0x55f072037390 2023-10-18T10:46:26.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.226+0000 7fb39bfff640 1 -- 172.21.15.89:0/3285838852 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a8051140 con 0x55f072037390 2023-10-18T10:46:26.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.226+0000 7fb39bfff640 1 -- 172.21.15.89:0/3285838852 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8058320 con 0x55f072037390 2023-10-18T10:46:26.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.226+0000 7fb3c3b0a000 1 -- 172.21.15.89:0/3285838852 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f072037390 msgr2=0x55f071fec930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:26.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.226+0000 7fb3c3b0a000 1 --2- 172.21.15.89:0/3285838852 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f072037390 0x55f071fec930 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8050dd0 tx=0x7fb3a8054360 comp rx=0 tx=0).stop 2023-10-18T10:46:26.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.226+0000 7fb3c3b0a000 1 -- 172.21.15.89:0/3285838852 shutdown_connections 2023-10-18T10:46:26.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.226+0000 7fb3c3b0a000 1 --2- 172.21.15.89:0/3285838852 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55f071fece70 0x55f071fed250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:26.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.226+0000 7fb3c3b0a000 1 --2- 172.21.15.89:0/3285838852 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f072037390 0x55f071fec930 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:26.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.226+0000 7fb3c3b0a000 1 --2- 172.21.15.89:0/3285838852 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f072036a70 0x55f072036e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:26.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.226+0000 7fb3c3b0a000 1 -- 172.21.15.89:0/3285838852 >> 172.21.15.89:0/3285838852 conn(0x55f071fe41e0 msgr2=0x55f071feb210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:26.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.226+0000 7fb3c3b0a000 1 -- 172.21.15.89:0/3285838852 shutdown_connections 2023-10-18T10:46:26.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.226+0000 7fb3c3b0a000 1 -- 172.21.15.89:0/3285838852 wait complete. 2023-10-18T10:46:26.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3c3b0a000 1 -- start start 2023-10-18T10:46:26.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3c3b0a000 1 Processor -- start 2023-10-18T10:46:26.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3c3b0a000 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f071fece70 0x55f071ff8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:26.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3c3b0a000 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55f072036a70 0x55f071ff8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:26.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3c3b0a000 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f072037390 0x55f071ff92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:26.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3c3b0a000 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f071e3d140 con 0x55f071fece70 2023-10-18T10:46:26.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3c3b0a000 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f071df7f60 con 0x55f072037390 2023-10-18T10:46:26.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3c3b0a000 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f071df7c00 con 0x55f072036a70 2023-10-18T10:46:26.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3c3b0a000 10 client.0 _ll_register_callbacks cb 0x55f071fd8350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T10:46:26.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3be53f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f071fece70 0x55f071ff8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:26.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3be53f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f071fece70 0x55f071ff8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59528/0 (socket says 172.21.15.89:59528) 2023-10-18T10:46:26.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3be53f640 1 -- 172.21.15.89:0/2436877590 learned_addr learned my addr 172.21.15.89:0/2436877590 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:26.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3bdd3e640 1 --2- 172.21.15.89:0/2436877590 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55f072036a70 0x55f071ff8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:26.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3bed40640 1 --2- 172.21.15.89:0/2436877590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f072037390 0x55f071ff92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:26.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3be53f640 1 -- 172.21.15.89:0/2436877590 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55f072036a70 msgr2=0x55f071ff8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:26.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3be53f640 1 --2- 172.21.15.89:0/2436877590 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55f072036a70 0x55f071ff8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:26.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3be53f640 1 -- 172.21.15.89:0/2436877590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f072037390 msgr2=0x55f071ff92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:26.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3be53f640 1 --2- 172.21.15.89:0/2436877590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f072037390 0x55f071ff92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:26.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3be53f640 1 -- 172.21.15.89:0/2436877590 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b815d120 con 0x55f071fece70 2023-10-18T10:46:26.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3bdd3e640 1 --2- 172.21.15.89:0/2436877590 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55f072036a70 0x55f071ff8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:26.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3bed40640 1 --2- 172.21.15.89:0/2436877590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f072037390 0x55f071ff92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:26.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3be53f640 1 --2- 172.21.15.89:0/2436877590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f071fece70 0x55f071ff8870 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4009870 tx=0x7fb3b401ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d7f0995a8bdb05 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:26.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3997fa640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-18T10:46:26.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3997fa640 1 -- 172.21.15.89:0/2436877590 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b40c5070 con 0x55f071fece70 2023-10-18T10:46:26.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3997fa640 1 -- 172.21.15.89:0/2436877590 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb3b40bf8d0 con 0x55f071fece70 2023-10-18T10:46:26.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3c3b0a000 1 -- 172.21.15.89:0/2436877590 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f071e43120 con 0x55f071fece70 2023-10-18T10:46:26.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3997fa640 1 -- 172.21.15.89:0/2436877590 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3b40bfa90 con 0x55f071fece70 2023-10-18T10:46:26.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3c3b0a000 10 client.4854 fetch_fsmap learned FSMap version 9 2023-10-18T10:46:26.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3c3b0a000 1 -- 172.21.15.89:0/2436877590 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x7fb3b8151e90 con 0x55f071fece70 2023-10-18T10:46:26.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3997fa640 1 -- 172.21.15.89:0/2436877590 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f071e43120 con 0x55f071fece70 2023-10-18T10:46:26.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3997fa640 1 -- 172.21.15.89:0/2436877590 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fb3b40bfbf0 con 0x55f071fece70 2023-10-18T10:46:26.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3c3b0a000 10 client.4854 fetch_fsmap finished waiting for FSMap version 9 2023-10-18T10:46:26.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3c3b0a000 10 client.4854 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-18T10:46:26.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3c3b0a000 20 client.4854 populate_metadata read hostname 'smithi089' 2023-10-18T10:46:26.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3c3b0a000 1 --2- 172.21.15.89:0/2436877590 >> [v2:172.21.15.89:6833/2906113039,v1:172.21.15.89:6835/2906113039] conn(0x55f071ff9d00 0x55f072048b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:26.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3c3b0a000 4 client.4854 mds_command: new command op to 4312 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-18T10:46:26.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3c3b0a000 1 -- 172.21.15.89:0/2436877590 --> [v2:172.21.15.89:6833/2906113039,v1:172.21.15.89:6835/2906113039] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f071e3ba90 con 0x55f071ff9d00 2023-10-18T10:46:26.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3bdd3e640 1 --2- 172.21.15.89:0/2436877590 >> [v2:172.21.15.89:6833/2906113039,v1:172.21.15.89:6835/2906113039] conn(0x55f071ff9d00 0x55f072048b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:26.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3997fa640 10 client.4854 ms_handle_connect on v2:172.21.15.89:6833/2906113039 2023-10-18T10:46:26.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.230+0000 7fb3bdd3e640 1 --2- 172.21.15.89:0/2436877590 >> [v2:172.21.15.89:6833/2906113039,v1:172.21.15.89:6835/2906113039] conn(0x55f071ff9d00 0x55f072048b20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90622f197a7e0984 server_cookie=7d2bc5f214f4183d in_seq=0 out_seq=0 2023-10-18T10:46:26.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.234+0000 7fb3997fa640 1 -- 172.21.15.89:0/2436877590 <== mds.0 v2:172.21.15.89:6833/2906113039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f071e3ba90 con 0x55f071ff9d00 2023-10-18T10:46:26.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.234+0000 7fb3997fa640 10 client.4854 handle_command_reply: tid=0 2023-10-18T10:46:26.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.234+0000 7fb3c3b0a000 1 client.4854 shutdown 2023-10-18T10:46:26.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.234+0000 7fb3c3b0a000 1 -- 172.21.15.89:0/2436877590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f071fece70 msgr2=0x55f071ff8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:26.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.234+0000 7fb3c3b0a000 1 --2- 172.21.15.89:0/2436877590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f071fece70 0x55f071ff8870 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4009870 tx=0x7fb3b401ad50 comp rx=0 tx=0).stop 2023-10-18T10:46:26.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.234+0000 7fb3c3b0a000 1 -- 172.21.15.89:0/2436877590 shutdown_connections 2023-10-18T10:46:26.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.234+0000 7fb3c3b0a000 1 --2- 172.21.15.89:0/2436877590 >> [v2:172.21.15.89:6833/2906113039,v1:172.21.15.89:6835/2906113039] conn(0x55f071ff9d00 0x55f072048b20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:26.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.234+0000 7fb3c3b0a000 1 --2- 172.21.15.89:0/2436877590 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55f072036a70 0x55f071ff8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:26.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.234+0000 7fb3c3b0a000 1 --2- 172.21.15.89:0/2436877590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f072037390 0x55f071ff92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:26.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.234+0000 7fb3c3b0a000 1 --2- 172.21.15.89:0/2436877590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f071fece70 0x55f071ff8870 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:26.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.234+0000 7fb3c3b0a000 1 -- 172.21.15.89:0/2436877590 >> 172.21.15.89:0/2436877590 conn(0x55f071fe41e0 msgr2=0x55f071feb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:26.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.234+0000 7fb3997fa640 0 client.4854 ms_handle_reset on v2:172.21.15.89:6833/2906113039 2023-10-18T10:46:26.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.234+0000 7fb3c3b0a000 1 -- 172.21.15.89:0/2436877590 shutdown_connections 2023-10-18T10:46:26.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.234+0000 7fb3c3b0a000 1 -- 172.21.15.89:0/2436877590 wait complete. 2023-10-18T10:46:26.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.234+0000 7fb3c3b0a000 20 client.4854 trim_cache size 0 max 16384 2023-10-18T10:46:26.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.246+0000 7fb3c3b0a000 1 Processor -- start 2023-10-18T10:46:26.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.246+0000 7fb3c3b0a000 1 -- start start 2023-10-18T10:46:26.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.250+0000 7fb3c3b0a000 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f071fece70 0x55f0720352c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:26.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.250+0000 7fb3c3b0a000 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f072036a70 0x55f072035800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:26.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.250+0000 7fb3c3b0a000 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55f072037390 0x55f072046b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:26.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.250+0000 7fb3c3b0a000 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f071af8940 con 0x55f071fece70 2023-10-18T10:46:26.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.250+0000 7fb3c3b0a000 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f072043a00 con 0x55f072036a70 2023-10-18T10:46:26.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.250+0000 7fb3c3b0a000 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f071e09fe0 con 0x55f072037390 2023-10-18T10:46:26.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.250+0000 7fb3be53f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f071fece70 0x55f0720352c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:26.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.250+0000 7fb3be53f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f071fece70 0x55f0720352c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59536/0 (socket says 172.21.15.89:59536) 2023-10-18T10:46:26.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.250+0000 7fb3be53f640 1 -- 172.21.15.89:0/65703212 learned_addr learned my addr 172.21.15.89:0/65703212 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:26.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.250+0000 7fb3be53f640 1 -- 172.21.15.89:0/65703212 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55f072037390 msgr2=0x55f072046b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:26.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.250+0000 7fb3be53f640 1 --2- 172.21.15.89:0/65703212 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55f072037390 0x55f072046b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:26.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.250+0000 7fb3be53f640 1 -- 172.21.15.89:0/65703212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f072036a70 msgr2=0x55f072035800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:26.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.250+0000 7fb3be53f640 1 --2- 172.21.15.89:0/65703212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f072036a70 0x55f072035800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:26.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.250+0000 7fb3be53f640 1 -- 172.21.15.89:0/65703212 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f071e00190 con 0x55f071fece70 2023-10-18T10:46:26.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.250+0000 7fb3be53f640 1 --2- 172.21.15.89:0/65703212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f071fece70 0x55f0720352c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb3b401a140 tx=0x7fb3b40c2050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5eff405613f287f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:26.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.250+0000 7fb3acff9640 1 -- 172.21.15.89:0/65703212 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b40bf7d0 con 0x55f071fece70 2023-10-18T10:46:26.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.250+0000 7fb3acff9640 1 -- 172.21.15.89:0/65703212 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b40bf930 con 0x55f071fece70 2023-10-18T10:46:26.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.250+0000 7fb3acff9640 1 -- 172.21.15.89:0/65703212 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b40bfaa0 con 0x55f071fece70 2023-10-18T10:46:26.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.250+0000 7fb3c3b0a000 1 -- 172.21.15.89:0/65703212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f071fece70 msgr2=0x55f0720352c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:26.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.250+0000 7fb3c3b0a000 1 --2- 172.21.15.89:0/65703212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f071fece70 0x55f0720352c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb3b401a140 tx=0x7fb3b40c2050 comp rx=0 tx=0).stop 2023-10-18T10:46:26.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.250+0000 7fb3c3b0a000 1 -- 172.21.15.89:0/65703212 shutdown_connections 2023-10-18T10:46:26.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.250+0000 7fb3c3b0a000 1 --2- 172.21.15.89:0/65703212 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55f072037390 0x55f072046b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:26.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.250+0000 7fb3c3b0a000 1 --2- 172.21.15.89:0/65703212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f072036a70 0x55f072035800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:26.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.250+0000 7fb3c3b0a000 1 --2- 172.21.15.89:0/65703212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f071fece70 0x55f0720352c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:26.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.250+0000 7fb3c3b0a000 1 -- 172.21.15.89:0/65703212 >> 172.21.15.89:0/65703212 conn(0x55f071fe41e0 msgr2=0x55f071ff8730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:26.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.250+0000 7fb3c3b0a000 1 -- 172.21.15.89:0/65703212 shutdown_connections 2023-10-18T10:46:26.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.250+0000 7fb3c3b0a000 1 -- 172.21.15.89:0/65703212 wait complete. 2023-10-18T10:46:26.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.250+0000 7fb3c3b0a000 1 -- start start 2023-10-18T10:46:26.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.250+0000 7fb3c3b0a000 1 Processor -- start 2023-10-18T10:46:26.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3c3b0a000 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f071fece70 0x55f072049b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:26.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3c3b0a000 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f072036a70 0x55f072051ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:26.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3c3b0a000 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55f072037390 0x55f072052420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:26.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3c3b0a000 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f071dfa740 con 0x55f071fece70 2023-10-18T10:46:26.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3c3b0a000 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f071e458f0 con 0x55f072036a70 2023-10-18T10:46:26.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3c3b0a000 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f071e41140 con 0x55f072037390 2023-10-18T10:46:26.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3c3b0a000 10 client.0 _ll_register_callbacks cb 0x7fb3a80546a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T10:46:26.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3be53f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f071fece70 0x55f072049b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:26.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3be53f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f071fece70 0x55f072049b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59548/0 (socket says 172.21.15.89:59548) 2023-10-18T10:46:26.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3be53f640 1 -- 172.21.15.89:0/3972070340 learned_addr learned my addr 172.21.15.89:0/3972070340 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:26.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3bed40640 1 --2- 172.21.15.89:0/3972070340 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55f072037390 0x55f072052420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:26.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3bdd3e640 1 --2- 172.21.15.89:0/3972070340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f072036a70 0x55f072051ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:26.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3be53f640 1 -- 172.21.15.89:0/3972070340 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55f072037390 msgr2=0x55f072052420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:26.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3be53f640 1 --2- 172.21.15.89:0/3972070340 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55f072037390 0x55f072052420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:26.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3be53f640 1 -- 172.21.15.89:0/3972070340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f072036a70 msgr2=0x55f072051ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:26.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3be53f640 1 --2- 172.21.15.89:0/3972070340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f072036a70 0x55f072051ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:26.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3be53f640 1 -- 172.21.15.89:0/3972070340 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b8151e90 con 0x55f071fece70 2023-10-18T10:46:26.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3bed40640 1 --2- 172.21.15.89:0/3972070340 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55f072037390 0x55f072052420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:26.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3bdd3e640 1 --2- 172.21.15.89:0/3972070340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f072036a70 0x55f072051ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:26.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3be53f640 1 --2- 172.21.15.89:0/3972070340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f071fece70 0x55f072049b40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb3b401ca80 tx=0x7fb3b4009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da74d0596c6576f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:26.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3997fa640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-18T10:46:26.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3997fa640 1 -- 172.21.15.89:0/3972070340 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4004250 con 0x55f071fece70 2023-10-18T10:46:26.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3997fa640 1 -- 172.21.15.89:0/3972070340 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb3b40043b0 con 0x55f071fece70 2023-10-18T10:46:26.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3c3b0a000 1 -- 172.21.15.89:0/3972070340 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f071e42030 con 0x55f071fece70 2023-10-18T10:46:26.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3997fa640 1 -- 172.21.15.89:0/3972070340 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3b4003620 con 0x55f071fece70 2023-10-18T10:46:26.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3c3b0a000 10 client.4860 fetch_fsmap learned FSMap version 9 2023-10-18T10:46:26.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3c3b0a000 1 -- 172.21.15.89:0/3972070340 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x55f071de6420 con 0x55f071fece70 2023-10-18T10:46:26.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3997fa640 1 -- 172.21.15.89:0/3972070340 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f071e42030 con 0x55f071fece70 2023-10-18T10:46:26.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3997fa640 1 -- 172.21.15.89:0/3972070340 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fb3b4003780 con 0x55f071fece70 2023-10-18T10:46:26.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3c3b0a000 10 client.4860 fetch_fsmap finished waiting for FSMap version 9 2023-10-18T10:46:26.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3c3b0a000 10 client.4860 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-18T10:46:26.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3c3b0a000 20 client.4860 populate_metadata read hostname 'smithi089' 2023-10-18T10:46:26.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3c3b0a000 1 --2- 172.21.15.89:0/3972070340 >> [v2:172.21.15.89:6833/2906113039,v1:172.21.15.89:6835/2906113039] conn(0x55f072095670 0x55f072095a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:26.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3c3b0a000 4 client.4860 mds_command: new command op to 4312 tid=0[{"prefix": "status"}] 2023-10-18T10:46:26.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3c3b0a000 1 -- 172.21.15.89:0/3972070340 --> [v2:172.21.15.89:6833/2906113039,v1:172.21.15.89:6835/2906113039] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f071e36120 con 0x55f072095670 2023-10-18T10:46:26.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3bdd3e640 1 --2- 172.21.15.89:0/3972070340 >> [v2:172.21.15.89:6833/2906113039,v1:172.21.15.89:6835/2906113039] conn(0x55f072095670 0x55f072095a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:26.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3997fa640 10 client.4860 ms_handle_connect on v2:172.21.15.89:6833/2906113039 2023-10-18T10:46:26.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3bdd3e640 1 --2- 172.21.15.89:0/3972070340 >> [v2:172.21.15.89:6833/2906113039,v1:172.21.15.89:6835/2906113039] conn(0x55f072095670 0x55f072095a50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5b53e3d0540a055 server_cookie=83d2dabd00e44465 in_seq=0 out_seq=0 2023-10-18T10:46:26.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3997fa640 1 -- 172.21.15.89:0/3972070340 <== mds.0 v2:172.21.15.89:6833/2906113039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55f071e36120 con 0x55f072095670 2023-10-18T10:46:26.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3997fa640 10 client.4860 handle_command_reply: tid=0 2023-10-18T10:46:26.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3c3b0a000 1 client.4860 shutdown 2023-10-18T10:46:26.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3c3b0a000 1 -- 172.21.15.89:0/3972070340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f071fece70 msgr2=0x55f072049b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:26.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3c3b0a000 1 --2- 172.21.15.89:0/3972070340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f071fece70 0x55f072049b40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb3b401ca80 tx=0x7fb3b4009240 comp rx=0 tx=0).stop 2023-10-18T10:46:26.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3c3b0a000 1 -- 172.21.15.89:0/3972070340 shutdown_connections 2023-10-18T10:46:26.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3c3b0a000 1 --2- 172.21.15.89:0/3972070340 >> [v2:172.21.15.89:6833/2906113039,v1:172.21.15.89:6835/2906113039] conn(0x55f072095670 0x55f072095a50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:26.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3c3b0a000 1 --2- 172.21.15.89:0/3972070340 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55f072037390 0x55f072052420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:26.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3c3b0a000 1 --2- 172.21.15.89:0/3972070340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f072036a70 0x55f072051ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:26.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3c3b0a000 1 --2- 172.21.15.89:0/3972070340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f071fece70 0x55f072049b40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:26.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3997fa640 0 client.4860 ms_handle_reset on v2:172.21.15.89:6833/2906113039 2023-10-18T10:46:26.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3c3b0a000 1 -- 172.21.15.89:0/3972070340 >> 172.21.15.89:0/3972070340 conn(0x55f071fe41e0 msgr2=0x55f0720342b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:26.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3c3b0a000 1 -- 172.21.15.89:0/3972070340 shutdown_connections 2023-10-18T10:46:26.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3c3b0a000 1 -- 172.21.15.89:0/3972070340 wait complete. 2023-10-18T10:46:26.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.254+0000 7fb3c3b0a000 20 client.4860 trim_cache size 0 max 16384 2023-10-18T10:46:26.274 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-18T10:46:26.274 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "1352f62a-2330-4e96-afcd-b8e87cd82667", 2023-10-18T10:46:26.274 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 0, 2023-10-18T10:46:26.274 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 4312, 2023-10-18T10:46:26.275 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2023-10-18T10:46:26.275 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2023-10-18T10:46:26.275 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2023-10-18T10:46:26.275 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 29.983661582, 2023-10-18T10:46:26.275 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 9, 2023-10-18T10:46:26.276 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 27, 2023-10-18T10:46:26.276 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 27, 2023-10-18T10:46:26.276 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 35.772572500000003 2023-10-18T10:46:26.276 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-18T10:46:26.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.258+0000 7fb3c07a3640 1 -- 172.21.15.89:0/2003820688 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb39c065680 msgr2=0x7fb39c067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:26.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.258+0000 7fb3c07a3640 1 --2- 172.21.15.89:0/2003820688 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb39c065680 0x7fb39c067b40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb3b40af000 tx=0x7fb3b40b6040 comp rx=0 tx=0).stop 2023-10-18T10:46:26.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.258+0000 7fb3c07a3640 1 -- 172.21.15.89:0/2003820688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3b8151190 msgr2=0x7fb3b8151950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:26.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.258+0000 7fb3c07a3640 1 --2- 172.21.15.89:0/2003820688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3b8151190 0x7fb3b8151950 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800e930 tx=0x7fb3a800edf0 comp rx=0 tx=0).stop 2023-10-18T10:46:26.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.258+0000 7fb3c07a3640 1 -- 172.21.15.89:0/2003820688 shutdown_connections 2023-10-18T10:46:26.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.258+0000 7fb3c07a3640 1 --2- 172.21.15.89:0/2003820688 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb39c065680 0x7fb39c067b40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:26.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.258+0000 7fb3c07a3640 1 --2- 172.21.15.89:0/2003820688 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3b81546c0 0x7fb3b8154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:26.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.258+0000 7fb3c07a3640 1 --2- 172.21.15.89:0/2003820688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3b8151190 0x7fb3b8151950 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:26.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.258+0000 7fb3c07a3640 1 --2- 172.21.15.89:0/2003820688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3b80d0ca0 0x7fb3b80b6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:26.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.258+0000 7fb3c07a3640 1 -- 172.21.15.89:0/2003820688 >> 172.21.15.89:0/2003820688 conn(0x7fb3b8089fd0 msgr2=0x7fb3b815e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:26.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.258+0000 7fb3c07a3640 1 -- 172.21.15.89:0/2003820688 shutdown_connections 2023-10-18T10:46:26.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:26.258+0000 7fb3c07a3640 1 -- 172.21.15.89:0/2003820688 wait complete. 2023-10-18T10:46:26.280 INFO:teuthology.run_tasks:Running task kclient... 2023-10-18T10:46:26.293 INFO:tasks.kclient:Mounting kernel clients... 2023-10-18T10:46:26.294 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-18T10:46:26.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:26.294 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-10-18T10:46:26.306 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-10-18T10:46:26.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:26.307 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-18T10:46:26.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:26.371 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link delete ceph-brx 2023-10-18T10:46:26.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:26.581 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-10-18T10:46:26.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:26.591 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link delete ceph-brx 2023-10-18T10:46:26.660 INFO:teuthology.orchestra.run.smithi089.stderr:Cannot find device "ceph-brx" 2023-10-18T10:46:26.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:46:26.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:26.663 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2023-10-18T10:46:26.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:26.674 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link delete ceph-brx 2023-10-18T10:46:26.741 INFO:teuthology.orchestra.run.smithi154.stderr:Cannot find device "ceph-brx" 2023-10-18T10:46:26.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:46:26.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:26.744 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2023-10-18T10:46:26.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:26.801 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link delete ceph-brx 2023-10-18T10:46:26.868 INFO:teuthology.orchestra.run.smithi154.stderr:Cannot find device "ceph-brx" 2023-10-18T10:46:26.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:46:26.871 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2023-10-18T10:46:26.871 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-18T10:46:26.872 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-18T10:46:26.872 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T10:46:27.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.006+0000 7f93974de640 1 Processor -- start 2023-10-18T10:46:27.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.010+0000 7f93974de640 1 -- start start 2023-10-18T10:46:27.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.010+0000 7f93974de640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f93900b7f00 0x7f93900cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:27.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.010+0000 7f93974de640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93900b5610 con 0x7f939015c980 2023-10-18T10:46:27.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.010+0000 7f93974de640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93901609f0 con 0x7f939015c5a0 2023-10-18T10:46:27.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.010+0000 7f93974de640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9390161280 con 0x7f93900b7f00 2023-10-18T10:46:27.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.010+0000 7f9395cdb640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f93900b7f00 0x7f93900cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:27.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.010+0000 7f9395cdb640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f93900b7f00 0x7f93900cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59554/0 (socket says 172.21.15.89:59554) 2023-10-18T10:46:27.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.010+0000 7f9395cdb640 1 -- 172.21.15.89:0/1095951051 learned_addr learned my addr 172.21.15.89:0/1095951051 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:27.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.010+0000 7f9395cdb640 1 -- 172.21.15.89:0/1095951051 >> v1:172.21.15.154:6789/0 conn(0x7f939015c5a0 legacy=0x7f93901602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:27.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.010+0000 7f9395cdb640 1 -- 172.21.15.89:0/1095951051 >> v1:172.21.15.89:6789/0 conn(0x7f939015c980 legacy=0x7f93900d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:27.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.010+0000 7f9395cdb640 1 -- 172.21.15.89:0/1095951051 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9390160cc0 con 0x7f93900b7f00 2023-10-18T10:46:27.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.014+0000 7f9395cdb640 1 --2- 172.21.15.89:0/1095951051 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93900b7f00 0x7f93900cd240 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9380006fd0 tx=0x7f938001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f0a4a02dcd9a16b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:27.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.014+0000 7f93954da640 1 -- 172.21.15.89:0/1095951051 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93800ab600 con 0x7f93900b7f00 2023-10-18T10:46:27.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.014+0000 7f93954da640 1 -- 172.21.15.89:0/1095951051 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93800a7050 con 0x7f93900b7f00 2023-10-18T10:46:27.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.014+0000 7f93954da640 1 -- 172.21.15.89:0/1095951051 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93800ab980 con 0x7f93900b7f00 2023-10-18T10:46:27.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.014+0000 7f93974de640 1 -- 172.21.15.89:0/1095951051 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93900b7f00 msgr2=0x7f93900cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:27.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.014+0000 7f93974de640 1 --2- 172.21.15.89:0/1095951051 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93900b7f00 0x7f93900cd240 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9380006fd0 tx=0x7f938001c420 comp rx=0 tx=0).stop 2023-10-18T10:46:27.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.014+0000 7f93974de640 1 -- 172.21.15.89:0/1095951051 shutdown_connections 2023-10-18T10:46:27.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.014+0000 7f93974de640 1 --2- 172.21.15.89:0/1095951051 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93900b7f00 0x7f93900cd240 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:27.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.014+0000 7f93974de640 1 -- 172.21.15.89:0/1095951051 >> 172.21.15.89:0/1095951051 conn(0x7f93900ceef0 msgr2=0x7f93900cf300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:27.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.014+0000 7f93974de640 1 -- 172.21.15.89:0/1095951051 shutdown_connections 2023-10-18T10:46:27.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.014+0000 7f93974de640 1 -- 172.21.15.89:0/1095951051 wait complete. 2023-10-18T10:46:27.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.014+0000 7f93974de640 1 Processor -- start 2023-10-18T10:46:27.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.014+0000 7f93974de640 1 -- start start 2023-10-18T10:46:27.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.018+0000 7f93974de640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93900b7f00 0x7f93901530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:27.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.018+0000 7f93974de640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f939015c5a0 0x7f9390147850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:27.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.018+0000 7f93974de640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f939015c980 0x7f939015d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:27.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.018+0000 7f93974de640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939015cf10 con 0x7f939015c980 2023-10-18T10:46:27.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.018+0000 7f93974de640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939015f790 con 0x7f93900b7f00 2023-10-18T10:46:27.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.018+0000 7f93974de640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f939015fa70 con 0x7f939015c5a0 2023-10-18T10:46:27.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.018+0000 7f9396cdd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f939015c980 0x7f939015d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:27.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.018+0000 7f9396cdd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f939015c980 0x7f939015d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59556/0 (socket says 172.21.15.89:59556) 2023-10-18T10:46:27.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.018+0000 7f9396cdd640 1 -- 172.21.15.89:0/955172961 learned_addr learned my addr 172.21.15.89:0/955172961 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:27.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.018+0000 7f93964dc640 1 --2- 172.21.15.89:0/955172961 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93900b7f00 0x7f93901530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:27.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.018+0000 7f9395cdb640 1 --2- 172.21.15.89:0/955172961 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f939015c5a0 0x7f9390147850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:27.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.018+0000 7f9396cdd640 1 -- 172.21.15.89:0/955172961 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f939015c5a0 msgr2=0x7f9390147850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:27.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.018+0000 7f9396cdd640 1 --2- 172.21.15.89:0/955172961 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f939015c5a0 0x7f9390147850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:27.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.018+0000 7f9396cdd640 1 -- 172.21.15.89:0/955172961 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93900b7f00 msgr2=0x7f93901530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:27.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.018+0000 7f9396cdd640 1 --2- 172.21.15.89:0/955172961 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93900b7f00 0x7f93901530a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:27.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.018+0000 7f9396cdd640 1 -- 172.21.15.89:0/955172961 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9380009480 con 0x7f939015c980 2023-10-18T10:46:27.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.018+0000 7f9396cdd640 1 --2- 172.21.15.89:0/955172961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f939015c980 0x7f939015d3a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f938800c730 tx=0x7f938800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:27.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.018+0000 7f9386ffd640 1 -- 172.21.15.89:0/955172961 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9388003b80 con 0x7f939015c980 2023-10-18T10:46:27.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.018+0000 7f9386ffd640 1 -- 172.21.15.89:0/955172961 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9388003ce0 con 0x7f939015c980 2023-10-18T10:46:27.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.018+0000 7f93974de640 1 -- 172.21.15.89:0/955172961 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f939014a330 con 0x7f939015c980 2023-10-18T10:46:27.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.018+0000 7f9386ffd640 1 -- 172.21.15.89:0/955172961 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9388005310 con 0x7f939015c980 2023-10-18T10:46:27.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.018+0000 7f93974de640 1 -- 172.21.15.89:0/955172961 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9390157ae0 con 0x7f939015c980 2023-10-18T10:46:27.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.018+0000 7f93974de640 1 -- 172.21.15.89:0/955172961 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f939015d080 con 0x7f939015c980 2023-10-18T10:46:27.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.018+0000 7f9386ffd640 1 -- 172.21.15.89:0/955172961 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f9388005680 con 0x7f939015c980 2023-10-18T10:46:27.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.022+0000 7f9386ffd640 1 --2- 172.21.15.89:0/955172961 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f93740656b0 0x7f9374067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:27.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.022+0000 7f9386ffd640 1 -- 172.21.15.89:0/955172961 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9388082e60 con 0x7f939015c980 2023-10-18T10:46:27.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.022+0000 7f93964dc640 1 --2- 172.21.15.89:0/955172961 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f93740656b0 0x7f9374067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:27.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.022+0000 7f93964dc640 1 --2- 172.21.15.89:0/955172961 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f93740656b0 0x7f9374067b70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f938c007fd0 tx=0x7f938c007e40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:27.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.026+0000 7f9386ffd640 1 -- 172.21.15.89:0/955172961 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f9388050770 con 0x7f939015c980 2023-10-18T10:46:27.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.206+0000 7f93974de640 1 -- 172.21.15.89:0/955172961 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9390073b50 con 0x7f939015c980 2023-10-18T10:46:27.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.206+0000 7f9386ffd640 1 -- 172.21.15.89:0/955172961 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f9388053d90 con 0x7f939015c980 2023-10-18T10:46:27.208 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-18T10:46:27.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.206+0000 7f93974de640 1 -- 172.21.15.89:0/955172961 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f93740656b0 msgr2=0x7f9374067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:27.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.206+0000 7f93974de640 1 --2- 172.21.15.89:0/955172961 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f93740656b0 0x7f9374067b70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f938c007fd0 tx=0x7f938c007e40 comp rx=0 tx=0).stop 2023-10-18T10:46:27.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.206+0000 7f93974de640 1 -- 172.21.15.89:0/955172961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f939015c980 msgr2=0x7f939015d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:27.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.206+0000 7f93974de640 1 --2- 172.21.15.89:0/955172961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f939015c980 0x7f939015d3a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f938800c730 tx=0x7f938800cbf0 comp rx=0 tx=0).stop 2023-10-18T10:46:27.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.206+0000 7f93974de640 1 -- 172.21.15.89:0/955172961 shutdown_connections 2023-10-18T10:46:27.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.206+0000 7f93974de640 1 --2- 172.21.15.89:0/955172961 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f93740656b0 0x7f9374067b70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:27.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.206+0000 7f93974de640 1 --2- 172.21.15.89:0/955172961 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f939015c5a0 0x7f9390147850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:27.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.206+0000 7f93974de640 1 --2- 172.21.15.89:0/955172961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f939015c980 0x7f939015d3a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:27.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.206+0000 7f93974de640 1 --2- 172.21.15.89:0/955172961 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93900b7f00 0x7f93901530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:27.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.206+0000 7f93974de640 1 -- 172.21.15.89:0/955172961 >> 172.21.15.89:0/955172961 conn(0x7f93900ceef0 msgr2=0x7f939015a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:27.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.206+0000 7f93974de640 1 -- 172.21.15.89:0/955172961 shutdown_connections 2023-10-18T10:46:27.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:27.206+0000 7f93974de640 1 -- 172.21.15.89:0/955172961 wait complete. 2023-10-18T10:46:27.228 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-18T10:46:27.228 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T10:46:27.229 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi089.front.sepia.ceph.com 2023-10-18T10:46:27.229 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-18T10:46:27.229 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-18T10:46:27.229 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-18T10:46:27.230 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-18T10:46:27.230 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T10:46:27.230 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T10:46:27.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:27.231 DEBUG:teuthology.orchestra.run.smithi089:> ip addr 2023-10-18T10:46:27.241 INFO:teuthology.orchestra.run.smithi089.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T10:46:27.241 INFO:teuthology.orchestra.run.smithi089.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T10:46:27.241 INFO:teuthology.orchestra.run.smithi089.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T10:46:27.242 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:27.242 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 ::1/128 scope host 2023-10-18T10:46:27.242 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:27.242 INFO:teuthology.orchestra.run.smithi089.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-18T10:46:27.242 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c6 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:46:27.243 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f0 2023-10-18T10:46:27.243 INFO:teuthology.orchestra.run.smithi089.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-18T10:46:27.243 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c7 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:46:27.243 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f1 2023-10-18T10:46:27.243 INFO:teuthology.orchestra.run.smithi089.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-18T10:46:27.244 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:00 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:46:27.244 INFO:teuthology.orchestra.run.smithi089.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T10:46:27.244 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:01 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:46:27.244 INFO:teuthology.orchestra.run.smithi089.stdout: inet 172.21.15.89/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-18T10:46:27.244 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft 42362sec preferred_lft 42362sec 2023-10-18T10:46:27.244 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec4:7aff:fe8f:d501/64 scope link 2023-10-18T10:46:27.245 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:27.245 INFO:teuthology.orchestra.run.smithi089.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-18T10:46:27.245 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 02:42:3f:fd:35:8d brd ff:ff:ff:ff:ff:ff 2023-10-18T10:46:27.245 INFO:teuthology.orchestra.run.smithi089.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-18T10:46:27.245 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:27.246 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-18T10:46:27.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:27.246 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-18T10:46:27.247 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-18T10:46:27.247 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link add name ceph-brx type bridge 2023-10-18T10:46:27.247 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip addr flush dev ceph-brx 2023-10-18T10:46:27.248 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set ceph-brx up 2023-10-18T10:46:27.248 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-18T10:46:27.248 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-18T10:46:27.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:27.356 DEBUG:teuthology.orchestra.run.smithi089:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T10:46:27.375 INFO:teuthology.orchestra.run.smithi089.stdout:1 2023-10-18T10:46:27.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:27.378 DEBUG:teuthology.orchestra.run.smithi089:> route 2023-10-18T10:46:27.436 INFO:teuthology.orchestra.run.smithi089.stdout:Kernel IP routing table 2023-10-18T10:46:27.436 INFO:teuthology.orchestra.run.smithi089.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T10:46:27.436 INFO:teuthology.orchestra.run.smithi089.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-18T10:46:27.437 INFO:teuthology.orchestra.run.smithi089.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T10:46:27.437 INFO:teuthology.orchestra.run.smithi089.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-18T10:46:27.437 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T10:46:27.437 INFO:teuthology.orchestra.run.smithi089.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-18T10:46:27.438 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-18T10:46:27.438 INFO:teuthology.orchestra.run.smithi089.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-18T10:46:27.438 INFO:teuthology.orchestra.run.smithi089.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T10:46:27.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:27.439 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-18T10:46:27.439 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-18T10:46:27.439 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T10:46:27.440 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T10:46:27.440 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T10:46:27.440 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-18T10:46:27.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:27.532 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-10-18T10:46:27.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:27.543 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list-id 2023-10-18T10:46:27.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:27.595 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-18T10:46:27.595 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-18T10:46:27.595 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:46:27.595 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-18T10:46:27.596 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-18T10:46:27.680 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-18T10:46:27.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:27.681 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-18T10:46:27.682 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-18T10:46:27.682 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-18T10:46:27.682 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-18T10:46:27.683 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-18T10:46:27.683 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-18T10:46:27.683 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-18T10:46:27.684 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-18T10:46:27.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:27.822 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-18T10:46:27.822 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-18T10:46:27.823 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set brx.0 up 2023-10-18T10:46:27.823 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set dev brx.0 master ceph-brx 2023-10-18T10:46:27.823 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-18T10:46:27.863 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:46:27.864 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-18T10:46:27.914 INFO:teuthology.orchestra.run.smithi089.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-18T10:46:27.914 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:46:27.915 DEBUG:teuthology.orchestra.run.smithi089:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-18T10:46:27.969 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-18T10:46:27.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:27.970 DEBUG:teuthology.orchestra.run.smithi089:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-18T10:46:28.062 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-18T10:46:28.063 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-18T10:46:28.063 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-18T10:46:28.392 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-18T10:46:28.392 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-18T10:46:28.393 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-18T10:46:28.393 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.154:3300,172.21.15.154:3301" 2023-10-18T10:46:28.393 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-18T10:46:28.404 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-18T10:46:28.404 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-18T10:46:28.404 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-18T10:46:28.405 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.154:3300,172.21.15.154:3301" 2023-10-18T10:46:28.405 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-18T10:46:28.405 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.154:6789,172.21.15.154:6790" 2023-10-18T10:46:28.405 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.154:6789,172.21.15.154:6790:/ 2023-10-18T10:46:28.405 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1352f62a-2330-4e96-afcd-b8e87cd82667" will pass to kernel 2023-10-18T10:46:28.407 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-18T10:46:28.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:28.408 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-18T10:46:28.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:28.429 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T10:46:28.481 INFO:teuthology.orchestra.run.smithi089.stdout:client4869 2023-10-18T10:46:28.481 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:46:28.482 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:46:28.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:28.530 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T10:46:28.584 INFO:teuthology.orchestra.run.smithi089.stdout:client4869 2023-10-18T10:46:28.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:28.585 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T10:46:28.636 INFO:teuthology.orchestra.run.smithi089.stdout:client4869 2023-10-18T10:46:28.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:28.637 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/1352f62a-2330-4e96-afcd-b8e87cd82667.client4869/status') 2023-10-18T10:46:28.697 INFO:teuthology.orchestra.run.smithi089.stdout:instance: client.4869 (3)192.168.0.1:0/3512926064 2023-10-18T10:46:28.698 INFO:teuthology.orchestra.run.smithi089.stdout:blocklisted: false 2023-10-18T10:46:28.698 INFO:teuthology.orchestra.run.smithi089.stderr:0+1 records in 2023-10-18T10:46:28.699 INFO:teuthology.orchestra.run.smithi089.stderr:0+1 records out 2023-10-18T10:46:28.699 INFO:teuthology.orchestra.run.smithi089.stderr:69 bytes copied, 7.1597e-05 s, 964 kB/s 2023-10-18T10:46:28.700 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2023-10-18T10:46:28.700 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-18T10:46:28.700 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-18T10:46:28.701 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T10:46:28.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.874+0000 7efc9e4cb640 1 Processor -- start 2023-10-18T10:46:28.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.878+0000 7efc9e4cb640 1 -- start start 2023-10-18T10:46:28.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.878+0000 7efc9e4cb640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efc980d0760 0x7efc980d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:28.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.878+0000 7efc9e4cb640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efc980cd160 0x7efc980cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:28.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.878+0000 7efc9e4cb640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc980d1100 con 0x7efc9815c940 2023-10-18T10:46:28.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.878+0000 7efc9e4cb640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc9815da50 con 0x7efc980cd160 2023-10-18T10:46:28.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.878+0000 7efc9e4cb640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc980d3110 con 0x7efc980d0760 2023-10-18T10:46:28.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.878+0000 7efc9ca68640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efc980cd160 0x7efc980cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:28.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.878+0000 7efc9ca68640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efc980cd160 0x7efc980cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59572/0 (socket says 172.21.15.89:59572) 2023-10-18T10:46:28.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.878+0000 7efc9ca68640 1 -- 172.21.15.89:0/1115524789 learned_addr learned my addr 172.21.15.89:0/1115524789 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:28.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.878+0000 7efc97fff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7efc9815c940 0x7efc980d2a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:45750/0 (socket says 172.21.15.89:45750) 2023-10-18T10:46:28.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.878+0000 7efc977fe640 1 --2- 172.21.15.89:0/1115524789 >> v2:172.21.15.154:3300/0 conn(0x7efc980d0760 0x7efc980d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:28.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.878+0000 7efc9ca68640 1 -- 172.21.15.89:0/1115524789 >> v2:172.21.15.154:3300/0 conn(0x7efc980d0760 msgr2=0x7efc980d0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:28.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.878+0000 7efc9ca68640 1 --2- 172.21.15.89:0/1115524789 >> v2:172.21.15.154:3300/0 conn(0x7efc980d0760 0x7efc980d0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:28.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.878+0000 7efc9ca68640 1 -- 172.21.15.89:0/1115524789 >> v1:172.21.15.89:6789/0 conn(0x7efc9815c940 legacy=0x7efc980d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:28.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.878+0000 7efc9ca68640 1 -- 172.21.15.89:0/1115524789 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc9815ce40 con 0x7efc980cd160 2023-10-18T10:46:28.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.878+0000 7efc96ffd640 1 -- 172.21.15.89:0/1115524789 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1400502169 0 0) 0x7efc980d1100 con 0x7efc9815c940 2023-10-18T10:46:28.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.882+0000 7efc9ca68640 1 --2- 172.21.15.89:0/1115524789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc980cd160 0x7efc980cd5f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efc980724f0 tx=0x7efc9001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1303e2227bd75aa1 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:28.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.882+0000 7efc96ffd640 1 -- 172.21.15.89:0/1115524789 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc900a5860 con 0x7efc980cd160 2023-10-18T10:46:28.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.882+0000 7efc96ffd640 1 -- 172.21.15.89:0/1115524789 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc900a59c0 con 0x7efc980cd160 2023-10-18T10:46:28.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.882+0000 7efc96ffd640 1 -- 172.21.15.89:0/1115524789 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc900a5d50 con 0x7efc980cd160 2023-10-18T10:46:28.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.882+0000 7efc9e4cb640 1 -- 172.21.15.89:0/1115524789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc980cd160 msgr2=0x7efc980cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:28.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.882+0000 7efc9e4cb640 1 --2- 172.21.15.89:0/1115524789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc980cd160 0x7efc980cd5f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efc980724f0 tx=0x7efc9001c9a0 comp rx=0 tx=0).stop 2023-10-18T10:46:28.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.882+0000 7efc9e4cb640 1 -- 172.21.15.89:0/1115524789 shutdown_connections 2023-10-18T10:46:28.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.882+0000 7efc9e4cb640 1 --2- 172.21.15.89:0/1115524789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc980cd160 0x7efc980cd5f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:28.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.882+0000 7efc9e4cb640 1 --2- 172.21.15.89:0/1115524789 >> v2:172.21.15.154:3300/0 conn(0x7efc980d0760 0x7efc980d0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:28.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.882+0000 7efc9e4cb640 1 -- 172.21.15.89:0/1115524789 >> 172.21.15.89:0/1115524789 conn(0x7efc98089fd0 msgr2=0x7efc9809b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:28.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.882+0000 7efc9e4cb640 1 -- 172.21.15.89:0/1115524789 shutdown_connections 2023-10-18T10:46:28.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.882+0000 7efc9e4cb640 1 -- 172.21.15.89:0/1115524789 wait complete. 2023-10-18T10:46:28.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.882+0000 7efc9e4cb640 1 Processor -- start 2023-10-18T10:46:28.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.882+0000 7efc9e4cb640 1 -- start start 2023-10-18T10:46:28.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.886+0000 7efc9e4cb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc980cd160 0x7efc981519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:28.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.886+0000 7efc9e4cb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc980d0760 0x7efc98069900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:28.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.886+0000 7efc9e4cb640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc98153670 0x7efc98069e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:28.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.886+0000 7efc9e4cb640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9815a150 con 0x7efc98153670 2023-10-18T10:46:28.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.886+0000 7efc9e4cb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc98153e80 con 0x7efc980d0760 2023-10-18T10:46:28.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.886+0000 7efc9e4cb640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc980b6df0 con 0x7efc980cd160 2023-10-18T10:46:28.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.886+0000 7efc9ca68640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc98153670 0x7efc98069e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:28.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.886+0000 7efc9ca68640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc98153670 0x7efc98069e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59574/0 (socket says 172.21.15.89:59574) 2023-10-18T10:46:28.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.886+0000 7efc9ca68640 1 -- 172.21.15.89:0/2212803557 learned_addr learned my addr 172.21.15.89:0/2212803557 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:28.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.886+0000 7efc977fe640 1 --2- 172.21.15.89:0/2212803557 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc980d0760 0x7efc98069900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:28.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.886+0000 7efc97fff640 1 --2- 172.21.15.89:0/2212803557 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc980cd160 0x7efc981519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:28.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.886+0000 7efc9ca68640 1 -- 172.21.15.89:0/2212803557 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc980cd160 msgr2=0x7efc981519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:28.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.886+0000 7efc9ca68640 1 --2- 172.21.15.89:0/2212803557 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc980cd160 0x7efc981519b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:28.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.886+0000 7efc9ca68640 1 -- 172.21.15.89:0/2212803557 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc980d0760 msgr2=0x7efc98069900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:28.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.886+0000 7efc9ca68640 1 --2- 172.21.15.89:0/2212803557 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc980d0760 0x7efc98069900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:28.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.886+0000 7efc9ca68640 1 -- 172.21.15.89:0/2212803557 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc90009b30 con 0x7efc98153670 2023-10-18T10:46:28.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.886+0000 7efc977fe640 1 --2- 172.21.15.89:0/2212803557 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc980d0760 0x7efc98069900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:28.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.886+0000 7efc9ca68640 1 --2- 172.21.15.89:0/2212803557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc98153670 0x7efc98069e40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7efc900af040 tx=0x7efc90019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:28.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.886+0000 7efc94ff9640 1 -- 172.21.15.89:0/2212803557 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc900b8670 con 0x7efc98153670 2023-10-18T10:46:28.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.886+0000 7efc94ff9640 1 -- 172.21.15.89:0/2212803557 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc900b87d0 con 0x7efc98153670 2023-10-18T10:46:28.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.886+0000 7efc9e4cb640 1 -- 172.21.15.89:0/2212803557 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc98151ef0 con 0x7efc98153670 2023-10-18T10:46:28.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.886+0000 7efc94ff9640 1 -- 172.21.15.89:0/2212803557 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc900b8b80 con 0x7efc98153670 2023-10-18T10:46:28.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.886+0000 7efc9e4cb640 1 -- 172.21.15.89:0/2212803557 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc980b6550 con 0x7efc98153670 2023-10-18T10:46:28.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.886+0000 7efc9e4cb640 1 -- 172.21.15.89:0/2212803557 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc980c9570 con 0x7efc98153670 2023-10-18T10:46:28.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.890+0000 7efc94ff9640 1 -- 172.21.15.89:0/2212803557 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7efc900b8ce0 con 0x7efc98153670 2023-10-18T10:46:28.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.890+0000 7efc94ff9640 1 --2- 172.21.15.89:0/2212803557 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efc84065680 0x7efc84067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:28.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.890+0000 7efc94ff9640 1 -- 172.21.15.89:0/2212803557 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7efc9011e8a0 con 0x7efc98153670 2023-10-18T10:46:28.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.890+0000 7efc97fff640 1 --2- 172.21.15.89:0/2212803557 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efc84065680 0x7efc84067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:28.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.890+0000 7efc97fff640 1 --2- 172.21.15.89:0/2212803557 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efc84065680 0x7efc84067b40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efc88000f70 tx=0x7efc88007c10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:28.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:28.894+0000 7efc94ff9640 1 -- 172.21.15.89:0/2212803557 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7efc900ec100 con 0x7efc98153670 2023-10-18T10:46:29.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:29.074+0000 7efc9e4cb640 1 -- 172.21.15.89:0/2212803557 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efc9815a290 con 0x7efc98153670 2023-10-18T10:46:29.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:29.074+0000 7efc94ff9640 1 -- 172.21.15.89:0/2212803557 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7efc9815a290 con 0x7efc98153670 2023-10-18T10:46:29.077 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-18T10:46:29.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:29.078+0000 7efc9e4cb640 1 -- 172.21.15.89:0/2212803557 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efc84065680 msgr2=0x7efc84067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:29.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:29.078+0000 7efc9e4cb640 1 --2- 172.21.15.89:0/2212803557 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efc84065680 0x7efc84067b40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efc88000f70 tx=0x7efc88007c10 comp rx=0 tx=0).stop 2023-10-18T10:46:29.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:29.078+0000 7efc9e4cb640 1 -- 172.21.15.89:0/2212803557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc98153670 msgr2=0x7efc98069e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:29.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:29.078+0000 7efc9e4cb640 1 --2- 172.21.15.89:0/2212803557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc98153670 0x7efc98069e40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7efc900af040 tx=0x7efc90019740 comp rx=0 tx=0).stop 2023-10-18T10:46:29.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:29.078+0000 7efc9e4cb640 1 -- 172.21.15.89:0/2212803557 shutdown_connections 2023-10-18T10:46:29.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:29.078+0000 7efc9e4cb640 1 --2- 172.21.15.89:0/2212803557 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efc84065680 0x7efc84067b40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:29.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:29.078+0000 7efc9e4cb640 1 --2- 172.21.15.89:0/2212803557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc98153670 0x7efc98069e40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:29.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:29.078+0000 7efc9e4cb640 1 --2- 172.21.15.89:0/2212803557 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc980d0760 0x7efc98069900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:29.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:29.078+0000 7efc9e4cb640 1 --2- 172.21.15.89:0/2212803557 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc980cd160 0x7efc981519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:29.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:29.078+0000 7efc9e4cb640 1 -- 172.21.15.89:0/2212803557 >> 172.21.15.89:0/2212803557 conn(0x7efc98089fd0 msgr2=0x7efc98158d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:29.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:29.078+0000 7efc9e4cb640 1 -- 172.21.15.89:0/2212803557 shutdown_connections 2023-10-18T10:46:29.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:29.078+0000 7efc9e4cb640 1 -- 172.21.15.89:0/2212803557 wait complete. 2023-10-18T10:46:29.098 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-18T10:46:29.098 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T10:46:29.099 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi089.front.sepia.ceph.com 2023-10-18T10:46:29.099 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-18T10:46:29.099 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-18T10:46:29.099 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-18T10:46:29.100 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-18T10:46:29.100 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T10:46:29.100 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-18T10:46:29.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:29.101 DEBUG:teuthology.orchestra.run.smithi089:> ip addr 2023-10-18T10:46:29.111 INFO:teuthology.orchestra.run.smithi089.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T10:46:29.111 INFO:teuthology.orchestra.run.smithi089.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T10:46:29.111 INFO:teuthology.orchestra.run.smithi089.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T10:46:29.111 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:29.112 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 ::1/128 scope host 2023-10-18T10:46:29.112 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:29.112 INFO:teuthology.orchestra.run.smithi089.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-18T10:46:29.112 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c6 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:46:29.112 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f0 2023-10-18T10:46:29.113 INFO:teuthology.orchestra.run.smithi089.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-18T10:46:29.113 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c7 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:46:29.113 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f1 2023-10-18T10:46:29.113 INFO:teuthology.orchestra.run.smithi089.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-18T10:46:29.113 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:00 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:46:29.114 INFO:teuthology.orchestra.run.smithi089.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T10:46:29.114 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:01 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:46:29.114 INFO:teuthology.orchestra.run.smithi089.stdout: inet 172.21.15.89/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-18T10:46:29.114 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft 42360sec preferred_lft 42360sec 2023-10-18T10:46:29.114 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec4:7aff:fe8f:d501/64 scope link 2023-10-18T10:46:29.115 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:29.115 INFO:teuthology.orchestra.run.smithi089.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-18T10:46:29.115 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 02:42:3f:fd:35:8d brd ff:ff:ff:ff:ff:ff 2023-10-18T10:46:29.115 INFO:teuthology.orchestra.run.smithi089.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-18T10:46:29.115 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:29.115 INFO:teuthology.orchestra.run.smithi089.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T10:46:29.116 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-18T10:46:29.116 INFO:teuthology.orchestra.run.smithi089.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T10:46:29.116 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:29.116 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link tentative 2023-10-18T10:46:29.116 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:29.117 INFO:teuthology.orchestra.run.smithi089.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T10:46:29.117 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:46:29.117 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link tentative 2023-10-18T10:46:29.117 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:29.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:29.118 DEBUG:teuthology.orchestra.run.smithi089:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T10:46:29.172 INFO:teuthology.orchestra.run.smithi089.stdout:1 2023-10-18T10:46:29.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:29.174 DEBUG:teuthology.orchestra.run.smithi089:> route 2023-10-18T10:46:29.228 INFO:teuthology.orchestra.run.smithi089.stdout:Kernel IP routing table 2023-10-18T10:46:29.228 INFO:teuthology.orchestra.run.smithi089.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T10:46:29.228 INFO:teuthology.orchestra.run.smithi089.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-18T10:46:29.228 INFO:teuthology.orchestra.run.smithi089.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T10:46:29.228 INFO:teuthology.orchestra.run.smithi089.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-18T10:46:29.229 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T10:46:29.229 INFO:teuthology.orchestra.run.smithi089.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-18T10:46:29.229 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-18T10:46:29.229 INFO:teuthology.orchestra.run.smithi089.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-18T10:46:29.230 INFO:teuthology.orchestra.run.smithi089.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T10:46:29.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:29.230 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-18T10:46:29.231 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-18T10:46:29.231 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T10:46:29.231 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T10:46:29.232 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T10:46:29.232 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-18T10:46:29.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:29.325 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-10-18T10:46:29.334 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-18T10:46:29.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:29.335 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list-id 2023-10-18T10:46:29.389 INFO:teuthology.orchestra.run.smithi089.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-18T10:46:29.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:29.391 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-18T10:46:29.391 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-18T10:46:29.392 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:46:29.392 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-18T10:46:29.392 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-18T10:46:29.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:29.478 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-18T10:46:29.542 INFO:teuthology.orchestra.run.smithi089.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T10:46:29.542 INFO:teuthology.orchestra.run.smithi089.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T10:46:29.543 INFO:teuthology.orchestra.run.smithi089.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T10:46:29.543 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:29.543 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 ::1/128 scope host 2023-10-18T10:46:29.543 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:29.544 INFO:teuthology.orchestra.run.smithi089.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T10:46:29.544 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 36:41:61:22:d1:22 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-18T10:46:29.544 INFO:teuthology.orchestra.run.smithi089.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-18T10:46:29.544 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:29.544 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::3441:61ff:fe22:d122/64 scope link 2023-10-18T10:46:29.545 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:29.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:29.546 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-18T10:46:29.610 INFO:teuthology.orchestra.run.smithi089.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T10:46:29.610 INFO:teuthology.orchestra.run.smithi089.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T10:46:29.611 INFO:teuthology.orchestra.run.smithi089.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T10:46:29.611 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:29.611 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 ::1/128 scope host 2023-10-18T10:46:29.611 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:29.611 INFO:teuthology.orchestra.run.smithi089.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T10:46:29.612 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 36:41:61:22:d1:22 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-18T10:46:29.612 INFO:teuthology.orchestra.run.smithi089.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-18T10:46:29.612 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:29.612 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::3441:61ff:fe22:d122/64 scope link 2023-10-18T10:46:29.612 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:29.613 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-18T10:46:29.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:29.614 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-18T10:46:29.615 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-18T10:46:29.615 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-18T10:46:29.615 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-18T10:46:29.615 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-18T10:46:29.616 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-18T10:46:29.616 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-18T10:46:29.616 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-18T10:46:29.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:29.759 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-18T10:46:29.759 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-18T10:46:29.760 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set brx.1 up 2023-10-18T10:46:29.760 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set dev brx.1 master ceph-brx 2023-10-18T10:46:29.760 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-18T10:46:29.801 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:46:29.802 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-18T10:46:29.854 INFO:teuthology.orchestra.run.smithi089.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-18T10:46:29.854 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:46:29.855 DEBUG:teuthology.orchestra.run.smithi089:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-18T10:46:29.905 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-18T10:46:29.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:29.906 DEBUG:teuthology.orchestra.run.smithi089:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-10-18T10:46:29.998 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-10-18T10:46:29.998 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-10-18T10:46:29.998 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-18T10:46:30.010 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-10-18T10:46:30.010 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-10-18T10:46:30.010 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-18T10:46:30.011 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.154:3300,172.21.15.154:3301" 2023-10-18T10:46:30.011 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-18T10:46:30.019 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-10-18T10:46:30.019 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-10-18T10:46:30.020 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-18T10:46:30.020 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.154:3300,172.21.15.154:3301" 2023-10-18T10:46:30.020 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-18T10:46:30.020 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.154:6789,172.21.15.154:6790" 2023-10-18T10:46:30.020 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.154:6789,172.21.15.154:6790:/ 2023-10-18T10:46:30.021 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=1352f62a-2330-4e96-afcd-b8e87cd82667" will pass to kernel 2023-10-18T10:46:30.023 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-18T10:46:30.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:30.023 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-18T10:46:30.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:30.044 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T10:46:30.100 INFO:teuthology.orchestra.run.smithi089.stdout:client4881 2023-10-18T10:46:30.101 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:46:30.101 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:46:30.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:30.150 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T10:46:30.206 INFO:teuthology.orchestra.run.smithi089.stdout:client4881 2023-10-18T10:46:30.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:30.207 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T10:46:30.260 INFO:teuthology.orchestra.run.smithi089.stdout:client4881 2023-10-18T10:46:30.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:30.261 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/1352f62a-2330-4e96-afcd-b8e87cd82667.client4881/status') 2023-10-18T10:46:30.326 INFO:teuthology.orchestra.run.smithi089.stdout:instance: client.4881 (3)192.168.0.2:0/655359641 2023-10-18T10:46:30.326 INFO:teuthology.orchestra.run.smithi089.stdout:blocklisted: false 2023-10-18T10:46:30.327 INFO:teuthology.orchestra.run.smithi089.stderr:0+1 records in 2023-10-18T10:46:30.327 INFO:teuthology.orchestra.run.smithi089.stderr:0+1 records out 2023-10-18T10:46:30.327 INFO:teuthology.orchestra.run.smithi089.stderr:68 bytes copied, 6.7086e-05 s, 1.0 MB/s 2023-10-18T10:46:30.328 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2023-10-18T10:46:30.328 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-18T10:46:30.328 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-18T10:46:30.329 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T10:46:30.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.509+0000 7f4506c9e640 1 Processor -- start 2023-10-18T10:46:30.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.509+0000 7f4506c9e640 1 -- start start 2023-10-18T10:46:30.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.509+0000 7f4506c9e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f450015c550 0x7f45000d0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:30.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.509+0000 7f4506c9e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f45000cf540 0x7f45000b5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:30.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.509+0000 7f4506c9e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45000d0e40 con 0x7f45000d1a40 2023-10-18T10:46:30.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.509+0000 7f4506c9e640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45000d1120 con 0x7f45000cf540 2023-10-18T10:46:30.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.509+0000 7f4506c9e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45000d1400 con 0x7f450015c550 2023-10-18T10:46:30.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.509+0000 7f450649d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f45000cf540 0x7f45000b5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:30.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.509+0000 7f450649d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f45000cf540 0x7f45000b5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59580/0 (socket says 172.21.15.89:59580) 2023-10-18T10:46:30.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.509+0000 7f450649d640 1 -- 172.21.15.89:0/1871868887 learned_addr learned my addr 172.21.15.89:0/1871868887 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:30.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.509+0000 7f4505c9c640 1 --2- 172.21.15.89:0/1871868887 >> v2:172.21.15.154:3300/0 conn(0x7f450015c550 0x7f45000d0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:30.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.509+0000 7f450649d640 1 -- 172.21.15.89:0/1871868887 >> v2:172.21.15.154:3300/0 conn(0x7f450015c550 msgr2=0x7f45000d0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:30.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.509+0000 7f450649d640 1 --2- 172.21.15.89:0/1871868887 >> v2:172.21.15.154:3300/0 conn(0x7f450015c550 0x7f45000d0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:30.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.509+0000 7f450649d640 1 -- 172.21.15.89:0/1871868887 >> v1:172.21.15.154:6789/0 conn(0x7f45000d1a40 legacy=0x7f45000cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:30.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.509+0000 7f450649d640 1 -- 172.21.15.89:0/1871868887 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45000b57c0 con 0x7f45000cf540 2023-10-18T10:46:30.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.513+0000 7f450649d640 1 --2- 172.21.15.89:0/1871868887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45000cf540 0x7f45000b5ce0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f45000d3570 tx=0x7f44f801c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=330f81e331f36c4a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:30.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.513+0000 7f4504c9a640 1 -- 172.21.15.89:0/1871868887 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f80a57b0 con 0x7f45000cf540 2023-10-18T10:46:30.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.513+0000 7f4504c9a640 1 -- 172.21.15.89:0/1871868887 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f80a5910 con 0x7f45000cf540 2023-10-18T10:46:30.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.513+0000 7f4504c9a640 1 -- 172.21.15.89:0/1871868887 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f80a5ca0 con 0x7f45000cf540 2023-10-18T10:46:30.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.513+0000 7f4506c9e640 1 -- 172.21.15.89:0/1871868887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45000cf540 msgr2=0x7f45000b5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:30.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.513+0000 7f4506c9e640 1 --2- 172.21.15.89:0/1871868887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45000cf540 0x7f45000b5ce0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f45000d3570 tx=0x7f44f801c9d0 comp rx=0 tx=0).stop 2023-10-18T10:46:30.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.513+0000 7f4506c9e640 1 -- 172.21.15.89:0/1871868887 shutdown_connections 2023-10-18T10:46:30.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.513+0000 7f4506c9e640 1 --2- 172.21.15.89:0/1871868887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45000cf540 0x7f45000b5ce0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:30.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.513+0000 7f4506c9e640 1 --2- 172.21.15.89:0/1871868887 >> v2:172.21.15.154:3300/0 conn(0x7f450015c550 0x7f45000d0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:30.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.513+0000 7f4506c9e640 1 -- 172.21.15.89:0/1871868887 >> 172.21.15.89:0/1871868887 conn(0x7f4500070320 msgr2=0x7f4500070730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:30.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.513+0000 7f4506c9e640 1 -- 172.21.15.89:0/1871868887 shutdown_connections 2023-10-18T10:46:30.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.513+0000 7f4506c9e640 1 -- 172.21.15.89:0/1871868887 wait complete. 2023-10-18T10:46:30.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.513+0000 7f4506c9e640 1 Processor -- start 2023-10-18T10:46:30.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.517+0000 7f4506c9e640 1 -- start start 2023-10-18T10:46:30.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.517+0000 7f4506c9e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45000cf540 0x7f4500158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:30.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.517+0000 7f4506c9e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45000d1a40 0x7f45000c99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:30.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.517+0000 7f4506c9e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f450015c550 0x7f45000b6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:30.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.517+0000 7f4506c9e640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45000ca3f0 con 0x7f450015c550 2023-10-18T10:46:30.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.517+0000 7f4506c9e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45000ca6d0 con 0x7f45000cf540 2023-10-18T10:46:30.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.517+0000 7f4506c9e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45000ca9b0 con 0x7f45000d1a40 2023-10-18T10:46:30.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.517+0000 7f450649d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f450015c550 0x7f45000b6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:30.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.517+0000 7f450649d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f450015c550 0x7f45000b6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59592/0 (socket says 172.21.15.89:59592) 2023-10-18T10:46:30.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.517+0000 7f450649d640 1 -- 172.21.15.89:0/1959450040 learned_addr learned my addr 172.21.15.89:0/1959450040 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:30.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.517+0000 7f4505c9c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45000cf540 0x7f4500158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:30.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.517+0000 7f450549b640 1 --2- 172.21.15.89:0/1959450040 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45000d1a40 0x7f45000c99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:30.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.517+0000 7f450649d640 1 -- 172.21.15.89:0/1959450040 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45000d1a40 msgr2=0x7f45000c99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:30.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.517+0000 7f450649d640 1 --2- 172.21.15.89:0/1959450040 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45000d1a40 0x7f45000c99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:30.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.517+0000 7f450649d640 1 -- 172.21.15.89:0/1959450040 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45000cf540 msgr2=0x7f4500158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:30.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.517+0000 7f450649d640 1 --2- 172.21.15.89:0/1959450040 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45000cf540 0x7f4500158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:30.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.517+0000 7f450649d640 1 -- 172.21.15.89:0/1959450040 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f8009b30 con 0x7f450015c550 2023-10-18T10:46:30.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.517+0000 7f4505c9c640 1 --2- 172.21.15.89:0/1959450040 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45000cf540 0x7f4500158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:46:30.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.517+0000 7f450649d640 1 --2- 172.21.15.89:0/1959450040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f450015c550 0x7f45000b6e50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f44f80af040 tx=0x7f44f8019770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:30.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.517+0000 7f44f67fc640 1 -- 172.21.15.89:0/1959450040 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f80b8670 con 0x7f450015c550 2023-10-18T10:46:30.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.517+0000 7f44f67fc640 1 -- 172.21.15.89:0/1959450040 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f80b87d0 con 0x7f450015c550 2023-10-18T10:46:30.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.517+0000 7f44f67fc640 1 -- 172.21.15.89:0/1959450040 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f80b8b80 con 0x7f450015c550 2023-10-18T10:46:30.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.517+0000 7f4506c9e640 1 -- 172.21.15.89:0/1959450040 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4500161540 con 0x7f450015c550 2023-10-18T10:46:30.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.517+0000 7f4506c9e640 1 -- 172.21.15.89:0/1959450040 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45000cb9e0 con 0x7f450015c550 2023-10-18T10:46:30.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.517+0000 7f4506c9e640 1 -- 172.21.15.89:0/1959450040 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4500161730 con 0x7f450015c550 2023-10-18T10:46:30.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.521+0000 7f44f67fc640 1 -- 172.21.15.89:0/1959450040 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f44f80b8ce0 con 0x7f450015c550 2023-10-18T10:46:30.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.521+0000 7f44f67fc640 1 --2- 172.21.15.89:0/1959450040 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f44e40656b0 0x7f44e4067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:30.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.521+0000 7f44f67fc640 1 -- 172.21.15.89:0/1959450040 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f44f811e4e0 con 0x7f450015c550 2023-10-18T10:46:30.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.521+0000 7f4505c9c640 1 --2- 172.21.15.89:0/1959450040 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f44e40656b0 0x7f44e4067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:30.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.521+0000 7f4505c9c640 1 --2- 172.21.15.89:0/1959450040 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f44e40656b0 0x7f44e4067b70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f44fc009eb0 tx=0x7f44fc009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:30.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.525+0000 7f44f67fc640 1 -- 172.21.15.89:0/1959450040 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f44f80ebd80 con 0x7f450015c550 2023-10-18T10:46:30.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.705+0000 7f4506c9e640 1 -- 172.21.15.89:0/1959450040 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f45000caaf0 con 0x7f450015c550 2023-10-18T10:46:30.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.705+0000 7f44f67fc640 1 -- 172.21.15.89:0/1959450040 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f45000caaf0 con 0x7f450015c550 2023-10-18T10:46:30.709 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-18T10:46:30.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.709+0000 7f4506c9e640 1 -- 172.21.15.89:0/1959450040 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f44e40656b0 msgr2=0x7f44e4067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:30.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.709+0000 7f4506c9e640 1 --2- 172.21.15.89:0/1959450040 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f44e40656b0 0x7f44e4067b70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f44fc009eb0 tx=0x7f44fc009190 comp rx=0 tx=0).stop 2023-10-18T10:46:30.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.709+0000 7f4506c9e640 1 -- 172.21.15.89:0/1959450040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f450015c550 msgr2=0x7f45000b6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:30.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.709+0000 7f4506c9e640 1 --2- 172.21.15.89:0/1959450040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f450015c550 0x7f45000b6e50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f44f80af040 tx=0x7f44f8019770 comp rx=0 tx=0).stop 2023-10-18T10:46:30.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.709+0000 7f4506c9e640 1 -- 172.21.15.89:0/1959450040 shutdown_connections 2023-10-18T10:46:30.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.709+0000 7f4506c9e640 1 --2- 172.21.15.89:0/1959450040 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f44e40656b0 0x7f44e4067b70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:30.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.709+0000 7f4506c9e640 1 --2- 172.21.15.89:0/1959450040 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45000d1a40 0x7f45000c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:30.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.709+0000 7f4506c9e640 1 --2- 172.21.15.89:0/1959450040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f450015c550 0x7f45000b6e50 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:30.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.709+0000 7f4506c9e640 1 --2- 172.21.15.89:0/1959450040 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45000cf540 0x7f4500158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:30.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.709+0000 7f4506c9e640 1 -- 172.21.15.89:0/1959450040 >> 172.21.15.89:0/1959450040 conn(0x7f4500070320 msgr2=0x7f45000b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:30.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.709+0000 7f4506c9e640 1 -- 172.21.15.89:0/1959450040 shutdown_connections 2023-10-18T10:46:30.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:30.709+0000 7f4506c9e640 1 -- 172.21.15.89:0/1959450040 wait complete. 2023-10-18T10:46:30.729 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-18T10:46:30.730 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T10:46:30.730 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2023-10-18T10:46:30.730 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-18T10:46:30.730 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-18T10:46:30.731 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-18T10:46:30.731 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-18T10:46:30.731 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T10:46:30.731 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-18T10:46:30.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:30.732 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2023-10-18T10:46:30.743 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T10:46:30.743 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T10:46:30.743 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T10:46:30.743 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:30.744 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2023-10-18T10:46:30.744 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:30.744 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-18T10:46:30.744 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:46:30.744 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f0 2023-10-18T10:46:30.745 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-18T10:46:30.745 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:46:30.745 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f1 2023-10-18T10:46:30.745 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-18T10:46:30.746 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:46:30.746 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T10:46:30.746 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:46:30.746 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-18T10:46:30.746 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 42389sec preferred_lft 42389sec 2023-10-18T10:46:30.747 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::ec4:7aff:fe88:7761/64 scope link 2023-10-18T10:46:30.747 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:30.747 INFO:teuthology.orchestra.run.smithi154.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-18T10:46:30.747 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 02:42:60:ea:f7:c5 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:46:30.747 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-18T10:46:30.748 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:30.749 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-18T10:46:30.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:30.749 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-10-18T10:46:30.750 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-10-18T10:46:30.750 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link add name ceph-brx type bridge 2023-10-18T10:46:30.750 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip addr flush dev ceph-brx 2023-10-18T10:46:30.751 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set ceph-brx up 2023-10-18T10:46:30.751 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-18T10:46:30.751 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-10-18T10:46:30.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:30.870 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T10:46:30.889 INFO:teuthology.orchestra.run.smithi154.stdout:1 2023-10-18T10:46:30.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:30.892 DEBUG:teuthology.orchestra.run.smithi154:> route 2023-10-18T10:46:30.991 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2023-10-18T10:46:30.991 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T10:46:30.992 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-18T10:46:30.992 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T10:46:30.992 INFO:teuthology.orchestra.run.smithi154.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-18T10:46:30.992 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T10:46:30.992 INFO:teuthology.orchestra.run.smithi154.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-18T10:46:30.993 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-18T10:46:30.993 INFO:teuthology.orchestra.run.smithi154.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-18T10:46:30.993 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T10:46:30.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:30.994 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-10-18T10:46:30.994 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-10-18T10:46:30.995 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T10:46:30.995 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T10:46:30.995 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T10:46:30.995 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-10-18T10:46:31.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:31.048 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2023-10-18T10:46:31.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:31.059 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list-id 2023-10-18T10:46:31.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:31.109 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-10-18T10:46:31.109 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-10-18T10:46:31.109 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-18T10:46:31.110 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-18T10:46:31.110 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-10-18T10:46:31.195 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-18T10:46:31.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:31.196 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-10-18T10:46:31.196 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-10-18T10:46:31.197 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-18T10:46:31.197 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-18T10:46:31.197 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-18T10:46:31.197 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-18T10:46:31.198 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-18T10:46:31.198 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-10-18T10:46:31.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:31.371 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-10-18T10:46:31.372 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-10-18T10:46:31.372 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.0 up 2023-10-18T10:46:31.372 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set dev brx.0 master ceph-brx 2023-10-18T10:46:31.372 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-10-18T10:46:31.413 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:46:31.414 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-18T10:46:31.467 INFO:teuthology.orchestra.run.smithi154.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-18T10:46:31.468 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:46:31.468 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-18T10:46:31.519 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-18T10:46:31.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:31.520 DEBUG:teuthology.orchestra.run.smithi154:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-18T10:46:31.638 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-18T10:46:31.638 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-18T10:46:31.638 INFO:teuthology.orchestra.run.smithi154.stdout:invalid new device string format 2023-10-18T10:46:31.773 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-18T10:46:31.774 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-18T10:46:31.774 INFO:teuthology.orchestra.run.smithi154.stdout:invalid new device string format 2023-10-18T10:46:31.774 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.154:3300,172.21.15.154:3301" 2023-10-18T10:46:31.774 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-18T10:46:31.784 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-18T10:46:31.785 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-18T10:46:31.785 INFO:teuthology.orchestra.run.smithi154.stdout:invalid new device string format 2023-10-18T10:46:31.785 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.154:3300,172.21.15.154:3301" 2023-10-18T10:46:31.785 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-18T10:46:31.785 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.154:6789,172.21.15.154:6790" 2023-10-18T10:46:31.786 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.154:6789,172.21.15.154:6790:/ 2023-10-18T10:46:31.786 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=1352f62a-2330-4e96-afcd-b8e87cd82667" will pass to kernel 2023-10-18T10:46:31.788 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-18T10:46:31.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:31.789 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-18T10:46:31.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:31.813 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T10:46:31.970 INFO:teuthology.orchestra.run.smithi154.stdout:client4670 2023-10-18T10:46:31.970 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T10:46:31.971 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:46:31.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:31.981 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T10:46:32.035 INFO:teuthology.orchestra.run.smithi154.stdout:client4670 2023-10-18T10:46:32.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:32.036 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T10:46:32.091 INFO:teuthology.orchestra.run.smithi154.stdout:client4670 2023-10-18T10:46:32.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:32.092 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/1352f62a-2330-4e96-afcd-b8e87cd82667.client4670/status') 2023-10-18T10:46:32.155 INFO:teuthology.orchestra.run.smithi154.stdout:instance: client.4670 (3)192.168.0.1:0/1538657397 2023-10-18T10:46:32.156 INFO:teuthology.orchestra.run.smithi154.stdout:blocklisted: false 2023-10-18T10:46:32.156 INFO:teuthology.orchestra.run.smithi154.stderr:0+1 records in 2023-10-18T10:46:32.156 INFO:teuthology.orchestra.run.smithi154.stderr:0+1 records out 2023-10-18T10:46:32.156 INFO:teuthology.orchestra.run.smithi154.stderr:69 bytes copied, 4.9753e-05 s, 1.4 MB/s 2023-10-18T10:46:32.157 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2023-10-18T10:46:32.158 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-18T10:46:32.158 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-18T10:46:32.158 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T10:46:32.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc70beca640 1 Processor -- start 2023-10-18T10:46:32.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc70beca640 1 -- start start 2023-10-18T10:46:32.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc70beca640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc70406ff00 0x7fc70415e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:32.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc70beca640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc7040ca340 0x7fc70415b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:32.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc70beca640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc70415acb0 con 0x7fc70415b680 2023-10-18T10:46:32.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc70beca640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc70415ef30 con 0x7fc7040ca340 2023-10-18T10:46:32.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc70beca640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc70415aa10 con 0x7fc70406ff00 2023-10-18T10:46:32.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc70a467640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc70415b680 0x7fc7040c8960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47874/0 (socket says 172.21.15.89:47874) 2023-10-18T10:46:32.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc70a467640 1 -- 172.21.15.89:0/3178964575 learned_addr learned my addr 172.21.15.89:0/3178964575 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:32.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc709465640 1 --2- 172.21.15.89:0/3178964575 >> v2:172.21.15.154:3300/0 conn(0x7fc7040ca340 0x7fc70415b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:32.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc709c66640 1 --2- 172.21.15.89:0/3178964575 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc70406ff00 0x7fc70415e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:32.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc708c64640 1 -- 172.21.15.89:0/3178964575 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2126131463 0 0) 0x7fc70415acb0 con 0x7fc70415b680 2023-10-18T10:46:32.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc708c64640 1 -- 172.21.15.89:0/3178964575 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6f0003530 con 0x7fc70415b680 2023-10-18T10:46:32.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc708c64640 1 -- 172.21.15.89:0/3178964575 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4060470872 0 0) 0x7fc6f0003530 con 0x7fc70415b680 2023-10-18T10:46:32.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc708c64640 1 -- 172.21.15.89:0/3178964575 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc70406ff00 msgr2=0x7fc70415e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:32.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc708c64640 1 --2- 172.21.15.89:0/3178964575 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc70406ff00 0x7fc70415e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:32.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc708c64640 1 -- 172.21.15.89:0/3178964575 >> v2:172.21.15.154:3300/0 conn(0x7fc7040ca340 msgr2=0x7fc70415b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:32.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc708c64640 1 --2- 172.21.15.89:0/3178964575 >> v2:172.21.15.154:3300/0 conn(0x7fc7040ca340 0x7fc70415b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:32.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc708c64640 1 -- 172.21.15.89:0/3178964575 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7040702e0 con 0x7fc70415b680 2023-10-18T10:46:32.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc708c64640 1 -- 172.21.15.89:0/3178964575 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7fc6fc001a00 con 0x7fc70415b680 2023-10-18T10:46:32.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc709c66640 1 --2- 172.21.15.89:0/3178964575 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc70406ff00 0x7fc70415e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:46:32.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc709465640 1 --2- 172.21.15.89:0/3178964575 >> v2:172.21.15.154:3300/0 conn(0x7fc7040ca340 0x7fc70415b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:46:32.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc708c64640 1 -- 172.21.15.89:0/3178964575 >> v1:172.21.15.89:6789/0 conn(0x7fc70415b680 legacy=0x7fc7040c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:32.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc708c64640 1 --2- 172.21.15.89:0/3178964575 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc6f0003980 0x7fc6f0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:32.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc708c64640 1 --2- 172.21.15.89:0/3178964575 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc70406ff00 0x7fc6f00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:32.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc708c64640 1 --2- 172.21.15.89:0/3178964575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6f0004d10 0x7fc6f0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:32.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc708c64640 1 -- 172.21.15.89:0/3178964575 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70415ef30 con 0x7fc6f0004d10 2023-10-18T10:46:32.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc708c64640 1 -- 172.21.15.89:0/3178964575 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70415aa10 con 0x7fc6f0003980 2023-10-18T10:46:32.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc708c64640 1 -- 172.21.15.89:0/3178964575 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f000b990 con 0x7fc70406ff00 2023-10-18T10:46:32.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc70a467640 1 --2- 172.21.15.89:0/3178964575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6f0004d10 0x7fc6f0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:32.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc709c66640 1 --2- 172.21.15.89:0/3178964575 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc6f0003980 0x7fc6f0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:32.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc709465640 1 --2- 172.21.15.89:0/3178964575 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc70406ff00 0x7fc6f00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:32.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc70a467640 1 -- 172.21.15.89:0/3178964575 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc70406ff00 msgr2=0x7fc6f00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:32.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc70a467640 1 --2- 172.21.15.89:0/3178964575 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc70406ff00 0x7fc6f00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:32.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc70a467640 1 -- 172.21.15.89:0/3178964575 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc6f0003980 msgr2=0x7fc6f0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:32.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc70a467640 1 --2- 172.21.15.89:0/3178964575 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc6f0003980 0x7fc6f0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:32.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.305+0000 7fc70a467640 1 -- 172.21.15.89:0/3178964575 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc700009420 con 0x7fc6f0004d10 2023-10-18T10:46:32.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.309+0000 7fc70a467640 1 --2- 172.21.15.89:0/3178964575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6f0004d10 0x7fc6f0005190 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00a810 tx=0x7fc6fc01fa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7282ddd8115b6624 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:32.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.309+0000 7fc708c64640 1 -- 172.21.15.89:0/3178964575 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc0aa6b0 con 0x7fc6f0004d10 2023-10-18T10:46:32.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.309+0000 7fc708c64640 1 -- 172.21.15.89:0/3178964575 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6fc0aa830 con 0x7fc6f0004d10 2023-10-18T10:46:32.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.309+0000 7fc708c64640 1 -- 172.21.15.89:0/3178964575 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc0aabe0 con 0x7fc6f0004d10 2023-10-18T10:46:32.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.309+0000 7fc70beca640 1 -- 172.21.15.89:0/3178964575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6f0004d10 msgr2=0x7fc6f0005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:32.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.309+0000 7fc70beca640 1 --2- 172.21.15.89:0/3178964575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6f0004d10 0x7fc6f0005190 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00a810 tx=0x7fc6fc01fa20 comp rx=0 tx=0).stop 2023-10-18T10:46:32.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.309+0000 7fc70a467640 1 -- 172.21.15.89:0/3178964575 reap_dead start 2023-10-18T10:46:32.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.309+0000 7fc70beca640 1 -- 172.21.15.89:0/3178964575 shutdown_connections 2023-10-18T10:46:32.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.309+0000 7fc70beca640 1 -- 172.21.15.89:0/3178964575 >> 172.21.15.89:0/3178964575 conn(0x7fc704069f00 msgr2=0x7fc7040cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:32.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.309+0000 7fc70beca640 1 -- 172.21.15.89:0/3178964575 shutdown_connections 2023-10-18T10:46:32.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.309+0000 7fc70beca640 1 -- 172.21.15.89:0/3178964575 wait complete. 2023-10-18T10:46:32.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.313+0000 7fc70beca640 1 Processor -- start 2023-10-18T10:46:32.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.313+0000 7fc70beca640 1 -- start start 2023-10-18T10:46:32.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.313+0000 7fc70beca640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc6f0004d10 0x7fc70414dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:32.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.313+0000 7fc70beca640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc70415d110 0x7fc70415d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:32.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.313+0000 7fc70beca640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc70414f910 0x7fc70414fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:32.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.313+0000 7fc70beca640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70415dad0 con 0x7fc70415d110 2023-10-18T10:46:32.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.313+0000 7fc70beca640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7040c5db0 con 0x7fc6f0004d10 2023-10-18T10:46:32.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.313+0000 7fc70beca640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7041502b0 con 0x7fc70414f910 2023-10-18T10:46:32.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.313+0000 7fc709465640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc70415d110 0x7fc70415d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:32.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.313+0000 7fc709465640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc70415d110 0x7fc70415d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46930/0 (socket says 172.21.15.89:46930) 2023-10-18T10:46:32.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.313+0000 7fc709465640 1 -- 172.21.15.89:0/775235935 learned_addr learned my addr 172.21.15.89:0/775235935 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:32.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.313+0000 7fc709c66640 1 --2- 172.21.15.89:0/775235935 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc6f0004d10 0x7fc70414dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:32.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.313+0000 7fc70a467640 1 --2- 172.21.15.89:0/775235935 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc70414f910 0x7fc70414fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:32.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.313+0000 7fc709465640 1 -- 172.21.15.89:0/775235935 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc70414f910 msgr2=0x7fc70414fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:32.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.313+0000 7fc709465640 1 --2- 172.21.15.89:0/775235935 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc70414f910 0x7fc70414fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:32.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.313+0000 7fc709465640 1 -- 172.21.15.89:0/775235935 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc6f0004d10 msgr2=0x7fc70414dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:32.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.313+0000 7fc709465640 1 --2- 172.21.15.89:0/775235935 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc6f0004d10 0x7fc70414dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:32.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.313+0000 7fc709465640 1 -- 172.21.15.89:0/775235935 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7040c67c0 con 0x7fc70415d110 2023-10-18T10:46:32.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.313+0000 7fc709465640 1 --2- 172.21.15.89:0/775235935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc70415d110 0x7fc70415d590 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4011cd0 tx=0x7fc6f400f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:32.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.313+0000 7fc6fa7fc640 1 -- 172.21.15.89:0/775235935 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f4011e80 con 0x7fc70415d110 2023-10-18T10:46:32.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.313+0000 7fc6fa7fc640 1 -- 172.21.15.89:0/775235935 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f4004020 con 0x7fc70415d110 2023-10-18T10:46:32.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.313+0000 7fc70beca640 1 -- 172.21.15.89:0/775235935 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc704150560 con 0x7fc70415d110 2023-10-18T10:46:32.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.313+0000 7fc6fa7fc640 1 -- 172.21.15.89:0/775235935 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f40043b0 con 0x7fc70415d110 2023-10-18T10:46:32.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.313+0000 7fc70beca640 1 -- 172.21.15.89:0/775235935 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7040c5760 con 0x7fc70415d110 2023-10-18T10:46:32.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.317+0000 7fc70beca640 1 -- 172.21.15.89:0/775235935 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc70404dbe0 con 0x7fc70415d110 2023-10-18T10:46:32.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.317+0000 7fc6fa7fc640 1 -- 172.21.15.89:0/775235935 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fc6f4003620 con 0x7fc70415d110 2023-10-18T10:46:32.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.317+0000 7fc6fa7fc640 1 --2- 172.21.15.89:0/775235935 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc6e8065680 0x7fc6e8067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:32.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.317+0000 7fc6fa7fc640 1 -- 172.21.15.89:0/775235935 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc6f4083870 con 0x7fc70415d110 2023-10-18T10:46:32.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.317+0000 7fc709c66640 1 --2- 172.21.15.89:0/775235935 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc6e8065680 0x7fc6e8067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:32.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.317+0000 7fc709c66640 1 --2- 172.21.15.89:0/775235935 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc6e8065680 0x7fc6e8067b40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc70414cb20 tx=0x7fc700009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:32.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.321+0000 7fc6fa7fc640 1 -- 172.21.15.89:0/775235935 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fc6f4051180 con 0x7fc70415d110 2023-10-18T10:46:32.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.501+0000 7fc70beca640 1 -- 172.21.15.89:0/775235935 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc70415a450 con 0x7fc70415d110 2023-10-18T10:46:32.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.505+0000 7fc6fa7fc640 1 -- 172.21.15.89:0/775235935 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fc6f4022070 con 0x7fc70415d110 2023-10-18T10:46:32.506 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-18T10:46:32.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.505+0000 7fc70beca640 1 -- 172.21.15.89:0/775235935 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc6e8065680 msgr2=0x7fc6e8067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:32.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.505+0000 7fc70beca640 1 --2- 172.21.15.89:0/775235935 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc6e8065680 0x7fc6e8067b40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc70414cb20 tx=0x7fc700009190 comp rx=0 tx=0).stop 2023-10-18T10:46:32.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.505+0000 7fc70beca640 1 -- 172.21.15.89:0/775235935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc70415d110 msgr2=0x7fc70415d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:32.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.505+0000 7fc70beca640 1 --2- 172.21.15.89:0/775235935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc70415d110 0x7fc70415d590 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4011cd0 tx=0x7fc6f400f610 comp rx=0 tx=0).stop 2023-10-18T10:46:32.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.505+0000 7fc70beca640 1 -- 172.21.15.89:0/775235935 shutdown_connections 2023-10-18T10:46:32.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.505+0000 7fc70beca640 1 --2- 172.21.15.89:0/775235935 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc6e8065680 0x7fc6e8067b40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:32.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.505+0000 7fc70beca640 1 --2- 172.21.15.89:0/775235935 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc70414f910 0x7fc70414fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:32.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.505+0000 7fc70beca640 1 --2- 172.21.15.89:0/775235935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc70415d110 0x7fc70415d590 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:32.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.505+0000 7fc70beca640 1 --2- 172.21.15.89:0/775235935 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc6f0004d10 0x7fc70414dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:32.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.505+0000 7fc70beca640 1 -- 172.21.15.89:0/775235935 >> 172.21.15.89:0/775235935 conn(0x7fc704069f00 msgr2=0x7fc70406b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:32.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.505+0000 7fc70beca640 1 -- 172.21.15.89:0/775235935 shutdown_connections 2023-10-18T10:46:32.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:32.505+0000 7fc70beca640 1 -- 172.21.15.89:0/775235935 wait complete. 2023-10-18T10:46:32.527 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-18T10:46:32.527 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T10:46:32.528 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2023-10-18T10:46:32.528 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-18T10:46:32.528 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-18T10:46:32.529 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-18T10:46:32.529 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-18T10:46:32.529 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T10:46:32.529 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-18T10:46:32.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:32.530 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2023-10-18T10:46:32.540 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T10:46:32.541 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T10:46:32.541 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T10:46:32.541 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:32.542 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2023-10-18T10:46:32.542 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:32.542 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-18T10:46:32.542 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:46:32.542 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f0 2023-10-18T10:46:32.543 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-18T10:46:32.543 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:46:32.543 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f1 2023-10-18T10:46:32.543 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-18T10:46:32.543 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:46:32.544 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T10:46:32.544 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:46:32.544 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-18T10:46:32.544 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 42387sec preferred_lft 42387sec 2023-10-18T10:46:32.544 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::ec4:7aff:fe88:7761/64 scope link 2023-10-18T10:46:32.545 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:32.545 INFO:teuthology.orchestra.run.smithi154.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-18T10:46:32.545 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 02:42:60:ea:f7:c5 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:46:32.545 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-18T10:46:32.545 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:32.546 INFO:teuthology.orchestra.run.smithi154.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T10:46:32.546 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-18T10:46:32.546 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T10:46:32.546 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:32.546 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link tentative 2023-10-18T10:46:32.547 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:32.547 INFO:teuthology.orchestra.run.smithi154.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T10:46:32.547 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-18T10:46:32.547 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link tentative 2023-10-18T10:46:32.547 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:32.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:32.549 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T10:46:32.611 INFO:teuthology.orchestra.run.smithi154.stdout:1 2023-10-18T10:46:32.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:32.613 DEBUG:teuthology.orchestra.run.smithi154:> route 2023-10-18T10:46:32.667 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2023-10-18T10:46:32.667 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T10:46:32.667 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-18T10:46:32.668 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T10:46:32.668 INFO:teuthology.orchestra.run.smithi154.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-18T10:46:32.669 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T10:46:32.669 INFO:teuthology.orchestra.run.smithi154.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-18T10:46:32.669 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-18T10:46:32.669 INFO:teuthology.orchestra.run.smithi154.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-18T10:46:32.669 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T10:46:32.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:32.670 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-10-18T10:46:32.671 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-10-18T10:46:32.671 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T10:46:32.671 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T10:46:32.672 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T10:46:32.672 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-10-18T10:46:32.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:32.766 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2023-10-18T10:46:32.775 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-18T10:46:32.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:32.776 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list-id 2023-10-18T10:46:32.827 INFO:teuthology.orchestra.run.smithi154.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-18T10:46:32.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:32.829 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-10-18T10:46:32.829 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-10-18T10:46:32.830 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-18T10:46:32.830 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-18T10:46:32.830 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-10-18T10:46:32.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:32.911 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-18T10:46:32.973 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T10:46:32.973 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T10:46:32.973 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T10:46:32.973 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:32.974 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2023-10-18T10:46:32.974 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:32.974 INFO:teuthology.orchestra.run.smithi154.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T10:46:32.974 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0e:2a:a1:bd:98:0c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-18T10:46:32.975 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-18T10:46:32.975 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:32.975 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::c2a:a1ff:febd:980c/64 scope link 2023-10-18T10:46:32.976 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:32.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:32.977 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-18T10:46:33.040 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T10:46:33.040 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T10:46:33.041 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T10:46:33.041 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:33.041 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2023-10-18T10:46:33.041 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:33.041 INFO:teuthology.orchestra.run.smithi154.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T10:46:33.042 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0e:2a:a1:bd:98:0c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-18T10:46:33.042 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-18T10:46:33.042 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:33.042 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::c2a:a1ff:febd:980c/64 scope link 2023-10-18T10:46:33.043 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:46:33.044 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-18T10:46:33.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:33.044 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-10-18T10:46:33.045 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-10-18T10:46:33.045 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-18T10:46:33.045 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-18T10:46:33.045 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-18T10:46:33.046 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-18T10:46:33.046 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-18T10:46:33.046 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-10-18T10:46:33.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:33.185 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-10-18T10:46:33.185 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-10-18T10:46:33.185 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.1 up 2023-10-18T10:46:33.186 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set dev brx.1 master ceph-brx 2023-10-18T10:46:33.186 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-10-18T10:46:33.226 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:46:33.227 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-18T10:46:33.276 INFO:teuthology.orchestra.run.smithi154.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-18T10:46:33.277 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:46:33.277 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-18T10:46:33.327 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-18T10:46:33.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:33.328 DEBUG:teuthology.orchestra.run.smithi154:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-10-18T10:46:33.420 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-10-18T10:46:33.421 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-10-18T10:46:33.421 INFO:teuthology.orchestra.run.smithi154.stdout:invalid new device string format 2023-10-18T10:46:33.436 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-10-18T10:46:33.436 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-10-18T10:46:33.436 INFO:teuthology.orchestra.run.smithi154.stdout:invalid new device string format 2023-10-18T10:46:33.437 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.154:3300,172.21.15.154:3301" 2023-10-18T10:46:33.437 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-18T10:46:33.445 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-10-18T10:46:33.445 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-10-18T10:46:33.445 INFO:teuthology.orchestra.run.smithi154.stdout:invalid new device string format 2023-10-18T10:46:33.446 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.154:3300,172.21.15.154:3301" 2023-10-18T10:46:33.446 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-18T10:46:33.446 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.154:6789,172.21.15.154:6790" 2023-10-18T10:46:33.446 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.154:6789,172.21.15.154:6790:/ 2023-10-18T10:46:33.447 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=1352f62a-2330-4e96-afcd-b8e87cd82667" will pass to kernel 2023-10-18T10:46:33.449 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-18T10:46:33.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:33.450 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-18T10:46:33.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:33.470 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T10:46:33.523 INFO:teuthology.orchestra.run.smithi154.stdout:client4822 2023-10-18T10:46:33.523 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T10:46:33.523 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:46:33.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:33.572 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T10:46:33.627 INFO:teuthology.orchestra.run.smithi154.stdout:client4822 2023-10-18T10:46:33.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:33.628 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T10:46:33.678 INFO:teuthology.orchestra.run.smithi154.stdout:client4822 2023-10-18T10:46:33.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:33.682 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/1352f62a-2330-4e96-afcd-b8e87cd82667.client4822/status') 2023-10-18T10:46:33.739 INFO:teuthology.orchestra.run.smithi154.stdout:instance: client.4822 (3)192.168.0.2:0/2685786461 2023-10-18T10:46:33.740 INFO:teuthology.orchestra.run.smithi154.stdout:blocklisted: false 2023-10-18T10:46:33.740 INFO:teuthology.orchestra.run.smithi154.stderr:0+1 records in 2023-10-18T10:46:33.740 INFO:teuthology.orchestra.run.smithi154.stderr:0+1 records out 2023-10-18T10:46:33.740 INFO:teuthology.orchestra.run.smithi154.stderr:69 bytes copied, 4.8821e-05 s, 1.4 MB/s 2023-10-18T10:46:33.742 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-18T10:46:33.763 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:46:33.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.909+0000 7f53fe89c640 1 Processor -- start 2023-10-18T10:46:33.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.909+0000 7f53fe89c640 1 -- start start 2023-10-18T10:46:33.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.909+0000 7f53fe89c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f53f815c940 0x7f53f80d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:33.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.909+0000 7f53fe89c640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53f815da50 con 0x7f53f80d0e70 2023-10-18T10:46:33.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.909+0000 7f53fe89c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53f815ce40 con 0x7f53f80d2f40 2023-10-18T10:46:33.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.909+0000 7f53fe89c640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53f815d120 con 0x7f53f815c940 2023-10-18T10:46:33.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.909+0000 7f53f7fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f53f815c940 0x7f53f80d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:33.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.909+0000 7f53f7fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f53f815c940 0x7f53f80d2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46944/0 (socket says 172.21.15.89:46944) 2023-10-18T10:46:33.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.909+0000 7f53f7fff640 1 -- 172.21.15.89:0/938797733 learned_addr learned my addr 172.21.15.89:0/938797733 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:33.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.909+0000 7f53fce39640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f53f80d0e70 0x7f53f80cd160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47878/0 (socket says 172.21.15.89:47878) 2023-10-18T10:46:33.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.909+0000 7f53f7fff640 1 -- 172.21.15.89:0/938797733 >> v1:172.21.15.154:6789/0 conn(0x7f53f80d2f40 legacy=0x7f53f80d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:33.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.909+0000 7f53f7fff640 1 -- 172.21.15.89:0/938797733 >> v1:172.21.15.89:6789/0 conn(0x7f53f80d0e70 legacy=0x7f53f80cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:33.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.909+0000 7f53f7fff640 1 -- 172.21.15.89:0/938797733 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f815d400 con 0x7f53f815c940 2023-10-18T10:46:33.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.909+0000 7f53f6ffd640 1 -- 172.21.15.89:0/938797733 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2837164819 0 0) 0x7f53f815da50 con 0x7f53f80d0e70 2023-10-18T10:46:33.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.913+0000 7f53f7fff640 1 --2- 172.21.15.89:0/938797733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53f815c940 0x7f53f80d2a00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009870 tx=0x7f53e801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f90c86f76979d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:33.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.913+0000 7f53f6ffd640 1 -- 172.21.15.89:0/938797733 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e80aa650 con 0x7f53f815c940 2023-10-18T10:46:33.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.913+0000 7f53f6ffd640 1 -- 172.21.15.89:0/938797733 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e80aa7b0 con 0x7f53f815c940 2023-10-18T10:46:33.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.913+0000 7f53f6ffd640 1 -- 172.21.15.89:0/938797733 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e80aab40 con 0x7f53f815c940 2023-10-18T10:46:33.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.913+0000 7f53fe89c640 1 -- 172.21.15.89:0/938797733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53f815c940 msgr2=0x7f53f80d2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:33.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.913+0000 7f53fe89c640 1 --2- 172.21.15.89:0/938797733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53f815c940 0x7f53f80d2a00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009870 tx=0x7f53e801c4d0 comp rx=0 tx=0).stop 2023-10-18T10:46:33.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.913+0000 7f53fe89c640 1 -- 172.21.15.89:0/938797733 shutdown_connections 2023-10-18T10:46:33.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.913+0000 7f53fe89c640 1 --2- 172.21.15.89:0/938797733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53f815c940 0x7f53f80d2a00 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:33.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.913+0000 7f53fe89c640 1 -- 172.21.15.89:0/938797733 >> 172.21.15.89:0/938797733 conn(0x7f53f8089fd0 msgr2=0x7f53f809b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:33.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.913+0000 7f53fe89c640 1 -- 172.21.15.89:0/938797733 shutdown_connections 2023-10-18T10:46:33.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.913+0000 7f53fe89c640 1 -- 172.21.15.89:0/938797733 wait complete. 2023-10-18T10:46:33.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.917+0000 7f53fe89c640 1 Processor -- start 2023-10-18T10:46:33.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.917+0000 7f53fe89c640 1 -- start start 2023-10-18T10:46:33.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.917+0000 7f53fe89c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53f80d0e70 0x7f53f8153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:33.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.917+0000 7f53fe89c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53f80c6210 0x7f53f8159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:33.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.917+0000 7f53fe89c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53f80c8450 0x7f53f8159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:33.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.917+0000 7f53fe89c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f80c8950 con 0x7f53f80c8450 2023-10-18T10:46:33.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.917+0000 7f53fe89c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f80c8c30 con 0x7f53f80c6210 2023-10-18T10:46:33.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.917+0000 7f53fe89c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f80c8f10 con 0x7f53f80d0e70 2023-10-18T10:46:33.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.917+0000 7f53fce39640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53f80c8450 0x7f53f8159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:33.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.917+0000 7f53fce39640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53f80c8450 0x7f53f8159680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46948/0 (socket says 172.21.15.89:46948) 2023-10-18T10:46:33.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.917+0000 7f53fce39640 1 -- 172.21.15.89:0/4168173997 learned_addr learned my addr 172.21.15.89:0/4168173997 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:33.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.917+0000 7f53f7fff640 1 --2- 172.21.15.89:0/4168173997 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53f80d0e70 0x7f53f8153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:33.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.917+0000 7f53f77fe640 1 --2- 172.21.15.89:0/4168173997 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53f80c6210 0x7f53f8159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:33.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.917+0000 7f53fce39640 1 -- 172.21.15.89:0/4168173997 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53f80d0e70 msgr2=0x7f53f8153360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:33.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.917+0000 7f53fce39640 1 --2- 172.21.15.89:0/4168173997 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53f80d0e70 0x7f53f8153360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:33.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.917+0000 7f53fce39640 1 -- 172.21.15.89:0/4168173997 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53f80c6210 msgr2=0x7f53f8159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:33.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.917+0000 7f53fce39640 1 --2- 172.21.15.89:0/4168173997 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53f80c6210 0x7f53f8159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:33.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.917+0000 7f53fce39640 1 -- 172.21.15.89:0/4168173997 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e80094d0 con 0x7f53f80c8450 2023-10-18T10:46:33.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.917+0000 7f53f77fe640 1 --2- 172.21.15.89:0/4168173997 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53f80c6210 0x7f53f8159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:33.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.917+0000 7f53fce39640 1 --2- 172.21.15.89:0/4168173997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53f80c8450 0x7f53f8159680 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f53f0001570 tx=0x7f53f000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:33.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.917+0000 7f53f4ff9640 1 -- 172.21.15.89:0/4168173997 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f000a640 con 0x7f53f80c8450 2023-10-18T10:46:33.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.917+0000 7f53f4ff9640 1 -- 172.21.15.89:0/4168173997 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f000a7a0 con 0x7f53f80c8450 2023-10-18T10:46:33.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.917+0000 7f53f4ff9640 1 -- 172.21.15.89:0/4168173997 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f000ab90 con 0x7f53f80c8450 2023-10-18T10:46:33.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.917+0000 7f53fe89c640 1 -- 172.21.15.89:0/4168173997 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53f80c91f0 con 0x7f53f80c8450 2023-10-18T10:46:33.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.917+0000 7f53fe89c640 1 -- 172.21.15.89:0/4168173997 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53f8153c00 con 0x7f53f80c8450 2023-10-18T10:46:33.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.921+0000 7f53fe89c640 1 -- 172.21.15.89:0/4168173997 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53f8153f30 con 0x7f53f80c8450 2023-10-18T10:46:33.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.921+0000 7f53f4ff9640 1 -- 172.21.15.89:0/4168173997 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f53f000acf0 con 0x7f53f80c8450 2023-10-18T10:46:33.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.921+0000 7f53f4ff9640 1 --2- 172.21.15.89:0/4168173997 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f53e4065680 0x7f53e4067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:33.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.921+0000 7f53f4ff9640 1 -- 172.21.15.89:0/4168173997 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f53f00825a0 con 0x7f53f80c8450 2023-10-18T10:46:33.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.921+0000 7f53f7fff640 1 --2- 172.21.15.89:0/4168173997 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f53e4065680 0x7f53e4067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:33.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.921+0000 7f53f7fff640 1 --2- 172.21.15.89:0/4168173997 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f53e4065680 0x7f53e4067b40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f53e80ae000 tx=0x7f53e80a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:33.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:33.925+0000 7f53f4ff9640 1 -- 172.21.15.89:0/4168173997 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f53f004fe10 con 0x7f53f80c8450 2023-10-18T10:46:34.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.081+0000 7f53fe89c640 1 -- 172.21.15.89:0/4168173997 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53f804dbe0 con 0x7f53f80c8450 2023-10-18T10:46:34.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.081+0000 7f53f4ff9640 1 -- 172.21.15.89:0/4168173997 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20096 (secure 0 0 0) 0x7f53f0053430 con 0x7f53f80c8450 2023-10-18T10:46:34.086 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:34.086 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":27,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:46:00.166702+0000","last_up_change":"2023-10-18T10:45:42.164688+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:45:53.309122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-18T10:45:54.659763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:45:57.796514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6824","nonce":773078364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6827","nonce":773078364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6831","nonce":773078364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6829","nonce":773078364}]},"public_addr":"172.21.15.154:6824/773078364","cluster_addr":"172.21.15.154:6827/773078364","heartbeat_back_addr":"172.21.15.154:6831/773078364","heartbeat_front_addr":"172.21.15.154:6829/773078364","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:46:34.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.085+0000 7f53fe89c640 1 -- 172.21.15.89:0/4168173997 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f53e4065680 msgr2=0x7f53e4067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:34.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.085+0000 7f53fe89c640 1 --2- 172.21.15.89:0/4168173997 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f53e4065680 0x7f53e4067b40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f53e80ae000 tx=0x7f53e80a7000 comp rx=0 tx=0).stop 2023-10-18T10:46:34.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.085+0000 7f53fe89c640 1 -- 172.21.15.89:0/4168173997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53f80c8450 msgr2=0x7f53f8159680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:34.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.085+0000 7f53fe89c640 1 --2- 172.21.15.89:0/4168173997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53f80c8450 0x7f53f8159680 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f53f0001570 tx=0x7f53f000cc70 comp rx=0 tx=0).stop 2023-10-18T10:46:34.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.085+0000 7f53fe89c640 1 -- 172.21.15.89:0/4168173997 shutdown_connections 2023-10-18T10:46:34.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.085+0000 7f53fe89c640 1 --2- 172.21.15.89:0/4168173997 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f53e4065680 0x7f53e4067b40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:34.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.085+0000 7f53fe89c640 1 --2- 172.21.15.89:0/4168173997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53f80c8450 0x7f53f8159680 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:34.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.085+0000 7f53fe89c640 1 --2- 172.21.15.89:0/4168173997 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53f80c6210 0x7f53f8159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:34.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.085+0000 7f53fe89c640 1 --2- 172.21.15.89:0/4168173997 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53f80d0e70 0x7f53f8153360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:34.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.085+0000 7f53fe89c640 1 -- 172.21.15.89:0/4168173997 >> 172.21.15.89:0/4168173997 conn(0x7f53f8089fd0 msgr2=0x7f53f8155070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:34.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.085+0000 7f53fe89c640 1 -- 172.21.15.89:0/4168173997 shutdown_connections 2023-10-18T10:46:34.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.085+0000 7f53fe89c640 1 -- 172.21.15.89:0/4168173997 wait complete. 2023-10-18T10:46:34.107 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:45:43.114184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-18T10:45:48.730968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:45:53.309122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:45:54.659763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T10:45:57.796514+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T10:46:34.108 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:46:34.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.245+0000 7ff3bd3db640 1 Processor -- start 2023-10-18T10:46:34.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.245+0000 7ff3bd3db640 1 -- start start 2023-10-18T10:46:34.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.249+0000 7ff3bd3db640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff3b80d0bc0 0x7ff3b80cd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:34.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.249+0000 7ff3bd3db640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b815da50 con 0x7ff3b80d0760 2023-10-18T10:46:34.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.249+0000 7ff3bd3db640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b80d3110 con 0x7ff3b815c940 2023-10-18T10:46:34.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.249+0000 7ff3bd3db640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3b815ce40 con 0x7ff3b80d0bc0 2023-10-18T10:46:34.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.249+0000 7ff3b77fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff3b80d0bc0 0x7ff3b80cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:34.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.249+0000 7ff3b67fc640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff3b80d0760 0x7ff3b80cd160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47884/0 (socket says 172.21.15.89:47884) 2023-10-18T10:46:34.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.249+0000 7ff3b67fc640 1 -- 172.21.15.89:0/2838415765 learned_addr learned my addr 172.21.15.89:0/2838415765 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:34.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.249+0000 7ff3b77fe640 1 -- 172.21.15.89:0/2838415765 >> v1:172.21.15.154:6789/0 conn(0x7ff3b815c940 legacy=0x7ff3b80d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:34.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.249+0000 7ff3b77fe640 1 -- 172.21.15.89:0/2838415765 >> v1:172.21.15.89:6789/0 conn(0x7ff3b80d0760 legacy=0x7ff3b80cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:34.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.249+0000 7ff3b77fe640 1 -- 172.21.15.89:0/2838415765 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b815d120 con 0x7ff3b80d0bc0 2023-10-18T10:46:34.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.249+0000 7ff3b5ffb640 1 -- 172.21.15.89:0/2838415765 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1726471167 0 0) 0x7ff3b815da50 con 0x7ff3b80d0760 2023-10-18T10:46:34.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.253+0000 7ff3b77fe640 1 --2- 172.21.15.89:0/2838415765 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3b80d0bc0 0x7ff3b80cd870 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff3b00097c0 tx=0x7ff3b001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a60905c49cad603 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:34.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.253+0000 7ff3b5ffb640 1 -- 172.21.15.89:0/2838415765 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b00ab650 con 0x7ff3b80d0bc0 2023-10-18T10:46:34.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.253+0000 7ff3b5ffb640 1 -- 172.21.15.89:0/2838415765 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b80ce5c0 con 0x7ff3b80d0bc0 2023-10-18T10:46:34.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.253+0000 7ff3b5ffb640 1 -- 172.21.15.89:0/2838415765 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b00ab9d0 con 0x7ff3b80d0bc0 2023-10-18T10:46:34.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.253+0000 7ff3bd3db640 1 -- 172.21.15.89:0/2838415765 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3b80d0bc0 msgr2=0x7ff3b80cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:34.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.253+0000 7ff3bd3db640 1 --2- 172.21.15.89:0/2838415765 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3b80d0bc0 0x7ff3b80cd870 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff3b00097c0 tx=0x7ff3b001c450 comp rx=0 tx=0).stop 2023-10-18T10:46:34.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.253+0000 7ff3bd3db640 1 -- 172.21.15.89:0/2838415765 shutdown_connections 2023-10-18T10:46:34.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.253+0000 7ff3bd3db640 1 --2- 172.21.15.89:0/2838415765 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3b80d0bc0 0x7ff3b80cd870 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:34.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.253+0000 7ff3bd3db640 1 -- 172.21.15.89:0/2838415765 >> 172.21.15.89:0/2838415765 conn(0x7ff3b8089fd0 msgr2=0x7ff3b809b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:34.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.253+0000 7ff3bd3db640 1 -- 172.21.15.89:0/2838415765 shutdown_connections 2023-10-18T10:46:34.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.253+0000 7ff3bd3db640 1 -- 172.21.15.89:0/2838415765 wait complete. 2023-10-18T10:46:34.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.253+0000 7ff3bd3db640 1 Processor -- start 2023-10-18T10:46:34.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.253+0000 7ff3bd3db640 1 -- start start 2023-10-18T10:46:34.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.257+0000 7ff3bd3db640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3b80d0760 0x7ff3b80b7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:34.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.257+0000 7ff3bd3db640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3b80d0bc0 0x7ff3b815e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:34.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.257+0000 7ff3bd3db640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3b80ca200 0x7ff3b80ca690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:34.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.257+0000 7ff3bd3db640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b80c9d70 con 0x7ff3b80d0760 2023-10-18T10:46:34.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.257+0000 7ff3bd3db640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b8153dc0 con 0x7ff3b80ca200 2023-10-18T10:46:34.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.257+0000 7ff3bd3db640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b80b81d0 con 0x7ff3b80d0bc0 2023-10-18T10:46:34.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.257+0000 7ff3b6ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3b80d0760 0x7ff3b80b7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:34.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.257+0000 7ff3b6ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3b80d0760 0x7ff3b80b7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46976/0 (socket says 172.21.15.89:46976) 2023-10-18T10:46:34.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.257+0000 7ff3b6ffd640 1 -- 172.21.15.89:0/3321301573 learned_addr learned my addr 172.21.15.89:0/3321301573 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:34.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.257+0000 7ff3b67fc640 1 --2- 172.21.15.89:0/3321301573 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3b80d0bc0 0x7ff3b815e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:34.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.257+0000 7ff3b77fe640 1 --2- 172.21.15.89:0/3321301573 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3b80ca200 0x7ff3b80ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:34.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.257+0000 7ff3b6ffd640 1 -- 172.21.15.89:0/3321301573 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3b80d0bc0 msgr2=0x7ff3b815e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:34.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.257+0000 7ff3b6ffd640 1 --2- 172.21.15.89:0/3321301573 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3b80d0bc0 0x7ff3b815e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:34.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.257+0000 7ff3b6ffd640 1 -- 172.21.15.89:0/3321301573 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3b80ca200 msgr2=0x7ff3b80ca690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:34.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.257+0000 7ff3b6ffd640 1 --2- 172.21.15.89:0/3321301573 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3b80ca200 0x7ff3b80ca690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:34.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.257+0000 7ff3b6ffd640 1 -- 172.21.15.89:0/3321301573 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b0009420 con 0x7ff3b80d0760 2023-10-18T10:46:34.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.257+0000 7ff3b6ffd640 1 --2- 172.21.15.89:0/3321301573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3b80d0760 0x7ff3b80b7c90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800c7d0 tx=0x7ff3a800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:34.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.257+0000 7ff39f7fe640 1 -- 172.21.15.89:0/3321301573 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a8003b80 con 0x7ff3b80d0760 2023-10-18T10:46:34.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.257+0000 7ff39f7fe640 1 -- 172.21.15.89:0/3321301573 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a8003ce0 con 0x7ff3b80d0760 2023-10-18T10:46:34.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.257+0000 7ff39f7fe640 1 -- 172.21.15.89:0/3321301573 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a800aa90 con 0x7ff3b80d0760 2023-10-18T10:46:34.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.257+0000 7ff3bd3db640 1 -- 172.21.15.89:0/3321301573 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b80cc0b0 con 0x7ff3b80d0760 2023-10-18T10:46:34.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.257+0000 7ff3bd3db640 1 -- 172.21.15.89:0/3321301573 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b80c9720 con 0x7ff3b80d0760 2023-10-18T10:46:34.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.257+0000 7ff3bd3db640 1 -- 172.21.15.89:0/3321301573 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3b80cc560 con 0x7ff3b80d0760 2023-10-18T10:46:34.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.257+0000 7ff39f7fe640 1 -- 172.21.15.89:0/3321301573 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7ff3a800abf0 con 0x7ff3b80d0760 2023-10-18T10:46:34.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.261+0000 7ff39f7fe640 1 --2- 172.21.15.89:0/3321301573 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff3a4065680 0x7ff3a4067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:34.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.261+0000 7ff39f7fe640 1 -- 172.21.15.89:0/3321301573 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff3a8082da0 con 0x7ff3b80d0760 2023-10-18T10:46:34.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.261+0000 7ff3b67fc640 1 --2- 172.21.15.89:0/3321301573 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff3a4065680 0x7ff3a4067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:34.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.261+0000 7ff3b67fc640 1 --2- 172.21.15.89:0/3321301573 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff3a4065680 0x7ff3a4067b40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff3b81544a0 tx=0x7ff3ac007240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:34.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.265+0000 7ff39f7fe640 1 -- 172.21.15.89:0/3321301573 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7ff3a8050620 con 0x7ff3b80d0760 2023-10-18T10:46:34.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.413+0000 7ff3bd3db640 1 -- 172.21.15.89:0/3321301573 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff3b804dbe0 con 0x7ff3b80d0760 2023-10-18T10:46:34.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.417+0000 7ff39f7fe640 1 -- 172.21.15.89:0/3321301573 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff3a8053c40 con 0x7ff3b80d0760 2023-10-18T10:46:34.419 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-18T10:46:34.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.417+0000 7ff3bd3db640 1 -- 172.21.15.89:0/3321301573 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff3a4065680 msgr2=0x7ff3a4067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:34.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.417+0000 7ff3bd3db640 1 --2- 172.21.15.89:0/3321301573 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff3a4065680 0x7ff3a4067b40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff3b81544a0 tx=0x7ff3ac007240 comp rx=0 tx=0).stop 2023-10-18T10:46:34.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.417+0000 7ff3bd3db640 1 -- 172.21.15.89:0/3321301573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3b80d0760 msgr2=0x7ff3b80b7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:34.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.417+0000 7ff3bd3db640 1 --2- 172.21.15.89:0/3321301573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3b80d0760 0x7ff3b80b7c90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800c7d0 tx=0x7ff3a800cc90 comp rx=0 tx=0).stop 2023-10-18T10:46:34.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.417+0000 7ff3bd3db640 1 -- 172.21.15.89:0/3321301573 shutdown_connections 2023-10-18T10:46:34.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.417+0000 7ff3bd3db640 1 --2- 172.21.15.89:0/3321301573 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff3a4065680 0x7ff3a4067b40 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:34.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.417+0000 7ff3bd3db640 1 --2- 172.21.15.89:0/3321301573 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3b80d0bc0 0x7ff3b815e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:34.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.417+0000 7ff3bd3db640 1 --2- 172.21.15.89:0/3321301573 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3b80ca200 0x7ff3b80ca690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:34.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.417+0000 7ff3bd3db640 1 --2- 172.21.15.89:0/3321301573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3b80d0760 0x7ff3b80b7c90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:34.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.417+0000 7ff3bd3db640 1 -- 172.21.15.89:0/3321301573 >> 172.21.15.89:0/3321301573 conn(0x7ff3b8089fd0 msgr2=0x7ff3b80b4400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:34.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.417+0000 7ff3bd3db640 1 -- 172.21.15.89:0/3321301573 shutdown_connections 2023-10-18T10:46:34.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.417+0000 7ff3bd3db640 1 -- 172.21.15.89:0/3321301573 wait complete. 2023-10-18T10:46:34.440 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-18T10:46:34.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.577+0000 7fb1e4140640 1 Processor -- start 2023-10-18T10:46:34.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.577+0000 7fb1e4140640 1 -- start start 2023-10-18T10:46:34.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.577+0000 7fb1e4140640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb1dc159490 0x7fb1dc15d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:34.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.577+0000 7fb1e4140640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb1dc070600 0x7fb1dc06b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:34.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.577+0000 7fb1e4140640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1dc15ddf0 con 0x7fb1dc06bd10 2023-10-18T10:46:34.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.577+0000 7fb1e4140640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1dc06c1a0 con 0x7fb1dc070600 2023-10-18T10:46:34.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.577+0000 7fb1e4140640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1dc070b90 con 0x7fb1dc159490 2023-10-18T10:46:34.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.581+0000 7fb1e16db640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb1dc070600 0x7fb1dc06b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:34.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.581+0000 7fb1e16db640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb1dc070600 0x7fb1dc06b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46990/0 (socket says 172.21.15.89:46990) 2023-10-18T10:46:34.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.581+0000 7fb1e16db640 1 -- 172.21.15.89:0/1333316347 learned_addr learned my addr 172.21.15.89:0/1333316347 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:34.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.581+0000 7fb1e1edc640 1 --2- 172.21.15.89:0/1333316347 >> v2:172.21.15.154:3300/0 conn(0x7fb1dc159490 0x7fb1dc15d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:34.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.581+0000 7fb1e16db640 1 -- 172.21.15.89:0/1333316347 >> v2:172.21.15.154:3300/0 conn(0x7fb1dc159490 msgr2=0x7fb1dc15d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:34.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.581+0000 7fb1e16db640 1 --2- 172.21.15.89:0/1333316347 >> v2:172.21.15.154:3300/0 conn(0x7fb1dc159490 0x7fb1dc15d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:34.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.581+0000 7fb1e16db640 1 -- 172.21.15.89:0/1333316347 >> v1:172.21.15.154:6789/0 conn(0x7fb1dc06bd10 legacy=0x7fb1dc0cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:34.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.581+0000 7fb1e16db640 1 -- 172.21.15.89:0/1333316347 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1dc070e70 con 0x7fb1dc070600 2023-10-18T10:46:34.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.581+0000 7fb1e16db640 1 --2- 172.21.15.89:0/1333316347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1dc070600 0x7fb1dc06b7d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc06c560 tx=0x7fb1cc01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e960eaec11a18 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:34.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.581+0000 7fb1e0eda640 1 -- 172.21.15.89:0/1333316347 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc0ab640 con 0x7fb1dc070600 2023-10-18T10:46:34.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.581+0000 7fb1e0eda640 1 -- 172.21.15.89:0/1333316347 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1cc0a7070 con 0x7fb1dc070600 2023-10-18T10:46:34.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.581+0000 7fb1e0eda640 1 -- 172.21.15.89:0/1333316347 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc0ab9c0 con 0x7fb1dc070600 2023-10-18T10:46:34.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.585+0000 7fb1e4140640 1 -- 172.21.15.89:0/1333316347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1dc070600 msgr2=0x7fb1dc06b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:34.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.585+0000 7fb1e4140640 1 --2- 172.21.15.89:0/1333316347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1dc070600 0x7fb1dc06b7d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc06c560 tx=0x7fb1cc01c440 comp rx=0 tx=0).stop 2023-10-18T10:46:34.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.585+0000 7fb1e4140640 1 -- 172.21.15.89:0/1333316347 shutdown_connections 2023-10-18T10:46:34.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.585+0000 7fb1e4140640 1 --2- 172.21.15.89:0/1333316347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1dc070600 0x7fb1dc06b7d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:34.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.585+0000 7fb1e4140640 1 --2- 172.21.15.89:0/1333316347 >> v2:172.21.15.154:3300/0 conn(0x7fb1dc159490 0x7fb1dc15d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:34.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.585+0000 7fb1e4140640 1 -- 172.21.15.89:0/1333316347 >> 172.21.15.89:0/1333316347 conn(0x7fb1dc069f00 msgr2=0x7fb1dc0cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:34.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.585+0000 7fb1e4140640 1 -- 172.21.15.89:0/1333316347 shutdown_connections 2023-10-18T10:46:34.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.585+0000 7fb1e4140640 1 -- 172.21.15.89:0/1333316347 wait complete. 2023-10-18T10:46:34.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.585+0000 7fb1e4140640 1 Processor -- start 2023-10-18T10:46:34.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.585+0000 7fb1e4140640 1 -- start start 2023-10-18T10:46:34.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.585+0000 7fb1e4140640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1dc06bd10 0x7fb1dc1525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:34.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.585+0000 7fb1e4140640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1dc070600 0x7fb1dc15bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:34.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.585+0000 7fb1e4140640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1dc159490 0x7fb1dc155360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:34.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.585+0000 7fb1e4140640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1dc153560 con 0x7fb1dc070600 2023-10-18T10:46:34.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.585+0000 7fb1e4140640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1dc152af0 con 0x7fb1dc06bd10 2023-10-18T10:46:34.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.585+0000 7fb1e4140640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1dc152c50 con 0x7fb1dc159490 2023-10-18T10:46:34.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.585+0000 7fb1e16db640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1dc070600 0x7fb1dc15bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:34.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.585+0000 7fb1e16db640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1dc070600 0x7fb1dc15bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47004/0 (socket says 172.21.15.89:47004) 2023-10-18T10:46:34.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.585+0000 7fb1e16db640 1 -- 172.21.15.89:0/445032503 learned_addr learned my addr 172.21.15.89:0/445032503 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:34.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.585+0000 7fb1e1edc640 1 --2- 172.21.15.89:0/445032503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1dc06bd10 0x7fb1dc1525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:34.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.585+0000 7fb1e26dd640 1 --2- 172.21.15.89:0/445032503 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1dc159490 0x7fb1dc155360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:34.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.585+0000 7fb1e16db640 1 -- 172.21.15.89:0/445032503 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1dc159490 msgr2=0x7fb1dc155360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:34.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.585+0000 7fb1e16db640 1 --2- 172.21.15.89:0/445032503 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1dc159490 0x7fb1dc155360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:34.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.585+0000 7fb1e16db640 1 -- 172.21.15.89:0/445032503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1dc06bd10 msgr2=0x7fb1dc1525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:34.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.585+0000 7fb1e16db640 1 --2- 172.21.15.89:0/445032503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1dc06bd10 0x7fb1dc1525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:34.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.585+0000 7fb1e16db640 1 -- 172.21.15.89:0/445032503 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1cc0094d0 con 0x7fb1dc070600 2023-10-18T10:46:34.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.589+0000 7fb1e16db640 1 --2- 172.21.15.89:0/445032503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1dc070600 0x7fb1dc15bf60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc01c920 tx=0x7fb1cc01ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:34.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.589+0000 7fb1d27fc640 1 -- 172.21.15.89:0/445032503 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc0be3f0 con 0x7fb1dc070600 2023-10-18T10:46:34.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.589+0000 7fb1d27fc640 1 -- 172.21.15.89:0/445032503 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1cc0be570 con 0x7fb1dc070600 2023-10-18T10:46:34.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.589+0000 7fb1e4140640 1 -- 172.21.15.89:0/445032503 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1dc15c4a0 con 0x7fb1dc070600 2023-10-18T10:46:34.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.589+0000 7fb1d27fc640 1 -- 172.21.15.89:0/445032503 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc0be920 con 0x7fb1dc070600 2023-10-18T10:46:34.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.589+0000 7fb1e4140640 1 -- 172.21.15.89:0/445032503 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1dc152ea0 con 0x7fb1dc070600 2023-10-18T10:46:34.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.589+0000 7fb1e4140640 1 -- 172.21.15.89:0/445032503 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1dc1536a0 con 0x7fb1dc070600 2023-10-18T10:46:34.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.589+0000 7fb1d27fc640 1 -- 172.21.15.89:0/445032503 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fb1cc0c7440 con 0x7fb1dc070600 2023-10-18T10:46:34.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.589+0000 7fb1d27fc640 1 --2- 172.21.15.89:0/445032503 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb1c0065680 0x7fb1c0067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:34.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.589+0000 7fb1d27fc640 1 -- 172.21.15.89:0/445032503 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb1cc11d970 con 0x7fb1dc070600 2023-10-18T10:46:34.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.589+0000 7fb1e1edc640 1 --2- 172.21.15.89:0/445032503 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb1c0065680 0x7fb1c0067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:34.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.593+0000 7fb1e1edc640 1 --2- 172.21.15.89:0/445032503 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb1c0065680 0x7fb1c0067b40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc0aca00 tx=0x7fb1d8009830 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:34.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.593+0000 7fb1d27fc640 1 -- 172.21.15.89:0/445032503 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fb1dc1536a0 con 0x7fb1dc070600 2023-10-18T10:46:34.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.741+0000 7fb1e4140640 1 -- 172.21.15.89:0/445032503 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb1dc1536a0 con 0x7fb1dc070600 2023-10-18T10:46:34.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.741+0000 7fb1d27fc640 1 -- 172.21.15.89:0/445032503 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb1dc1536a0 con 0x7fb1dc070600 2023-10-18T10:46:34.746 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-10-18T10:46:34.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.745+0000 7fb1e4140640 1 -- 172.21.15.89:0/445032503 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb1c0065680 msgr2=0x7fb1c0067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:34.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.745+0000 7fb1e4140640 1 --2- 172.21.15.89:0/445032503 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb1c0065680 0x7fb1c0067b40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc0aca00 tx=0x7fb1d8009830 comp rx=0 tx=0).stop 2023-10-18T10:46:34.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.745+0000 7fb1e4140640 1 -- 172.21.15.89:0/445032503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1dc070600 msgr2=0x7fb1dc15bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:34.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.745+0000 7fb1e4140640 1 --2- 172.21.15.89:0/445032503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1dc070600 0x7fb1dc15bf60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc01c920 tx=0x7fb1cc01ac80 comp rx=0 tx=0).stop 2023-10-18T10:46:34.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.745+0000 7fb1e4140640 1 -- 172.21.15.89:0/445032503 shutdown_connections 2023-10-18T10:46:34.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.745+0000 7fb1e4140640 1 --2- 172.21.15.89:0/445032503 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb1c0065680 0x7fb1c0067b40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:34.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.745+0000 7fb1e4140640 1 --2- 172.21.15.89:0/445032503 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1dc159490 0x7fb1dc155360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:34.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.745+0000 7fb1e4140640 1 --2- 172.21.15.89:0/445032503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1dc070600 0x7fb1dc15bf60 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:34.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.745+0000 7fb1e4140640 1 --2- 172.21.15.89:0/445032503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1dc06bd10 0x7fb1dc1525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:34.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.745+0000 7fb1e4140640 1 -- 172.21.15.89:0/445032503 >> 172.21.15.89:0/445032503 conn(0x7fb1dc069f00 msgr2=0x7fb1dc0ab8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:34.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.745+0000 7fb1e4140640 1 -- 172.21.15.89:0/445032503 shutdown_connections 2023-10-18T10:46:34.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.745+0000 7fb1e4140640 1 -- 172.21.15.89:0/445032503 wait complete. 2023-10-18T10:46:34.767 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:46:34.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.905+0000 7f231c1d5640 1 Processor -- start 2023-10-18T10:46:34.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.905+0000 7f231c1d5640 1 -- start start 2023-10-18T10:46:34.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.905+0000 7f231c1d5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f23140afd90 0x7f23140b0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:34.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.905+0000 7f231c1d5640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2314161160 con 0x7f23140ad5b0 2023-10-18T10:46:34.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.905+0000 7f231c1d5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23141605c0 con 0x7f231415c7f0 2023-10-18T10:46:34.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.905+0000 7f231c1d5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23140b06b0 con 0x7f23140afd90 2023-10-18T10:46:34.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.905+0000 7f231a772640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f23140ad5b0 0x7f2314160a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47890/0 (socket says 172.21.15.89:47890) 2023-10-18T10:46:34.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.905+0000 7f231a772640 1 -- 172.21.15.89:0/1444983578 learned_addr learned my addr 172.21.15.89:0/1444983578 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:34.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.905+0000 7f2319f71640 1 --2- 172.21.15.89:0/1444983578 >> v2:172.21.15.154:3300/0 conn(0x7f23140afd90 0x7f23140b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:34.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.905+0000 7f2318f6f640 1 -- 172.21.15.89:0/1444983578 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3250852730 0 0) 0x7f230c0014a0 con 0x7f23140ad5b0 2023-10-18T10:46:34.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.905+0000 7f2318f6f640 1 -- 172.21.15.89:0/1444983578 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2300003530 con 0x7f23140ad5b0 2023-10-18T10:46:34.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.905+0000 7f2318f6f640 1 -- 172.21.15.89:0/1444983578 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1188651578 0 0) 0x7f2300003530 con 0x7f23140ad5b0 2023-10-18T10:46:34.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.905+0000 7f2318f6f640 1 -- 172.21.15.89:0/1444983578 >> v2:172.21.15.154:3300/0 conn(0x7f23140afd90 msgr2=0x7f23140b0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:34.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.905+0000 7f2318f6f640 1 --2- 172.21.15.89:0/1444983578 >> v2:172.21.15.154:3300/0 conn(0x7f23140afd90 0x7f23140b0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:34.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.905+0000 7f2318f6f640 1 -- 172.21.15.89:0/1444983578 >> v1:172.21.15.154:6789/0 conn(0x7f231415c7f0 legacy=0x7f231406b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:34.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.905+0000 7f2318f6f640 1 -- 172.21.15.89:0/1444983578 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f231406c000 con 0x7f23140ad5b0 2023-10-18T10:46:34.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.905+0000 7f2318f6f640 1 -- 172.21.15.89:0/1444983578 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7f230c001b90 con 0x7f23140ad5b0 2023-10-18T10:46:34.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.905+0000 7f2318f6f640 1 -- 172.21.15.89:0/1444983578 >> v1:172.21.15.89:6789/0 conn(0x7f23140ad5b0 legacy=0x7f2314160a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:34.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.905+0000 7f2318f6f640 1 --2- 172.21.15.89:0/1444983578 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2300003980 0x7f2300003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:34.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.905+0000 7f2318f6f640 1 --2- 172.21.15.89:0/1444983578 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23000047d0 0x7f2300004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:34.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.905+0000 7f2318f6f640 1 --2- 172.21.15.89:0/1444983578 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2300005190 0x7f230000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:34.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.905+0000 7f2318f6f640 1 -- 172.21.15.89:0/1444983578 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23140b06b0 con 0x7f2300003980 2023-10-18T10:46:34.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.905+0000 7f2318f6f640 1 -- 172.21.15.89:0/1444983578 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230000c130 con 0x7f23000047d0 2023-10-18T10:46:34.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.905+0000 7f2318f6f640 1 -- 172.21.15.89:0/1444983578 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f230000c2f0 con 0x7f2300005190 2023-10-18T10:46:34.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.905+0000 7f2319f71640 1 --2- 172.21.15.89:0/1444983578 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2300003980 0x7f2300003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:34.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.905+0000 7f231a772640 1 --2- 172.21.15.89:0/1444983578 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2300005190 0x7f230000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:34.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.905+0000 7f2319770640 1 --2- 172.21.15.89:0/1444983578 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23000047d0 0x7f2300004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:34.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.905+0000 7f2319f71640 1 -- 172.21.15.89:0/1444983578 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2300005190 msgr2=0x7f230000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:34.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.905+0000 7f2319f71640 1 --2- 172.21.15.89:0/1444983578 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2300005190 0x7f230000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:34.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.905+0000 7f2319f71640 1 -- 172.21.15.89:0/1444983578 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23000047d0 msgr2=0x7f2300004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:34.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.905+0000 7f2319f71640 1 --2- 172.21.15.89:0/1444983578 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23000047d0 0x7f2300004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:34.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.905+0000 7f2319f71640 1 -- 172.21.15.89:0/1444983578 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2300003530 con 0x7f2300003980 2023-10-18T10:46:34.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.905+0000 7f231a772640 1 --2- 172.21.15.89:0/1444983578 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2300005190 0x7f230000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:34.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.905+0000 7f231a772640 1 -- 172.21.15.89:0/1444983578 reap_dead start 2023-10-18T10:46:34.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.909+0000 7f2319f71640 1 --2- 172.21.15.89:0/1444983578 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2300003980 0x7f2300003da0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2310009ae0 tx=0x7f231001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd5ab718e7ee2bc5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:34.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.909+0000 7f2318f6f640 1 -- 172.21.15.89:0/1444983578 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23100a8460 con 0x7f2300003980 2023-10-18T10:46:34.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.909+0000 7f2318f6f640 1 -- 172.21.15.89:0/1444983578 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23100a85c0 con 0x7f2300003980 2023-10-18T10:46:34.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.909+0000 7f2318f6f640 1 -- 172.21.15.89:0/1444983578 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23100a8970 con 0x7f2300003980 2023-10-18T10:46:34.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.909+0000 7f231c1d5640 1 -- 172.21.15.89:0/1444983578 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2300003980 msgr2=0x7f2300003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:34.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.909+0000 7f231c1d5640 1 --2- 172.21.15.89:0/1444983578 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2300003980 0x7f2300003da0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2310009ae0 tx=0x7f231001ca50 comp rx=0 tx=0).stop 2023-10-18T10:46:34.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.909+0000 7f231c1d5640 1 -- 172.21.15.89:0/1444983578 shutdown_connections 2023-10-18T10:46:34.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.909+0000 7f231c1d5640 1 --2- 172.21.15.89:0/1444983578 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2300003980 0x7f2300003da0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:34.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.909+0000 7f231c1d5640 1 -- 172.21.15.89:0/1444983578 >> 172.21.15.89:0/1444983578 conn(0x7f2314069f00 msgr2=0x7f23140cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:34.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.909+0000 7f231c1d5640 1 -- 172.21.15.89:0/1444983578 shutdown_connections 2023-10-18T10:46:34.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.909+0000 7f231c1d5640 1 -- 172.21.15.89:0/1444983578 wait complete. 2023-10-18T10:46:34.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.913+0000 7f231c1d5640 1 Processor -- start 2023-10-18T10:46:34.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.913+0000 7f231c1d5640 1 -- start start 2023-10-18T10:46:34.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.913+0000 7f231c1d5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2300003980 0x7f231415f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:34.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.913+0000 7f231c1d5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23140b1f30 0x7f231415e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:34.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.913+0000 7f231c1d5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23140c2e90 0x7f23140c3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:34.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.913+0000 7f231c1d5640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231415db30 con 0x7f23140b1f30 2023-10-18T10:46:34.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.913+0000 7f231c1d5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23140b23b0 con 0x7f23140c2e90 2023-10-18T10:46:34.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.913+0000 7f231c1d5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2314158d20 con 0x7f2300003980 2023-10-18T10:46:34.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.913+0000 7f2319770640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23140b1f30 0x7f231415e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:34.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.913+0000 7f2319770640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23140b1f30 0x7f231415e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47026/0 (socket says 172.21.15.89:47026) 2023-10-18T10:46:34.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.913+0000 7f2319770640 1 -- 172.21.15.89:0/950792845 learned_addr learned my addr 172.21.15.89:0/950792845 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:34.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.913+0000 7f2319f71640 1 --2- 172.21.15.89:0/950792845 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2300003980 0x7f231415f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:34.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.913+0000 7f231a772640 1 --2- 172.21.15.89:0/950792845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23140c2e90 0x7f23140c3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:34.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.913+0000 7f2319770640 1 -- 172.21.15.89:0/950792845 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2300003980 msgr2=0x7f231415f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:34.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.913+0000 7f2319770640 1 --2- 172.21.15.89:0/950792845 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2300003980 0x7f231415f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:34.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.913+0000 7f2319770640 1 -- 172.21.15.89:0/950792845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23140c2e90 msgr2=0x7f23140c3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:34.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.913+0000 7f2319770640 1 --2- 172.21.15.89:0/950792845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23140c2e90 0x7f23140c3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:34.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.913+0000 7f2319770640 1 -- 172.21.15.89:0/950792845 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f231415e570 con 0x7f23140b1f30 2023-10-18T10:46:34.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.913+0000 7f231a772640 1 --2- 172.21.15.89:0/950792845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23140c2e90 0x7f23140c3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:34.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.913+0000 7f2319770640 1 --2- 172.21.15.89:0/950792845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23140b1f30 0x7f231415e030 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f230400e4d0 tx=0x7f230400e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:34.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.913+0000 7f230a7fc640 1 -- 172.21.15.89:0/950792845 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23040053a0 con 0x7f23140b1f30 2023-10-18T10:46:34.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.913+0000 7f230a7fc640 1 -- 172.21.15.89:0/950792845 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2304003600 con 0x7f23140b1f30 2023-10-18T10:46:34.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.913+0000 7f231c1d5640 1 -- 172.21.15.89:0/950792845 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f231415e830 con 0x7f23140b1f30 2023-10-18T10:46:34.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.913+0000 7f230a7fc640 1 -- 172.21.15.89:0/950792845 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2304003990 con 0x7f23140b1f30 2023-10-18T10:46:34.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.913+0000 7f231c1d5640 1 -- 172.21.15.89:0/950792845 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f231415edf0 con 0x7f23140b1f30 2023-10-18T10:46:34.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.917+0000 7f231c1d5640 1 -- 172.21.15.89:0/950792845 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f231404dbe0 con 0x7f23140b1f30 2023-10-18T10:46:34.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.917+0000 7f230a7fc640 1 -- 172.21.15.89:0/950792845 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f2304003af0 con 0x7f23140b1f30 2023-10-18T10:46:34.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.917+0000 7f230a7fc640 1 --2- 172.21.15.89:0/950792845 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f22f80656b0 0x7f22f8067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:34.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.917+0000 7f230a7fc640 1 -- 172.21.15.89:0/950792845 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2304082eb0 con 0x7f23140b1f30 2023-10-18T10:46:34.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.917+0000 7f2319f71640 1 --2- 172.21.15.89:0/950792845 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f22f80656b0 0x7f22f8067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:34.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.921+0000 7f2319f71640 1 --2- 172.21.15.89:0/950792845 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f22f80656b0 0x7f22f8067b70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f23100af000 tx=0x7f23100b3040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:34.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:34.921+0000 7f230a7fc640 1 -- 172.21.15.89:0/950792845 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f23040505e0 con 0x7f23140b1f30 2023-10-18T10:46:35.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.073+0000 7f231c1d5640 1 -- 172.21.15.89:0/950792845 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f23140cba40 con 0x7f23140b1f30 2023-10-18T10:46:35.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.073+0000 7f230a7fc640 1 -- 172.21.15.89:0/950792845 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2304053c00 con 0x7f23140b1f30 2023-10-18T10:46:35.077 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-18T10:46:35.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.077+0000 7f231c1d5640 1 -- 172.21.15.89:0/950792845 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f22f80656b0 msgr2=0x7f22f8067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:35.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.077+0000 7f231c1d5640 1 --2- 172.21.15.89:0/950792845 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f22f80656b0 0x7f22f8067b70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f23100af000 tx=0x7f23100b3040 comp rx=0 tx=0).stop 2023-10-18T10:46:35.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.077+0000 7f231c1d5640 1 -- 172.21.15.89:0/950792845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23140b1f30 msgr2=0x7f231415e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:35.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.077+0000 7f231c1d5640 1 --2- 172.21.15.89:0/950792845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23140b1f30 0x7f231415e030 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f230400e4d0 tx=0x7f230400e990 comp rx=0 tx=0).stop 2023-10-18T10:46:35.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.077+0000 7f231c1d5640 1 -- 172.21.15.89:0/950792845 shutdown_connections 2023-10-18T10:46:35.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.077+0000 7f231c1d5640 1 --2- 172.21.15.89:0/950792845 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f22f80656b0 0x7f22f8067b70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:35.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.077+0000 7f231c1d5640 1 --2- 172.21.15.89:0/950792845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23140c2e90 0x7f23140c3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:35.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.077+0000 7f231c1d5640 1 --2- 172.21.15.89:0/950792845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23140b1f30 0x7f231415e030 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:35.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.077+0000 7f231c1d5640 1 --2- 172.21.15.89:0/950792845 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2300003980 0x7f231415f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:35.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.077+0000 7f231c1d5640 1 -- 172.21.15.89:0/950792845 >> 172.21.15.89:0/950792845 conn(0x7f2314069f00 msgr2=0x7f23140ca410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:35.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.077+0000 7f231c1d5640 1 -- 172.21.15.89:0/950792845 shutdown_connections 2023-10-18T10:46:35.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.077+0000 7f231c1d5640 1 -- 172.21.15.89:0/950792845 wait complete. 2023-10-18T10:46:35.097 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:46:35.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.237+0000 7fcd099ba640 1 Processor -- start 2023-10-18T10:46:35.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.237+0000 7fcd099ba640 1 -- start start 2023-10-18T10:46:35.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.237+0000 7fcd099ba640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcd040d0760 0x7fcd040d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:35.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.237+0000 7fcd099ba640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd040d1100 con 0x7fcd0415c940 2023-10-18T10:46:35.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.237+0000 7fcd099ba640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd0415da50 con 0x7fcd040cd160 2023-10-18T10:46:35.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.237+0000 7fcd099ba640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd040d3110 con 0x7fcd040d0760 2023-10-18T10:46:35.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.237+0000 7fccfbfff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcd040d0760 0x7fcd040d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:35.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.237+0000 7fccfbfff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcd040d0760 0x7fcd040d0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47032/0 (socket says 172.21.15.89:47032) 2023-10-18T10:46:35.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.237+0000 7fcd02ffd640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fcd0415c940 0x7fcd040d2a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47902/0 (socket says 172.21.15.89:47902) 2023-10-18T10:46:35.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.237+0000 7fccfbfff640 1 -- 172.21.15.89:0/3936316196 learned_addr learned my addr 172.21.15.89:0/3936316196 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:35.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.237+0000 7fccfbfff640 1 -- 172.21.15.89:0/3936316196 >> v1:172.21.15.154:6789/0 conn(0x7fcd040cd160 legacy=0x7fcd040cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:35.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.237+0000 7fccfbfff640 1 -- 172.21.15.89:0/3936316196 >> v1:172.21.15.89:6789/0 conn(0x7fcd0415c940 legacy=0x7fcd040d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:35.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.237+0000 7fccfbfff640 1 -- 172.21.15.89:0/3936316196 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd0415ce40 con 0x7fcd040d0760 2023-10-18T10:46:35.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.241+0000 7fccfbfff640 1 --2- 172.21.15.89:0/3936316196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd040d0760 0x7fcd040d0bc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fccec009870 tx=0x7fccec01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d3e322e47a8ac97 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:35.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.241+0000 7fcd027fc640 1 -- 172.21.15.89:0/3936316196 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec0aa430 con 0x7fcd040d0760 2023-10-18T10:46:35.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.241+0000 7fcd027fc640 1 -- 172.21.15.89:0/3936316196 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccec0aa590 con 0x7fcd040d0760 2023-10-18T10:46:35.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.241+0000 7fcd027fc640 1 -- 172.21.15.89:0/3936316196 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec0aa920 con 0x7fcd040d0760 2023-10-18T10:46:35.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.241+0000 7fcd099ba640 1 -- 172.21.15.89:0/3936316196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd040d0760 msgr2=0x7fcd040d0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:35.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.241+0000 7fcd099ba640 1 --2- 172.21.15.89:0/3936316196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd040d0760 0x7fcd040d0bc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fccec009870 tx=0x7fccec01c4d0 comp rx=0 tx=0).stop 2023-10-18T10:46:35.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.241+0000 7fcd099ba640 1 -- 172.21.15.89:0/3936316196 shutdown_connections 2023-10-18T10:46:35.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.241+0000 7fcd099ba640 1 --2- 172.21.15.89:0/3936316196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd040d0760 0x7fcd040d0bc0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:35.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.241+0000 7fcd099ba640 1 -- 172.21.15.89:0/3936316196 >> 172.21.15.89:0/3936316196 conn(0x7fcd04089fd0 msgr2=0x7fcd0409b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:35.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.241+0000 7fcd099ba640 1 -- 172.21.15.89:0/3936316196 shutdown_connections 2023-10-18T10:46:35.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.241+0000 7fcd099ba640 1 -- 172.21.15.89:0/3936316196 wait complete. 2023-10-18T10:46:35.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.241+0000 7fcd099ba640 1 Processor -- start 2023-10-18T10:46:35.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.241+0000 7fcd099ba640 1 -- start start 2023-10-18T10:46:35.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.245+0000 7fcd099ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd040cd160 0x7fcd040c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:35.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.245+0000 7fcd099ba640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd040d0760 0x7fcd041546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:35.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.245+0000 7fcd099ba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd04153c90 0x7fcd041561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:35.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.245+0000 7fcd099ba640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd040c9390 con 0x7fcd040d0760 2023-10-18T10:46:35.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.245+0000 7fcd099ba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd040cc4a0 con 0x7fcd040cd160 2023-10-18T10:46:35.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.245+0000 7fcd099ba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd040cb920 con 0x7fcd04153c90 2023-10-18T10:46:35.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.245+0000 7fccfbfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd040d0760 0x7fcd041546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:35.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.245+0000 7fccfbfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd040d0760 0x7fcd041546d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47044/0 (socket says 172.21.15.89:47044) 2023-10-18T10:46:35.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.245+0000 7fccfbfff640 1 -- 172.21.15.89:0/4004965759 learned_addr learned my addr 172.21.15.89:0/4004965759 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:35.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.245+0000 7fcd02ffd640 1 --2- 172.21.15.89:0/4004965759 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd040cd160 0x7fcd040c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:35.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.245+0000 7fcd037fe640 1 --2- 172.21.15.89:0/4004965759 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd04153c90 0x7fcd041561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:35.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.245+0000 7fccfbfff640 1 -- 172.21.15.89:0/4004965759 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd04153c90 msgr2=0x7fcd041561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:35.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.245+0000 7fccfbfff640 1 --2- 172.21.15.89:0/4004965759 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd04153c90 0x7fcd041561c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:35.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.245+0000 7fccfbfff640 1 -- 172.21.15.89:0/4004965759 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd040cd160 msgr2=0x7fcd040c8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:35.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.245+0000 7fccfbfff640 1 --2- 172.21.15.89:0/4004965759 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd040cd160 0x7fcd040c8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:35.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.245+0000 7fccfbfff640 1 -- 172.21.15.89:0/4004965759 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccec0094d0 con 0x7fcd040d0760 2023-10-18T10:46:35.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.245+0000 7fcd037fe640 1 --2- 172.21.15.89:0/4004965759 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd04153c90 0x7fcd041561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:35.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.245+0000 7fccfbfff640 1 --2- 172.21.15.89:0/4004965759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd040d0760 0x7fcd041546d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fccec0ac040 tx=0x7fccec01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:35.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.245+0000 7fcd00ff9640 1 -- 172.21.15.89:0/4004965759 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec0b38c0 con 0x7fcd040d0760 2023-10-18T10:46:35.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.245+0000 7fcd00ff9640 1 -- 172.21.15.89:0/4004965759 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccec0b3a40 con 0x7fcd040d0760 2023-10-18T10:46:35.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.245+0000 7fcd00ff9640 1 -- 172.21.15.89:0/4004965759 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec0b3df0 con 0x7fcd040d0760 2023-10-18T10:46:35.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.245+0000 7fcd099ba640 1 -- 172.21.15.89:0/4004965759 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd040cbc00 con 0x7fcd040d0760 2023-10-18T10:46:35.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.245+0000 7fcd099ba640 1 -- 172.21.15.89:0/4004965759 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd040cc1c0 con 0x7fcd040d0760 2023-10-18T10:46:35.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.245+0000 7fcd099ba640 1 -- 172.21.15.89:0/4004965759 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd0404dbe0 con 0x7fcd040d0760 2023-10-18T10:46:35.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.249+0000 7fcd00ff9640 1 -- 172.21.15.89:0/4004965759 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fccec0b2380 con 0x7fcd040d0760 2023-10-18T10:46:35.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.249+0000 7fcd00ff9640 1 --2- 172.21.15.89:0/4004965759 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fcce4065680 0x7fcce4067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:35.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.249+0000 7fcd00ff9640 1 -- 172.21.15.89:0/4004965759 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fccec11d750 con 0x7fcd040d0760 2023-10-18T10:46:35.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.249+0000 7fcd02ffd640 1 --2- 172.21.15.89:0/4004965759 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fcce4065680 0x7fcce4067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:35.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.249+0000 7fcd02ffd640 1 --2- 172.21.15.89:0/4004965759 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fcce4065680 0x7fcce4067b40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fccf4000f70 tx=0x7fccf4007e50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:35.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.253+0000 7fcd00ff9640 1 -- 172.21.15.89:0/4004965759 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fccec0eafc0 con 0x7fcd040d0760 2023-10-18T10:46:35.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.405+0000 7fcd099ba640 1 -- 172.21.15.89:0/4004965759 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcd040d3330 con 0x7fcd040d0760 2023-10-18T10:46:35.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.405+0000 7fcd00ff9640 1 -- 172.21.15.89:0/4004965759 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fccec0ee5e0 con 0x7fcd040d0760 2023-10-18T10:46:35.409 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-18T10:46:35.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.405+0000 7fcd099ba640 1 -- 172.21.15.89:0/4004965759 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fcce4065680 msgr2=0x7fcce4067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:35.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.405+0000 7fcd099ba640 1 --2- 172.21.15.89:0/4004965759 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fcce4065680 0x7fcce4067b40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fccf4000f70 tx=0x7fccf4007e50 comp rx=0 tx=0).stop 2023-10-18T10:46:35.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.405+0000 7fcd099ba640 1 -- 172.21.15.89:0/4004965759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd040d0760 msgr2=0x7fcd041546d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:35.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.405+0000 7fcd099ba640 1 --2- 172.21.15.89:0/4004965759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd040d0760 0x7fcd041546d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fccec0ac040 tx=0x7fccec01ad10 comp rx=0 tx=0).stop 2023-10-18T10:46:35.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.405+0000 7fcd099ba640 1 -- 172.21.15.89:0/4004965759 shutdown_connections 2023-10-18T10:46:35.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.405+0000 7fcd099ba640 1 --2- 172.21.15.89:0/4004965759 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fcce4065680 0x7fcce4067b40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:35.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.405+0000 7fcd099ba640 1 --2- 172.21.15.89:0/4004965759 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd04153c90 0x7fcd041561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:35.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.405+0000 7fcd099ba640 1 --2- 172.21.15.89:0/4004965759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd040d0760 0x7fcd041546d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:35.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.405+0000 7fcd099ba640 1 --2- 172.21.15.89:0/4004965759 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd040cd160 0x7fcd040c8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:35.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.405+0000 7fcd099ba640 1 -- 172.21.15.89:0/4004965759 >> 172.21.15.89:0/4004965759 conn(0x7fcd04089fd0 msgr2=0x7fcd040b3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:35.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.405+0000 7fcd099ba640 1 -- 172.21.15.89:0/4004965759 shutdown_connections 2023-10-18T10:46:35.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.405+0000 7fcd099ba640 1 -- 172.21.15.89:0/4004965759 wait complete. 2023-10-18T10:46:35.430 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T10:46:35.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.569+0000 7ffaaed67640 1 Processor -- start 2023-10-18T10:46:35.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.569+0000 7ffaaed67640 1 -- start start 2023-10-18T10:46:35.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.569+0000 7ffaaed67640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffaa80c9970 0x7ffaa8164230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:35.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.569+0000 7ffaaed67640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaa8070b90 con 0x7ffaa8070600 2023-10-18T10:46:35.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.569+0000 7ffaaed67640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaa8070e70 con 0x7ffaa8159490 2023-10-18T10:46:35.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.569+0000 7ffaaed67640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa8071150 con 0x7ffaa80c9970 2023-10-18T10:46:35.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.569+0000 7ffaa7fff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ffaa8070600 0x7ffaa806b7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47910/0 (socket says 172.21.15.89:47910) 2023-10-18T10:46:35.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.569+0000 7ffaa7fff640 1 -- 172.21.15.89:0/2354703161 learned_addr learned my addr 172.21.15.89:0/2354703161 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:35.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.569+0000 7ffaad304640 1 --2- 172.21.15.89:0/2354703161 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffaa80c9970 0x7ffaa8164230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:35.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.569+0000 7ffaa77fe640 1 -- 172.21.15.89:0/2354703161 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1107210997 0 0) 0x7ffaa8070b90 con 0x7ffaa8070600 2023-10-18T10:46:35.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.569+0000 7ffaa77fe640 1 -- 172.21.15.89:0/2354703161 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa90003530 con 0x7ffaa8070600 2023-10-18T10:46:35.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.569+0000 7ffaa77fe640 1 -- 172.21.15.89:0/2354703161 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1748511041 0 0) 0x7ffaa8070e70 con 0x7ffaa8159490 2023-10-18T10:46:35.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.569+0000 7ffaa77fe640 1 -- 172.21.15.89:0/2354703161 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffaa8070b90 con 0x7ffaa8159490 2023-10-18T10:46:35.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.569+0000 7ffaad304640 1 -- 172.21.15.89:0/2354703161 >> v1:172.21.15.154:6789/0 conn(0x7ffaa8159490 legacy=0x7ffaa815d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:35.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.569+0000 7ffaad304640 1 -- 172.21.15.89:0/2354703161 >> v1:172.21.15.89:6789/0 conn(0x7ffaa8070600 legacy=0x7ffaa806b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:35.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.569+0000 7ffaa77fe640 1 -- 172.21.15.89:0/2354703161 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2355548727 0 0) 0x7ffa90003530 con 0x7ffaa8070600 2023-10-18T10:46:35.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.569+0000 7ffaad304640 1 -- 172.21.15.89:0/2354703161 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa8071430 con 0x7ffaa80c9970 2023-10-18T10:46:35.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.573+0000 7ffaad304640 1 --2- 172.21.15.89:0/2354703161 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffaa80c9970 0x7ffaa8164230 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0009dc0 tx=0x7ffaa001c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3697cdca7630d755 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:35.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.573+0000 7ffaa77fe640 1 -- 172.21.15.89:0/2354703161 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c001680 con 0x7ffaa80c9970 2023-10-18T10:46:35.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.573+0000 7ffaa77fe640 1 -- 172.21.15.89:0/2354703161 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa00a55f0 con 0x7ffaa80c9970 2023-10-18T10:46:35.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.573+0000 7ffaa77fe640 1 -- 172.21.15.89:0/2354703161 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa00a5980 con 0x7ffaa80c9970 2023-10-18T10:46:35.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.573+0000 7ffaaed67640 1 -- 172.21.15.89:0/2354703161 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffaa80c9970 msgr2=0x7ffaa8164230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:35.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.573+0000 7ffaaed67640 1 --2- 172.21.15.89:0/2354703161 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffaa80c9970 0x7ffaa8164230 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0009dc0 tx=0x7ffaa001c7d0 comp rx=0 tx=0).stop 2023-10-18T10:46:35.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.573+0000 7ffaaed67640 1 -- 172.21.15.89:0/2354703161 shutdown_connections 2023-10-18T10:46:35.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.573+0000 7ffaaed67640 1 --2- 172.21.15.89:0/2354703161 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffaa80c9970 0x7ffaa8164230 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:35.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.573+0000 7ffaaed67640 1 -- 172.21.15.89:0/2354703161 >> 172.21.15.89:0/2354703161 conn(0x7ffaa8069f00 msgr2=0x7ffaa80cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:35.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.573+0000 7ffaaed67640 1 -- 172.21.15.89:0/2354703161 shutdown_connections 2023-10-18T10:46:35.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.573+0000 7ffaaed67640 1 -- 172.21.15.89:0/2354703161 wait complete. 2023-10-18T10:46:35.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.577+0000 7ffaaed67640 1 Processor -- start 2023-10-18T10:46:35.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.577+0000 7ffaaed67640 1 -- start start 2023-10-18T10:46:35.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.577+0000 7ffaaed67640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffaa8070600 0x7ffaa80c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:35.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.577+0000 7ffaaed67640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffaa80c9970 0x7ffaa81f7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:35.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.577+0000 7ffaaed67640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffaa8159490 0x7ffaa81432d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:35.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.577+0000 7ffaaed67640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa8155070 con 0x7ffaa80c9970 2023-10-18T10:46:35.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.577+0000 7ffaaed67640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa8159e10 con 0x7ffaa8159490 2023-10-18T10:46:35.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.577+0000 7ffaaed67640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa815a0f0 con 0x7ffaa8070600 2023-10-18T10:46:35.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.577+0000 7ffaa7fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffaa80c9970 0x7ffaa81f7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:35.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.577+0000 7ffaa7fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffaa80c9970 0x7ffaa81f7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47048/0 (socket says 172.21.15.89:47048) 2023-10-18T10:46:35.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.577+0000 7ffaa7fff640 1 -- 172.21.15.89:0/803700549 learned_addr learned my addr 172.21.15.89:0/803700549 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:35.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.577+0000 7ffaacb03640 1 --2- 172.21.15.89:0/803700549 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffaa8070600 0x7ffaa80c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:35.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.577+0000 7ffaad304640 1 --2- 172.21.15.89:0/803700549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffaa8159490 0x7ffaa81432d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:35.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.577+0000 7ffaa7fff640 1 -- 172.21.15.89:0/803700549 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffaa8070600 msgr2=0x7ffaa80c6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:35.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.577+0000 7ffaa7fff640 1 --2- 172.21.15.89:0/803700549 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffaa8070600 0x7ffaa80c6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:35.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.577+0000 7ffaa7fff640 1 -- 172.21.15.89:0/803700549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffaa8159490 msgr2=0x7ffaa81432d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:35.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.577+0000 7ffaa7fff640 1 --2- 172.21.15.89:0/803700549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffaa8159490 0x7ffaa81432d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:35.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.577+0000 7ffaa7fff640 1 -- 172.21.15.89:0/803700549 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa0009a70 con 0x7ffaa80c9970 2023-10-18T10:46:35.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.577+0000 7ffaa7fff640 1 --2- 172.21.15.89:0/803700549 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffaa80c9970 0x7ffaa81f7e90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c00c750 tx=0x7ffa9c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:35.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.577+0000 7ffaa57fa640 1 -- 172.21.15.89:0/803700549 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c005af0 con 0x7ffaa80c9970 2023-10-18T10:46:35.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.577+0000 7ffaa57fa640 1 -- 172.21.15.89:0/803700549 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9c005c70 con 0x7ffaa80c9970 2023-10-18T10:46:35.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.577+0000 7ffaa57fa640 1 -- 172.21.15.89:0/803700549 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c007070 con 0x7ffaa80c9970 2023-10-18T10:46:35.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.577+0000 7ffaaed67640 1 -- 172.21.15.89:0/803700549 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa815a3d0 con 0x7ffaa80c9970 2023-10-18T10:46:35.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.577+0000 7ffaaed67640 1 -- 172.21.15.89:0/803700549 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa815a990 con 0x7ffaa80c9970 2023-10-18T10:46:35.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.581+0000 7ffaaed67640 1 -- 172.21.15.89:0/803700549 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaa815abb0 con 0x7ffaa80c9970 2023-10-18T10:46:35.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.581+0000 7ffaa57fa640 1 -- 172.21.15.89:0/803700549 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7ffa9c0071d0 con 0x7ffaa80c9970 2023-10-18T10:46:35.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.581+0000 7ffaa57fa640 1 --2- 172.21.15.89:0/803700549 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ffa94065680 0x7ffa94067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:35.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.581+0000 7ffaa57fa640 1 -- 172.21.15.89:0/803700549 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ffa9c082da0 con 0x7ffaa80c9970 2023-10-18T10:46:35.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.581+0000 7ffaacb03640 1 --2- 172.21.15.89:0/803700549 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ffa94065680 0x7ffa94067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:35.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.581+0000 7ffaacb03640 1 --2- 172.21.15.89:0/803700549 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ffa94065680 0x7ffa94067b40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffa980011f0 tx=0x7ffa98009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:35.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.585+0000 7ffaa57fa640 1 -- 172.21.15.89:0/803700549 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7ffa9c050610 con 0x7ffaa80c9970 2023-10-18T10:46:35.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.733+0000 7ffaaed67640 1 -- 172.21.15.89:0/803700549 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ffaa804dbe0 con 0x7ffaa80c9970 2023-10-18T10:46:35.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.733+0000 7ffaa57fa640 1 -- 172.21.15.89:0/803700549 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7ffa9c053c30 con 0x7ffaa80c9970 2023-10-18T10:46:35.736 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-18T10:46:35.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.733+0000 7ffaaed67640 1 -- 172.21.15.89:0/803700549 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ffa94065680 msgr2=0x7ffa94067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:35.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.733+0000 7ffaaed67640 1 --2- 172.21.15.89:0/803700549 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ffa94065680 0x7ffa94067b40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffa980011f0 tx=0x7ffa98009040 comp rx=0 tx=0).stop 2023-10-18T10:46:35.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.733+0000 7ffaaed67640 1 -- 172.21.15.89:0/803700549 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffaa80c9970 msgr2=0x7ffaa81f7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:35.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.733+0000 7ffaaed67640 1 --2- 172.21.15.89:0/803700549 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffaa80c9970 0x7ffaa81f7e90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c00c750 tx=0x7ffa9c00cc10 comp rx=0 tx=0).stop 2023-10-18T10:46:35.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.733+0000 7ffaaed67640 1 -- 172.21.15.89:0/803700549 shutdown_connections 2023-10-18T10:46:35.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.733+0000 7ffaaed67640 1 --2- 172.21.15.89:0/803700549 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ffa94065680 0x7ffa94067b40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:35.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.733+0000 7ffaaed67640 1 --2- 172.21.15.89:0/803700549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffaa8159490 0x7ffaa81432d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:35.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.733+0000 7ffaaed67640 1 --2- 172.21.15.89:0/803700549 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffaa80c9970 0x7ffaa81f7e90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:35.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.733+0000 7ffaaed67640 1 --2- 172.21.15.89:0/803700549 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffaa8070600 0x7ffaa80c6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:35.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.733+0000 7ffaaed67640 1 -- 172.21.15.89:0/803700549 >> 172.21.15.89:0/803700549 conn(0x7ffaa8069f00 msgr2=0x7ffaa80aedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:35.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.733+0000 7ffaaed67640 1 -- 172.21.15.89:0/803700549 shutdown_connections 2023-10-18T10:46:35.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.733+0000 7ffaaed67640 1 -- 172.21.15.89:0/803700549 wait complete. 2023-10-18T10:46:35.758 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:46:35.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.893+0000 7f2b9b8ca640 1 Processor -- start 2023-10-18T10:46:35.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.897+0000 7f2b9b8ca640 1 -- start start 2023-10-18T10:46:35.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.897+0000 7f2b9b8ca640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2b9415df70 0x7f2b940c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:35.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.897+0000 7f2b9b8ca640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b940afd90 0x7f2b940b01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:35.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.897+0000 7f2b9b8ca640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b940b0720 con 0x7f2b940c98d0 2023-10-18T10:46:35.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.897+0000 7f2b9b8ca640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b940cac00 con 0x7f2b9415df70 2023-10-18T10:46:35.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.897+0000 7f2b9b8ca640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b940c9cb0 con 0x7f2b940afd90 2023-10-18T10:46:35.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.897+0000 7f2b99666640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2b940c98d0 0x7f2b940ca4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47918/0 (socket says 172.21.15.89:47918) 2023-10-18T10:46:35.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.897+0000 7f2b99666640 1 -- 172.21.15.89:0/2243038848 learned_addr learned my addr 172.21.15.89:0/2243038848 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:35.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.897+0000 7f2b99e67640 1 --2- 172.21.15.89:0/2243038848 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b940afd90 0x7f2b940b01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:35.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.897+0000 7f2b98e65640 1 --2- 172.21.15.89:0/2243038848 >> v2:172.21.15.154:3300/0 conn(0x7f2b9415df70 0x7f2b940c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:35.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.897+0000 7f2b8bfff640 1 -- 172.21.15.89:0/2243038848 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1130652050 0 0) 0x7f2b90001120 con 0x7f2b940c98d0 2023-10-18T10:46:35.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.897+0000 7f2b8bfff640 1 -- 172.21.15.89:0/2243038848 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b80003530 con 0x7f2b940c98d0 2023-10-18T10:46:35.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.897+0000 7f2b98e65640 1 -- 172.21.15.89:0/2243038848 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b940afd90 msgr2=0x7f2b940b01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:35.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.897+0000 7f2b98e65640 1 --2- 172.21.15.89:0/2243038848 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b940afd90 0x7f2b940b01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:35.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.897+0000 7f2b98e65640 1 -- 172.21.15.89:0/2243038848 >> v1:172.21.15.89:6789/0 conn(0x7f2b940c98d0 legacy=0x7f2b940ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:35.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.897+0000 7f2b98e65640 1 -- 172.21.15.89:0/2243038848 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b9415e4a0 con 0x7f2b9415df70 2023-10-18T10:46:35.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.897+0000 7f2b99e67640 1 --2- 172.21.15.89:0/2243038848 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b940afd90 0x7f2b940b01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:46:35.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.901+0000 7f2b98e65640 1 --2- 172.21.15.89:0/2243038848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b9415df70 0x7f2b940c8960 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2b840097c0 tx=0x7f2b8401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ca249ec72bf1b8a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:35.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.901+0000 7f2b8bfff640 1 -- 172.21.15.89:0/2243038848 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b840ab650 con 0x7f2b9415df70 2023-10-18T10:46:35.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.901+0000 7f2b8bfff640 1 -- 172.21.15.89:0/2243038848 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b840ab7b0 con 0x7f2b9415df70 2023-10-18T10:46:35.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.901+0000 7f2b8bfff640 1 -- 172.21.15.89:0/2243038848 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b840abb20 con 0x7f2b9415df70 2023-10-18T10:46:35.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.901+0000 7f2b9b8ca640 1 -- 172.21.15.89:0/2243038848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b9415df70 msgr2=0x7f2b940c8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:35.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.901+0000 7f2b9b8ca640 1 --2- 172.21.15.89:0/2243038848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b9415df70 0x7f2b940c8960 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2b840097c0 tx=0x7f2b8401c450 comp rx=0 tx=0).stop 2023-10-18T10:46:35.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.901+0000 7f2b9b8ca640 1 -- 172.21.15.89:0/2243038848 shutdown_connections 2023-10-18T10:46:35.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.901+0000 7f2b9b8ca640 1 --2- 172.21.15.89:0/2243038848 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b940afd90 0x7f2b940b01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:35.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.901+0000 7f2b9b8ca640 1 --2- 172.21.15.89:0/2243038848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b9415df70 0x7f2b940c8960 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:35.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.901+0000 7f2b9b8ca640 1 -- 172.21.15.89:0/2243038848 >> 172.21.15.89:0/2243038848 conn(0x7f2b94069f00 msgr2=0x7f2b940cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:35.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.901+0000 7f2b9b8ca640 1 -- 172.21.15.89:0/2243038848 shutdown_connections 2023-10-18T10:46:35.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.901+0000 7f2b9b8ca640 1 -- 172.21.15.89:0/2243038848 wait complete. 2023-10-18T10:46:35.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.905+0000 7f2b9b8ca640 1 Processor -- start 2023-10-18T10:46:35.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.905+0000 7f2b9b8ca640 1 -- start start 2023-10-18T10:46:35.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.905+0000 7f2b9b8ca640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b940afd90 0x7f2b9415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:35.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.905+0000 7f2b9b8ca640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b940c98d0 0x7f2b940c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:35.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.905+0000 7f2b9b8ca640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b9415df70 0x7f2b940c4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:35.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.905+0000 7f2b9b8ca640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9415c420 con 0x7f2b940c98d0 2023-10-18T10:46:35.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.905+0000 7f2b9b8ca640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b940b2a60 con 0x7f2b9415df70 2023-10-18T10:46:35.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.905+0000 7f2b9b8ca640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b940b2d00 con 0x7f2b940afd90 2023-10-18T10:46:35.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.905+0000 7f2b98e65640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b940c98d0 0x7f2b940c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:35.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.905+0000 7f2b98e65640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b940c98d0 0x7f2b940c3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47062/0 (socket says 172.21.15.89:47062) 2023-10-18T10:46:35.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.905+0000 7f2b98e65640 1 -- 172.21.15.89:0/3428876276 learned_addr learned my addr 172.21.15.89:0/3428876276 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:35.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.905+0000 7f2b99e67640 1 --2- 172.21.15.89:0/3428876276 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b9415df70 0x7f2b940c4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:35.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.905+0000 7f2b99666640 1 --2- 172.21.15.89:0/3428876276 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b940afd90 0x7f2b9415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:35.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.905+0000 7f2b98e65640 1 -- 172.21.15.89:0/3428876276 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b940afd90 msgr2=0x7f2b9415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:35.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.905+0000 7f2b98e65640 1 --2- 172.21.15.89:0/3428876276 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b940afd90 0x7f2b9415bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:35.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.905+0000 7f2b98e65640 1 -- 172.21.15.89:0/3428876276 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b9415df70 msgr2=0x7f2b940c4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:35.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.905+0000 7f2b98e65640 1 --2- 172.21.15.89:0/3428876276 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b9415df70 0x7f2b940c4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:35.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.905+0000 7f2b98e65640 1 -- 172.21.15.89:0/3428876276 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b8c009a70 con 0x7f2b940c98d0 2023-10-18T10:46:35.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.905+0000 7f2b99e67640 1 --2- 172.21.15.89:0/3428876276 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b9415df70 0x7f2b940c4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:35.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.905+0000 7f2b98e65640 1 --2- 172.21.15.89:0/3428876276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b940c98d0 0x7f2b940c3bd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2b8401ccf0 tx=0x7f2b8401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:35.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.905+0000 7f2b89ffb640 1 -- 172.21.15.89:0/3428876276 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b840bf610 con 0x7f2b940c98d0 2023-10-18T10:46:35.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.905+0000 7f2b89ffb640 1 -- 172.21.15.89:0/3428876276 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b840bf770 con 0x7f2b940c98d0 2023-10-18T10:46:35.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.905+0000 7f2b89ffb640 1 -- 172.21.15.89:0/3428876276 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b840bfae0 con 0x7f2b940c98d0 2023-10-18T10:46:35.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.905+0000 7f2b9b8ca640 1 -- 172.21.15.89:0/3428876276 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b84009420 con 0x7f2b940c98d0 2023-10-18T10:46:35.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.905+0000 7f2b9b8ca640 1 -- 172.21.15.89:0/3428876276 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b940b32c0 con 0x7f2b940c98d0 2023-10-18T10:46:35.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.905+0000 7f2b9b8ca640 1 -- 172.21.15.89:0/3428876276 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b9404dbe0 con 0x7f2b940c98d0 2023-10-18T10:46:35.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.909+0000 7f2b89ffb640 1 -- 172.21.15.89:0/3428876276 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f2b840bfc40 con 0x7f2b940c98d0 2023-10-18T10:46:35.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.909+0000 7f2b89ffb640 1 --2- 172.21.15.89:0/3428876276 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2b78065680 0x7f2b78067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:35.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.909+0000 7f2b89ffb640 1 -- 172.21.15.89:0/3428876276 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2b8411ded0 con 0x7f2b940c98d0 2023-10-18T10:46:35.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.909+0000 7f2b99666640 1 --2- 172.21.15.89:0/3428876276 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2b78065680 0x7f2b78067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:35.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.909+0000 7f2b99666640 1 --2- 172.21.15.89:0/3428876276 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2b78065680 0x7f2b78067b40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2b90005890 tx=0x7f2b90009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:35.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:35.913+0000 7f2b89ffb640 1 -- 172.21.15.89:0/3428876276 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f2b840eb690 con 0x7f2b940c98d0 2023-10-18T10:46:36.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.069+0000 7f2b9b8ca640 1 -- 172.21.15.89:0/3428876276 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b940cb890 con 0x7f2b940c98d0 2023-10-18T10:46:36.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.069+0000 7f2b89ffb640 1 -- 172.21.15.89:0/3428876276 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+20192 (secure 0 0 0) 0x7f2b840c4020 con 0x7f2b940c98d0 2023-10-18T10:46:36.074 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:36.074 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":28,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:46:35.951496+0000","last_up_change":"2023-10-18T10:45:42.164688+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:45:53.309122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-18T10:45:54.659763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:45:57.796514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6824","nonce":773078364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6827","nonce":773078364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6831","nonce":773078364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6829","nonce":773078364}]},"public_addr":"172.21.15.154:6824/773078364","cluster_addr":"172.21.15.154:6827/773078364","heartbeat_back_addr":"172.21.15.154:6831/773078364","heartbeat_front_addr":"172.21.15.154:6829/773078364","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":1,"to":5}]},{"pgid":"4.1a","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:46:36.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.073+0000 7f2b9b8ca640 1 -- 172.21.15.89:0/3428876276 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2b78065680 msgr2=0x7f2b78067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:36.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.073+0000 7f2b9b8ca640 1 --2- 172.21.15.89:0/3428876276 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2b78065680 0x7f2b78067b40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2b90005890 tx=0x7f2b90009040 comp rx=0 tx=0).stop 2023-10-18T10:46:36.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.073+0000 7f2b9b8ca640 1 -- 172.21.15.89:0/3428876276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b940c98d0 msgr2=0x7f2b940c3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:36.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.073+0000 7f2b9b8ca640 1 --2- 172.21.15.89:0/3428876276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b940c98d0 0x7f2b940c3bd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2b8401ccf0 tx=0x7f2b8401ac90 comp rx=0 tx=0).stop 2023-10-18T10:46:36.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.073+0000 7f2b9b8ca640 1 -- 172.21.15.89:0/3428876276 shutdown_connections 2023-10-18T10:46:36.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.073+0000 7f2b9b8ca640 1 --2- 172.21.15.89:0/3428876276 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2b78065680 0x7f2b78067b40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:36.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.073+0000 7f2b9b8ca640 1 --2- 172.21.15.89:0/3428876276 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b9415df70 0x7f2b940c4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:36.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.073+0000 7f2b9b8ca640 1 --2- 172.21.15.89:0/3428876276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b940c98d0 0x7f2b940c3bd0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:36.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.073+0000 7f2b9b8ca640 1 --2- 172.21.15.89:0/3428876276 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b940afd90 0x7f2b9415bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:36.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.073+0000 7f2b9b8ca640 1 -- 172.21.15.89:0/3428876276 >> 172.21.15.89:0/3428876276 conn(0x7f2b94069f00 msgr2=0x7f2b9415cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:36.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.073+0000 7f2b9b8ca640 1 -- 172.21.15.89:0/3428876276 shutdown_connections 2023-10-18T10:46:36.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.073+0000 7f2b9b8ca640 1 -- 172.21.15.89:0/3428876276 wait complete. 2023-10-18T10:46:36.095 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:45:43.114184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-18T10:45:48.730968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:45:53.309122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:45:54.659763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T10:45:57.796514+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T10:46:36.096 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:46:36.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.233+0000 7f1e63300640 1 Processor -- start 2023-10-18T10:46:36.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.233+0000 7f1e63300640 1 -- start start 2023-10-18T10:46:36.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.233+0000 7f1e63300640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e5c06ff00 0x7f1e5c15e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:36.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.233+0000 7f1e63300640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1e5c0ca340 0x7f1e5c15b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:36.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.233+0000 7f1e63300640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1e5c15b680 0x7f1e5c0c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:36.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.233+0000 7f1e63300640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e5c15ef30 con 0x7f1e5c15b680 2023-10-18T10:46:36.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.233+0000 7f1e63300640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e5c15aa10 con 0x7f1e5c0ca340 2023-10-18T10:46:36.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.233+0000 7f1e63300640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c15acb0 con 0x7f1e5c06ff00 2023-10-18T10:46:36.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.237+0000 7f1e6189d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1e5c15b680 0x7f1e5c0c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:36.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.237+0000 7f1e6189d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1e5c15b680 0x7f1e5c0c8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47074/0 (socket says 172.21.15.89:47074) 2023-10-18T10:46:36.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.237+0000 7f1e6189d640 1 -- 172.21.15.89:0/3795231263 learned_addr learned my addr 172.21.15.89:0/3795231263 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:36.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.237+0000 7f1e6189d640 1 -- 172.21.15.89:0/3795231263 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e5c06ff00 msgr2=0x7f1e5c15e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:36.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.237+0000 7f1e6189d640 1 --2- 172.21.15.89:0/3795231263 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e5c06ff00 0x7f1e5c15e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:36.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.237+0000 7f1e6189d640 1 -- 172.21.15.89:0/3795231263 >> v2:172.21.15.154:3300/0 conn(0x7f1e5c0ca340 msgr2=0x7f1e5c15b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:36.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.237+0000 7f1e6189d640 1 --2- 172.21.15.89:0/3795231263 >> v2:172.21.15.154:3300/0 conn(0x7f1e5c0ca340 0x7f1e5c15b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:36.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.237+0000 7f1e6189d640 1 -- 172.21.15.89:0/3795231263 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e5c0702e0 con 0x7f1e5c15b680 2023-10-18T10:46:36.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.237+0000 7f1e6189d640 1 --2- 172.21.15.89:0/3795231263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e5c15b680 0x7f1e5c0c8960 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c0b0f00 tx=0x7f1e5401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=579d6eafe3276f58 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:36.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.237+0000 7f1e53fff640 1 -- 172.21.15.89:0/3795231263 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e540ac640 con 0x7f1e5c15b680 2023-10-18T10:46:36.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.237+0000 7f1e53fff640 1 -- 172.21.15.89:0/3795231263 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e540ac7a0 con 0x7f1e5c15b680 2023-10-18T10:46:36.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.237+0000 7f1e53fff640 1 -- 172.21.15.89:0/3795231263 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e540acb30 con 0x7f1e5c15b680 2023-10-18T10:46:36.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.237+0000 7f1e63300640 1 -- 172.21.15.89:0/3795231263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e5c15b680 msgr2=0x7f1e5c0c8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:36.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.237+0000 7f1e63300640 1 --2- 172.21.15.89:0/3795231263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e5c15b680 0x7f1e5c0c8960 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c0b0f00 tx=0x7f1e5401c440 comp rx=0 tx=0).stop 2023-10-18T10:46:36.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.241+0000 7f1e63300640 1 -- 172.21.15.89:0/3795231263 shutdown_connections 2023-10-18T10:46:36.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.241+0000 7f1e63300640 1 --2- 172.21.15.89:0/3795231263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e5c15b680 0x7f1e5c0c8960 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:36.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.241+0000 7f1e63300640 1 --2- 172.21.15.89:0/3795231263 >> v2:172.21.15.154:3300/0 conn(0x7f1e5c0ca340 0x7f1e5c15b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:36.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.241+0000 7f1e63300640 1 --2- 172.21.15.89:0/3795231263 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e5c06ff00 0x7f1e5c15e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:36.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.241+0000 7f1e63300640 1 -- 172.21.15.89:0/3795231263 >> 172.21.15.89:0/3795231263 conn(0x7f1e5c069f00 msgr2=0x7f1e5c0cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:36.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.241+0000 7f1e63300640 1 -- 172.21.15.89:0/3795231263 shutdown_connections 2023-10-18T10:46:36.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.241+0000 7f1e63300640 1 -- 172.21.15.89:0/3795231263 wait complete. 2023-10-18T10:46:36.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.241+0000 7f1e63300640 1 Processor -- start 2023-10-18T10:46:36.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.241+0000 7f1e63300640 1 -- start start 2023-10-18T10:46:36.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.241+0000 7f1e63300640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e5c06ff00 0x7f1e5c14d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:36.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.241+0000 7f1e63300640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e5c0ca340 0x7f1e5c0b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:36.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.241+0000 7f1e63300640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e5c0c39a0 0x7f1e5c0c3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:36.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.241+0000 7f1e63300640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c0c4c80 con 0x7f1e5c0ca340 2023-10-18T10:46:36.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.241+0000 7f1e63300640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c0c4670 con 0x7f1e5c06ff00 2023-10-18T10:46:36.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.241+0000 7f1e63300640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c0c2140 con 0x7f1e5c0c39a0 2023-10-18T10:46:36.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.241+0000 7f1e6089b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e5c0ca340 0x7f1e5c0b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:36.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.241+0000 7f1e6089b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e5c0ca340 0x7f1e5c0b2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47084/0 (socket says 172.21.15.89:47084) 2023-10-18T10:46:36.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.241+0000 7f1e6089b640 1 -- 172.21.15.89:0/4235094050 learned_addr learned my addr 172.21.15.89:0/4235094050 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:36.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.241+0000 7f1e6189d640 1 --2- 172.21.15.89:0/4235094050 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e5c0c39a0 0x7f1e5c0c3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:36.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.241+0000 7f1e6109c640 1 --2- 172.21.15.89:0/4235094050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e5c06ff00 0x7f1e5c14d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:36.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.241+0000 7f1e6089b640 1 -- 172.21.15.89:0/4235094050 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e5c0c39a0 msgr2=0x7f1e5c0c3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:36.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.241+0000 7f1e6089b640 1 --2- 172.21.15.89:0/4235094050 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e5c0c39a0 0x7f1e5c0c3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:36.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.241+0000 7f1e6089b640 1 -- 172.21.15.89:0/4235094050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e5c06ff00 msgr2=0x7f1e5c14d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:36.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.241+0000 7f1e6089b640 1 --2- 172.21.15.89:0/4235094050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e5c06ff00 0x7f1e5c14d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:36.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.241+0000 7f1e6089b640 1 -- 172.21.15.89:0/4235094050 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e540094d0 con 0x7f1e5c0ca340 2023-10-18T10:46:36.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.241+0000 7f1e6189d640 1 --2- 172.21.15.89:0/4235094050 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e5c0c39a0 0x7f1e5c0c3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:36.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.241+0000 7f1e6109c640 1 --2- 172.21.15.89:0/4235094050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e5c06ff00 0x7f1e5c14d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:36.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.241+0000 7f1e6089b640 1 --2- 172.21.15.89:0/4235094050 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e5c0ca340 0x7f1e5c0b2df0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c00b550 tx=0x7f1e4c00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:36.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.245+0000 7f1e51ffb640 1 -- 172.21.15.89:0/4235094050 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c010870 con 0x7f1e5c0ca340 2023-10-18T10:46:36.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.245+0000 7f1e51ffb640 1 -- 172.21.15.89:0/4235094050 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e4c0109f0 con 0x7f1e5c0ca340 2023-10-18T10:46:36.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.245+0000 7f1e51ffb640 1 -- 172.21.15.89:0/4235094050 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c010da0 con 0x7f1e5c0ca340 2023-10-18T10:46:36.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.245+0000 7f1e63300640 1 -- 172.21.15.89:0/4235094050 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e5c0c4910 con 0x7f1e5c0ca340 2023-10-18T10:46:36.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.245+0000 7f1e63300640 1 -- 172.21.15.89:0/4235094050 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e5c0b2100 con 0x7f1e5c0ca340 2023-10-18T10:46:36.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.245+0000 7f1e63300640 1 -- 172.21.15.89:0/4235094050 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e5c0c4e30 con 0x7f1e5c0ca340 2023-10-18T10:46:36.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.245+0000 7f1e51ffb640 1 -- 172.21.15.89:0/4235094050 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f1e4c00f3c0 con 0x7f1e5c0ca340 2023-10-18T10:46:36.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.245+0000 7f1e51ffb640 1 --2- 172.21.15.89:0/4235094050 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1e40065680 0x7f1e40067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:36.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.245+0000 7f1e51ffb640 1 -- 172.21.15.89:0/4235094050 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1e4c083110 con 0x7f1e5c0ca340 2023-10-18T10:46:36.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.245+0000 7f1e6109c640 1 --2- 172.21.15.89:0/4235094050 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1e40065680 0x7f1e40067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:36.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.249+0000 7f1e6109c640 1 --2- 172.21.15.89:0/4235094050 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1e40065680 0x7f1e40067b40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1e58007eb0 tx=0x7f1e580073c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:36.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.249+0000 7f1e51ffb640 1 -- 172.21.15.89:0/4235094050 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f1e5c0c4e30 con 0x7f1e5c0ca340 2023-10-18T10:46:36.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.401+0000 7f1e63300640 1 -- 172.21.15.89:0/4235094050 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1e5c04dbe0 con 0x7f1e5c0ca340 2023-10-18T10:46:36.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.401+0000 7f1e51ffb640 1 -- 172.21.15.89:0/4235094050 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1e4c054330 con 0x7f1e5c0ca340 2023-10-18T10:46:36.404 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-18T10:46:36.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.401+0000 7f1e63300640 1 -- 172.21.15.89:0/4235094050 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1e40065680 msgr2=0x7f1e40067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:36.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.401+0000 7f1e63300640 1 --2- 172.21.15.89:0/4235094050 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1e40065680 0x7f1e40067b40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1e58007eb0 tx=0x7f1e580073c0 comp rx=0 tx=0).stop 2023-10-18T10:46:36.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.401+0000 7f1e63300640 1 -- 172.21.15.89:0/4235094050 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e5c0ca340 msgr2=0x7f1e5c0b2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:36.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.401+0000 7f1e63300640 1 --2- 172.21.15.89:0/4235094050 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e5c0ca340 0x7f1e5c0b2df0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c00b550 tx=0x7f1e4c00ba10 comp rx=0 tx=0).stop 2023-10-18T10:46:36.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.401+0000 7f1e63300640 1 -- 172.21.15.89:0/4235094050 shutdown_connections 2023-10-18T10:46:36.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.401+0000 7f1e63300640 1 --2- 172.21.15.89:0/4235094050 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1e40065680 0x7f1e40067b40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:36.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.401+0000 7f1e63300640 1 --2- 172.21.15.89:0/4235094050 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e5c0c39a0 0x7f1e5c0c3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:36.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.401+0000 7f1e63300640 1 --2- 172.21.15.89:0/4235094050 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e5c0ca340 0x7f1e5c0b2df0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:36.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.401+0000 7f1e63300640 1 --2- 172.21.15.89:0/4235094050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e5c06ff00 0x7f1e5c14d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:36.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.401+0000 7f1e63300640 1 -- 172.21.15.89:0/4235094050 >> 172.21.15.89:0/4235094050 conn(0x7f1e5c069f00 msgr2=0x7f1e5c06bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:36.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.401+0000 7f1e63300640 1 -- 172.21.15.89:0/4235094050 shutdown_connections 2023-10-18T10:46:36.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.401+0000 7f1e63300640 1 -- 172.21.15.89:0/4235094050 wait complete. 2023-10-18T10:46:36.425 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-18T10:46:36.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.561+0000 7fd7a9e93640 1 Processor -- start 2023-10-18T10:46:36.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.561+0000 7fd7a9e93640 1 -- start start 2023-10-18T10:46:36.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.561+0000 7fd7a9e93640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7a4159490 0x7fd7a415d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:36.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.561+0000 7fd7a9e93640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd7a406bee0 0x7fd7a40c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:36.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.561+0000 7fd7a9e93640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7a415ddf0 con 0x7fd7a4070600 2023-10-18T10:46:36.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.561+0000 7fd7a9e93640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7a4070b90 con 0x7fd7a406bee0 2023-10-18T10:46:36.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.561+0000 7fd7a9e93640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a4070e70 con 0x7fd7a4159490 2023-10-18T10:46:36.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.561+0000 7fd7a3fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd7a406bee0 0x7fd7a40c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:36.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.561+0000 7fd7a3fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd7a406bee0 0x7fd7a40c8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47094/0 (socket says 172.21.15.89:47094) 2023-10-18T10:46:36.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.561+0000 7fd7a3fff640 1 -- 172.21.15.89:0/3125160466 learned_addr learned my addr 172.21.15.89:0/3125160466 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:36.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.561+0000 7fd7a37fe640 1 --2- 172.21.15.89:0/3125160466 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7a4159490 0x7fd7a415d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:36.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.561+0000 7fd7a3fff640 1 -- 172.21.15.89:0/3125160466 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7a4159490 msgr2=0x7fd7a415d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:36.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.561+0000 7fd7a3fff640 1 --2- 172.21.15.89:0/3125160466 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7a4159490 0x7fd7a415d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:36.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.565+0000 7fd7a3fff640 1 -- 172.21.15.89:0/3125160466 >> v1:172.21.15.154:6789/0 conn(0x7fd7a4070600 legacy=0x7fd7a406b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:36.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.565+0000 7fd7a3fff640 1 -- 172.21.15.89:0/3125160466 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a4071150 con 0x7fd7a406bee0 2023-10-18T10:46:36.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.565+0000 7fd7a3fff640 1 --2- 172.21.15.89:0/3125160466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7a406bee0 0x7fd7a40c8960 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009e50 tx=0x7fd79c01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=662beb660228acf server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:36.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.565+0000 7fd7a27fc640 1 -- 172.21.15.89:0/3125160466 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c0a5810 con 0x7fd7a406bee0 2023-10-18T10:46:36.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.565+0000 7fd7a27fc640 1 -- 172.21.15.89:0/3125160466 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd79c0a5970 con 0x7fd7a406bee0 2023-10-18T10:46:36.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.565+0000 7fd7a27fc640 1 -- 172.21.15.89:0/3125160466 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c0a5d00 con 0x7fd7a406bee0 2023-10-18T10:46:36.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.565+0000 7fd7a9e93640 1 -- 172.21.15.89:0/3125160466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7a406bee0 msgr2=0x7fd7a40c8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:36.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.565+0000 7fd7a9e93640 1 --2- 172.21.15.89:0/3125160466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7a406bee0 0x7fd7a40c8960 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009e50 tx=0x7fd79c01c950 comp rx=0 tx=0).stop 2023-10-18T10:46:36.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.565+0000 7fd7a9e93640 1 -- 172.21.15.89:0/3125160466 shutdown_connections 2023-10-18T10:46:36.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.565+0000 7fd7a9e93640 1 --2- 172.21.15.89:0/3125160466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7a406bee0 0x7fd7a40c8960 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:36.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.565+0000 7fd7a9e93640 1 --2- 172.21.15.89:0/3125160466 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7a4159490 0x7fd7a415d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:36.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.565+0000 7fd7a9e93640 1 -- 172.21.15.89:0/3125160466 >> 172.21.15.89:0/3125160466 conn(0x7fd7a4069f00 msgr2=0x7fd7a40cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:36.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.565+0000 7fd7a9e93640 1 -- 172.21.15.89:0/3125160466 shutdown_connections 2023-10-18T10:46:36.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.565+0000 7fd7a9e93640 1 -- 172.21.15.89:0/3125160466 wait complete. 2023-10-18T10:46:36.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.569+0000 7fd7a9e93640 1 Processor -- start 2023-10-18T10:46:36.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.569+0000 7fd7a9e93640 1 -- start start 2023-10-18T10:46:36.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.569+0000 7fd7a9e93640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7a406bee0 0x7fd7a4150600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:36.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.569+0000 7fd7a9e93640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd7a4070600 0x7fd7a40b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:36.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.569+0000 7fd7a9e93640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7a4159490 0x7fd7a40ae340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:36.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.569+0000 7fd7a9e93640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a4150e40 con 0x7fd7a4159490 2023-10-18T10:46:36.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.569+0000 7fd7a9e93640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a4151ea0 con 0x7fd7a4070600 2023-10-18T10:46:36.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.569+0000 7fd7a9e93640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a41510e0 con 0x7fd7a406bee0 2023-10-18T10:46:36.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.569+0000 7fd7a3fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7a4159490 0x7fd7a40ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:36.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.569+0000 7fd7a3fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7a4159490 0x7fd7a40ae340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47096/0 (socket says 172.21.15.89:47096) 2023-10-18T10:46:36.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.569+0000 7fd7a3fff640 1 -- 172.21.15.89:0/3245817692 learned_addr learned my addr 172.21.15.89:0/3245817692 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:36.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.569+0000 7fd7a37fe640 1 --2- 172.21.15.89:0/3245817692 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7a406bee0 0x7fd7a4150600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:36.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.569+0000 7fd7a2ffd640 1 --2- 172.21.15.89:0/3245817692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd7a4070600 0x7fd7a40b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:36.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.569+0000 7fd7a3fff640 1 -- 172.21.15.89:0/3245817692 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7a406bee0 msgr2=0x7fd7a4150600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:36.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.569+0000 7fd7a3fff640 1 --2- 172.21.15.89:0/3245817692 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7a406bee0 0x7fd7a4150600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:36.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.569+0000 7fd7a3fff640 1 -- 172.21.15.89:0/3245817692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd7a4070600 msgr2=0x7fd7a40b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:36.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.569+0000 7fd7a3fff640 1 --2- 172.21.15.89:0/3245817692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd7a4070600 0x7fd7a40b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:36.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.569+0000 7fd7a3fff640 1 -- 172.21.15.89:0/3245817692 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd79c009b30 con 0x7fd7a4159490 2023-10-18T10:46:36.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.569+0000 7fd7a37fe640 1 --2- 172.21.15.89:0/3245817692 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7a406bee0 0x7fd7a4150600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:36.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.569+0000 7fd7a2ffd640 1 --2- 172.21.15.89:0/3245817692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd7a4070600 0x7fd7a40b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:36.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.569+0000 7fd7a3fff640 1 --2- 172.21.15.89:0/3245817692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7a4159490 0x7fd7a40ae340 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd79c0af040 tx=0x7fd79c0196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:36.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.573+0000 7fd78bfff640 1 -- 172.21.15.89:0/3245817692 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c0b8670 con 0x7fd7a4159490 2023-10-18T10:46:36.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.573+0000 7fd78bfff640 1 -- 172.21.15.89:0/3245817692 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd79c0b87d0 con 0x7fd7a4159490 2023-10-18T10:46:36.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.573+0000 7fd7a9e93640 1 -- 172.21.15.89:0/3245817692 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7a41513c0 con 0x7fd7a4159490 2023-10-18T10:46:36.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.573+0000 7fd78bfff640 1 -- 172.21.15.89:0/3245817692 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c0b8b80 con 0x7fd7a4159490 2023-10-18T10:46:36.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.573+0000 7fd7a9e93640 1 -- 172.21.15.89:0/3245817692 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a4151980 con 0x7fd7a4159490 2023-10-18T10:46:36.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.573+0000 7fd7a9e93640 1 -- 172.21.15.89:0/3245817692 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7a4151b70 con 0x7fd7a4159490 2023-10-18T10:46:36.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.573+0000 7fd78bfff640 1 -- 172.21.15.89:0/3245817692 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fd79c0b8ce0 con 0x7fd7a4159490 2023-10-18T10:46:36.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.573+0000 7fd78bfff640 1 --2- 172.21.15.89:0/3245817692 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd7900656b0 0x7fd790067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:36.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.573+0000 7fd78bfff640 1 -- 172.21.15.89:0/3245817692 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd79c11e3b0 con 0x7fd7a4159490 2023-10-18T10:46:36.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.573+0000 7fd7a37fe640 1 --2- 172.21.15.89:0/3245817692 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd7900656b0 0x7fd790067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:36.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.577+0000 7fd7a37fe640 1 --2- 172.21.15.89:0/3245817692 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd7900656b0 0x7fd790067b70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd794009b10 tx=0x7fd794008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:36.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.577+0000 7fd78bfff640 1 -- 172.21.15.89:0/3245817692 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fd79c0ebbd0 con 0x7fd7a4159490 2023-10-18T10:46:36.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.729+0000 7fd7a9e93640 1 -- 172.21.15.89:0/3245817692 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd7a404dbe0 con 0x7fd7a4159490 2023-10-18T10:46:36.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.729+0000 7fd78bfff640 1 -- 172.21.15.89:0/3245817692 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v28) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd79c0ef1f0 con 0x7fd7a4159490 2023-10-18T10:46:36.732 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-10-18T10:46:36.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.729+0000 7fd7a9e93640 1 -- 172.21.15.89:0/3245817692 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd7900656b0 msgr2=0x7fd790067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:36.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.729+0000 7fd7a9e93640 1 --2- 172.21.15.89:0/3245817692 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd7900656b0 0x7fd790067b70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd794009b10 tx=0x7fd794008040 comp rx=0 tx=0).stop 2023-10-18T10:46:36.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.729+0000 7fd7a9e93640 1 -- 172.21.15.89:0/3245817692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7a4159490 msgr2=0x7fd7a40ae340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:36.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.729+0000 7fd7a9e93640 1 --2- 172.21.15.89:0/3245817692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7a4159490 0x7fd7a40ae340 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd79c0af040 tx=0x7fd79c0196f0 comp rx=0 tx=0).stop 2023-10-18T10:46:36.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.729+0000 7fd7a9e93640 1 -- 172.21.15.89:0/3245817692 shutdown_connections 2023-10-18T10:46:36.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.729+0000 7fd7a9e93640 1 --2- 172.21.15.89:0/3245817692 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd7900656b0 0x7fd790067b70 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:36.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.729+0000 7fd7a9e93640 1 --2- 172.21.15.89:0/3245817692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7a4159490 0x7fd7a40ae340 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:36.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.729+0000 7fd7a9e93640 1 --2- 172.21.15.89:0/3245817692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd7a4070600 0x7fd7a40b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:36.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.729+0000 7fd7a9e93640 1 --2- 172.21.15.89:0/3245817692 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7a406bee0 0x7fd7a4150600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:36.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.729+0000 7fd7a9e93640 1 -- 172.21.15.89:0/3245817692 >> 172.21.15.89:0/3245817692 conn(0x7fd7a4069f00 msgr2=0x7fd7a406b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:36.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.729+0000 7fd7a9e93640 1 -- 172.21.15.89:0/3245817692 shutdown_connections 2023-10-18T10:46:36.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.729+0000 7fd7a9e93640 1 -- 172.21.15.89:0/3245817692 wait complete. 2023-10-18T10:46:36.753 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:46:36.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.889+0000 7fee7180b640 1 Processor -- start 2023-10-18T10:46:36.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.889+0000 7fee7180b640 1 -- start start 2023-10-18T10:46:36.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.889+0000 7fee7180b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fee6c070600 0x7fee6c06b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:36.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.889+0000 7fee7180b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fee6c06bd10 0x7fee6c0c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:36.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.889+0000 7fee7180b640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee6c070b90 con 0x7fee6c159490 2023-10-18T10:46:36.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.889+0000 7fee7180b640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee6c06c1a0 con 0x7fee6c070600 2023-10-18T10:46:36.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.889+0000 7fee7180b640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee6c070e70 con 0x7fee6c06bd10 2023-10-18T10:46:36.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.893+0000 7fee6a7fc640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fee6c070600 0x7fee6c06b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:36.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.893+0000 7fee6affd640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fee6c159490 0x7fee6c15d8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47934/0 (socket says 172.21.15.89:47934) 2023-10-18T10:46:36.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.893+0000 7fee6a7fc640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fee6c070600 0x7fee6c06b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47108/0 (socket says 172.21.15.89:47108) 2023-10-18T10:46:36.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.893+0000 7fee6affd640 1 -- 172.21.15.89:0/2035634537 learned_addr learned my addr 172.21.15.89:0/2035634537 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:36.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.893+0000 7fee6b7fe640 1 --2- 172.21.15.89:0/2035634537 >> v2:172.21.15.154:3300/0 conn(0x7fee6c06bd10 0x7fee6c0c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:36.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.893+0000 7fee6a7fc640 1 -- 172.21.15.89:0/2035634537 >> v2:172.21.15.154:3300/0 conn(0x7fee6c06bd10 msgr2=0x7fee6c0c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:36.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.893+0000 7fee6a7fc640 1 --2- 172.21.15.89:0/2035634537 >> v2:172.21.15.154:3300/0 conn(0x7fee6c06bd10 0x7fee6c0c8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:36.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.893+0000 7fee6a7fc640 1 -- 172.21.15.89:0/2035634537 >> v1:172.21.15.89:6789/0 conn(0x7fee6c159490 legacy=0x7fee6c15d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:36.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.893+0000 7fee6a7fc640 1 -- 172.21.15.89:0/2035634537 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6c071150 con 0x7fee6c070600 2023-10-18T10:46:36.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.893+0000 7fee69ffb640 1 -- 172.21.15.89:0/2035634537 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2636396285 0 0) 0x7fee6c070b90 con 0x7fee6c159490 2023-10-18T10:46:36.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.893+0000 7fee6a7fc640 1 --2- 172.21.15.89:0/2035634537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee6c070600 0x7fee6c06b7d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fee6c0c88f0 tx=0x7fee6001ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cfa4201f0208724 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:36.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.893+0000 7fee69ffb640 1 -- 172.21.15.89:0/2035634537 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee600a7b40 con 0x7fee6c070600 2023-10-18T10:46:36.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.893+0000 7fee69ffb640 1 -- 172.21.15.89:0/2035634537 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee600a7ca0 con 0x7fee6c070600 2023-10-18T10:46:36.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.893+0000 7fee69ffb640 1 -- 172.21.15.89:0/2035634537 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee600af660 con 0x7fee6c070600 2023-10-18T10:46:36.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.897+0000 7fee7180b640 1 -- 172.21.15.89:0/2035634537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee6c070600 msgr2=0x7fee6c06b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:36.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.897+0000 7fee7180b640 1 --2- 172.21.15.89:0/2035634537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee6c070600 0x7fee6c06b7d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fee6c0c88f0 tx=0x7fee6001ea70 comp rx=0 tx=0).stop 2023-10-18T10:46:36.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.897+0000 7fee7180b640 1 -- 172.21.15.89:0/2035634537 shutdown_connections 2023-10-18T10:46:36.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.897+0000 7fee7180b640 1 --2- 172.21.15.89:0/2035634537 >> v2:172.21.15.154:3300/0 conn(0x7fee6c06bd10 0x7fee6c0c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:36.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.897+0000 7fee7180b640 1 --2- 172.21.15.89:0/2035634537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee6c070600 0x7fee6c06b7d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:36.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.897+0000 7fee7180b640 1 -- 172.21.15.89:0/2035634537 >> 172.21.15.89:0/2035634537 conn(0x7fee6c069f00 msgr2=0x7fee6c0cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:36.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.897+0000 7fee7180b640 1 -- 172.21.15.89:0/2035634537 shutdown_connections 2023-10-18T10:46:36.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.897+0000 7fee7180b640 1 -- 172.21.15.89:0/2035634537 wait complete. 2023-10-18T10:46:36.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.897+0000 7fee7180b640 1 Processor -- start 2023-10-18T10:46:36.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.897+0000 7fee7180b640 1 -- start start 2023-10-18T10:46:36.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.897+0000 7fee7180b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee6c06bd10 0x7fee6c157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:36.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.897+0000 7fee7180b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee6c070600 0x7fee6c0c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:36.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.897+0000 7fee7180b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee6c159490 0x7fee6c15d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:36.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.897+0000 7fee7180b640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c1f83f0 con 0x7fee6c070600 2023-10-18T10:46:36.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.897+0000 7fee7180b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c1f86d0 con 0x7fee6c06bd10 2023-10-18T10:46:36.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.897+0000 7fee7180b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c1f89b0 con 0x7fee6c159490 2023-10-18T10:46:36.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.897+0000 7fee6a7fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee6c070600 0x7fee6c0c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:36.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.897+0000 7fee6a7fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee6c070600 0x7fee6c0c84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47114/0 (socket says 172.21.15.89:47114) 2023-10-18T10:46:36.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.897+0000 7fee6a7fc640 1 -- 172.21.15.89:0/1337340669 learned_addr learned my addr 172.21.15.89:0/1337340669 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:36.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.897+0000 7fee6affd640 1 --2- 172.21.15.89:0/1337340669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee6c06bd10 0x7fee6c157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:36.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.897+0000 7fee6b7fe640 1 --2- 172.21.15.89:0/1337340669 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee6c159490 0x7fee6c15d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:36.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.901+0000 7fee6a7fc640 1 -- 172.21.15.89:0/1337340669 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee6c159490 msgr2=0x7fee6c15d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:36.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.901+0000 7fee6a7fc640 1 --2- 172.21.15.89:0/1337340669 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee6c159490 0x7fee6c15d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:36.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.901+0000 7fee6a7fc640 1 -- 172.21.15.89:0/1337340669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee6c06bd10 msgr2=0x7fee6c157bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:36.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.901+0000 7fee6a7fc640 1 --2- 172.21.15.89:0/1337340669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee6c06bd10 0x7fee6c157bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:36.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.901+0000 7fee6a7fc640 1 -- 172.21.15.89:0/1337340669 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6000b6c0 con 0x7fee6c070600 2023-10-18T10:46:36.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.901+0000 7fee6a7fc640 1 --2- 172.21.15.89:0/1337340669 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee6c070600 0x7fee6c0c84a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fee600b2040 tx=0x7fee6001b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:36.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.901+0000 7fee53fff640 1 -- 172.21.15.89:0/1337340669 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee600afd70 con 0x7fee6c070600 2023-10-18T10:46:36.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.901+0000 7fee53fff640 1 -- 172.21.15.89:0/1337340669 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee600ba470 con 0x7fee6c070600 2023-10-18T10:46:36.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.901+0000 7fee7180b640 1 -- 172.21.15.89:0/1337340669 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee6c1f8c90 con 0x7fee6c070600 2023-10-18T10:46:36.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.901+0000 7fee53fff640 1 -- 172.21.15.89:0/1337340669 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee600ba820 con 0x7fee6c070600 2023-10-18T10:46:36.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.901+0000 7fee7180b640 1 -- 172.21.15.89:0/1337340669 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee6c158100 con 0x7fee6c070600 2023-10-18T10:46:36.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.901+0000 7fee7180b640 1 -- 172.21.15.89:0/1337340669 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee6c1f8e20 con 0x7fee6c070600 2023-10-18T10:46:36.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.901+0000 7fee53fff640 1 -- 172.21.15.89:0/1337340669 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fee600b6070 con 0x7fee6c070600 2023-10-18T10:46:36.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.901+0000 7fee53fff640 1 --2- 172.21.15.89:0/1337340669 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fee58065680 0x7fee58067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:36.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.901+0000 7fee53fff640 1 -- 172.21.15.89:0/1337340669 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fee6011e530 con 0x7fee6c070600 2023-10-18T10:46:36.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.901+0000 7fee6affd640 1 --2- 172.21.15.89:0/1337340669 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fee58065680 0x7fee58067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:36.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.905+0000 7fee6affd640 1 --2- 172.21.15.89:0/1337340669 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fee58065680 0x7fee58067b40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fee5c000f70 tx=0x7fee5c007c10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:36.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:36.905+0000 7fee53fff640 1 -- 172.21.15.89:0/1337340669 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fee600ebde0 con 0x7fee6c070600 2023-10-18T10:46:37.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.061+0000 7fee7180b640 1 -- 172.21.15.89:0/1337340669 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fee6c04dbe0 con 0x7fee6c070600 2023-10-18T10:46:37.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.061+0000 7fee53fff640 1 -- 172.21.15.89:0/1337340669 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7fee600ef400 con 0x7fee6c070600 2023-10-18T10:46:37.066 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-18T10:46:37.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.065+0000 7fee7180b640 1 -- 172.21.15.89:0/1337340669 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fee58065680 msgr2=0x7fee58067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:37.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.065+0000 7fee7180b640 1 --2- 172.21.15.89:0/1337340669 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fee58065680 0x7fee58067b40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fee5c000f70 tx=0x7fee5c007c10 comp rx=0 tx=0).stop 2023-10-18T10:46:37.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.065+0000 7fee7180b640 1 -- 172.21.15.89:0/1337340669 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee6c070600 msgr2=0x7fee6c0c84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:37.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.065+0000 7fee7180b640 1 --2- 172.21.15.89:0/1337340669 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee6c070600 0x7fee6c0c84a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fee600b2040 tx=0x7fee6001b810 comp rx=0 tx=0).stop 2023-10-18T10:46:37.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.065+0000 7fee7180b640 1 -- 172.21.15.89:0/1337340669 shutdown_connections 2023-10-18T10:46:37.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.065+0000 7fee7180b640 1 --2- 172.21.15.89:0/1337340669 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fee58065680 0x7fee58067b40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:37.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.065+0000 7fee7180b640 1 --2- 172.21.15.89:0/1337340669 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee6c159490 0x7fee6c15d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:37.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.065+0000 7fee7180b640 1 --2- 172.21.15.89:0/1337340669 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee6c070600 0x7fee6c0c84a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:37.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.065+0000 7fee7180b640 1 --2- 172.21.15.89:0/1337340669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee6c06bd10 0x7fee6c157bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:37.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.065+0000 7fee7180b640 1 -- 172.21.15.89:0/1337340669 >> 172.21.15.89:0/1337340669 conn(0x7fee6c069f00 msgr2=0x7fee6c0c1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:37.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.065+0000 7fee7180b640 1 -- 172.21.15.89:0/1337340669 shutdown_connections 2023-10-18T10:46:37.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.065+0000 7fee7180b640 1 -- 172.21.15.89:0/1337340669 wait complete. 2023-10-18T10:46:37.086 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:46:37.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.225+0000 7efc7866a640 1 Processor -- start 2023-10-18T10:46:37.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.225+0000 7efc7866a640 1 -- start start 2023-10-18T10:46:37.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.229+0000 7efc7866a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc700d0760 0x7efc700d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:37.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.229+0000 7efc7866a640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efc700cd160 0x7efc700cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:37.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.229+0000 7efc7866a640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc700d1100 con 0x7efc7015c940 2023-10-18T10:46:37.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.229+0000 7efc7866a640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc7015da50 con 0x7efc700cd160 2023-10-18T10:46:37.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.229+0000 7efc7866a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc700d3110 con 0x7efc700d0760 2023-10-18T10:46:37.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.229+0000 7efc76c07640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efc700cd160 0x7efc700cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:37.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.229+0000 7efc76c07640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efc700cd160 0x7efc700cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47116/0 (socket says 172.21.15.89:47116) 2023-10-18T10:46:37.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.229+0000 7efc76c07640 1 -- 172.21.15.89:0/1656800184 learned_addr learned my addr 172.21.15.89:0/1656800184 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:37.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.229+0000 7efc76406640 1 --1- 172.21.15.89:0/1656800184 >> v1:172.21.15.89:6789/0 conn(0x7efc7015c940 0x7efc700d2a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47940/0 (socket says 172.21.15.89:47940) 2023-10-18T10:46:37.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.229+0000 7efc75c05640 1 --2- 172.21.15.89:0/1656800184 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc700d0760 0x7efc700d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:37.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.229+0000 7efc76c07640 1 -- 172.21.15.89:0/1656800184 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc700d0760 msgr2=0x7efc700d0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:37.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.229+0000 7efc76c07640 1 --2- 172.21.15.89:0/1656800184 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc700d0760 0x7efc700d0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:37.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.229+0000 7efc76c07640 1 -- 172.21.15.89:0/1656800184 >> v1:172.21.15.89:6789/0 conn(0x7efc7015c940 legacy=0x7efc700d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:37.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.229+0000 7efc76c07640 1 -- 172.21.15.89:0/1656800184 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc7015ce40 con 0x7efc700cd160 2023-10-18T10:46:37.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.229+0000 7efc76c07640 1 --2- 172.21.15.89:0/1656800184 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc700cd160 0x7efc700cd5f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7efc68009e50 tx=0x7efc6801c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b9115b6963eb7a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:37.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.233+0000 7efc75404640 1 -- 172.21.15.89:0/1656800184 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc680a58a0 con 0x7efc700cd160 2023-10-18T10:46:37.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.233+0000 7efc75404640 1 -- 172.21.15.89:0/1656800184 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc680a5a00 con 0x7efc700cd160 2023-10-18T10:46:37.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.233+0000 7efc75404640 1 -- 172.21.15.89:0/1656800184 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc680a5d90 con 0x7efc700cd160 2023-10-18T10:46:37.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.233+0000 7efc7866a640 1 -- 172.21.15.89:0/1656800184 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc700cd160 msgr2=0x7efc700cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:37.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.233+0000 7efc7866a640 1 --2- 172.21.15.89:0/1656800184 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc700cd160 0x7efc700cd5f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7efc68009e50 tx=0x7efc6801c9e0 comp rx=0 tx=0).stop 2023-10-18T10:46:37.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.233+0000 7efc7866a640 1 -- 172.21.15.89:0/1656800184 shutdown_connections 2023-10-18T10:46:37.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.233+0000 7efc7866a640 1 --2- 172.21.15.89:0/1656800184 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc700cd160 0x7efc700cd5f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:37.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.233+0000 7efc7866a640 1 --2- 172.21.15.89:0/1656800184 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc700d0760 0x7efc700d0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:37.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.233+0000 7efc7866a640 1 -- 172.21.15.89:0/1656800184 >> 172.21.15.89:0/1656800184 conn(0x7efc70089fd0 msgr2=0x7efc7009b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:37.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.233+0000 7efc7866a640 1 -- 172.21.15.89:0/1656800184 shutdown_connections 2023-10-18T10:46:37.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.233+0000 7efc7866a640 1 -- 172.21.15.89:0/1656800184 wait complete. 2023-10-18T10:46:37.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.233+0000 7efc7866a640 1 Processor -- start 2023-10-18T10:46:37.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.233+0000 7efc7866a640 1 -- start start 2023-10-18T10:46:37.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.233+0000 7efc7866a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc700cd160 0x7efc7015aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:37.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.233+0000 7efc7866a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc700d0760 0x7efc700b6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:37.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.233+0000 7efc7866a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc700c6460 0x7efc700c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:37.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.233+0000 7efc7866a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7009c370 con 0x7efc700cd160 2023-10-18T10:46:37.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.233+0000 7efc7866a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7009c980 con 0x7efc700d0760 2023-10-18T10:46:37.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.233+0000 7efc7866a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc7015c1a0 con 0x7efc700c6460 2023-10-18T10:46:37.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.233+0000 7efc76406640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc700cd160 0x7efc7015aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:37.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.233+0000 7efc76406640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc700cd160 0x7efc7015aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47120/0 (socket says 172.21.15.89:47120) 2023-10-18T10:46:37.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.233+0000 7efc76406640 1 -- 172.21.15.89:0/570448601 learned_addr learned my addr 172.21.15.89:0/570448601 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:37.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.237+0000 7efc75c05640 1 --2- 172.21.15.89:0/570448601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc700d0760 0x7efc700b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:37.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.237+0000 7efc76c07640 1 --2- 172.21.15.89:0/570448601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc700c6460 0x7efc700c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:37.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.237+0000 7efc76406640 1 -- 172.21.15.89:0/570448601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc700c6460 msgr2=0x7efc700c68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:37.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.237+0000 7efc76406640 1 --2- 172.21.15.89:0/570448601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc700c6460 0x7efc700c68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:37.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.237+0000 7efc76406640 1 -- 172.21.15.89:0/570448601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc700d0760 msgr2=0x7efc700b6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:37.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.237+0000 7efc76406640 1 --2- 172.21.15.89:0/570448601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc700d0760 0x7efc700b6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:37.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.237+0000 7efc76406640 1 -- 172.21.15.89:0/570448601 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc68009b30 con 0x7efc700cd160 2023-10-18T10:46:37.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.237+0000 7efc75c05640 1 --2- 172.21.15.89:0/570448601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc700d0760 0x7efc700b6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:37.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.237+0000 7efc76c07640 1 --2- 172.21.15.89:0/570448601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc700c6460 0x7efc700c68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:37.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.237+0000 7efc76406640 1 --2- 172.21.15.89:0/570448601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc700cd160 0x7efc7015aa00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7efc6c00c430 tx=0x7efc6c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:37.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.237+0000 7efc66ffd640 1 -- 172.21.15.89:0/570448601 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6c0074e0 con 0x7efc700cd160 2023-10-18T10:46:37.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.237+0000 7efc66ffd640 1 -- 172.21.15.89:0/570448601 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc6c007640 con 0x7efc700cd160 2023-10-18T10:46:37.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.237+0000 7efc66ffd640 1 -- 172.21.15.89:0/570448601 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6c004a10 con 0x7efc700cd160 2023-10-18T10:46:37.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.237+0000 7efc7866a640 1 -- 172.21.15.89:0/570448601 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc7009c610 con 0x7efc700cd160 2023-10-18T10:46:37.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.237+0000 7efc7866a640 1 -- 172.21.15.89:0/570448601 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc7015e4b0 con 0x7efc700cd160 2023-10-18T10:46:37.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.237+0000 7efc7866a640 1 -- 172.21.15.89:0/570448601 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc7009c7d0 con 0x7efc700cd160 2023-10-18T10:46:37.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.237+0000 7efc66ffd640 1 -- 172.21.15.89:0/570448601 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7efc6c004b70 con 0x7efc700cd160 2023-10-18T10:46:37.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.237+0000 7efc66ffd640 1 --2- 172.21.15.89:0/570448601 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efc540656b0 0x7efc54067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:37.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.237+0000 7efc66ffd640 1 -- 172.21.15.89:0/570448601 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7efc6c082ba0 con 0x7efc700cd160 2023-10-18T10:46:37.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.237+0000 7efc75c05640 1 --2- 172.21.15.89:0/570448601 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efc540656b0 0x7efc54067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:37.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.241+0000 7efc75c05640 1 --2- 172.21.15.89:0/570448601 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efc540656b0 0x7efc54067b70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efc700b6eb0 tx=0x7efc60008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:37.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.241+0000 7efc66ffd640 1 -- 172.21.15.89:0/570448601 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7efc6c050290 con 0x7efc700cd160 2023-10-18T10:46:37.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.389+0000 7efc7866a640 1 -- 172.21.15.89:0/570448601 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efc7004dbe0 con 0x7efc700cd160 2023-10-18T10:46:37.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.393+0000 7efc66ffd640 1 -- 172.21.15.89:0/570448601 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7efc6c0538b0 con 0x7efc700cd160 2023-10-18T10:46:37.395 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-18T10:46:37.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.393+0000 7efc7866a640 1 -- 172.21.15.89:0/570448601 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efc540656b0 msgr2=0x7efc54067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:37.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.393+0000 7efc7866a640 1 --2- 172.21.15.89:0/570448601 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efc540656b0 0x7efc54067b70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efc700b6eb0 tx=0x7efc60008040 comp rx=0 tx=0).stop 2023-10-18T10:46:37.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.393+0000 7efc7866a640 1 -- 172.21.15.89:0/570448601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc700cd160 msgr2=0x7efc7015aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:37.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.393+0000 7efc7866a640 1 --2- 172.21.15.89:0/570448601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc700cd160 0x7efc7015aa00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7efc6c00c430 tx=0x7efc6c00c8f0 comp rx=0 tx=0).stop 2023-10-18T10:46:37.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.393+0000 7efc7866a640 1 -- 172.21.15.89:0/570448601 shutdown_connections 2023-10-18T10:46:37.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.393+0000 7efc7866a640 1 --2- 172.21.15.89:0/570448601 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efc540656b0 0x7efc54067b70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:37.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.393+0000 7efc7866a640 1 --2- 172.21.15.89:0/570448601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc700c6460 0x7efc700c68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:37.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.393+0000 7efc7866a640 1 --2- 172.21.15.89:0/570448601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc700d0760 0x7efc700b6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:37.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.393+0000 7efc7866a640 1 --2- 172.21.15.89:0/570448601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc700cd160 0x7efc7015aa00 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:37.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.393+0000 7efc7866a640 1 -- 172.21.15.89:0/570448601 >> 172.21.15.89:0/570448601 conn(0x7efc70089fd0 msgr2=0x7efc7015e820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:37.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.393+0000 7efc7866a640 1 -- 172.21.15.89:0/570448601 shutdown_connections 2023-10-18T10:46:37.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.393+0000 7efc7866a640 1 -- 172.21.15.89:0/570448601 wait complete. 2023-10-18T10:46:37.415 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T10:46:37.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.553+0000 7fede14cb640 1 Processor -- start 2023-10-18T10:46:37.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.553+0000 7fede14cb640 1 -- start start 2023-10-18T10:46:37.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.553+0000 7fede14cb640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7feddc0b1860 0x7feddc0cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:37.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.553+0000 7fede14cb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feddc0b1cc0 0x7feddc0cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:37.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.553+0000 7fede14cb640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feddc0d1e50 con 0x7feddc15c550 2023-10-18T10:46:37.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.553+0000 7fede14cb640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feddc0d2130 con 0x7feddc0b1860 2023-10-18T10:46:37.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.553+0000 7fede14cb640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feddc0d2410 con 0x7feddc0b1cc0 2023-10-18T10:46:37.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.553+0000 7feddaffd640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7feddc15c550 0x7feddc0d2ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47942/0 (socket says 172.21.15.89:47942) 2023-10-18T10:46:37.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.553+0000 7feddaffd640 1 -- 172.21.15.89:0/3172771696 learned_addr learned my addr 172.21.15.89:0/3172771696 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:37.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.553+0000 7fedda7fc640 1 --2- 172.21.15.89:0/3172771696 >> v2:172.21.15.154:3300/0 conn(0x7feddc0b1860 0x7feddc0cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:37.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.553+0000 7feddb7fe640 1 --2- 172.21.15.89:0/3172771696 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feddc0b1cc0 0x7feddc0cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:37.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.553+0000 7fedd9ffb640 1 -- 172.21.15.89:0/3172771696 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2891055298 0 0) 0x7feddc0d1e50 con 0x7feddc15c550 2023-10-18T10:46:37.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.553+0000 7fedd9ffb640 1 -- 172.21.15.89:0/3172771696 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedc4003530 con 0x7feddc15c550 2023-10-18T10:46:37.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.553+0000 7feddb7fe640 1 -- 172.21.15.89:0/3172771696 >> v2:172.21.15.154:3300/0 conn(0x7feddc0b1860 msgr2=0x7feddc0cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:37.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.553+0000 7feddb7fe640 1 --2- 172.21.15.89:0/3172771696 >> v2:172.21.15.154:3300/0 conn(0x7feddc0b1860 0x7feddc0cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:37.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.553+0000 7feddb7fe640 1 -- 172.21.15.89:0/3172771696 >> v1:172.21.15.89:6789/0 conn(0x7feddc15c550 legacy=0x7feddc0d2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:37.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.553+0000 7fedd9ffb640 1 -- 172.21.15.89:0/3172771696 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 280169319 0 0) 0x7fedc4003530 con 0x7feddc15c550 2023-10-18T10:46:37.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.553+0000 7feddb7fe640 1 -- 172.21.15.89:0/3172771696 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feddc0d26f0 con 0x7feddc0b1cc0 2023-10-18T10:46:37.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.553+0000 7fedda7fc640 1 --2- 172.21.15.89:0/3172771696 >> v2:172.21.15.154:3300/0 conn(0x7feddc0b1860 0x7feddc0cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:46:37.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.557+0000 7feddb7fe640 1 --2- 172.21.15.89:0/3172771696 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feddc0b1cc0 0x7feddc0cfb50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fedd40097c0 tx=0x7fedd401c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af4cf4908d676446 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:37.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.557+0000 7fedd9ffb640 1 -- 172.21.15.89:0/3172771696 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedcc001a00 con 0x7feddc0b1cc0 2023-10-18T10:46:37.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.557+0000 7fedd9ffb640 1 -- 172.21.15.89:0/3172771696 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd40a7050 con 0x7feddc0b1cc0 2023-10-18T10:46:37.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.557+0000 7fedd9ffb640 1 -- 172.21.15.89:0/3172771696 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd40ab600 con 0x7feddc0b1cc0 2023-10-18T10:46:37.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.557+0000 7fede14cb640 1 -- 172.21.15.89:0/3172771696 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feddc0b1cc0 msgr2=0x7feddc0cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:37.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.557+0000 7fede14cb640 1 --2- 172.21.15.89:0/3172771696 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feddc0b1cc0 0x7feddc0cfb50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fedd40097c0 tx=0x7fedd401c370 comp rx=0 tx=0).stop 2023-10-18T10:46:37.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.557+0000 7fede14cb640 1 -- 172.21.15.89:0/3172771696 shutdown_connections 2023-10-18T10:46:37.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.557+0000 7fede14cb640 1 --2- 172.21.15.89:0/3172771696 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feddc0b1cc0 0x7feddc0cfb50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:37.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.557+0000 7fede14cb640 1 --2- 172.21.15.89:0/3172771696 >> v2:172.21.15.154:3300/0 conn(0x7feddc0b1860 0x7feddc0cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:37.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.557+0000 7fede14cb640 1 -- 172.21.15.89:0/3172771696 >> 172.21.15.89:0/3172771696 conn(0x7feddc069d00 msgr2=0x7feddc06a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:37.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.557+0000 7fede14cb640 1 -- 172.21.15.89:0/3172771696 shutdown_connections 2023-10-18T10:46:37.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.557+0000 7fede14cb640 1 -- 172.21.15.89:0/3172771696 wait complete. 2023-10-18T10:46:37.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.561+0000 7fede14cb640 1 Processor -- start 2023-10-18T10:46:37.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.561+0000 7fede14cb640 1 -- start start 2023-10-18T10:46:37.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.561+0000 7fede14cb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feddc0b1860 0x7feddc158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:37.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.561+0000 7fede14cb640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feddc0b1cc0 0x7feddc158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:37.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.561+0000 7fede14cb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feddc15c550 0x7feddc0b6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:37.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.561+0000 7fede14cb640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feddc1603e0 con 0x7feddc0b1cc0 2023-10-18T10:46:37.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.561+0000 7fede14cb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feddc162180 con 0x7feddc0b1860 2023-10-18T10:46:37.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.561+0000 7fede14cb640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feddc162460 con 0x7feddc15c550 2023-10-18T10:46:37.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.561+0000 7fedda7fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feddc0b1cc0 0x7feddc158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:37.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.561+0000 7fedda7fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feddc0b1cc0 0x7feddc158a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47132/0 (socket says 172.21.15.89:47132) 2023-10-18T10:46:37.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.561+0000 7fedda7fc640 1 -- 172.21.15.89:0/2209519100 learned_addr learned my addr 172.21.15.89:0/2209519100 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:37.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.561+0000 7feddaffd640 1 --2- 172.21.15.89:0/2209519100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feddc0b1860 0x7feddc158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:37.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.561+0000 7feddb7fe640 1 --2- 172.21.15.89:0/2209519100 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feddc15c550 0x7feddc0b6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:37.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.561+0000 7fedda7fc640 1 -- 172.21.15.89:0/2209519100 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feddc15c550 msgr2=0x7feddc0b6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:37.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.561+0000 7fedda7fc640 1 --2- 172.21.15.89:0/2209519100 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feddc15c550 0x7feddc0b6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:37.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.561+0000 7fedda7fc640 1 -- 172.21.15.89:0/2209519100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feddc0b1860 msgr2=0x7feddc158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:37.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.561+0000 7fedda7fc640 1 --2- 172.21.15.89:0/2209519100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feddc0b1860 0x7feddc158240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:37.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.561+0000 7fedda7fc640 1 -- 172.21.15.89:0/2209519100 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd0009420 con 0x7feddc0b1cc0 2023-10-18T10:46:37.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.561+0000 7fedda7fc640 1 --2- 172.21.15.89:0/2209519100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feddc0b1cc0 0x7feddc158a70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fedd000ebe0 tx=0x7fedd000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:37.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.565+0000 7fedc37fe640 1 -- 172.21.15.89:0/2209519100 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd000eee0 con 0x7feddc0b1cc0 2023-10-18T10:46:37.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.565+0000 7fedc37fe640 1 -- 172.21.15.89:0/2209519100 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd000cc30 con 0x7feddc0b1cc0 2023-10-18T10:46:37.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.565+0000 7fedc37fe640 1 -- 172.21.15.89:0/2209519100 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd0013470 con 0x7feddc0b1cc0 2023-10-18T10:46:37.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.565+0000 7fede14cb640 1 -- 172.21.15.89:0/2209519100 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedd4009420 con 0x7feddc0b1cc0 2023-10-18T10:46:37.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.565+0000 7fede14cb640 1 -- 172.21.15.89:0/2209519100 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feddc158780 con 0x7feddc0b1cc0 2023-10-18T10:46:37.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.565+0000 7fede14cb640 1 -- 172.21.15.89:0/2209519100 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feddc0b8000 con 0x7feddc0b1cc0 2023-10-18T10:46:37.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.565+0000 7fedc37fe640 1 -- 172.21.15.89:0/2209519100 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fedd00135d0 con 0x7feddc0b1cc0 2023-10-18T10:46:37.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.565+0000 7fedc37fe640 1 --2- 172.21.15.89:0/2209519100 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fedc80656b0 0x7fedc8067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:37.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.565+0000 7fedc37fe640 1 -- 172.21.15.89:0/2209519100 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fedd0083340 con 0x7feddc0b1cc0 2023-10-18T10:46:37.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.565+0000 7feddaffd640 1 --2- 172.21.15.89:0/2209519100 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fedc80656b0 0x7fedc8067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:37.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.565+0000 7feddaffd640 1 --2- 172.21.15.89:0/2209519100 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fedc80656b0 0x7fedc8067b70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00e270 tx=0x7fedcc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:37.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.569+0000 7fedc37fe640 1 -- 172.21.15.89:0/2209519100 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fedd0050b60 con 0x7feddc0b1cc0 2023-10-18T10:46:37.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.717+0000 7fede14cb640 1 -- 172.21.15.89:0/2209519100 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7feddc0b77c0 con 0x7feddc0b1cc0 2023-10-18T10:46:37.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.717+0000 7fedc37fe640 1 -- 172.21.15.89:0/2209519100 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7fedd0054180 con 0x7feddc0b1cc0 2023-10-18T10:46:37.722 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-18T10:46:37.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.721+0000 7fede14cb640 1 -- 172.21.15.89:0/2209519100 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fedc80656b0 msgr2=0x7fedc8067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:37.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.721+0000 7fede14cb640 1 --2- 172.21.15.89:0/2209519100 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fedc80656b0 0x7fedc8067b70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00e270 tx=0x7fedcc009040 comp rx=0 tx=0).stop 2023-10-18T10:46:37.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.721+0000 7fede14cb640 1 -- 172.21.15.89:0/2209519100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feddc0b1cc0 msgr2=0x7feddc158a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:37.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.721+0000 7fede14cb640 1 --2- 172.21.15.89:0/2209519100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feddc0b1cc0 0x7feddc158a70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fedd000ebe0 tx=0x7fedd000c520 comp rx=0 tx=0).stop 2023-10-18T10:46:37.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.721+0000 7fede14cb640 1 -- 172.21.15.89:0/2209519100 shutdown_connections 2023-10-18T10:46:37.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.721+0000 7fede14cb640 1 --2- 172.21.15.89:0/2209519100 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fedc80656b0 0x7fedc8067b70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:37.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.721+0000 7fede14cb640 1 --2- 172.21.15.89:0/2209519100 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7feddc15c550 0x7feddc0b6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:37.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.721+0000 7fede14cb640 1 --2- 172.21.15.89:0/2209519100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feddc0b1cc0 0x7feddc158a70 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:37.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.721+0000 7fede14cb640 1 --2- 172.21.15.89:0/2209519100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feddc0b1860 0x7feddc158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:37.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.721+0000 7fede14cb640 1 -- 172.21.15.89:0/2209519100 >> 172.21.15.89:0/2209519100 conn(0x7feddc069d00 msgr2=0x7feddc06a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:37.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.721+0000 7fede14cb640 1 -- 172.21.15.89:0/2209519100 shutdown_connections 2023-10-18T10:46:37.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.721+0000 7fede14cb640 1 -- 172.21.15.89:0/2209519100 wait complete. 2023-10-18T10:46:37.743 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:46:37.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.877+0000 7f0c183fa640 1 Processor -- start 2023-10-18T10:46:37.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.877+0000 7f0c183fa640 1 -- start start 2023-10-18T10:46:37.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.877+0000 7f0c183fa640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0c100d0760 0x7f0c100d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:37.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.877+0000 7f0c183fa640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c100d1100 con 0x7f0c1015c940 2023-10-18T10:46:37.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.877+0000 7f0c183fa640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c1015da50 con 0x7f0c100cd160 2023-10-18T10:46:37.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.877+0000 7f0c183fa640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c100d3110 con 0x7f0c100d0760 2023-10-18T10:46:37.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.877+0000 7f0c16196640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0c1015c940 0x7f0c100d2a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47948/0 (socket says 172.21.15.89:47948) 2023-10-18T10:46:37.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.877+0000 7f0c16196640 1 -- 172.21.15.89:0/968683675 learned_addr learned my addr 172.21.15.89:0/968683675 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:37.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.877+0000 7f0c15995640 1 --2- 172.21.15.89:0/968683675 >> v2:172.21.15.154:3300/0 conn(0x7f0c100d0760 0x7f0c100d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:37.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.877+0000 7f0c15194640 1 -- 172.21.15.89:0/968683675 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1216049122 0 0) 0x7f0c100d1100 con 0x7f0c1015c940 2023-10-18T10:46:37.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.877+0000 7f0c15194640 1 -- 172.21.15.89:0/968683675 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bfc003530 con 0x7f0c1015c940 2023-10-18T10:46:37.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.877+0000 7f0c15995640 1 -- 172.21.15.89:0/968683675 >> v1:172.21.15.154:6789/0 conn(0x7f0c100cd160 legacy=0x7f0c100cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:37.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.877+0000 7f0c15995640 1 -- 172.21.15.89:0/968683675 >> v1:172.21.15.89:6789/0 conn(0x7f0c1015c940 legacy=0x7f0c100d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:37.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.877+0000 7f0c15995640 1 -- 172.21.15.89:0/968683675 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c1015ce40 con 0x7f0c100d0760 2023-10-18T10:46:37.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.881+0000 7f0c15995640 1 --2- 172.21.15.89:0/968683675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c100d0760 0x7f0c100d0bc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0c000097c0 tx=0x7f0c0001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0c3b3341d481b20 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:37.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.881+0000 7f0c15194640 1 -- 172.21.15.89:0/968683675 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c000ab610 con 0x7f0c100d0760 2023-10-18T10:46:37.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.881+0000 7f0c15194640 1 -- 172.21.15.89:0/968683675 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c000ab770 con 0x7f0c100d0760 2023-10-18T10:46:37.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.881+0000 7f0c15194640 1 -- 172.21.15.89:0/968683675 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c000abb00 con 0x7f0c100d0760 2023-10-18T10:46:37.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.881+0000 7f0c183fa640 1 -- 172.21.15.89:0/968683675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c100d0760 msgr2=0x7f0c100d0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:37.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.881+0000 7f0c183fa640 1 --2- 172.21.15.89:0/968683675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c100d0760 0x7f0c100d0bc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0c000097c0 tx=0x7f0c0001c410 comp rx=0 tx=0).stop 2023-10-18T10:46:37.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.881+0000 7f0c183fa640 1 -- 172.21.15.89:0/968683675 shutdown_connections 2023-10-18T10:46:37.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.881+0000 7f0c183fa640 1 --2- 172.21.15.89:0/968683675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c100d0760 0x7f0c100d0bc0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:37.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.881+0000 7f0c183fa640 1 -- 172.21.15.89:0/968683675 >> 172.21.15.89:0/968683675 conn(0x7f0c10089fd0 msgr2=0x7f0c1009b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:37.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.881+0000 7f0c183fa640 1 -- 172.21.15.89:0/968683675 shutdown_connections 2023-10-18T10:46:37.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.881+0000 7f0c183fa640 1 -- 172.21.15.89:0/968683675 wait complete. 2023-10-18T10:46:37.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.885+0000 7f0c183fa640 1 Processor -- start 2023-10-18T10:46:37.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.885+0000 7f0c183fa640 1 -- start start 2023-10-18T10:46:37.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.885+0000 7f0c183fa640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c100cd160 0x7f0c100b7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:37.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.885+0000 7f0c183fa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c100d0760 0x7f0c100b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:37.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.885+0000 7f0c183fa640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c101531b0 0x7f0c10153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:37.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.885+0000 7f0c183fa640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c100ca810 con 0x7f0c101531b0 2023-10-18T10:46:37.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.885+0000 7f0c183fa640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c100cbea0 con 0x7f0c100cd160 2023-10-18T10:46:37.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.885+0000 7f0c183fa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c100ca530 con 0x7f0c100d0760 2023-10-18T10:46:37.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.885+0000 7f0c16997640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c101531b0 0x7f0c10153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:37.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.885+0000 7f0c16997640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c101531b0 0x7f0c10153640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47148/0 (socket says 172.21.15.89:47148) 2023-10-18T10:46:37.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.885+0000 7f0c16997640 1 -- 172.21.15.89:0/1712383347 learned_addr learned my addr 172.21.15.89:0/1712383347 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:37.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.885+0000 7f0c15995640 1 --2- 172.21.15.89:0/1712383347 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c100d0760 0x7f0c100b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:37.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.885+0000 7f0c16196640 1 --2- 172.21.15.89:0/1712383347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c100cd160 0x7f0c100b7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:37.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.885+0000 7f0c16997640 1 -- 172.21.15.89:0/1712383347 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c100d0760 msgr2=0x7f0c100b7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:37.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.885+0000 7f0c16997640 1 --2- 172.21.15.89:0/1712383347 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c100d0760 0x7f0c100b7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:37.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.885+0000 7f0c16997640 1 -- 172.21.15.89:0/1712383347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c100cd160 msgr2=0x7f0c100b7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:37.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.885+0000 7f0c16997640 1 --2- 172.21.15.89:0/1712383347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c100cd160 0x7f0c100b7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:37.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.885+0000 7f0c16997640 1 -- 172.21.15.89:0/1712383347 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c00009420 con 0x7f0c101531b0 2023-10-18T10:46:37.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.885+0000 7f0c16997640 1 --2- 172.21.15.89:0/1712383347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c101531b0 0x7f0c10153640 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0c0800c780 tx=0x7f0c0800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:37.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.885+0000 7f0c06ffd640 1 -- 172.21.15.89:0/1712383347 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c08004b70 con 0x7f0c101531b0 2023-10-18T10:46:37.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.885+0000 7f0c06ffd640 1 -- 172.21.15.89:0/1712383347 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c0800ce90 con 0x7f0c101531b0 2023-10-18T10:46:37.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.885+0000 7f0c06ffd640 1 -- 172.21.15.89:0/1712383347 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0800a850 con 0x7f0c101531b0 2023-10-18T10:46:37.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.885+0000 7f0c183fa640 1 -- 172.21.15.89:0/1712383347 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c100b6e70 con 0x7f0c101531b0 2023-10-18T10:46:37.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.885+0000 7f0c183fa640 1 -- 172.21.15.89:0/1712383347 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c100b7430 con 0x7f0c101531b0 2023-10-18T10:46:37.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.889+0000 7f0c183fa640 1 -- 172.21.15.89:0/1712383347 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c100b6cd0 con 0x7f0c101531b0 2023-10-18T10:46:37.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.889+0000 7f0c06ffd640 1 -- 172.21.15.89:0/1712383347 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f0c0800a9b0 con 0x7f0c101531b0 2023-10-18T10:46:37.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.889+0000 7f0c06ffd640 1 --2- 172.21.15.89:0/1712383347 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0bf4065680 0x7f0bf4067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:37.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.889+0000 7f0c06ffd640 1 -- 172.21.15.89:0/1712383347 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0c08083410 con 0x7f0c101531b0 2023-10-18T10:46:37.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.889+0000 7f0c16196640 1 --2- 172.21.15.89:0/1712383347 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0bf4065680 0x7f0bf4067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:37.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.893+0000 7f0c16196640 1 --2- 172.21.15.89:0/1712383347 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0bf4065680 0x7f0bf4067b40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0011f0 tx=0x7f0c0c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:37.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:37.893+0000 7f0c06ffd640 1 -- 172.21.15.89:0/1712383347 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f0c100b6cd0 con 0x7f0c101531b0 2023-10-18T10:46:38.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.049+0000 7f0c183fa640 1 -- 172.21.15.89:0/1712383347 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c100b6cd0 con 0x7f0c101531b0 2023-10-18T10:46:38.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.053+0000 7f0c06ffd640 1 -- 172.21.15.89:0/1712383347 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20192 (secure 0 0 0) 0x7f0c100b6cd0 con 0x7f0c101531b0 2023-10-18T10:46:38.056 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:38.056 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":29,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:46:36.958604+0000","last_up_change":"2023-10-18T10:45:42.164688+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:45:53.309122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-18T10:45:54.659763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:45:57.796514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6824","nonce":773078364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6827","nonce":773078364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6831","nonce":773078364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6829","nonce":773078364}]},"public_addr":"172.21.15.154:6824/773078364","cluster_addr":"172.21.15.154:6827/773078364","heartbeat_back_addr":"172.21.15.154:6831/773078364","heartbeat_front_addr":"172.21.15.154:6829/773078364","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":1,"to":5}]},{"pgid":"4.1a","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:46:38.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.053+0000 7f0c183fa640 1 -- 172.21.15.89:0/1712383347 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0bf4065680 msgr2=0x7f0bf4067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:38.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.053+0000 7f0c183fa640 1 --2- 172.21.15.89:0/1712383347 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0bf4065680 0x7f0bf4067b40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0011f0 tx=0x7f0c0c009040 comp rx=0 tx=0).stop 2023-10-18T10:46:38.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.053+0000 7f0c183fa640 1 -- 172.21.15.89:0/1712383347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c101531b0 msgr2=0x7f0c10153640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:38.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.053+0000 7f0c183fa640 1 --2- 172.21.15.89:0/1712383347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c101531b0 0x7f0c10153640 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0c0800c780 tx=0x7f0c0800cc40 comp rx=0 tx=0).stop 2023-10-18T10:46:38.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.053+0000 7f0c183fa640 1 -- 172.21.15.89:0/1712383347 shutdown_connections 2023-10-18T10:46:38.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.053+0000 7f0c183fa640 1 --2- 172.21.15.89:0/1712383347 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0bf4065680 0x7f0bf4067b40 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:38.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.053+0000 7f0c183fa640 1 --2- 172.21.15.89:0/1712383347 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c100d0760 0x7f0c100b7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:38.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.053+0000 7f0c183fa640 1 --2- 172.21.15.89:0/1712383347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c101531b0 0x7f0c10153640 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:38.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.053+0000 7f0c183fa640 1 --2- 172.21.15.89:0/1712383347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c100cd160 0x7f0c100b7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:38.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.053+0000 7f0c183fa640 1 -- 172.21.15.89:0/1712383347 >> 172.21.15.89:0/1712383347 conn(0x7f0c10089fd0 msgr2=0x7f0c1015fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:38.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.053+0000 7f0c183fa640 1 -- 172.21.15.89:0/1712383347 shutdown_connections 2023-10-18T10:46:38.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.053+0000 7f0c183fa640 1 -- 172.21.15.89:0/1712383347 wait complete. 2023-10-18T10:46:38.076 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:45:43.114184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-18T10:45:48.730968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:45:53.309122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:45:54.659763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T10:45:57.796514+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T10:46:38.077 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:46:38.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.213+0000 7fca0b46a640 1 Processor -- start 2023-10-18T10:46:38.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.213+0000 7fca0b46a640 1 -- start start 2023-10-18T10:46:38.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.217+0000 7fca0b46a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fca040cd4f0 0x7fca04143470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:38.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.217+0000 7fca0b46a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca04145b70 0x7fca04160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:38.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.217+0000 7fca0b46a640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fca040ce070 0x7fca040ce500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:38.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.217+0000 7fca0b46a640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca040cea40 con 0x7fca040ce070 2023-10-18T10:46:38.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.217+0000 7fca0b46a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca04069900 con 0x7fca040cd4f0 2023-10-18T10:46:38.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.217+0000 7fca0b46a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca040cd8d0 con 0x7fca04145b70 2023-10-18T10:46:38.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.217+0000 7fca09a07640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fca040ce070 0x7fca040ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:38.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.217+0000 7fca09a07640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fca040ce070 0x7fca040ce500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47156/0 (socket says 172.21.15.89:47156) 2023-10-18T10:46:38.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.217+0000 7fca09a07640 1 -- 172.21.15.89:0/1873976088 learned_addr learned my addr 172.21.15.89:0/1873976088 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:38.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.217+0000 7fca08a05640 1 --2- 172.21.15.89:0/1873976088 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca04145b70 0x7fca04160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:38.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.217+0000 7fca09206640 1 --2- 172.21.15.89:0/1873976088 >> v2:172.21.15.154:3300/0 conn(0x7fca040cd4f0 0x7fca04143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:38.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.217+0000 7fca09a07640 1 -- 172.21.15.89:0/1873976088 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca04145b70 msgr2=0x7fca04160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:38.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.217+0000 7fca09a07640 1 --2- 172.21.15.89:0/1873976088 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca04145b70 0x7fca04160cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:38.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.217+0000 7fca09a07640 1 -- 172.21.15.89:0/1873976088 >> v2:172.21.15.154:3300/0 conn(0x7fca040cd4f0 msgr2=0x7fca04143470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:38.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.217+0000 7fca09a07640 1 --2- 172.21.15.89:0/1873976088 >> v2:172.21.15.154:3300/0 conn(0x7fca040cd4f0 0x7fca04143470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:38.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.217+0000 7fca09a07640 1 -- 172.21.15.89:0/1873976088 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca0415cbf0 con 0x7fca040ce070 2023-10-18T10:46:38.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.217+0000 7fca09a07640 1 --2- 172.21.15.89:0/1873976088 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca040ce070 0x7fca040ce500 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fca040716d0 tx=0x7fc9fc01c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eabc97da5c5d4aa server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:38.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.217+0000 7fc9fbfff640 1 -- 172.21.15.89:0/1873976088 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc0a5740 con 0x7fca040ce070 2023-10-18T10:46:38.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.221+0000 7fc9fbfff640 1 -- 172.21.15.89:0/1873976088 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9fc0a58a0 con 0x7fca040ce070 2023-10-18T10:46:38.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.221+0000 7fc9fbfff640 1 -- 172.21.15.89:0/1873976088 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc0a5c30 con 0x7fca040ce070 2023-10-18T10:46:38.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.221+0000 7fca0b46a640 1 -- 172.21.15.89:0/1873976088 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca040ce070 msgr2=0x7fca040ce500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:38.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.221+0000 7fca0b46a640 1 --2- 172.21.15.89:0/1873976088 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca040ce070 0x7fca040ce500 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fca040716d0 tx=0x7fc9fc01c9d0 comp rx=0 tx=0).stop 2023-10-18T10:46:38.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.221+0000 7fca0b46a640 1 -- 172.21.15.89:0/1873976088 shutdown_connections 2023-10-18T10:46:38.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.221+0000 7fca0b46a640 1 --2- 172.21.15.89:0/1873976088 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca040ce070 0x7fca040ce500 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:38.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.221+0000 7fca0b46a640 1 --2- 172.21.15.89:0/1873976088 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca04145b70 0x7fca04160cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:38.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.221+0000 7fca0b46a640 1 --2- 172.21.15.89:0/1873976088 >> v2:172.21.15.154:3300/0 conn(0x7fca040cd4f0 0x7fca04143470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:38.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.221+0000 7fca0b46a640 1 -- 172.21.15.89:0/1873976088 >> 172.21.15.89:0/1873976088 conn(0x7fca0406dad0 msgr2=0x7fca0406dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:38.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.221+0000 7fca0b46a640 1 -- 172.21.15.89:0/1873976088 shutdown_connections 2023-10-18T10:46:38.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.221+0000 7fca0b46a640 1 -- 172.21.15.89:0/1873976088 wait complete. 2023-10-18T10:46:38.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.221+0000 7fca0b46a640 1 Processor -- start 2023-10-18T10:46:38.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.221+0000 7fca0b46a640 1 -- start start 2023-10-18T10:46:38.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.221+0000 7fca0b46a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca040cd4f0 0x7fca0415f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:38.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.221+0000 7fca0b46a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca040ce070 0x7fca040cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:38.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.221+0000 7fca0b46a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca04145b70 0x7fca040c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:38.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.221+0000 7fca0b46a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0415f0c0 con 0x7fca040cd4f0 2023-10-18T10:46:38.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.221+0000 7fca0b46a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca040b26d0 con 0x7fca040ce070 2023-10-18T10:46:38.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.221+0000 7fca0b46a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca0415fa90 con 0x7fca04145b70 2023-10-18T10:46:38.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.221+0000 7fca09206640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca040cd4f0 0x7fca0415f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:38.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.221+0000 7fca09206640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca040cd4f0 0x7fca0415f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47160/0 (socket says 172.21.15.89:47160) 2023-10-18T10:46:38.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.221+0000 7fca09206640 1 -- 172.21.15.89:0/3108202119 learned_addr learned my addr 172.21.15.89:0/3108202119 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:38.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.221+0000 7fca08a05640 1 --2- 172.21.15.89:0/3108202119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca040ce070 0x7fca040cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:38.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.221+0000 7fca09a07640 1 --2- 172.21.15.89:0/3108202119 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca04145b70 0x7fca040c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:38.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.225+0000 7fca09206640 1 -- 172.21.15.89:0/3108202119 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca04145b70 msgr2=0x7fca040c3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:38.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.225+0000 7fca09206640 1 --2- 172.21.15.89:0/3108202119 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca04145b70 0x7fca040c3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:38.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.225+0000 7fca09206640 1 -- 172.21.15.89:0/3108202119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca040ce070 msgr2=0x7fca040cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:38.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.225+0000 7fca09206640 1 --2- 172.21.15.89:0/3108202119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca040ce070 0x7fca040cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:38.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.225+0000 7fca09206640 1 -- 172.21.15.89:0/3108202119 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9fc009b30 con 0x7fca040cd4f0 2023-10-18T10:46:38.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.225+0000 7fca09a07640 1 --2- 172.21.15.89:0/3108202119 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca04145b70 0x7fca040c3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:38.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.225+0000 7fca09206640 1 --2- 172.21.15.89:0/3108202119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca040cd4f0 0x7fca0415f550 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fca0000d620 tx=0x7fca0000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:38.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.225+0000 7fc9f9ffb640 1 -- 172.21.15.89:0/3108202119 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0000b620 con 0x7fca040cd4f0 2023-10-18T10:46:38.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.225+0000 7fc9f9ffb640 1 -- 172.21.15.89:0/3108202119 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0000b780 con 0x7fca040cd4f0 2023-10-18T10:46:38.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.225+0000 7fc9f9ffb640 1 -- 172.21.15.89:0/3108202119 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0000bb10 con 0x7fca040cd4f0 2023-10-18T10:46:38.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.225+0000 7fca0b46a640 1 -- 172.21.15.89:0/3108202119 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca040b2940 con 0x7fca040cd4f0 2023-10-18T10:46:38.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.225+0000 7fca0b46a640 1 -- 172.21.15.89:0/3108202119 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca040cc990 con 0x7fca040cd4f0 2023-10-18T10:46:38.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.225+0000 7fca0b46a640 1 -- 172.21.15.89:0/3108202119 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca0404dbe0 con 0x7fca040cd4f0 2023-10-18T10:46:38.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.225+0000 7fc9f9ffb640 1 -- 172.21.15.89:0/3108202119 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fca0000bc70 con 0x7fca040cd4f0 2023-10-18T10:46:38.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.225+0000 7fc9f9ffb640 1 --2- 172.21.15.89:0/3108202119 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc9e80656b0 0x7fc9e8067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:38.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.225+0000 7fc9f9ffb640 1 -- 172.21.15.89:0/3108202119 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fca000836c0 con 0x7fca040cd4f0 2023-10-18T10:46:38.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.229+0000 7fca08a05640 1 --2- 172.21.15.89:0/3108202119 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc9e80656b0 0x7fc9e8067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:38.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.229+0000 7fca08a05640 1 --2- 172.21.15.89:0/3108202119 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc9e80656b0 0x7fc9e8067b70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fca040c6f90 tx=0x7fc9f4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:38.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.229+0000 7fc9f9ffb640 1 -- 172.21.15.89:0/3108202119 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fca00050db0 con 0x7fca040cd4f0 2023-10-18T10:46:38.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.381+0000 7fca0b46a640 1 -- 172.21.15.89:0/3108202119 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fca040b6420 con 0x7fca040cd4f0 2023-10-18T10:46:38.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.381+0000 7fc9f9ffb640 1 -- 172.21.15.89:0/3108202119 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7fca000543d0 con 0x7fca040cd4f0 2023-10-18T10:46:38.385 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-18T10:46:38.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.385+0000 7fca0b46a640 1 -- 172.21.15.89:0/3108202119 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc9e80656b0 msgr2=0x7fc9e8067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:38.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.385+0000 7fca0b46a640 1 --2- 172.21.15.89:0/3108202119 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc9e80656b0 0x7fc9e8067b70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fca040c6f90 tx=0x7fc9f4008040 comp rx=0 tx=0).stop 2023-10-18T10:46:38.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.385+0000 7fca0b46a640 1 -- 172.21.15.89:0/3108202119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca040cd4f0 msgr2=0x7fca0415f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:38.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.385+0000 7fca0b46a640 1 --2- 172.21.15.89:0/3108202119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca040cd4f0 0x7fca0415f550 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fca0000d620 tx=0x7fca0000dae0 comp rx=0 tx=0).stop 2023-10-18T10:46:38.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.385+0000 7fca0b46a640 1 -- 172.21.15.89:0/3108202119 shutdown_connections 2023-10-18T10:46:38.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.385+0000 7fca0b46a640 1 --2- 172.21.15.89:0/3108202119 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc9e80656b0 0x7fc9e8067b70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:38.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.385+0000 7fca0b46a640 1 --2- 172.21.15.89:0/3108202119 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca04145b70 0x7fca040c3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:38.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.385+0000 7fca0b46a640 1 --2- 172.21.15.89:0/3108202119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca040ce070 0x7fca040cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:38.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.385+0000 7fca0b46a640 1 --2- 172.21.15.89:0/3108202119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca040cd4f0 0x7fca0415f550 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:38.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.385+0000 7fca0b46a640 1 -- 172.21.15.89:0/3108202119 >> 172.21.15.89:0/3108202119 conn(0x7fca0406dad0 msgr2=0x7fca04156e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:38.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.385+0000 7fca0b46a640 1 -- 172.21.15.89:0/3108202119 shutdown_connections 2023-10-18T10:46:38.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.385+0000 7fca0b46a640 1 -- 172.21.15.89:0/3108202119 wait complete. 2023-10-18T10:46:38.407 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-18T10:46:38.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.545+0000 7f06683ba640 1 Processor -- start 2023-10-18T10:46:38.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.545+0000 7f06683ba640 1 -- start start 2023-10-18T10:46:38.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.545+0000 7f06683ba640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f066006ff00 0x7f066015e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:38.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.545+0000 7f06683ba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06600ca340 0x7f066015b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:38.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.545+0000 7f06683ba640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f066015b680 0x7f06600c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:38.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.545+0000 7f06683ba640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f066015ef30 con 0x7f066006ff00 2023-10-18T10:46:38.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.545+0000 7f06683ba640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f066015aa10 con 0x7f066015b680 2023-10-18T10:46:38.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.545+0000 7f06683ba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f066015acb0 con 0x7f06600ca340 2023-10-18T10:46:38.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.545+0000 7f0666156640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f066006ff00 0x7f066015e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:38.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.545+0000 7f0666156640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f066006ff00 0x7f066015e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47166/0 (socket says 172.21.15.89:47166) 2023-10-18T10:46:38.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.545+0000 7f0666156640 1 -- 172.21.15.89:0/1278587690 learned_addr learned my addr 172.21.15.89:0/1278587690 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:38.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.549+0000 7f0666957640 1 --2- 172.21.15.89:0/1278587690 >> v2:172.21.15.154:3300/0 conn(0x7f066015b680 0x7f06600c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:38.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.549+0000 7f0665955640 1 --2- 172.21.15.89:0/1278587690 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06600ca340 0x7f066015b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:38.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.549+0000 7f0666156640 1 -- 172.21.15.89:0/1278587690 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06600ca340 msgr2=0x7f066015b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:38.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.549+0000 7f0666156640 1 --2- 172.21.15.89:0/1278587690 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06600ca340 0x7f066015b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:38.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.549+0000 7f0666156640 1 -- 172.21.15.89:0/1278587690 >> v2:172.21.15.154:3300/0 conn(0x7f066015b680 msgr2=0x7f06600c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:38.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.549+0000 7f0666156640 1 --2- 172.21.15.89:0/1278587690 >> v2:172.21.15.154:3300/0 conn(0x7f066015b680 0x7f06600c8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:38.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.549+0000 7f0666156640 1 -- 172.21.15.89:0/1278587690 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06600702e0 con 0x7f066006ff00 2023-10-18T10:46:38.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.549+0000 7f0666957640 1 --2- 172.21.15.89:0/1278587690 >> v2:172.21.15.154:3300/0 conn(0x7f066015b680 0x7f06600c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:38.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.549+0000 7f0666156640 1 --2- 172.21.15.89:0/1278587690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f066006ff00 0x7f066015e000 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f06600ce2a0 tx=0x7f065c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ad0417eb1f4827 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:38.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.549+0000 7f0665154640 1 -- 172.21.15.89:0/1278587690 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065c0ac640 con 0x7f066006ff00 2023-10-18T10:46:38.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.549+0000 7f0665154640 1 -- 172.21.15.89:0/1278587690 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f065c0ac7a0 con 0x7f066006ff00 2023-10-18T10:46:38.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.549+0000 7f0665154640 1 -- 172.21.15.89:0/1278587690 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065c0acb30 con 0x7f066006ff00 2023-10-18T10:46:38.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.549+0000 7f06683ba640 1 -- 172.21.15.89:0/1278587690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f066006ff00 msgr2=0x7f066015e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:38.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.549+0000 7f06683ba640 1 --2- 172.21.15.89:0/1278587690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f066006ff00 0x7f066015e000 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f06600ce2a0 tx=0x7f065c01c440 comp rx=0 tx=0).stop 2023-10-18T10:46:38.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.553+0000 7f06683ba640 1 -- 172.21.15.89:0/1278587690 shutdown_connections 2023-10-18T10:46:38.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.553+0000 7f06683ba640 1 --2- 172.21.15.89:0/1278587690 >> v2:172.21.15.154:3300/0 conn(0x7f066015b680 0x7f06600c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:38.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.553+0000 7f06683ba640 1 --2- 172.21.15.89:0/1278587690 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f06600ca340 0x7f066015b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:38.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.553+0000 7f06683ba640 1 --2- 172.21.15.89:0/1278587690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f066006ff00 0x7f066015e000 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:38.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.553+0000 7f06683ba640 1 -- 172.21.15.89:0/1278587690 >> 172.21.15.89:0/1278587690 conn(0x7f0660069f00 msgr2=0x7f06600cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:38.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.553+0000 7f06683ba640 1 -- 172.21.15.89:0/1278587690 shutdown_connections 2023-10-18T10:46:38.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.553+0000 7f06683ba640 1 -- 172.21.15.89:0/1278587690 wait complete. 2023-10-18T10:46:38.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.553+0000 7f06683ba640 1 Processor -- start 2023-10-18T10:46:38.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.553+0000 7f06683ba640 1 -- start start 2023-10-18T10:46:38.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.553+0000 7f06683ba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f066006ff00 0x7f06600c4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:38.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.553+0000 7f06683ba640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06600ca340 0x7f06600cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:38.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.553+0000 7f06683ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06600c7f10 0x7f06600cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:38.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.553+0000 7f06683ba640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066015c870 con 0x7f06600ca340 2023-10-18T10:46:38.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.553+0000 7f06683ba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06600c2f10 con 0x7f06600c7f10 2023-10-18T10:46:38.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.553+0000 7f06683ba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f066015c1b0 con 0x7f066006ff00 2023-10-18T10:46:38.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.553+0000 7f0665955640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06600ca340 0x7f06600cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:38.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.553+0000 7f0665955640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06600ca340 0x7f06600cd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47178/0 (socket says 172.21.15.89:47178) 2023-10-18T10:46:38.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.553+0000 7f0665955640 1 -- 172.21.15.89:0/2215091836 learned_addr learned my addr 172.21.15.89:0/2215091836 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:38.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.553+0000 7f0666156640 1 --2- 172.21.15.89:0/2215091836 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f066006ff00 0x7f06600c4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:38.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.553+0000 7f0666957640 1 --2- 172.21.15.89:0/2215091836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06600c7f10 0x7f06600cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:38.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.553+0000 7f0665955640 1 -- 172.21.15.89:0/2215091836 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f066006ff00 msgr2=0x7f06600c4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:38.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.553+0000 7f0665955640 1 --2- 172.21.15.89:0/2215091836 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f066006ff00 0x7f06600c4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:38.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.553+0000 7f0665955640 1 -- 172.21.15.89:0/2215091836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06600c7f10 msgr2=0x7f06600cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:38.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.553+0000 7f0665955640 1 --2- 172.21.15.89:0/2215091836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06600c7f10 0x7f06600cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:38.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.553+0000 7f0665955640 1 -- 172.21.15.89:0/2215091836 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065c0094d0 con 0x7f06600ca340 2023-10-18T10:46:38.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.553+0000 7f0665955640 1 --2- 172.21.15.89:0/2215091836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06600ca340 0x7f06600cd0b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f065000d800 tx=0x7f065000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:38.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.557+0000 7f0656ffd640 1 -- 172.21.15.89:0/2215091836 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06500096a0 con 0x7f06600ca340 2023-10-18T10:46:38.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.557+0000 7f0656ffd640 1 -- 172.21.15.89:0/2215091836 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0650009800 con 0x7f06600ca340 2023-10-18T10:46:38.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.557+0000 7f06683ba640 1 -- 172.21.15.89:0/2215091836 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f066015c490 con 0x7f06600ca340 2023-10-18T10:46:38.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.557+0000 7f0656ffd640 1 -- 172.21.15.89:0/2215091836 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0650009b70 con 0x7f06600ca340 2023-10-18T10:46:38.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.557+0000 7f06683ba640 1 -- 172.21.15.89:0/2215091836 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06600c31f0 con 0x7f06600ca340 2023-10-18T10:46:38.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.557+0000 7f06683ba640 1 -- 172.21.15.89:0/2215091836 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f066004dbe0 con 0x7f06600ca340 2023-10-18T10:46:38.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.557+0000 7f0656ffd640 1 -- 172.21.15.89:0/2215091836 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f0650009cd0 con 0x7f06600ca340 2023-10-18T10:46:38.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.557+0000 7f0656ffd640 1 --2- 172.21.15.89:0/2215091836 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0644065680 0x7f0644067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:38.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.557+0000 7f0656ffd640 1 -- 172.21.15.89:0/2215091836 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0650083730 con 0x7f06600ca340 2023-10-18T10:46:38.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.557+0000 7f0666156640 1 --2- 172.21.15.89:0/2215091836 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0644065680 0x7f0644067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:38.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.561+0000 7f0666156640 1 --2- 172.21.15.89:0/2215091836 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0644065680 0x7f0644067b40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f065c0ae000 tx=0x7f065c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:38.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.561+0000 7f0656ffd640 1 -- 172.21.15.89:0/2215091836 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f0650051000 con 0x7f06600ca340 2023-10-18T10:46:38.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.717+0000 7f06683ba640 1 -- 172.21.15.89:0/2215091836 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0660165170 con 0x7f06600ca340 2023-10-18T10:46:38.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.717+0000 7f0656ffd640 1 -- 172.21.15.89:0/2215091836 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0650054620 con 0x7f06600ca340 2023-10-18T10:46:38.720 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-10-18T10:46:38.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.717+0000 7f06683ba640 1 -- 172.21.15.89:0/2215091836 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0644065680 msgr2=0x7f0644067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:38.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.717+0000 7f06683ba640 1 --2- 172.21.15.89:0/2215091836 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0644065680 0x7f0644067b40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f065c0ae000 tx=0x7f065c008040 comp rx=0 tx=0).stop 2023-10-18T10:46:38.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.717+0000 7f06683ba640 1 -- 172.21.15.89:0/2215091836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06600ca340 msgr2=0x7f06600cd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:38.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.717+0000 7f06683ba640 1 --2- 172.21.15.89:0/2215091836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06600ca340 0x7f06600cd0b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f065000d800 tx=0x7f065000dcc0 comp rx=0 tx=0).stop 2023-10-18T10:46:38.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.717+0000 7f06683ba640 1 -- 172.21.15.89:0/2215091836 shutdown_connections 2023-10-18T10:46:38.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.717+0000 7f06683ba640 1 --2- 172.21.15.89:0/2215091836 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0644065680 0x7f0644067b40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:38.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.717+0000 7f06683ba640 1 --2- 172.21.15.89:0/2215091836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06600c7f10 0x7f06600cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:38.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.717+0000 7f06683ba640 1 --2- 172.21.15.89:0/2215091836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06600ca340 0x7f06600cd0b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:38.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.717+0000 7f06683ba640 1 --2- 172.21.15.89:0/2215091836 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f066006ff00 0x7f06600c4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:38.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.717+0000 7f06683ba640 1 -- 172.21.15.89:0/2215091836 >> 172.21.15.89:0/2215091836 conn(0x7f0660069f00 msgr2=0x7f0660153b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:38.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.717+0000 7f06683ba640 1 -- 172.21.15.89:0/2215091836 shutdown_connections 2023-10-18T10:46:38.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.717+0000 7f06683ba640 1 -- 172.21.15.89:0/2215091836 wait complete. 2023-10-18T10:46:38.741 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:46:38.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.877+0000 7fa334461640 1 Processor -- start 2023-10-18T10:46:38.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.877+0000 7fa334461640 1 -- start start 2023-10-18T10:46:38.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.881+0000 7fa334461640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa32c0ca9e0 0x7fa32c0cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:38.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.881+0000 7fa334461640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa32c0cc140 con 0x7fa32c0cbc40 2023-10-18T10:46:38.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.881+0000 7fa334461640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa32c0cc420 con 0x7fa32c0cd7c0 2023-10-18T10:46:38.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.881+0000 7fa334461640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c0cc700 con 0x7fa32c0ca9e0 2023-10-18T10:46:38.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.881+0000 7fa3319fc640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa32c0cbc40 0x7fa32c15d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47956/0 (socket says 172.21.15.89:47956) 2023-10-18T10:46:38.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.881+0000 7fa3319fc640 1 -- 172.21.15.89:0/225258606 learned_addr learned my addr 172.21.15.89:0/225258606 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:38.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.881+0000 7fa3321fd640 1 --2- 172.21.15.89:0/225258606 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa32c0ca9e0 0x7fa32c0cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:38.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.881+0000 7fa3311fb640 1 -- 172.21.15.89:0/225258606 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4253231990 0 0) 0x7fa32c0cc140 con 0x7fa32c0cbc40 2023-10-18T10:46:38.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.881+0000 7fa3311fb640 1 -- 172.21.15.89:0/225258606 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa318003530 con 0x7fa32c0cbc40 2023-10-18T10:46:38.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.881+0000 7fa3321fd640 1 -- 172.21.15.89:0/225258606 >> v1:172.21.15.154:6789/0 conn(0x7fa32c0cd7c0 legacy=0x7fa32c159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:38.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.881+0000 7fa3321fd640 1 -- 172.21.15.89:0/225258606 >> v1:172.21.15.89:6789/0 conn(0x7fa32c0cbc40 legacy=0x7fa32c15d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:38.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.881+0000 7fa3321fd640 1 -- 172.21.15.89:0/225258606 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa32c0cc9e0 con 0x7fa32c0ca9e0 2023-10-18T10:46:38.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.885+0000 7fa3321fd640 1 --2- 172.21.15.89:0/225258606 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa32c0ca9e0 0x7fa32c0cadc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa3200097f0 tx=0x7fa32001c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1980917da99de43c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:38.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.885+0000 7fa3311fb640 1 -- 172.21.15.89:0/225258606 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3200ac600 con 0x7fa32c0ca9e0 2023-10-18T10:46:38.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.885+0000 7fa3311fb640 1 -- 172.21.15.89:0/225258606 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3200a8050 con 0x7fa32c0ca9e0 2023-10-18T10:46:38.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.885+0000 7fa3311fb640 1 -- 172.21.15.89:0/225258606 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3200ac980 con 0x7fa32c0ca9e0 2023-10-18T10:46:38.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.885+0000 7fa334461640 1 -- 172.21.15.89:0/225258606 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa32c0ca9e0 msgr2=0x7fa32c0cadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:38.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.885+0000 7fa334461640 1 --2- 172.21.15.89:0/225258606 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa32c0ca9e0 0x7fa32c0cadc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa3200097f0 tx=0x7fa32001c490 comp rx=0 tx=0).stop 2023-10-18T10:46:38.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.885+0000 7fa334461640 1 -- 172.21.15.89:0/225258606 shutdown_connections 2023-10-18T10:46:38.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.885+0000 7fa334461640 1 --2- 172.21.15.89:0/225258606 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa32c0ca9e0 0x7fa32c0cadc0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:38.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.885+0000 7fa334461640 1 -- 172.21.15.89:0/225258606 >> 172.21.15.89:0/225258606 conn(0x7fa32c0afbc0 msgr2=0x7fa32c0affd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:38.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.885+0000 7fa334461640 1 -- 172.21.15.89:0/225258606 shutdown_connections 2023-10-18T10:46:38.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.885+0000 7fa334461640 1 -- 172.21.15.89:0/225258606 wait complete. 2023-10-18T10:46:38.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.885+0000 7fa334461640 1 Processor -- start 2023-10-18T10:46:38.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.885+0000 7fa334461640 1 -- start start 2023-10-18T10:46:38.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.889+0000 7fa334461640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa32c0ca9e0 0x7fa32c14fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:38.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.889+0000 7fa334461640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa32c0cbc40 0x7fa32c0acc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:38.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.889+0000 7fa334461640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa32c0cd7c0 0x7fa32c0ad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:38.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.889+0000 7fa334461640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c150010 con 0x7fa32c0cd7c0 2023-10-18T10:46:38.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.889+0000 7fa334461640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c0ab8c0 con 0x7fa32c0ca9e0 2023-10-18T10:46:38.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.889+0000 7fa334461640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c0abba0 con 0x7fa32c0cbc40 2023-10-18T10:46:38.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.889+0000 7fa3329fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa32c0cd7c0 0x7fa32c0ad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:38.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.889+0000 7fa3329fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa32c0cd7c0 0x7fa32c0ad1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47184/0 (socket says 172.21.15.89:47184) 2023-10-18T10:46:38.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.889+0000 7fa3329fe640 1 -- 172.21.15.89:0/1639359274 learned_addr learned my addr 172.21.15.89:0/1639359274 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:38.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.889+0000 7fa3321fd640 1 --2- 172.21.15.89:0/1639359274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa32c0ca9e0 0x7fa32c14fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:38.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.889+0000 7fa3319fc640 1 --2- 172.21.15.89:0/1639359274 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa32c0cbc40 0x7fa32c0acc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:38.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.889+0000 7fa3329fe640 1 -- 172.21.15.89:0/1639359274 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa32c0cbc40 msgr2=0x7fa32c0acc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:38.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.889+0000 7fa3329fe640 1 --2- 172.21.15.89:0/1639359274 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa32c0cbc40 0x7fa32c0acc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:38.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.889+0000 7fa3329fe640 1 -- 172.21.15.89:0/1639359274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa32c0ca9e0 msgr2=0x7fa32c14fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:38.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.889+0000 7fa3329fe640 1 --2- 172.21.15.89:0/1639359274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa32c0ca9e0 0x7fa32c14fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:38.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.889+0000 7fa3329fe640 1 -- 172.21.15.89:0/1639359274 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa320009420 con 0x7fa32c0cd7c0 2023-10-18T10:46:38.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.889+0000 7fa3329fe640 1 --2- 172.21.15.89:0/1639359274 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa32c0cd7c0 0x7fa32c0ad1c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa32400c770 tx=0x7fa32400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:38.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.889+0000 7fa31effd640 1 -- 172.21.15.89:0/1639359274 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa324003b80 con 0x7fa32c0cd7c0 2023-10-18T10:46:38.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.889+0000 7fa31effd640 1 -- 172.21.15.89:0/1639359274 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa324003ce0 con 0x7fa32c0cd7c0 2023-10-18T10:46:38.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.889+0000 7fa334461640 1 -- 172.21.15.89:0/1639359274 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa32c0abe80 con 0x7fa32c0cd7c0 2023-10-18T10:46:38.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.889+0000 7fa31effd640 1 -- 172.21.15.89:0/1639359274 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32400aa90 con 0x7fa32c0cd7c0 2023-10-18T10:46:38.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.889+0000 7fa334461640 1 -- 172.21.15.89:0/1639359274 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa32c0ac440 con 0x7fa32c0cd7c0 2023-10-18T10:46:38.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.889+0000 7fa334461640 1 -- 172.21.15.89:0/1639359274 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa32c073b50 con 0x7fa32c0cd7c0 2023-10-18T10:46:38.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.889+0000 7fa31effd640 1 -- 172.21.15.89:0/1639359274 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fa32400abf0 con 0x7fa32c0cd7c0 2023-10-18T10:46:38.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.893+0000 7fa31effd640 1 --2- 172.21.15.89:0/1639359274 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa310065680 0x7fa310067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:38.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.893+0000 7fa31effd640 1 -- 172.21.15.89:0/1639359274 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa324082f70 con 0x7fa32c0cd7c0 2023-10-18T10:46:38.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.893+0000 7fa3321fd640 1 --2- 172.21.15.89:0/1639359274 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa310065680 0x7fa310067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:38.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.893+0000 7fa3321fd640 1 --2- 172.21.15.89:0/1639359274 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa310065680 0x7fa310067b40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa3200ae000 tx=0x7fa3200a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:38.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:38.897+0000 7fa31effd640 1 -- 172.21.15.89:0/1639359274 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fa324050660 con 0x7fa32c0cd7c0 2023-10-18T10:46:39.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.045+0000 7fa334461640 1 -- 172.21.15.89:0/1639359274 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa32c0c96d0 con 0x7fa32c0cd7c0 2023-10-18T10:46:39.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.045+0000 7fa31effd640 1 -- 172.21.15.89:0/1639359274 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa324053c80 con 0x7fa32c0cd7c0 2023-10-18T10:46:39.051 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-18T10:46:39.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.049+0000 7fa334461640 1 -- 172.21.15.89:0/1639359274 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa310065680 msgr2=0x7fa310067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:39.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.049+0000 7fa334461640 1 --2- 172.21.15.89:0/1639359274 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa310065680 0x7fa310067b40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa3200ae000 tx=0x7fa3200a7000 comp rx=0 tx=0).stop 2023-10-18T10:46:39.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.049+0000 7fa334461640 1 -- 172.21.15.89:0/1639359274 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa32c0cd7c0 msgr2=0x7fa32c0ad1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:39.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.049+0000 7fa334461640 1 --2- 172.21.15.89:0/1639359274 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa32c0cd7c0 0x7fa32c0ad1c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa32400c770 tx=0x7fa32400cc30 comp rx=0 tx=0).stop 2023-10-18T10:46:39.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.049+0000 7fa334461640 1 -- 172.21.15.89:0/1639359274 shutdown_connections 2023-10-18T10:46:39.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.049+0000 7fa334461640 1 --2- 172.21.15.89:0/1639359274 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa310065680 0x7fa310067b40 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:39.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.049+0000 7fa334461640 1 --2- 172.21.15.89:0/1639359274 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa32c0cbc40 0x7fa32c0acc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:39.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.049+0000 7fa334461640 1 --2- 172.21.15.89:0/1639359274 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa32c0cd7c0 0x7fa32c0ad1c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:39.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.049+0000 7fa334461640 1 --2- 172.21.15.89:0/1639359274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa32c0ca9e0 0x7fa32c14fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:39.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.049+0000 7fa334461640 1 -- 172.21.15.89:0/1639359274 >> 172.21.15.89:0/1639359274 conn(0x7fa32c0afbc0 msgr2=0x7fa32c06b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:39.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.049+0000 7fa334461640 1 -- 172.21.15.89:0/1639359274 shutdown_connections 2023-10-18T10:46:39.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.049+0000 7fa334461640 1 -- 172.21.15.89:0/1639359274 wait complete. 2023-10-18T10:46:39.071 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:46:39.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.209+0000 7fdaaf093640 1 Processor -- start 2023-10-18T10:46:39.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.209+0000 7fdaaf093640 1 -- start start 2023-10-18T10:46:39.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.209+0000 7fdaaf093640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdaa8145b70 0x7fdaa8143470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:39.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.209+0000 7fdaaf093640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdaa80ce070 0x7fdaa80ce500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:39.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.209+0000 7fdaaf093640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaa80cd8d0 con 0x7fdaa80cd4f0 2023-10-18T10:46:39.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.209+0000 7fdaaf093640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaa80cea40 con 0x7fdaa80ce070 2023-10-18T10:46:39.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.209+0000 7fdaaf093640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa8069900 con 0x7fdaa8145b70 2023-10-18T10:46:39.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.209+0000 7fdaad630640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdaa80ce070 0x7fdaa80ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:39.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.209+0000 7fdaad630640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdaa80ce070 0x7fdaa80ce500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47190/0 (socket says 172.21.15.89:47190) 2023-10-18T10:46:39.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.209+0000 7fdaad630640 1 -- 172.21.15.89:0/2501253168 learned_addr learned my addr 172.21.15.89:0/2501253168 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:39.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.209+0000 7fdaa7fff640 1 --2- 172.21.15.89:0/2501253168 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdaa8145b70 0x7fdaa8143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:39.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.209+0000 7fdaad630640 1 -- 172.21.15.89:0/2501253168 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdaa8145b70 msgr2=0x7fdaa8143470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:39.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.209+0000 7fdaad630640 1 --2- 172.21.15.89:0/2501253168 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdaa8145b70 0x7fdaa8143470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:39.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.209+0000 7fdaad630640 1 -- 172.21.15.89:0/2501253168 >> v1:172.21.15.154:6789/0 conn(0x7fdaa80cd4f0 legacy=0x7fdaa8160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:39.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.209+0000 7fdaad630640 1 -- 172.21.15.89:0/2501253168 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa815cbf0 con 0x7fdaa80ce070 2023-10-18T10:46:39.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.213+0000 7fdaad630640 1 --2- 172.21.15.89:0/2501253168 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaa80ce070 0x7fdaa80ce500 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009e50 tx=0x7fdaa001c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f145ad5c684cdf server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:39.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.213+0000 7fdaa77fe640 1 -- 172.21.15.89:0/2501253168 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa00a58a0 con 0x7fdaa80ce070 2023-10-18T10:46:39.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.213+0000 7fdaa77fe640 1 -- 172.21.15.89:0/2501253168 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa00a5a00 con 0x7fdaa80ce070 2023-10-18T10:46:39.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.213+0000 7fdaa77fe640 1 -- 172.21.15.89:0/2501253168 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa00a5d90 con 0x7fdaa80ce070 2023-10-18T10:46:39.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.213+0000 7fdaaf093640 1 -- 172.21.15.89:0/2501253168 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaa80ce070 msgr2=0x7fdaa80ce500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:39.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.213+0000 7fdaaf093640 1 --2- 172.21.15.89:0/2501253168 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaa80ce070 0x7fdaa80ce500 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009e50 tx=0x7fdaa001c9e0 comp rx=0 tx=0).stop 2023-10-18T10:46:39.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.213+0000 7fdaaf093640 1 -- 172.21.15.89:0/2501253168 shutdown_connections 2023-10-18T10:46:39.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.213+0000 7fdaaf093640 1 --2- 172.21.15.89:0/2501253168 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaa80ce070 0x7fdaa80ce500 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:39.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.213+0000 7fdaaf093640 1 --2- 172.21.15.89:0/2501253168 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdaa8145b70 0x7fdaa8143470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:39.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.213+0000 7fdaaf093640 1 -- 172.21.15.89:0/2501253168 >> 172.21.15.89:0/2501253168 conn(0x7fdaa806dad0 msgr2=0x7fdaa806dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:39.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.213+0000 7fdaaf093640 1 -- 172.21.15.89:0/2501253168 shutdown_connections 2023-10-18T10:46:39.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.213+0000 7fdaaf093640 1 -- 172.21.15.89:0/2501253168 wait complete. 2023-10-18T10:46:39.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.217+0000 7fdaaf093640 1 Processor -- start 2023-10-18T10:46:39.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.217+0000 7fdaaf093640 1 -- start start 2023-10-18T10:46:39.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.217+0000 7fdaaf093640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaa80cd4f0 0x7fdaa815aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:39.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.217+0000 7fdaaf093640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdaa80ce070 0x7fdaa8156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:39.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.217+0000 7fdaaf093640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdaa8145b70 0x7fdaa81565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:39.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.217+0000 7fdaaf093640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa8153700 con 0x7fdaa80cd4f0 2023-10-18T10:46:39.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.217+0000 7fdaaf093640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa815b460 con 0x7fdaa80ce070 2023-10-18T10:46:39.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.217+0000 7fdaaf093640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa815b740 con 0x7fdaa8145b70 2023-10-18T10:46:39.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.217+0000 7fdaace2f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaa80cd4f0 0x7fdaa815aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:39.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.217+0000 7fdaace2f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaa80cd4f0 0x7fdaa815aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47200/0 (socket says 172.21.15.89:47200) 2023-10-18T10:46:39.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.217+0000 7fdaace2f640 1 -- 172.21.15.89:0/3834320619 learned_addr learned my addr 172.21.15.89:0/3834320619 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:39.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.217+0000 7fdaad630640 1 --2- 172.21.15.89:0/3834320619 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdaa8145b70 0x7fdaa81565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:39.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.217+0000 7fdaa7fff640 1 --2- 172.21.15.89:0/3834320619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdaa80ce070 0x7fdaa8156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:39.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.217+0000 7fdaace2f640 1 -- 172.21.15.89:0/3834320619 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdaa8145b70 msgr2=0x7fdaa81565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:39.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.217+0000 7fdaace2f640 1 --2- 172.21.15.89:0/3834320619 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdaa8145b70 0x7fdaa81565a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:39.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.217+0000 7fdaace2f640 1 -- 172.21.15.89:0/3834320619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdaa80ce070 msgr2=0x7fdaa8156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:39.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.217+0000 7fdaace2f640 1 --2- 172.21.15.89:0/3834320619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdaa80ce070 0x7fdaa8156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:39.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.217+0000 7fdaace2f640 1 -- 172.21.15.89:0/3834320619 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa0009b30 con 0x7fdaa80cd4f0 2023-10-18T10:46:39.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.217+0000 7fdaace2f640 1 --2- 172.21.15.89:0/3834320619 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaa80cd4f0 0x7fdaa815aa30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fda9c00c3c0 tx=0x7fda9c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:39.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.217+0000 7fdaa57fa640 1 -- 172.21.15.89:0/3834320619 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9c0052d0 con 0x7fdaa80cd4f0 2023-10-18T10:46:39.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.217+0000 7fdaa57fa640 1 -- 172.21.15.89:0/3834320619 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda9c005430 con 0x7fdaa80cd4f0 2023-10-18T10:46:39.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.217+0000 7fdaaf093640 1 -- 172.21.15.89:0/3834320619 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaa815ba20 con 0x7fdaa80cd4f0 2023-10-18T10:46:39.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.217+0000 7fdaa57fa640 1 -- 172.21.15.89:0/3834320619 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9c003840 con 0x7fdaa80cd4f0 2023-10-18T10:46:39.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.217+0000 7fdaaf093640 1 -- 172.21.15.89:0/3834320619 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa815af70 con 0x7fdaa80cd4f0 2023-10-18T10:46:39.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.217+0000 7fdaaf093640 1 -- 172.21.15.89:0/3834320619 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaa804dbe0 con 0x7fdaa80cd4f0 2023-10-18T10:46:39.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.221+0000 7fdaa57fa640 1 -- 172.21.15.89:0/3834320619 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fda9c0039a0 con 0x7fdaa80cd4f0 2023-10-18T10:46:39.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.221+0000 7fdaa57fa640 1 --2- 172.21.15.89:0/3834320619 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fda8c065680 0x7fda8c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:39.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.221+0000 7fdaa57fa640 1 -- 172.21.15.89:0/3834320619 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fda9c082510 con 0x7fdaa80cd4f0 2023-10-18T10:46:39.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.221+0000 7fdaa7fff640 1 --2- 172.21.15.89:0/3834320619 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fda8c065680 0x7fda8c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:39.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.221+0000 7fdaa7fff640 1 --2- 172.21.15.89:0/3834320619 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fda8c065680 0x7fda8c067b40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80c3f20 tx=0x7fda94008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:39.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.225+0000 7fdaa57fa640 1 -- 172.21.15.89:0/3834320619 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fda9c04fde0 con 0x7fdaa80cd4f0 2023-10-18T10:46:39.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.373+0000 7fdaaf093640 1 -- 172.21.15.89:0/3834320619 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdaa80b6420 con 0x7fdaa80cd4f0 2023-10-18T10:46:39.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.373+0000 7fdaa57fa640 1 -- 172.21.15.89:0/3834320619 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7fda9c053400 con 0x7fdaa80cd4f0 2023-10-18T10:46:39.377 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-18T10:46:39.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.373+0000 7fdaaf093640 1 -- 172.21.15.89:0/3834320619 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fda8c065680 msgr2=0x7fda8c067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:39.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.373+0000 7fdaaf093640 1 --2- 172.21.15.89:0/3834320619 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fda8c065680 0x7fda8c067b40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80c3f20 tx=0x7fda94008040 comp rx=0 tx=0).stop 2023-10-18T10:46:39.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.373+0000 7fdaaf093640 1 -- 172.21.15.89:0/3834320619 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaa80cd4f0 msgr2=0x7fdaa815aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:39.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.373+0000 7fdaaf093640 1 --2- 172.21.15.89:0/3834320619 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaa80cd4f0 0x7fdaa815aa30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fda9c00c3c0 tx=0x7fda9c00c880 comp rx=0 tx=0).stop 2023-10-18T10:46:39.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.377+0000 7fdaaf093640 1 -- 172.21.15.89:0/3834320619 shutdown_connections 2023-10-18T10:46:39.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.377+0000 7fdaaf093640 1 --2- 172.21.15.89:0/3834320619 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fda8c065680 0x7fda8c067b40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:39.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.377+0000 7fdaaf093640 1 --2- 172.21.15.89:0/3834320619 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdaa8145b70 0x7fdaa81565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:39.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.377+0000 7fdaaf093640 1 --2- 172.21.15.89:0/3834320619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdaa80ce070 0x7fdaa8156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:39.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.377+0000 7fdaaf093640 1 --2- 172.21.15.89:0/3834320619 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaa80cd4f0 0x7fdaa815aa30 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:39.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.377+0000 7fdaaf093640 1 -- 172.21.15.89:0/3834320619 >> 172.21.15.89:0/3834320619 conn(0x7fdaa806dad0 msgr2=0x7fdaa815d250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:39.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.377+0000 7fdaaf093640 1 -- 172.21.15.89:0/3834320619 shutdown_connections 2023-10-18T10:46:39.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.377+0000 7fdaaf093640 1 -- 172.21.15.89:0/3834320619 wait complete. 2023-10-18T10:46:39.398 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T10:46:39.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.533+0000 7f98796d8640 1 Processor -- start 2023-10-18T10:46:39.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.537+0000 7f98796d8640 1 -- start start 2023-10-18T10:46:39.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.537+0000 7f98796d8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98740d0760 0x7f98740d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:39.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.537+0000 7f98796d8640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f98740cd160 0x7f98740cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:39.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.537+0000 7f98796d8640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98740d1100 con 0x7f987415c940 2023-10-18T10:46:39.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.537+0000 7f98796d8640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f987415da50 con 0x7f98740cd160 2023-10-18T10:46:39.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.537+0000 7f98796d8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98740d3110 con 0x7f98740d0760 2023-10-18T10:46:39.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.537+0000 7f9872ffd640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f987415c940 0x7f98740d2a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47958/0 (socket says 172.21.15.89:47958) 2023-10-18T10:46:39.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.537+0000 7f9872ffd640 1 -- 172.21.15.89:0/220320737 learned_addr learned my addr 172.21.15.89:0/220320737 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:39.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.537+0000 7f98737fe640 1 --2- 172.21.15.89:0/220320737 >> v2:172.21.15.89:3300/0 conn(0x7f98740cd160 0x7f98740cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:39.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.537+0000 7f986a7fc640 1 --2- 172.21.15.89:0/220320737 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98740d0760 0x7f98740d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:39.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.537+0000 7f98727fc640 1 -- 172.21.15.89:0/220320737 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2715787360 0 0) 0x7f98740d1100 con 0x7f987415c940 2023-10-18T10:46:39.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.537+0000 7f98727fc640 1 -- 172.21.15.89:0/220320737 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9860003530 con 0x7f987415c940 2023-10-18T10:46:39.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.537+0000 7f98737fe640 1 -- 172.21.15.89:0/220320737 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98740d0760 msgr2=0x7f98740d0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:39.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.537+0000 7f98737fe640 1 --2- 172.21.15.89:0/220320737 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98740d0760 0x7f98740d0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:39.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.537+0000 7f98737fe640 1 -- 172.21.15.89:0/220320737 >> v1:172.21.15.89:6789/0 conn(0x7f987415c940 legacy=0x7f98740d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:39.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.537+0000 7f98737fe640 1 -- 172.21.15.89:0/220320737 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f987415ce40 con 0x7f98740cd160 2023-10-18T10:46:39.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.541+0000 7f98737fe640 1 --2- 172.21.15.89:0/220320737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98740cd160 0x7f98740cd5f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f986c0097c0 tx=0x7f986c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b745dbf9b9dce52 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:39.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.541+0000 7f98727fc640 1 -- 172.21.15.89:0/220320737 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c0ab650 con 0x7f98740cd160 2023-10-18T10:46:39.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.541+0000 7f98727fc640 1 -- 172.21.15.89:0/220320737 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986c0ab7b0 con 0x7f98740cd160 2023-10-18T10:46:39.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.541+0000 7f98727fc640 1 -- 172.21.15.89:0/220320737 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c0abb40 con 0x7f98740cd160 2023-10-18T10:46:39.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.541+0000 7f98796d8640 1 -- 172.21.15.89:0/220320737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98740cd160 msgr2=0x7f98740cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:39.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.541+0000 7f98796d8640 1 --2- 172.21.15.89:0/220320737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98740cd160 0x7f98740cd5f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f986c0097c0 tx=0x7f986c01c450 comp rx=0 tx=0).stop 2023-10-18T10:46:39.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.541+0000 7f98796d8640 1 -- 172.21.15.89:0/220320737 shutdown_connections 2023-10-18T10:46:39.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.541+0000 7f98796d8640 1 --2- 172.21.15.89:0/220320737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98740cd160 0x7f98740cd5f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:39.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.541+0000 7f98796d8640 1 --2- 172.21.15.89:0/220320737 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98740d0760 0x7f98740d0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:39.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.541+0000 7f98796d8640 1 -- 172.21.15.89:0/220320737 >> 172.21.15.89:0/220320737 conn(0x7f9874089fd0 msgr2=0x7f987409b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:39.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.541+0000 7f98796d8640 1 -- 172.21.15.89:0/220320737 shutdown_connections 2023-10-18T10:46:39.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.541+0000 7f98796d8640 1 -- 172.21.15.89:0/220320737 wait complete. 2023-10-18T10:46:39.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.541+0000 7f98796d8640 1 Processor -- start 2023-10-18T10:46:39.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.541+0000 7f98796d8640 1 -- start start 2023-10-18T10:46:39.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.545+0000 7f98796d8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98740cd160 0x7f9874150dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:39.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.545+0000 7f98796d8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98740d0760 0x7f98740b5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:39.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.545+0000 7f98796d8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98740b57b0 0x7f98740b6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:39.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.545+0000 7f98796d8640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98740b68d0 con 0x7f98740d0760 2023-10-18T10:46:39.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.545+0000 7f98796d8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9874151310 con 0x7f98740b57b0 2023-10-18T10:46:39.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.545+0000 7f98796d8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98740b5c50 con 0x7f98740cd160 2023-10-18T10:46:39.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.545+0000 7f986a7fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98740d0760 0x7f98740b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:39.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.545+0000 7f986a7fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98740d0760 0x7f98740b5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47228/0 (socket says 172.21.15.89:47228) 2023-10-18T10:46:39.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.545+0000 7f986a7fc640 1 -- 172.21.15.89:0/430175795 learned_addr learned my addr 172.21.15.89:0/430175795 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:39.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.545+0000 7f98737fe640 1 --2- 172.21.15.89:0/430175795 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98740b57b0 0x7f98740b6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:39.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.545+0000 7f9872ffd640 1 --2- 172.21.15.89:0/430175795 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98740cd160 0x7f9874150dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:39.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.545+0000 7f986a7fc640 1 -- 172.21.15.89:0/430175795 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98740cd160 msgr2=0x7f9874150dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:39.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.545+0000 7f986a7fc640 1 --2- 172.21.15.89:0/430175795 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98740cd160 0x7f9874150dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:39.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.545+0000 7f986a7fc640 1 -- 172.21.15.89:0/430175795 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98740b57b0 msgr2=0x7f98740b6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:39.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.545+0000 7f986a7fc640 1 --2- 172.21.15.89:0/430175795 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98740b57b0 0x7f98740b6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:39.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.545+0000 7f986a7fc640 1 -- 172.21.15.89:0/430175795 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f986c009420 con 0x7f98740d0760 2023-10-18T10:46:39.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.545+0000 7f986a7fc640 1 --2- 172.21.15.89:0/430175795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98740d0760 0x7f98740b5270 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f985c007ef0 tx=0x7f985c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:39.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.545+0000 7f986bfff640 1 -- 172.21.15.89:0/430175795 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985c014620 con 0x7f98740d0760 2023-10-18T10:46:39.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.545+0000 7f986bfff640 1 -- 172.21.15.89:0/430175795 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f985c014780 con 0x7f98740d0760 2023-10-18T10:46:39.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.545+0000 7f98796d8640 1 -- 172.21.15.89:0/430175795 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98740b5f30 con 0x7f98740d0760 2023-10-18T10:46:39.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.545+0000 7f986bfff640 1 -- 172.21.15.89:0/430175795 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985c014b30 con 0x7f98740d0760 2023-10-18T10:46:39.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.545+0000 7f98796d8640 1 -- 172.21.15.89:0/430175795 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98740b64f0 con 0x7f98740d0760 2023-10-18T10:46:39.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.545+0000 7f98796d8640 1 -- 172.21.15.89:0/430175795 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98740c9570 con 0x7f98740d0760 2023-10-18T10:46:39.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.545+0000 7f986bfff640 1 -- 172.21.15.89:0/430175795 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f985c014c90 con 0x7f98740d0760 2023-10-18T10:46:39.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.549+0000 7f986bfff640 1 --2- 172.21.15.89:0/430175795 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f98540656b0 0x7f9854067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:39.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.549+0000 7f986bfff640 1 -- 172.21.15.89:0/430175795 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f985c083b10 con 0x7f98740d0760 2023-10-18T10:46:39.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.549+0000 7f9872ffd640 1 --2- 172.21.15.89:0/430175795 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f98540656b0 0x7f9854067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:39.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.549+0000 7f9872ffd640 1 --2- 172.21.15.89:0/430175795 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f98540656b0 0x7f9854067b70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f98640011f0 tx=0x7f9864009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:39.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.553+0000 7f986bfff640 1 -- 172.21.15.89:0/430175795 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f985c0512a0 con 0x7f98740d0760 2023-10-18T10:46:39.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.697+0000 7f98796d8640 1 -- 172.21.15.89:0/430175795 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f987404dbe0 con 0x7f98740d0760 2023-10-18T10:46:39.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.697+0000 7f986bfff640 1 -- 172.21.15.89:0/430175795 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f985c0548c0 con 0x7f98740d0760 2023-10-18T10:46:39.702 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-18T10:46:39.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.701+0000 7f98796d8640 1 -- 172.21.15.89:0/430175795 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f98540656b0 msgr2=0x7f9854067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:39.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.701+0000 7f98796d8640 1 --2- 172.21.15.89:0/430175795 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f98540656b0 0x7f9854067b70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f98640011f0 tx=0x7f9864009040 comp rx=0 tx=0).stop 2023-10-18T10:46:39.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.701+0000 7f98796d8640 1 -- 172.21.15.89:0/430175795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98740d0760 msgr2=0x7f98740b5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:39.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.701+0000 7f98796d8640 1 --2- 172.21.15.89:0/430175795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98740d0760 0x7f98740b5270 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f985c007ef0 tx=0x7f985c00e4b0 comp rx=0 tx=0).stop 2023-10-18T10:46:39.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.701+0000 7f98796d8640 1 -- 172.21.15.89:0/430175795 shutdown_connections 2023-10-18T10:46:39.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.701+0000 7f98796d8640 1 --2- 172.21.15.89:0/430175795 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f98540656b0 0x7f9854067b70 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:39.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.701+0000 7f98796d8640 1 --2- 172.21.15.89:0/430175795 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98740b57b0 0x7f98740b6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:39.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.701+0000 7f98796d8640 1 --2- 172.21.15.89:0/430175795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98740d0760 0x7f98740b5270 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:39.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.701+0000 7f98796d8640 1 --2- 172.21.15.89:0/430175795 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98740cd160 0x7f9874150dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:39.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.701+0000 7f98796d8640 1 -- 172.21.15.89:0/430175795 >> 172.21.15.89:0/430175795 conn(0x7f9874089fd0 msgr2=0x7f987406b180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:39.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.701+0000 7f98796d8640 1 -- 172.21.15.89:0/430175795 shutdown_connections 2023-10-18T10:46:39.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.701+0000 7f98796d8640 1 -- 172.21.15.89:0/430175795 wait complete. 2023-10-18T10:46:39.723 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:46:39.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.861+0000 7f14882cb640 1 Processor -- start 2023-10-18T10:46:39.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.861+0000 7f14882cb640 1 -- start start 2023-10-18T10:46:39.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.861+0000 7f14882cb640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f148015c670 0x7f14800713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:39.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.861+0000 7f14882cb640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1480060a30 0x7f14800c9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:39.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.861+0000 7f14882cb640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14801602a0 con 0x7f1480160a10 2023-10-18T10:46:39.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.861+0000 7f14882cb640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1480071900 con 0x7f1480060a30 2023-10-18T10:46:39.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.861+0000 7f14882cb640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14800cd2b0 con 0x7f148015c670 2023-10-18T10:46:39.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.861+0000 7f1486868640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1480060a30 0x7f14800c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:39.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.861+0000 7f1486868640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1480060a30 0x7f14800c9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47240/0 (socket says 172.21.15.89:47240) 2023-10-18T10:46:39.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.861+0000 7f1486868640 1 -- 172.21.15.89:0/2950590417 learned_addr learned my addr 172.21.15.89:0/2950590417 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:39.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.865+0000 7f1486067640 1 --2- 172.21.15.89:0/2950590417 >> v2:172.21.15.154:3300/0 conn(0x7f148015c670 0x7f14800713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:39.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.865+0000 7f1486868640 1 -- 172.21.15.89:0/2950590417 >> v2:172.21.15.154:3300/0 conn(0x7f148015c670 msgr2=0x7f14800713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:39.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.865+0000 7f1486868640 1 --2- 172.21.15.89:0/2950590417 >> v2:172.21.15.154:3300/0 conn(0x7f148015c670 0x7f14800713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:39.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.865+0000 7f1486868640 1 -- 172.21.15.89:0/2950590417 >> v1:172.21.15.154:6789/0 conn(0x7f1480160a10 legacy=0x7f1480060320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:39.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.865+0000 7f1486868640 1 -- 172.21.15.89:0/2950590417 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14800cd590 con 0x7f1480060a30 2023-10-18T10:46:39.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.865+0000 7f1486868640 1 --2- 172.21.15.89:0/2950590417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1480060a30 0x7f14800c9990 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f148006c4e0 tx=0x7f147801f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca2ec75661b4926 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:39.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.865+0000 7f1485065640 1 -- 172.21.15.89:0/2950590417 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14780af400 con 0x7f1480060a30 2023-10-18T10:46:39.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.865+0000 7f1485065640 1 -- 172.21.15.89:0/2950590417 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14780ab050 con 0x7f1480060a30 2023-10-18T10:46:39.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.865+0000 7f1485065640 1 -- 172.21.15.89:0/2950590417 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14780af780 con 0x7f1480060a30 2023-10-18T10:46:39.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.869+0000 7f14882cb640 1 -- 172.21.15.89:0/2950590417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1480060a30 msgr2=0x7f14800c9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:39.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.869+0000 7f14882cb640 1 --2- 172.21.15.89:0/2950590417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1480060a30 0x7f14800c9990 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f148006c4e0 tx=0x7f147801f1c0 comp rx=0 tx=0).stop 2023-10-18T10:46:39.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.869+0000 7f14882cb640 1 -- 172.21.15.89:0/2950590417 shutdown_connections 2023-10-18T10:46:39.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.869+0000 7f14882cb640 1 --2- 172.21.15.89:0/2950590417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1480060a30 0x7f14800c9990 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:39.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.869+0000 7f14882cb640 1 --2- 172.21.15.89:0/2950590417 >> v2:172.21.15.154:3300/0 conn(0x7f148015c670 0x7f14800713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:39.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.869+0000 7f14882cb640 1 -- 172.21.15.89:0/2950590417 >> 172.21.15.89:0/2950590417 conn(0x7f14800cbdc0 msgr2=0x7f14800cc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:39.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.869+0000 7f14882cb640 1 -- 172.21.15.89:0/2950590417 shutdown_connections 2023-10-18T10:46:39.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.869+0000 7f14882cb640 1 -- 172.21.15.89:0/2950590417 wait complete. 2023-10-18T10:46:39.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.869+0000 7f14882cb640 1 Processor -- start 2023-10-18T10:46:39.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.869+0000 7f14882cb640 1 -- start start 2023-10-18T10:46:39.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.869+0000 7f14882cb640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1480060a30 0x7f148015b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:39.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.869+0000 7f14882cb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f148015c670 0x7f148015ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:39.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.869+0000 7f14882cb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1480160a10 0x7f148015efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:39.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.869+0000 7f14882cb640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14801ecd90 con 0x7f1480060a30 2023-10-18T10:46:39.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.869+0000 7f14882cb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14801e98c0 con 0x7f148015c670 2023-10-18T10:46:39.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.869+0000 7f14882cb640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14801e9e40 con 0x7f1480160a10 2023-10-18T10:46:39.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.869+0000 7f1486067640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1480060a30 0x7f148015b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:39.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.869+0000 7f1486067640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1480060a30 0x7f148015b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47246/0 (socket says 172.21.15.89:47246) 2023-10-18T10:46:39.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.869+0000 7f1486067640 1 -- 172.21.15.89:0/3083886333 learned_addr learned my addr 172.21.15.89:0/3083886333 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:39.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.869+0000 7f1485866640 1 --2- 172.21.15.89:0/3083886333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f148015c670 0x7f148015ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:39.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.869+0000 7f1486868640 1 --2- 172.21.15.89:0/3083886333 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1480160a10 0x7f148015efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:39.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.869+0000 7f1486067640 1 -- 172.21.15.89:0/3083886333 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1480160a10 msgr2=0x7f148015efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:39.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.869+0000 7f1486067640 1 --2- 172.21.15.89:0/3083886333 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1480160a10 0x7f148015efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:39.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.869+0000 7f1486067640 1 -- 172.21.15.89:0/3083886333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f148015c670 msgr2=0x7f148015ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:39.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.869+0000 7f1486067640 1 --2- 172.21.15.89:0/3083886333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f148015c670 0x7f148015ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:39.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.869+0000 7f1486067640 1 -- 172.21.15.89:0/3083886333 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1478009b30 con 0x7f1480060a30 2023-10-18T10:46:39.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.873+0000 7f1486868640 1 --2- 172.21.15.89:0/3083886333 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1480160a10 0x7f148015efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:39.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.873+0000 7f1486067640 1 --2- 172.21.15.89:0/3083886333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1480060a30 0x7f148015b900 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f147c00e930 tx=0x7f147c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:39.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.873+0000 7f1476ffd640 1 -- 172.21.15.89:0/3083886333 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c009690 con 0x7f1480060a30 2023-10-18T10:46:39.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.873+0000 7f1476ffd640 1 -- 172.21.15.89:0/3083886333 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f147c00ca00 con 0x7f1480060a30 2023-10-18T10:46:39.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.873+0000 7f1476ffd640 1 -- 172.21.15.89:0/3083886333 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c00cd70 con 0x7f1480060a30 2023-10-18T10:46:39.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.873+0000 7f14882cb640 1 -- 172.21.15.89:0/3083886333 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f148015be40 con 0x7f1480060a30 2023-10-18T10:46:39.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.873+0000 7f14882cb640 1 -- 172.21.15.89:0/3083886333 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14801ecab0 con 0x7f1480060a30 2023-10-18T10:46:39.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.873+0000 7f14882cb640 1 -- 172.21.15.89:0/3083886333 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f148004dbe0 con 0x7f1480060a30 2023-10-18T10:46:39.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.873+0000 7f1476ffd640 1 -- 172.21.15.89:0/3083886333 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f147c013470 con 0x7f1480060a30 2023-10-18T10:46:39.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.873+0000 7f1476ffd640 1 --2- 172.21.15.89:0/3083886333 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f14640656b0 0x7f1464067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:39.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.873+0000 7f1476ffd640 1 -- 172.21.15.89:0/3083886333 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f147c0832e0 con 0x7f1480060a30 2023-10-18T10:46:39.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.877+0000 7f1485866640 1 --2- 172.21.15.89:0/3083886333 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f14640656b0 0x7f1464067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:39.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.877+0000 7f1485866640 1 --2- 172.21.15.89:0/3083886333 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f14640656b0 0x7f1464067b70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f148015f8b0 tx=0x7f1470009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:39.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:39.881+0000 7f1476ffd640 1 -- 172.21.15.89:0/3083886333 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f147c050be0 con 0x7f1480060a30 2023-10-18T10:46:40.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.029+0000 7f14882cb640 1 -- 172.21.15.89:0/3083886333 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1480071c60 con 0x7f1480060a30 2023-10-18T10:46:40.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.033+0000 7f1476ffd640 1 -- 172.21.15.89:0/3083886333 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20192 (secure 0 0 0) 0x7f147c054200 con 0x7f1480060a30 2023-10-18T10:46:40.035 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:40.036 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":29,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:46:36.958604+0000","last_up_change":"2023-10-18T10:45:42.164688+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:45:53.309122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-18T10:45:54.659763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:45:57.796514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6824","nonce":773078364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6827","nonce":773078364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6831","nonce":773078364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6829","nonce":773078364}]},"public_addr":"172.21.15.154:6824/773078364","cluster_addr":"172.21.15.154:6827/773078364","heartbeat_back_addr":"172.21.15.154:6831/773078364","heartbeat_front_addr":"172.21.15.154:6829/773078364","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":1,"to":5}]},{"pgid":"4.1a","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:46:40.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.033+0000 7f14882cb640 1 -- 172.21.15.89:0/3083886333 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f14640656b0 msgr2=0x7f1464067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:40.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.033+0000 7f14882cb640 1 --2- 172.21.15.89:0/3083886333 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f14640656b0 0x7f1464067b70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f148015f8b0 tx=0x7f1470009040 comp rx=0 tx=0).stop 2023-10-18T10:46:40.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.033+0000 7f14882cb640 1 -- 172.21.15.89:0/3083886333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1480060a30 msgr2=0x7f148015b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:40.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.033+0000 7f14882cb640 1 --2- 172.21.15.89:0/3083886333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1480060a30 0x7f148015b900 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f147c00e930 tx=0x7f147c00edf0 comp rx=0 tx=0).stop 2023-10-18T10:46:40.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.033+0000 7f14882cb640 1 -- 172.21.15.89:0/3083886333 shutdown_connections 2023-10-18T10:46:40.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.033+0000 7f14882cb640 1 --2- 172.21.15.89:0/3083886333 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f14640656b0 0x7f1464067b70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:40.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.033+0000 7f14882cb640 1 --2- 172.21.15.89:0/3083886333 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1480160a10 0x7f148015efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:40.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.033+0000 7f14882cb640 1 --2- 172.21.15.89:0/3083886333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f148015c670 0x7f148015ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:40.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.033+0000 7f14882cb640 1 --2- 172.21.15.89:0/3083886333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1480060a30 0x7f148015b900 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:40.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.033+0000 7f14882cb640 1 -- 172.21.15.89:0/3083886333 >> 172.21.15.89:0/3083886333 conn(0x7f14800cbdc0 msgr2=0x7f14800b07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:40.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.033+0000 7f14882cb640 1 -- 172.21.15.89:0/3083886333 shutdown_connections 2023-10-18T10:46:40.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.033+0000 7f14882cb640 1 -- 172.21.15.89:0/3083886333 wait complete. 2023-10-18T10:46:40.056 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:45:43.114184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-18T10:45:48.730968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:45:53.309122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:45:54.659763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T10:45:57.796514+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T10:46:40.057 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:46:40.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.193+0000 7fe765974640 1 Processor -- start 2023-10-18T10:46:40.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.193+0000 7fe765974640 1 -- start start 2023-10-18T10:46:40.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.197+0000 7fe765974640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe76015c7d0 0x7fe7600cd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:40.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.197+0000 7fe765974640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe7600ce440 0x7fe7600b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:40.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.197+0000 7fe765974640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7600b5750 con 0x7fe7600b4490 2023-10-18T10:46:40.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.197+0000 7fe765974640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7600ce0f0 con 0x7fe76015c7d0 2023-10-18T10:46:40.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.197+0000 7fe765974640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7600b4920 con 0x7fe7600ce440 2023-10-18T10:46:40.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.197+0000 7fe75f7fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe7600b4490 0x7fe760069900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47962/0 (socket says 172.21.15.89:47962) 2023-10-18T10:46:40.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.197+0000 7fe75f7fe640 1 -- 172.21.15.89:0/1416421695 learned_addr learned my addr 172.21.15.89:0/1416421695 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:40.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.197+0000 7fe7567fc640 1 --2- 172.21.15.89:0/1416421695 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe7600ce440 0x7fe7600b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:40.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.197+0000 7fe75effd640 1 --2- 172.21.15.89:0/1416421695 >> v2:172.21.15.154:3300/0 conn(0x7fe76015c7d0 0x7fe7600cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:40.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.197+0000 7fe75e7fc640 1 -- 172.21.15.89:0/1416421695 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3403519490 0 0) 0x7fe7600b5750 con 0x7fe7600b4490 2023-10-18T10:46:40.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.197+0000 7fe75e7fc640 1 -- 172.21.15.89:0/1416421695 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe74c003530 con 0x7fe7600b4490 2023-10-18T10:46:40.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.197+0000 7fe75e7fc640 1 -- 172.21.15.89:0/1416421695 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1633707498 0 0) 0x7fe74c003530 con 0x7fe7600b4490 2023-10-18T10:46:40.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.197+0000 7fe75e7fc640 1 -- 172.21.15.89:0/1416421695 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe7600ce440 msgr2=0x7fe7600b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:40.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.197+0000 7fe75e7fc640 1 --2- 172.21.15.89:0/1416421695 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe7600ce440 0x7fe7600b3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:40.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.197+0000 7fe75e7fc640 1 -- 172.21.15.89:0/1416421695 >> v2:172.21.15.154:3300/0 conn(0x7fe76015c7d0 msgr2=0x7fe7600cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:40.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.197+0000 7fe75e7fc640 1 --2- 172.21.15.89:0/1416421695 >> v2:172.21.15.154:3300/0 conn(0x7fe76015c7d0 0x7fe7600cd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:40.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.197+0000 7fe75e7fc640 1 -- 172.21.15.89:0/1416421695 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7600b5420 con 0x7fe7600b4490 2023-10-18T10:46:40.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.197+0000 7fe75e7fc640 1 -- 172.21.15.89:0/1416421695 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7fe758001680 con 0x7fe7600b4490 2023-10-18T10:46:40.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.197+0000 7fe75e7fc640 1 -- 172.21.15.89:0/1416421695 >> v1:172.21.15.89:6789/0 conn(0x7fe7600b4490 legacy=0x7fe760069900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:40.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.197+0000 7fe75e7fc640 1 --2- 172.21.15.89:0/1416421695 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe7600ce440 0x7fe74c003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:40.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.197+0000 7fe75e7fc640 1 --2- 172.21.15.89:0/1416421695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe74c003f00 0x7fe74c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:40.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.197+0000 7fe75e7fc640 1 --2- 172.21.15.89:0/1416421695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe74c004d10 0x7fe74c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:40.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.197+0000 7fe75e7fc640 1 -- 172.21.15.89:0/1416421695 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7600ce0f0 con 0x7fe74c004d10 2023-10-18T10:46:40.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.197+0000 7fe75e7fc640 1 -- 172.21.15.89:0/1416421695 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7600b4920 con 0x7fe74c003f00 2023-10-18T10:46:40.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.197+0000 7fe75e7fc640 1 -- 172.21.15.89:0/1416421695 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe74c00b9c0 con 0x7fe7600ce440 2023-10-18T10:46:40.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.197+0000 7fe75f7fe640 1 --2- 172.21.15.89:0/1416421695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe74c004d10 0x7fe74c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:40.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.197+0000 7fe7567fc640 1 --2- 172.21.15.89:0/1416421695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe74c003f00 0x7fe74c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:40.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.197+0000 7fe75effd640 1 --2- 172.21.15.89:0/1416421695 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe7600ce440 0x7fe74c003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:40.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.197+0000 7fe75f7fe640 1 -- 172.21.15.89:0/1416421695 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe7600ce440 msgr2=0x7fe74c003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:40.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.197+0000 7fe75f7fe640 1 --2- 172.21.15.89:0/1416421695 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe7600ce440 0x7fe74c003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:40.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.197+0000 7fe75f7fe640 1 -- 172.21.15.89:0/1416421695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe74c003f00 msgr2=0x7fe74c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:40.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.197+0000 7fe75f7fe640 1 --2- 172.21.15.89:0/1416421695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe74c003f00 0x7fe74c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:40.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.197+0000 7fe75f7fe640 1 -- 172.21.15.89:0/1416421695 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe74c003530 con 0x7fe74c004d10 2023-10-18T10:46:40.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.201+0000 7fe75f7fe640 1 --2- 172.21.15.89:0/1416421695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe74c004d10 0x7fe74c005190 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe75800c290 tx=0x7fe75801cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cacc725cbda1c4bf server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:40.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.201+0000 7fe75e7fc640 1 -- 172.21.15.89:0/1416421695 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7580a5950 con 0x7fe74c004d10 2023-10-18T10:46:40.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.201+0000 7fe75e7fc640 1 -- 172.21.15.89:0/1416421695 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7580a5ab0 con 0x7fe74c004d10 2023-10-18T10:46:40.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.201+0000 7fe75e7fc640 1 -- 172.21.15.89:0/1416421695 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7580a5e60 con 0x7fe74c004d10 2023-10-18T10:46:40.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.201+0000 7fe765974640 1 -- 172.21.15.89:0/1416421695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe74c004d10 msgr2=0x7fe74c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:40.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.201+0000 7fe765974640 1 --2- 172.21.15.89:0/1416421695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe74c004d10 0x7fe74c005190 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe75800c290 tx=0x7fe75801cce0 comp rx=0 tx=0).stop 2023-10-18T10:46:40.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.201+0000 7fe75f7fe640 1 -- 172.21.15.89:0/1416421695 reap_dead start 2023-10-18T10:46:40.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.201+0000 7fe765974640 1 -- 172.21.15.89:0/1416421695 shutdown_connections 2023-10-18T10:46:40.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.201+0000 7fe765974640 1 -- 172.21.15.89:0/1416421695 >> 172.21.15.89:0/1416421695 conn(0x7fe760070320 msgr2=0x7fe760070730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:40.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.201+0000 7fe765974640 1 -- 172.21.15.89:0/1416421695 shutdown_connections 2023-10-18T10:46:40.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.201+0000 7fe765974640 1 -- 172.21.15.89:0/1416421695 wait complete. 2023-10-18T10:46:40.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.201+0000 7fe765974640 1 Processor -- start 2023-10-18T10:46:40.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.205+0000 7fe765974640 1 -- start start 2023-10-18T10:46:40.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.205+0000 7fe765974640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe74c004d10 0x7fe760152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:40.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.205+0000 7fe765974640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe760155170 0x7fe76015f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:40.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.205+0000 7fe765974640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe76015f8e0 0x7fe7601f4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:40.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.205+0000 7fe765974640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7600c8370 con 0x7fe74c004d10 2023-10-18T10:46:40.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.205+0000 7fe765974640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7600c7a60 con 0x7fe76015f8e0 2023-10-18T10:46:40.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.205+0000 7fe765974640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7600c5db0 con 0x7fe760155170 2023-10-18T10:46:40.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.205+0000 7fe75effd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe74c004d10 0x7fe760152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:40.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.205+0000 7fe75effd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe74c004d10 0x7fe760152310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47264/0 (socket says 172.21.15.89:47264) 2023-10-18T10:46:40.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.205+0000 7fe75effd640 1 -- 172.21.15.89:0/1938858921 learned_addr learned my addr 172.21.15.89:0/1938858921 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:40.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.205+0000 7fe7567fc640 1 --2- 172.21.15.89:0/1938858921 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe760155170 0x7fe76015f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:40.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.205+0000 7fe75f7fe640 1 --2- 172.21.15.89:0/1938858921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe76015f8e0 0x7fe7601f4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:40.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.205+0000 7fe75effd640 1 -- 172.21.15.89:0/1938858921 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe760155170 msgr2=0x7fe76015f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:40.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.205+0000 7fe75effd640 1 --2- 172.21.15.89:0/1938858921 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe760155170 0x7fe76015f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:40.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.205+0000 7fe75effd640 1 -- 172.21.15.89:0/1938858921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe76015f8e0 msgr2=0x7fe7601f4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:40.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.205+0000 7fe75effd640 1 --2- 172.21.15.89:0/1938858921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe76015f8e0 0x7fe7601f4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:40.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.205+0000 7fe75effd640 1 -- 172.21.15.89:0/1938858921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe760152960 con 0x7fe74c004d10 2023-10-18T10:46:40.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.205+0000 7fe75f7fe640 1 --2- 172.21.15.89:0/1938858921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe76015f8e0 0x7fe7601f4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:40.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.205+0000 7fe75effd640 1 --2- 172.21.15.89:0/1938858921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe74c004d10 0x7fe760152310 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe750011e60 tx=0x7fe75000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:40.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.205+0000 7fe75cff9640 1 -- 172.21.15.89:0/1938858921 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75000e040 con 0x7fe74c004d10 2023-10-18T10:46:40.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.205+0000 7fe75cff9640 1 -- 172.21.15.89:0/1938858921 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75000f040 con 0x7fe74c004d10 2023-10-18T10:46:40.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.205+0000 7fe75cff9640 1 -- 172.21.15.89:0/1938858921 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe750015850 con 0x7fe74c004d10 2023-10-18T10:46:40.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.205+0000 7fe765974640 1 -- 172.21.15.89:0/1938858921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7600c7cb0 con 0x7fe74c004d10 2023-10-18T10:46:40.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.205+0000 7fe765974640 1 -- 172.21.15.89:0/1938858921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7601555f0 con 0x7fe74c004d10 2023-10-18T10:46:40.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.205+0000 7fe765974640 1 -- 172.21.15.89:0/1938858921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7600c85e0 con 0x7fe74c004d10 2023-10-18T10:46:40.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.209+0000 7fe75cff9640 1 -- 172.21.15.89:0/1938858921 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fe7500159b0 con 0x7fe74c004d10 2023-10-18T10:46:40.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.209+0000 7fe75cff9640 1 --2- 172.21.15.89:0/1938858921 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe740065680 0x7fe740067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:40.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.209+0000 7fe75cff9640 1 -- 172.21.15.89:0/1938858921 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe750083fe0 con 0x7fe74c004d10 2023-10-18T10:46:40.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.209+0000 7fe7567fc640 1 --2- 172.21.15.89:0/1938858921 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe740065680 0x7fe740067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:40.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.209+0000 7fe7567fc640 1 --2- 172.21.15.89:0/1938858921 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe740065680 0x7fe740067b40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe760148060 tx=0x7fe748008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:40.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.213+0000 7fe75cff9640 1 -- 172.21.15.89:0/1938858921 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fe750051730 con 0x7fe74c004d10 2023-10-18T10:46:40.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.365+0000 7fe765974640 1 -- 172.21.15.89:0/1938858921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe7600ced50 con 0x7fe74c004d10 2023-10-18T10:46:40.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.365+0000 7fe75cff9640 1 -- 172.21.15.89:0/1938858921 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe750054d50 con 0x7fe74c004d10 2023-10-18T10:46:40.369 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-18T10:46:40.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.365+0000 7fe765974640 1 -- 172.21.15.89:0/1938858921 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe740065680 msgr2=0x7fe740067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:40.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.365+0000 7fe765974640 1 --2- 172.21.15.89:0/1938858921 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe740065680 0x7fe740067b40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe760148060 tx=0x7fe748008040 comp rx=0 tx=0).stop 2023-10-18T10:46:40.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.365+0000 7fe765974640 1 -- 172.21.15.89:0/1938858921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe74c004d10 msgr2=0x7fe760152310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:40.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.365+0000 7fe765974640 1 --2- 172.21.15.89:0/1938858921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe74c004d10 0x7fe760152310 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe750011e60 tx=0x7fe75000f580 comp rx=0 tx=0).stop 2023-10-18T10:46:40.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.365+0000 7fe765974640 1 -- 172.21.15.89:0/1938858921 shutdown_connections 2023-10-18T10:46:40.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.365+0000 7fe765974640 1 --2- 172.21.15.89:0/1938858921 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe740065680 0x7fe740067b40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:40.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.365+0000 7fe765974640 1 --2- 172.21.15.89:0/1938858921 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe760155170 0x7fe76015f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:40.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.365+0000 7fe765974640 1 --2- 172.21.15.89:0/1938858921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe76015f8e0 0x7fe7601f4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:40.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.365+0000 7fe765974640 1 --2- 172.21.15.89:0/1938858921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe74c004d10 0x7fe760152310 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:40.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.365+0000 7fe765974640 1 -- 172.21.15.89:0/1938858921 >> 172.21.15.89:0/1938858921 conn(0x7fe760070320 msgr2=0x7fe7600ca890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:40.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.369+0000 7fe765974640 1 -- 172.21.15.89:0/1938858921 shutdown_connections 2023-10-18T10:46:40.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.369+0000 7fe765974640 1 -- 172.21.15.89:0/1938858921 wait complete. 2023-10-18T10:46:40.389 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-18T10:46:40.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.525+0000 7f9db99ba640 1 Processor -- start 2023-10-18T10:46:40.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.525+0000 7f9db99ba640 1 -- start start 2023-10-18T10:46:40.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.529+0000 7f9db99ba640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9db415c940 0x7f9db40d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:40.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.529+0000 7f9db99ba640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9db40d2f40 0x7f9db40d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:40.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.529+0000 7f9db99ba640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9db40d10f0 con 0x7f9db40d0ca0 2023-10-18T10:46:40.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.529+0000 7f9db99ba640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9db415da50 con 0x7f9db415c940 2023-10-18T10:46:40.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.529+0000 7f9db99ba640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9db415ce40 con 0x7f9db40d2f40 2023-10-18T10:46:40.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.529+0000 7f9db2ffd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9db415c940 0x7f9db40d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:40.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.529+0000 7f9db2ffd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9db415c940 0x7f9db40d2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47280/0 (socket says 172.21.15.89:47280) 2023-10-18T10:46:40.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.529+0000 7f9db2ffd640 1 -- 172.21.15.89:0/770620587 learned_addr learned my addr 172.21.15.89:0/770620587 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:40.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.529+0000 7f9db27fc640 1 --2- 172.21.15.89:0/770620587 >> v2:172.21.15.154:3300/0 conn(0x7f9db40d2f40 0x7f9db40d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:40.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.529+0000 7f9db2ffd640 1 -- 172.21.15.89:0/770620587 >> v2:172.21.15.154:3300/0 conn(0x7f9db40d2f40 msgr2=0x7f9db40d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:40.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.529+0000 7f9db2ffd640 1 --2- 172.21.15.89:0/770620587 >> v2:172.21.15.154:3300/0 conn(0x7f9db40d2f40 0x7f9db40d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:40.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.529+0000 7f9db2ffd640 1 -- 172.21.15.89:0/770620587 >> v1:172.21.15.154:6789/0 conn(0x7f9db40d0ca0 legacy=0x7f9db40cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:40.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.529+0000 7f9db2ffd640 1 -- 172.21.15.89:0/770620587 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db415d120 con 0x7f9db415c940 2023-10-18T10:46:40.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.529+0000 7f9db2ffd640 1 --2- 172.21.15.89:0/770620587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9db415c940 0x7f9db40d2a00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9db40724f0 tx=0x7f9da401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eeae3248223b431 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:40.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.533+0000 7f9db1ffb640 1 -- 172.21.15.89:0/770620587 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da40a5730 con 0x7f9db415c940 2023-10-18T10:46:40.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.533+0000 7f9db1ffb640 1 -- 172.21.15.89:0/770620587 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da40a5890 con 0x7f9db415c940 2023-10-18T10:46:40.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.533+0000 7f9db1ffb640 1 -- 172.21.15.89:0/770620587 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da40a5c20 con 0x7f9db415c940 2023-10-18T10:46:40.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.533+0000 7f9db99ba640 1 -- 172.21.15.89:0/770620587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9db415c940 msgr2=0x7f9db40d2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:40.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.533+0000 7f9db99ba640 1 --2- 172.21.15.89:0/770620587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9db415c940 0x7f9db40d2a00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9db40724f0 tx=0x7f9da401c960 comp rx=0 tx=0).stop 2023-10-18T10:46:40.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.533+0000 7f9db99ba640 1 -- 172.21.15.89:0/770620587 shutdown_connections 2023-10-18T10:46:40.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.533+0000 7f9db99ba640 1 --2- 172.21.15.89:0/770620587 >> v2:172.21.15.154:3300/0 conn(0x7f9db40d2f40 0x7f9db40d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:40.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.533+0000 7f9db99ba640 1 --2- 172.21.15.89:0/770620587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9db415c940 0x7f9db40d2a00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:40.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.533+0000 7f9db99ba640 1 -- 172.21.15.89:0/770620587 >> 172.21.15.89:0/770620587 conn(0x7f9db4089fd0 msgr2=0x7f9db409b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:40.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.533+0000 7f9db99ba640 1 -- 172.21.15.89:0/770620587 shutdown_connections 2023-10-18T10:46:40.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.533+0000 7f9db99ba640 1 -- 172.21.15.89:0/770620587 wait complete. 2023-10-18T10:46:40.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.533+0000 7f9db99ba640 1 Processor -- start 2023-10-18T10:46:40.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.533+0000 7f9db99ba640 1 -- start start 2023-10-18T10:46:40.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.533+0000 7f9db99ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9db40d0ca0 0x7f9db415b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:40.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.533+0000 7f9db99ba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9db40b76f0 0x7f9db40b7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:40.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.533+0000 7f9db99ba640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9db40caea0 0x7f9db40cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:40.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.533+0000 7f9db99ba640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db4151850 con 0x7f9db40caea0 2023-10-18T10:46:40.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.533+0000 7f9db99ba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db4152db0 con 0x7f9db40d0ca0 2023-10-18T10:46:40.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.533+0000 7f9db99ba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db4152920 con 0x7f9db40b76f0 2023-10-18T10:46:40.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.533+0000 7f9db37fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9db40caea0 0x7f9db40cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:40.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.533+0000 7f9db37fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9db40caea0 0x7f9db40cb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47290/0 (socket says 172.21.15.89:47290) 2023-10-18T10:46:40.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.533+0000 7f9db37fe640 1 -- 172.21.15.89:0/3361022407 learned_addr learned my addr 172.21.15.89:0/3361022407 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:40.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.537+0000 7f9db2ffd640 1 --2- 172.21.15.89:0/3361022407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9db40d0ca0 0x7f9db415b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:40.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.537+0000 7f9db27fc640 1 --2- 172.21.15.89:0/3361022407 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9db40b76f0 0x7f9db40b7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:40.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.537+0000 7f9db37fe640 1 -- 172.21.15.89:0/3361022407 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9db40b76f0 msgr2=0x7f9db40b7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:40.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.537+0000 7f9db37fe640 1 --2- 172.21.15.89:0/3361022407 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9db40b76f0 0x7f9db40b7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:40.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.537+0000 7f9db37fe640 1 -- 172.21.15.89:0/3361022407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9db40d0ca0 msgr2=0x7f9db415b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:40.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.537+0000 7f9db37fe640 1 --2- 172.21.15.89:0/3361022407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9db40d0ca0 0x7f9db415b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:40.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.537+0000 7f9db37fe640 1 -- 172.21.15.89:0/3361022407 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da4009b30 con 0x7f9db40caea0 2023-10-18T10:46:40.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.537+0000 7f9db27fc640 1 --2- 172.21.15.89:0/3361022407 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9db40b76f0 0x7f9db40b7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:40.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.537+0000 7f9db37fe640 1 --2- 172.21.15.89:0/3361022407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9db40caea0 0x7f9db40cb320 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9dac00c3f0 tx=0x7f9dac00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:40.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.537+0000 7f9d9bfff640 1 -- 172.21.15.89:0/3361022407 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac011790 con 0x7f9db40caea0 2023-10-18T10:46:40.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.537+0000 7f9d9bfff640 1 -- 172.21.15.89:0/3361022407 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dac0118f0 con 0x7f9db40caea0 2023-10-18T10:46:40.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.537+0000 7f9d9bfff640 1 -- 172.21.15.89:0/3361022407 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac011ca0 con 0x7f9db40caea0 2023-10-18T10:46:40.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.537+0000 7f9db99ba640 1 -- 172.21.15.89:0/3361022407 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db40cb860 con 0x7f9db40caea0 2023-10-18T10:46:40.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.537+0000 7f9db99ba640 1 -- 172.21.15.89:0/3361022407 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9db4151ff0 con 0x7f9db40caea0 2023-10-18T10:46:40.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.537+0000 7f9db99ba640 1 -- 172.21.15.89:0/3361022407 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9db4152a90 con 0x7f9db40caea0 2023-10-18T10:46:40.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.537+0000 7f9d9bfff640 1 -- 172.21.15.89:0/3361022407 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f9dac01c460 con 0x7f9db40caea0 2023-10-18T10:46:40.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.537+0000 7f9d9bfff640 1 --2- 172.21.15.89:0/3361022407 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f9da00656b0 0x7f9da0067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:40.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.537+0000 7f9d9bfff640 1 -- 172.21.15.89:0/3361022407 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9dac082e60 con 0x7f9db40caea0 2023-10-18T10:46:40.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.537+0000 7f9db2ffd640 1 --2- 172.21.15.89:0/3361022407 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f9da00656b0 0x7f9da0067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:40.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.541+0000 7f9db2ffd640 1 --2- 172.21.15.89:0/3361022407 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f9da00656b0 0x7f9da0067b70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9da40ae000 tx=0x7f9da40a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:40.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.541+0000 7f9d9bfff640 1 -- 172.21.15.89:0/3361022407 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f9dac054110 con 0x7f9db40caea0 2023-10-18T10:46:40.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.692+0000 7f9db99ba640 1 -- 172.21.15.89:0/3361022407 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9db404dbe0 con 0x7f9db40caea0 2023-10-18T10:46:40.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.692+0000 7f9d9bfff640 1 -- 172.21.15.89:0/3361022407 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9dac003810 con 0x7f9db40caea0 2023-10-18T10:46:40.698 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-10-18T10:46:40.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.696+0000 7f9db99ba640 1 -- 172.21.15.89:0/3361022407 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f9da00656b0 msgr2=0x7f9da0067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:40.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.696+0000 7f9db99ba640 1 --2- 172.21.15.89:0/3361022407 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f9da00656b0 0x7f9da0067b70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9da40ae000 tx=0x7f9da40a7070 comp rx=0 tx=0).stop 2023-10-18T10:46:40.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.696+0000 7f9db99ba640 1 -- 172.21.15.89:0/3361022407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9db40caea0 msgr2=0x7f9db40cb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:40.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.696+0000 7f9db99ba640 1 --2- 172.21.15.89:0/3361022407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9db40caea0 0x7f9db40cb320 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9dac00c3f0 tx=0x7f9dac00c8b0 comp rx=0 tx=0).stop 2023-10-18T10:46:40.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.696+0000 7f9db99ba640 1 -- 172.21.15.89:0/3361022407 shutdown_connections 2023-10-18T10:46:40.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.696+0000 7f9db99ba640 1 --2- 172.21.15.89:0/3361022407 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f9da00656b0 0x7f9da0067b70 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:40.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.696+0000 7f9db99ba640 1 --2- 172.21.15.89:0/3361022407 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9db40b76f0 0x7f9db40b7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:40.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.696+0000 7f9db99ba640 1 --2- 172.21.15.89:0/3361022407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9db40caea0 0x7f9db40cb320 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:40.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.696+0000 7f9db99ba640 1 --2- 172.21.15.89:0/3361022407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9db40d0ca0 0x7f9db415b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:40.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.696+0000 7f9db99ba640 1 -- 172.21.15.89:0/3361022407 >> 172.21.15.89:0/3361022407 conn(0x7f9db4089fd0 msgr2=0x7f9db41600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:40.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.696+0000 7f9db99ba640 1 -- 172.21.15.89:0/3361022407 shutdown_connections 2023-10-18T10:46:40.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.696+0000 7f9db99ba640 1 -- 172.21.15.89:0/3361022407 wait complete. 2023-10-18T10:46:40.719 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:46:40.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.856+0000 7fe524e93640 1 Processor -- start 2023-10-18T10:46:40.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.856+0000 7fe524e93640 1 -- start start 2023-10-18T10:46:40.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.856+0000 7fe524e93640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5200cd4f0 0x7fe520143470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:40.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.856+0000 7fe524e93640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5200cd8d0 con 0x7fe5200ce070 2023-10-18T10:46:40.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.856+0000 7fe524e93640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe52015cbf0 con 0x7fe520145b70 2023-10-18T10:46:40.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.856+0000 7fe524e93640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe520069900 con 0x7fe5200cd4f0 2023-10-18T10:46:40.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.856+0000 7fe51ed9d640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe5200ce070 0x7fe52006abb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47976/0 (socket says 172.21.15.89:47976) 2023-10-18T10:46:40.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.856+0000 7fe51ed9d640 1 -- 172.21.15.89:0/3527821720 learned_addr learned my addr 172.21.15.89:0/3527821720 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:40.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.856+0000 7fe51e59c640 1 --2- 172.21.15.89:0/3527821720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5200cd4f0 0x7fe520143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:40.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.856+0000 7fe51dd9b640 1 -- 172.21.15.89:0/3527821720 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 230822988 0 0) 0x7fe5200cd8d0 con 0x7fe5200ce070 2023-10-18T10:46:40.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.856+0000 7fe51dd9b640 1 -- 172.21.15.89:0/3527821720 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe50c003530 con 0x7fe5200ce070 2023-10-18T10:46:40.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.856+0000 7fe51dd9b640 1 -- 172.21.15.89:0/3527821720 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3056566830 0 0) 0x7fe50c003530 con 0x7fe5200ce070 2023-10-18T10:46:40.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.856+0000 7fe51dd9b640 1 -- 172.21.15.89:0/3527821720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5200cd4f0 msgr2=0x7fe520143470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:40.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.856+0000 7fe51dd9b640 1 --2- 172.21.15.89:0/3527821720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5200cd4f0 0x7fe520143470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:40.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.856+0000 7fe51dd9b640 1 -- 172.21.15.89:0/3527821720 >> v1:172.21.15.154:6789/0 conn(0x7fe520145b70 legacy=0x7fe520160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:40.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.856+0000 7fe51dd9b640 1 -- 172.21.15.89:0/3527821720 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe520069b10 con 0x7fe5200ce070 2023-10-18T10:46:40.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.856+0000 7fe51dd9b640 1 -- 172.21.15.89:0/3527821720 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7fe518001a00 con 0x7fe5200ce070 2023-10-18T10:46:40.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.856+0000 7fe51e59c640 1 --2- 172.21.15.89:0/3527821720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5200cd4f0 0x7fe520143470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:46:40.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.856+0000 7fe51dd9b640 1 -- 172.21.15.89:0/3527821720 >> v1:172.21.15.89:6789/0 conn(0x7fe5200ce070 legacy=0x7fe52006abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:40.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.856+0000 7fe51dd9b640 1 --2- 172.21.15.89:0/3527821720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe50c0039b0 0x7fe50c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:40.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.856+0000 7fe51dd9b640 1 --2- 172.21.15.89:0/3527821720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5200cd4f0 0x7fe50c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:40.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.856+0000 7fe51dd9b640 1 --2- 172.21.15.89:0/3527821720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe50c004d10 0x7fe50c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:40.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.856+0000 7fe51dd9b640 1 -- 172.21.15.89:0/3527821720 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe520069900 con 0x7fe50c004d10 2023-10-18T10:46:40.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.856+0000 7fe51dd9b640 1 -- 172.21.15.89:0/3527821720 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50c00b990 con 0x7fe50c0039b0 2023-10-18T10:46:40.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.856+0000 7fe51dd9b640 1 -- 172.21.15.89:0/3527821720 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe50c00bb00 con 0x7fe5200cd4f0 2023-10-18T10:46:40.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.856+0000 7fe51ed9d640 1 --2- 172.21.15.89:0/3527821720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe50c004d10 0x7fe50c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:40.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.856+0000 7fe51e59c640 1 --2- 172.21.15.89:0/3527821720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe50c0039b0 0x7fe50c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:40.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.856+0000 7fe517fff640 1 --2- 172.21.15.89:0/3527821720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5200cd4f0 0x7fe50c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:40.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.860+0000 7fe51ed9d640 1 -- 172.21.15.89:0/3527821720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5200cd4f0 msgr2=0x7fe50c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:40.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.860+0000 7fe51ed9d640 1 --2- 172.21.15.89:0/3527821720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5200cd4f0 0x7fe50c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:40.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.860+0000 7fe51ed9d640 1 -- 172.21.15.89:0/3527821720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe50c0039b0 msgr2=0x7fe50c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:40.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.860+0000 7fe51ed9d640 1 --2- 172.21.15.89:0/3527821720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe50c0039b0 0x7fe50c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:40.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.860+0000 7fe51ed9d640 1 -- 172.21.15.89:0/3527821720 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe510009420 con 0x7fe50c004d10 2023-10-18T10:46:40.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.860+0000 7fe517fff640 1 --2- 172.21.15.89:0/3527821720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5200cd4f0 0x7fe50c0047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:40.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.860+0000 7fe51ed9d640 1 --2- 172.21.15.89:0/3527821720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe50c004d10 0x7fe50c005190 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe51800a830 tx=0x7fe51801faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead68d1aac3d1c2e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:40.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.860+0000 7fe51dd9b640 1 -- 172.21.15.89:0/3527821720 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5180aa6d0 con 0x7fe50c004d10 2023-10-18T10:46:40.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.860+0000 7fe51dd9b640 1 -- 172.21.15.89:0/3527821720 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5180aa850 con 0x7fe50c004d10 2023-10-18T10:46:40.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.860+0000 7fe51dd9b640 1 -- 172.21.15.89:0/3527821720 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5180aac00 con 0x7fe50c004d10 2023-10-18T10:46:40.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.860+0000 7fe524e93640 1 -- 172.21.15.89:0/3527821720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe50c004d10 msgr2=0x7fe50c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:40.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.860+0000 7fe524e93640 1 --2- 172.21.15.89:0/3527821720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe50c004d10 0x7fe50c005190 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe51800a830 tx=0x7fe51801faa0 comp rx=0 tx=0).stop 2023-10-18T10:46:40.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.860+0000 7fe51ed9d640 1 -- 172.21.15.89:0/3527821720 reap_dead start 2023-10-18T10:46:40.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.860+0000 7fe524e93640 1 -- 172.21.15.89:0/3527821720 shutdown_connections 2023-10-18T10:46:40.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.860+0000 7fe524e93640 1 -- 172.21.15.89:0/3527821720 >> 172.21.15.89:0/3527821720 conn(0x7fe52006dad0 msgr2=0x7fe52006dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:40.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.860+0000 7fe524e93640 1 -- 172.21.15.89:0/3527821720 shutdown_connections 2023-10-18T10:46:40.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.860+0000 7fe524e93640 1 -- 172.21.15.89:0/3527821720 wait complete. 2023-10-18T10:46:40.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.864+0000 7fe524e93640 1 Processor -- start 2023-10-18T10:46:40.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.864+0000 7fe524e93640 1 -- start start 2023-10-18T10:46:40.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.864+0000 7fe524e93640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe50c004d10 0x7fe5200c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:40.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.864+0000 7fe524e93640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5200c64f0 0x7fe52015a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:40.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.864+0000 7fe524e93640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe52015a750 0x7fe52015abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:40.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.864+0000 7fe524e93640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5200c6fc0 con 0x7fe50c004d10 2023-10-18T10:46:40.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.864+0000 7fe524e93640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe520157250 con 0x7fe52015a750 2023-10-18T10:46:40.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.864+0000 7fe524e93640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5200c6a00 con 0x7fe5200c64f0 2023-10-18T10:46:40.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.864+0000 7fe51e59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe50c004d10 0x7fe5200c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:40.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.864+0000 7fe51e59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe50c004d10 0x7fe5200c5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47302/0 (socket says 172.21.15.89:47302) 2023-10-18T10:46:40.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.864+0000 7fe51e59c640 1 -- 172.21.15.89:0/258600688 learned_addr learned my addr 172.21.15.89:0/258600688 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:40.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.864+0000 7fe51ed9d640 1 --2- 172.21.15.89:0/258600688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe52015a750 0x7fe52015abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:40.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.864+0000 7fe517fff640 1 --2- 172.21.15.89:0/258600688 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5200c64f0 0x7fe52015a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:40.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.864+0000 7fe51e59c640 1 -- 172.21.15.89:0/258600688 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5200c64f0 msgr2=0x7fe52015a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:40.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.864+0000 7fe51e59c640 1 --2- 172.21.15.89:0/258600688 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5200c64f0 0x7fe52015a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:40.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.864+0000 7fe51e59c640 1 -- 172.21.15.89:0/258600688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe52015a750 msgr2=0x7fe52015abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:40.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.864+0000 7fe51e59c640 1 --2- 172.21.15.89:0/258600688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe52015a750 0x7fe52015abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:40.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.864+0000 7fe51e59c640 1 -- 172.21.15.89:0/258600688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5200c6ce0 con 0x7fe50c004d10 2023-10-18T10:46:40.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.864+0000 7fe51e59c640 1 --2- 172.21.15.89:0/258600688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe50c004d10 0x7fe5200c5fb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe510011cd0 tx=0x7fe51000f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:40.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.864+0000 7fe516ffd640 1 -- 172.21.15.89:0/258600688 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51000fd20 con 0x7fe50c004d10 2023-10-18T10:46:40.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.868+0000 7fe516ffd640 1 -- 172.21.15.89:0/258600688 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51000fe80 con 0x7fe50c004d10 2023-10-18T10:46:40.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.868+0000 7fe524e93640 1 -- 172.21.15.89:0/258600688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5200c7730 con 0x7fe50c004d10 2023-10-18T10:46:40.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.868+0000 7fe516ffd640 1 -- 172.21.15.89:0/258600688 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe510004240 con 0x7fe50c004d10 2023-10-18T10:46:40.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.868+0000 7fe524e93640 1 -- 172.21.15.89:0/258600688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5201e6ad0 con 0x7fe50c004d10 2023-10-18T10:46:40.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.868+0000 7fe524e93640 1 -- 172.21.15.89:0/258600688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5200c7550 con 0x7fe50c004d10 2023-10-18T10:46:40.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.868+0000 7fe516ffd640 1 -- 172.21.15.89:0/258600688 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fe5100043a0 con 0x7fe50c004d10 2023-10-18T10:46:40.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.868+0000 7fe516ffd640 1 --2- 172.21.15.89:0/258600688 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe5000656b0 0x7fe500067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:40.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.868+0000 7fe516ffd640 1 -- 172.21.15.89:0/258600688 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe510083820 con 0x7fe50c004d10 2023-10-18T10:46:40.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.868+0000 7fe517fff640 1 --2- 172.21.15.89:0/258600688 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe5000656b0 0x7fe500067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:40.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.872+0000 7fe517fff640 1 --2- 172.21.15.89:0/258600688 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe5000656b0 0x7fe500067b70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe5200c6930 tx=0x7fe508009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:40.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:40.872+0000 7fe516ffd640 1 -- 172.21.15.89:0/258600688 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fe510050f10 con 0x7fe50c004d10 2023-10-18T10:46:41.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.020+0000 7fe524e93640 1 -- 172.21.15.89:0/258600688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe52004dbe0 con 0x7fe50c004d10 2023-10-18T10:46:41.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.020+0000 7fe516ffd640 1 -- 172.21.15.89:0/258600688 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe510054530 con 0x7fe50c004d10 2023-10-18T10:46:41.026 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-18T10:46:41.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.024+0000 7fe524e93640 1 -- 172.21.15.89:0/258600688 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe5000656b0 msgr2=0x7fe500067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:41.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.024+0000 7fe524e93640 1 --2- 172.21.15.89:0/258600688 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe5000656b0 0x7fe500067b70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe5200c6930 tx=0x7fe508009040 comp rx=0 tx=0).stop 2023-10-18T10:46:41.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.024+0000 7fe524e93640 1 -- 172.21.15.89:0/258600688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe50c004d10 msgr2=0x7fe5200c5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:41.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.024+0000 7fe524e93640 1 --2- 172.21.15.89:0/258600688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe50c004d10 0x7fe5200c5fb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe510011cd0 tx=0x7fe51000f610 comp rx=0 tx=0).stop 2023-10-18T10:46:41.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.024+0000 7fe524e93640 1 -- 172.21.15.89:0/258600688 shutdown_connections 2023-10-18T10:46:41.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.024+0000 7fe524e93640 1 --2- 172.21.15.89:0/258600688 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe5000656b0 0x7fe500067b70 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:41.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.024+0000 7fe524e93640 1 --2- 172.21.15.89:0/258600688 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5200c64f0 0x7fe52015a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:41.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.024+0000 7fe524e93640 1 --2- 172.21.15.89:0/258600688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe52015a750 0x7fe52015abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:41.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.024+0000 7fe524e93640 1 --2- 172.21.15.89:0/258600688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe50c004d10 0x7fe5200c5fb0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:41.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.024+0000 7fe524e93640 1 -- 172.21.15.89:0/258600688 >> 172.21.15.89:0/258600688 conn(0x7fe52006dad0 msgr2=0x7fe5200d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:41.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.024+0000 7fe524e93640 1 -- 172.21.15.89:0/258600688 shutdown_connections 2023-10-18T10:46:41.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.024+0000 7fe524e93640 1 -- 172.21.15.89:0/258600688 wait complete. 2023-10-18T10:46:41.047 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:46:41.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.184+0000 7f34cf3e8640 1 Processor -- start 2023-10-18T10:46:41.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.184+0000 7f34cf3e8640 1 -- start start 2023-10-18T10:46:41.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.184+0000 7f34cf3e8640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f34c80c9070 0x7f34c815e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:41.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.184+0000 7f34cf3e8640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34c815acb0 con 0x7f34c806ff00 2023-10-18T10:46:41.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.184+0000 7f34cf3e8640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34c80702e0 con 0x7f34c80ca340 2023-10-18T10:46:41.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.184+0000 7f34cf3e8640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34c815ef30 con 0x7f34c80c9070 2023-10-18T10:46:41.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.184+0000 7f34cd184640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f34c806ff00 0x7f34c815b140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47984/0 (socket says 172.21.15.89:47984) 2023-10-18T10:46:41.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.184+0000 7f34cd184640 1 -- 172.21.15.89:0/2107829030 learned_addr learned my addr 172.21.15.89:0/2107829030 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:41.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.184+0000 7f34cd985640 1 --2- 172.21.15.89:0/2107829030 >> v2:172.21.15.89:3300/0 conn(0x7f34c80c9070 0x7f34c815e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:41.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.184+0000 7f34cd985640 1 -- 172.21.15.89:0/2107829030 >> v1:172.21.15.154:6789/0 conn(0x7f34c80ca340 legacy=0x7f34c80c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:41.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.184+0000 7f34cd985640 1 -- 172.21.15.89:0/2107829030 >> v1:172.21.15.89:6789/0 conn(0x7f34c806ff00 legacy=0x7f34c815b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:41.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.184+0000 7f34bffff640 1 -- 172.21.15.89:0/2107829030 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 223522159 0 0) 0x7f34c815acb0 con 0x7f34c806ff00 2023-10-18T10:46:41.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.184+0000 7f34cd985640 1 -- 172.21.15.89:0/2107829030 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c815b850 con 0x7f34c80c9070 2023-10-18T10:46:41.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.188+0000 7f34cd985640 1 --2- 172.21.15.89:0/2107829030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34c80c9070 0x7f34c815e000 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f34c0009d90 tx=0x7f34c001c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6248eea9bb6bb6e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:41.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.188+0000 7f34bffff640 1 -- 172.21.15.89:0/2107829030 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c00a5830 con 0x7f34c80c9070 2023-10-18T10:46:41.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.188+0000 7f34bffff640 1 -- 172.21.15.89:0/2107829030 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c00a5990 con 0x7f34c80c9070 2023-10-18T10:46:41.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.188+0000 7f34bffff640 1 -- 172.21.15.89:0/2107829030 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c00a5d20 con 0x7f34c80c9070 2023-10-18T10:46:41.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.188+0000 7f34cf3e8640 1 -- 172.21.15.89:0/2107829030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34c80c9070 msgr2=0x7f34c815e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:41.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.188+0000 7f34cf3e8640 1 --2- 172.21.15.89:0/2107829030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34c80c9070 0x7f34c815e000 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f34c0009d90 tx=0x7f34c001c880 comp rx=0 tx=0).stop 2023-10-18T10:46:41.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.188+0000 7f34cf3e8640 1 -- 172.21.15.89:0/2107829030 shutdown_connections 2023-10-18T10:46:41.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.188+0000 7f34cf3e8640 1 --2- 172.21.15.89:0/2107829030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34c80c9070 0x7f34c815e000 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:41.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.188+0000 7f34cf3e8640 1 -- 172.21.15.89:0/2107829030 >> 172.21.15.89:0/2107829030 conn(0x7f34c8069f00 msgr2=0x7f34c80cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:41.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.188+0000 7f34cf3e8640 1 -- 172.21.15.89:0/2107829030 shutdown_connections 2023-10-18T10:46:41.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.188+0000 7f34cf3e8640 1 -- 172.21.15.89:0/2107829030 wait complete. 2023-10-18T10:46:41.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.192+0000 7f34cf3e8640 1 Processor -- start 2023-10-18T10:46:41.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.192+0000 7f34cf3e8640 1 -- start start 2023-10-18T10:46:41.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.192+0000 7f34cf3e8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34c806ff00 0x7f34c814f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:41.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.192+0000 7f34cf3e8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34c80ca340 0x7f34c814eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:41.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.192+0000 7f34cf3e8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34c80b3600 0x7f34c80c2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:41.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.192+0000 7f34cf3e8640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c814f070 con 0x7f34c806ff00 2023-10-18T10:46:41.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.192+0000 7f34cf3e8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c814fae0 con 0x7f34c80b3600 2023-10-18T10:46:41.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.192+0000 7f34cf3e8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34c80b3d50 con 0x7f34c80ca340 2023-10-18T10:46:41.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.192+0000 7f34cd184640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34c806ff00 0x7f34c814f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:41.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.192+0000 7f34cd184640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34c806ff00 0x7f34c814f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47308/0 (socket says 172.21.15.89:47308) 2023-10-18T10:46:41.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.192+0000 7f34cd184640 1 -- 172.21.15.89:0/168316328 learned_addr learned my addr 172.21.15.89:0/168316328 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:41.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.192+0000 7f34cd985640 1 --2- 172.21.15.89:0/168316328 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34c80b3600 0x7f34c80c2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:41.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.192+0000 7f34cd184640 1 -- 172.21.15.89:0/168316328 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34c80ca340 msgr2=0x7f34c814eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:41.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.192+0000 7f34cd184640 1 --2- 172.21.15.89:0/168316328 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34c80ca340 0x7f34c814eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:41.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.192+0000 7f34cd184640 1 -- 172.21.15.89:0/168316328 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34c80b3600 msgr2=0x7f34c80c2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:41.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.192+0000 7f34cd184640 1 --2- 172.21.15.89:0/168316328 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34c80b3600 0x7f34c80c2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:41.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.192+0000 7f34cd184640 1 -- 172.21.15.89:0/168316328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c0009a70 con 0x7f34c806ff00 2023-10-18T10:46:41.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.192+0000 7f34cd184640 1 --2- 172.21.15.89:0/168316328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34c806ff00 0x7f34c814f1d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f34c400c520 tx=0x7f34c400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:41.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.192+0000 7f34bdffb640 1 -- 172.21.15.89:0/168316328 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c40074e0 con 0x7f34c806ff00 2023-10-18T10:46:41.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.192+0000 7f34bdffb640 1 -- 172.21.15.89:0/168316328 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c4007640 con 0x7f34c806ff00 2023-10-18T10:46:41.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.192+0000 7f34cf3e8640 1 -- 172.21.15.89:0/168316328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34c80b3a80 con 0x7f34c806ff00 2023-10-18T10:46:41.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.192+0000 7f34bdffb640 1 -- 172.21.15.89:0/168316328 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c40049f0 con 0x7f34c806ff00 2023-10-18T10:46:41.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.192+0000 7f34cf3e8640 1 -- 172.21.15.89:0/168316328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34c80b2320 con 0x7f34c806ff00 2023-10-18T10:46:41.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.192+0000 7f34cf3e8640 1 -- 172.21.15.89:0/168316328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34c804dbe0 con 0x7f34c806ff00 2023-10-18T10:46:41.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.196+0000 7f34bdffb640 1 -- 172.21.15.89:0/168316328 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f34c4004b50 con 0x7f34c806ff00 2023-10-18T10:46:41.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.196+0000 7f34bdffb640 1 --2- 172.21.15.89:0/168316328 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f34ac065680 0x7f34ac067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:41.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.196+0000 7f34bdffb640 1 -- 172.21.15.89:0/168316328 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f34c4082810 con 0x7f34c806ff00 2023-10-18T10:46:41.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.196+0000 7f34cc983640 1 --2- 172.21.15.89:0/168316328 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f34ac065680 0x7f34ac067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:41.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.196+0000 7f34cc983640 1 --2- 172.21.15.89:0/168316328 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f34ac065680 0x7f34ac067b40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f34c80b0930 tx=0x7f34b8006bd0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:41.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.200+0000 7f34bdffb640 1 -- 172.21.15.89:0/168316328 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f34c404ff00 con 0x7f34c806ff00 2023-10-18T10:46:41.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.348+0000 7f34cf3e8640 1 -- 172.21.15.89:0/168316328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f34c80cb820 con 0x7f34c806ff00 2023-10-18T10:46:41.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.348+0000 7f34bdffb640 1 -- 172.21.15.89:0/168316328 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f34c4018020 con 0x7f34c806ff00 2023-10-18T10:46:41.352 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-18T10:46:41.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.348+0000 7f34cf3e8640 1 -- 172.21.15.89:0/168316328 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f34ac065680 msgr2=0x7f34ac067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:41.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.348+0000 7f34cf3e8640 1 --2- 172.21.15.89:0/168316328 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f34ac065680 0x7f34ac067b40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f34c80b0930 tx=0x7f34b8006bd0 comp rx=0 tx=0).stop 2023-10-18T10:46:41.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.348+0000 7f34cf3e8640 1 -- 172.21.15.89:0/168316328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34c806ff00 msgr2=0x7f34c814f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:41.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.348+0000 7f34cf3e8640 1 --2- 172.21.15.89:0/168316328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34c806ff00 0x7f34c814f1d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f34c400c520 tx=0x7f34c400c9e0 comp rx=0 tx=0).stop 2023-10-18T10:46:41.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.348+0000 7f34cf3e8640 1 -- 172.21.15.89:0/168316328 shutdown_connections 2023-10-18T10:46:41.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.348+0000 7f34cf3e8640 1 --2- 172.21.15.89:0/168316328 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f34ac065680 0x7f34ac067b40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:41.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.348+0000 7f34cf3e8640 1 --2- 172.21.15.89:0/168316328 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34c80ca340 0x7f34c814eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:41.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.348+0000 7f34cf3e8640 1 --2- 172.21.15.89:0/168316328 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34c80b3600 0x7f34c80c2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:41.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.348+0000 7f34cf3e8640 1 --2- 172.21.15.89:0/168316328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34c806ff00 0x7f34c814f1d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:41.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.348+0000 7f34cf3e8640 1 -- 172.21.15.89:0/168316328 >> 172.21.15.89:0/168316328 conn(0x7f34c8069f00 msgr2=0x7f34c80ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:41.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.348+0000 7f34cf3e8640 1 -- 172.21.15.89:0/168316328 shutdown_connections 2023-10-18T10:46:41.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.348+0000 7f34cf3e8640 1 -- 172.21.15.89:0/168316328 wait complete. 2023-10-18T10:46:41.373 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T10:46:41.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.508+0000 7fdb058cb640 1 Processor -- start 2023-10-18T10:46:41.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.508+0000 7fdb058cb640 1 -- start start 2023-10-18T10:46:41.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.508+0000 7fdb058cb640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdb000d0760 0x7fdb000d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:41.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.508+0000 7fdb058cb640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb000d1100 con 0x7fdb000cd160 2023-10-18T10:46:41.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.508+0000 7fdb058cb640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb0015da50 con 0x7fdb0015c940 2023-10-18T10:46:41.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.508+0000 7fdb058cb640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb000d3110 con 0x7fdb000d0760 2023-10-18T10:46:41.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.508+0000 7fdaf67fc640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdb000d0760 0x7fdb000d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:41.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.508+0000 7fdaf67fc640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdb000d0760 0x7fdb000d0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47318/0 (socket says 172.21.15.89:47318) 2023-10-18T10:46:41.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.508+0000 7fdaf67fc640 1 -- 172.21.15.89:0/689730252 learned_addr learned my addr 172.21.15.89:0/689730252 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:41.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.508+0000 7fdaff7fe640 1 --1- 172.21.15.89:0/689730252 >> v1:172.21.15.89:6789/0 conn(0x7fdb000cd160 0x7fdb000cd5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47990/0 (socket says 172.21.15.89:47990) 2023-10-18T10:46:41.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.508+0000 7fdaf67fc640 1 -- 172.21.15.89:0/689730252 >> v1:172.21.15.154:6789/0 conn(0x7fdb0015c940 legacy=0x7fdb000d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:41.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.508+0000 7fdaf67fc640 1 -- 172.21.15.89:0/689730252 >> v1:172.21.15.89:6789/0 conn(0x7fdb000cd160 legacy=0x7fdb000cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:41.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.508+0000 7fdaf67fc640 1 -- 172.21.15.89:0/689730252 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0015ce40 con 0x7fdb000d0760 2023-10-18T10:46:41.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.512+0000 7fdaf67fc640 1 --2- 172.21.15.89:0/689730252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb000d0760 0x7fdb000d0bc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009e50 tx=0x7fdae801c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f18864a89d09ad3 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:41.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.512+0000 7fdafe7fc640 1 -- 172.21.15.89:0/689730252 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae80a58a0 con 0x7fdb000d0760 2023-10-18T10:46:41.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.512+0000 7fdafe7fc640 1 -- 172.21.15.89:0/689730252 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae80a5a00 con 0x7fdb000d0760 2023-10-18T10:46:41.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.512+0000 7fdafe7fc640 1 -- 172.21.15.89:0/689730252 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae80a5d90 con 0x7fdb000d0760 2023-10-18T10:46:41.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.512+0000 7fdb058cb640 1 -- 172.21.15.89:0/689730252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb000d0760 msgr2=0x7fdb000d0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:41.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.512+0000 7fdb058cb640 1 --2- 172.21.15.89:0/689730252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb000d0760 0x7fdb000d0bc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009e50 tx=0x7fdae801c9e0 comp rx=0 tx=0).stop 2023-10-18T10:46:41.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.512+0000 7fdb058cb640 1 -- 172.21.15.89:0/689730252 shutdown_connections 2023-10-18T10:46:41.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.512+0000 7fdb058cb640 1 --2- 172.21.15.89:0/689730252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb000d0760 0x7fdb000d0bc0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:41.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.512+0000 7fdb058cb640 1 -- 172.21.15.89:0/689730252 >> 172.21.15.89:0/689730252 conn(0x7fdb00089fd0 msgr2=0x7fdb0009b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:41.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.512+0000 7fdb058cb640 1 -- 172.21.15.89:0/689730252 shutdown_connections 2023-10-18T10:46:41.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.512+0000 7fdb058cb640 1 -- 172.21.15.89:0/689730252 wait complete. 2023-10-18T10:46:41.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.516+0000 7fdb058cb640 1 Processor -- start 2023-10-18T10:46:41.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.516+0000 7fdb058cb640 1 -- start start 2023-10-18T10:46:41.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.516+0000 7fdb058cb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb000cd160 0x7fdb000c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:41.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.516+0000 7fdb058cb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb000d0760 0x7fdb0013ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:41.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.516+0000 7fdb058cb640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb000b54d0 0x7fdb00151030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:41.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.516+0000 7fdb058cb640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb000ca5d0 con 0x7fdb000b54d0 2023-10-18T10:46:41.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.516+0000 7fdb058cb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb000b7400 con 0x7fdb000d0760 2023-10-18T10:46:41.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.516+0000 7fdb058cb640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb000b76e0 con 0x7fdb000cd160 2023-10-18T10:46:41.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.516+0000 7fdaff7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb000b54d0 0x7fdb00151030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:41.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.516+0000 7fdaff7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb000b54d0 0x7fdb00151030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47330/0 (socket says 172.21.15.89:47330) 2023-10-18T10:46:41.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.516+0000 7fdaff7fe640 1 -- 172.21.15.89:0/4200567412 learned_addr learned my addr 172.21.15.89:0/4200567412 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:41.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.516+0000 7fdaf67fc640 1 --2- 172.21.15.89:0/4200567412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb000d0760 0x7fdb0013ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:41.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.516+0000 7fdafeffd640 1 --2- 172.21.15.89:0/4200567412 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb000cd160 0x7fdb000c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:41.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.516+0000 7fdaff7fe640 1 -- 172.21.15.89:0/4200567412 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb000cd160 msgr2=0x7fdb000c9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:41.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.516+0000 7fdaff7fe640 1 --2- 172.21.15.89:0/4200567412 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb000cd160 0x7fdb000c9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:41.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.516+0000 7fdaff7fe640 1 -- 172.21.15.89:0/4200567412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb000d0760 msgr2=0x7fdb0013ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:41.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.516+0000 7fdaff7fe640 1 --2- 172.21.15.89:0/4200567412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb000d0760 0x7fdb0013ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:41.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.516+0000 7fdaff7fe640 1 -- 172.21.15.89:0/4200567412 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae8009b30 con 0x7fdb000b54d0 2023-10-18T10:46:41.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.516+0000 7fdaf67fc640 1 --2- 172.21.15.89:0/4200567412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb000d0760 0x7fdb0013ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:41.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.516+0000 7fdaff7fe640 1 --2- 172.21.15.89:0/4200567412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb000b54d0 0x7fdb00151030 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdaf800c410 tx=0x7fdaf800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:41.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.516+0000 7fdafcff9640 1 -- 172.21.15.89:0/4200567412 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf80074e0 con 0x7fdb000b54d0 2023-10-18T10:46:41.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.516+0000 7fdafcff9640 1 -- 172.21.15.89:0/4200567412 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf8007640 con 0x7fdb000b54d0 2023-10-18T10:46:41.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.516+0000 7fdb058cb640 1 -- 172.21.15.89:0/4200567412 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb000b79c0 con 0x7fdb000b54d0 2023-10-18T10:46:41.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.516+0000 7fdafcff9640 1 -- 172.21.15.89:0/4200567412 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8004a10 con 0x7fdb000b54d0 2023-10-18T10:46:41.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.516+0000 7fdb058cb640 1 -- 172.21.15.89:0/4200567412 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb000b81c0 con 0x7fdb000b54d0 2023-10-18T10:46:41.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.520+0000 7fdb058cb640 1 -- 172.21.15.89:0/4200567412 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb000b83b0 con 0x7fdb000b54d0 2023-10-18T10:46:41.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.520+0000 7fdafcff9640 1 -- 172.21.15.89:0/4200567412 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fdaf8004b70 con 0x7fdb000b54d0 2023-10-18T10:46:41.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.520+0000 7fdafcff9640 1 --2- 172.21.15.89:0/4200567412 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdae0065680 0x7fdae0067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:41.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.520+0000 7fdafcff9640 1 -- 172.21.15.89:0/4200567412 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fdaf8082c20 con 0x7fdb000b54d0 2023-10-18T10:46:41.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.520+0000 7fdafeffd640 1 --2- 172.21.15.89:0/4200567412 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdae0065680 0x7fdae0067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:41.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.520+0000 7fdafeffd640 1 --2- 172.21.15.89:0/4200567412 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdae0065680 0x7fdae0067b40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0000f70 tx=0x7fdaf0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:41.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.524+0000 7fdafcff9640 1 -- 172.21.15.89:0/4200567412 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fdaf8050310 con 0x7fdb000b54d0 2023-10-18T10:46:41.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.672+0000 7fdb058cb640 1 -- 172.21.15.89:0/4200567412 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdb0004dbe0 con 0x7fdb000b54d0 2023-10-18T10:46:41.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.672+0000 7fdafcff9640 1 -- 172.21.15.89:0/4200567412 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdaf8053930 con 0x7fdb000b54d0 2023-10-18T10:46:41.676 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-18T10:46:41.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.672+0000 7fdb058cb640 1 -- 172.21.15.89:0/4200567412 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdae0065680 msgr2=0x7fdae0067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:41.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.672+0000 7fdb058cb640 1 --2- 172.21.15.89:0/4200567412 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdae0065680 0x7fdae0067b40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0000f70 tx=0x7fdaf0009040 comp rx=0 tx=0).stop 2023-10-18T10:46:41.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.672+0000 7fdb058cb640 1 -- 172.21.15.89:0/4200567412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb000b54d0 msgr2=0x7fdb00151030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:41.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.672+0000 7fdb058cb640 1 --2- 172.21.15.89:0/4200567412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb000b54d0 0x7fdb00151030 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdaf800c410 tx=0x7fdaf800c8d0 comp rx=0 tx=0).stop 2023-10-18T10:46:41.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.672+0000 7fdb058cb640 1 -- 172.21.15.89:0/4200567412 shutdown_connections 2023-10-18T10:46:41.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.672+0000 7fdb058cb640 1 --2- 172.21.15.89:0/4200567412 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdae0065680 0x7fdae0067b40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:41.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.672+0000 7fdb058cb640 1 --2- 172.21.15.89:0/4200567412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb000b54d0 0x7fdb00151030 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:41.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.672+0000 7fdb058cb640 1 --2- 172.21.15.89:0/4200567412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb000d0760 0x7fdb0013ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:41.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.672+0000 7fdb058cb640 1 --2- 172.21.15.89:0/4200567412 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdb000cd160 0x7fdb000c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:41.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.672+0000 7fdb058cb640 1 -- 172.21.15.89:0/4200567412 >> 172.21.15.89:0/4200567412 conn(0x7fdb00089fd0 msgr2=0x7fdb000b3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:41.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.672+0000 7fdb058cb640 1 -- 172.21.15.89:0/4200567412 shutdown_connections 2023-10-18T10:46:41.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.672+0000 7fdb058cb640 1 -- 172.21.15.89:0/4200567412 wait complete. 2023-10-18T10:46:41.703 INFO:tasks.cephfs_test_runner:Starting test: test_network_death (tasks.cephfs.test_client_recovery.TestClientNetworkRecovery) Simulate software freeze or temporary network failure. 2023-10-18T10:46:41.704 INFO:tasks.cephfs_test_runner:test_network_death (tasks.cephfs.test_client_recovery.TestClientNetworkRecovery) 2023-10-18T10:46:41.704 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_recovery.TestClientNetworkRecovery.test_network_death' 2023-10-18T10:46:41.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.836+0000 7f45e0cbd640 1 Processor -- start 2023-10-18T10:46:41.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.836+0000 7f45e0cbd640 1 -- start start 2023-10-18T10:46:41.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.836+0000 7f45e0cbd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f45dc0cd4f0 0x7f45dc143470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:41.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.840+0000 7f45e0cbd640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45dc0cd8d0 con 0x7f45dc0ce070 2023-10-18T10:46:41.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.840+0000 7f45e0cbd640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45dc15cbf0 con 0x7f45dc145b70 2023-10-18T10:46:41.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.840+0000 7f45e0cbd640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45dc069900 con 0x7f45dc0cd4f0 2023-10-18T10:46:41.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.840+0000 7f45dad9d640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f45dc0ce070 0x7f45dc06abb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48000/0 (socket says 172.21.15.89:48000) 2023-10-18T10:46:41.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.840+0000 7f45dad9d640 1 -- 172.21.15.89:0/1161318397 learned_addr learned my addr 172.21.15.89:0/1161318397 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:41.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.840+0000 7f45da59c640 1 --2- 172.21.15.89:0/1161318397 >> v2:172.21.15.154:3300/0 conn(0x7f45dc0cd4f0 0x7f45dc143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:41.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.840+0000 7f45d959a640 1 -- 172.21.15.89:0/1161318397 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 40774368 0 0) 0x7f45dc0cd8d0 con 0x7f45dc0ce070 2023-10-18T10:46:41.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.840+0000 7f45d959a640 1 -- 172.21.15.89:0/1161318397 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45c4003530 con 0x7f45dc0ce070 2023-10-18T10:46:41.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.840+0000 7f45d959a640 1 -- 172.21.15.89:0/1161318397 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3404677482 0 0) 0x7f45dc15cbf0 con 0x7f45dc145b70 2023-10-18T10:46:41.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.840+0000 7f45d959a640 1 -- 172.21.15.89:0/1161318397 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45dc0cd8d0 con 0x7f45dc145b70 2023-10-18T10:46:41.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.840+0000 7f45d959a640 1 -- 172.21.15.89:0/1161318397 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 698662785 0 0) 0x7f45c4003530 con 0x7f45dc0ce070 2023-10-18T10:46:41.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.840+0000 7f45d959a640 1 -- 172.21.15.89:0/1161318397 >> v2:172.21.15.154:3300/0 conn(0x7f45dc0cd4f0 msgr2=0x7f45dc143470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:41.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.840+0000 7f45d959a640 1 --2- 172.21.15.89:0/1161318397 >> v2:172.21.15.154:3300/0 conn(0x7f45dc0cd4f0 0x7f45dc143470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:41.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.840+0000 7f45d959a640 1 -- 172.21.15.89:0/1161318397 >> v1:172.21.15.154:6789/0 conn(0x7f45dc145b70 legacy=0x7f45dc160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:41.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.840+0000 7f45d959a640 1 -- 172.21.15.89:0/1161318397 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45dc069b10 con 0x7f45dc0ce070 2023-10-18T10:46:41.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.840+0000 7f45d959a640 1 -- 172.21.15.89:0/1161318397 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7f45d40016a0 con 0x7f45dc0ce070 2023-10-18T10:46:41.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.840+0000 7f45d959a640 1 -- 172.21.15.89:0/1161318397 >> v1:172.21.15.89:6789/0 conn(0x7f45dc0ce070 legacy=0x7f45dc06abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:41.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.840+0000 7f45d959a640 1 --2- 172.21.15.89:0/1161318397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45c4003b10 0x7f45c4003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:41.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.840+0000 7f45d959a640 1 --2- 172.21.15.89:0/1161318397 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45c40047d0 0x7f45c4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:41.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.840+0000 7f45d959a640 1 --2- 172.21.15.89:0/1161318397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45c4005190 0x7f45c400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:41.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.840+0000 7f45d959a640 1 -- 172.21.15.89:0/1161318397 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc069900 con 0x7f45c4005190 2023-10-18T10:46:41.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.840+0000 7f45d959a640 1 -- 172.21.15.89:0/1161318397 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c400c010 con 0x7f45c4003b10 2023-10-18T10:46:41.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.840+0000 7f45d959a640 1 -- 172.21.15.89:0/1161318397 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c400c1d0 con 0x7f45c40047d0 2023-10-18T10:46:41.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.840+0000 7f45dad9d640 1 --2- 172.21.15.89:0/1161318397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45c4005190 0x7f45c400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:41.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.840+0000 7f45da59c640 1 --2- 172.21.15.89:0/1161318397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45c4003b10 0x7f45c4003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:41.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.840+0000 7f45d9d9b640 1 --2- 172.21.15.89:0/1161318397 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45c40047d0 0x7f45c4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:41.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.840+0000 7f45dad9d640 1 -- 172.21.15.89:0/1161318397 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45c40047d0 msgr2=0x7f45c4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:41.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.840+0000 7f45dad9d640 1 --2- 172.21.15.89:0/1161318397 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45c40047d0 0x7f45c4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:41.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.840+0000 7f45dad9d640 1 -- 172.21.15.89:0/1161318397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45c4003b10 msgr2=0x7f45c4003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:41.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.840+0000 7f45dad9d640 1 --2- 172.21.15.89:0/1161318397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45c4003b10 0x7f45c4003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:41.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.840+0000 7f45dad9d640 1 -- 172.21.15.89:0/1161318397 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c4003530 con 0x7f45c4005190 2023-10-18T10:46:41.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.840+0000 7f45d9d9b640 1 --2- 172.21.15.89:0/1161318397 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45c40047d0 0x7f45c4004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:41.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.844+0000 7f45dad9d640 1 -- 172.21.15.89:0/1161318397 reap_dead start 2023-10-18T10:46:41.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.844+0000 7f45dad9d640 1 --2- 172.21.15.89:0/1161318397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45c4005190 0x7f45c400b990 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f45dc0cda60 tx=0x7f45d401f2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d75713dfcfc982 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:41.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.844+0000 7f45d959a640 1 -- 172.21.15.89:0/1161318397 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d4005ae0 con 0x7f45c4005190 2023-10-18T10:46:41.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.844+0000 7f45d959a640 1 -- 172.21.15.89:0/1161318397 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45d40ab070 con 0x7f45c4005190 2023-10-18T10:46:41.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.844+0000 7f45d959a640 1 -- 172.21.15.89:0/1161318397 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d4005ea0 con 0x7f45c4005190 2023-10-18T10:46:41.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.844+0000 7f45e0cbd640 1 -- 172.21.15.89:0/1161318397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45c4005190 msgr2=0x7f45c400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:41.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.844+0000 7f45e0cbd640 1 --2- 172.21.15.89:0/1161318397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45c4005190 0x7f45c400b990 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f45dc0cda60 tx=0x7f45d401f2f0 comp rx=0 tx=0).stop 2023-10-18T10:46:41.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.844+0000 7f45e0cbd640 1 -- 172.21.15.89:0/1161318397 shutdown_connections 2023-10-18T10:46:41.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.844+0000 7f45e0cbd640 1 --2- 172.21.15.89:0/1161318397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45c4005190 0x7f45c400b990 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:41.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.844+0000 7f45e0cbd640 1 -- 172.21.15.89:0/1161318397 >> 172.21.15.89:0/1161318397 conn(0x7f45dc06dad0 msgr2=0x7f45dc06dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:41.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.844+0000 7f45e0cbd640 1 -- 172.21.15.89:0/1161318397 shutdown_connections 2023-10-18T10:46:41.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.844+0000 7f45e0cbd640 1 -- 172.21.15.89:0/1161318397 wait complete. 2023-10-18T10:46:41.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.844+0000 7f45e0cbd640 1 Processor -- start 2023-10-18T10:46:41.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.844+0000 7f45e0cbd640 1 -- start start 2023-10-18T10:46:41.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.848+0000 7f45e0cbd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45c4005190 0x7f45dc158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:41.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.848+0000 7f45e0cbd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45dc158590 0x7f45dc0c6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:41.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.848+0000 7f45e0cbd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45dc0c6540 0x7f45dc0c69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:41.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.848+0000 7f45e0cbd640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc0c58b0 con 0x7f45dc0c6540 2023-10-18T10:46:41.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.848+0000 7f45e0cbd640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc0c4c30 con 0x7f45dc158590 2023-10-18T10:46:41.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.848+0000 7f45e0cbd640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc0c4f10 con 0x7f45c4005190 2023-10-18T10:46:41.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.848+0000 7f45dad9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45dc0c6540 0x7f45dc0c69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:41.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.848+0000 7f45dad9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45dc0c6540 0x7f45dc0c69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47344/0 (socket says 172.21.15.89:47344) 2023-10-18T10:46:41.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.848+0000 7f45dad9d640 1 -- 172.21.15.89:0/3775658291 learned_addr learned my addr 172.21.15.89:0/3775658291 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:41.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.848+0000 7f45da59c640 1 --2- 172.21.15.89:0/3775658291 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45c4005190 0x7f45dc158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:41.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.848+0000 7f45d9d9b640 1 --2- 172.21.15.89:0/3775658291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45dc158590 0x7f45dc0c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:41.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.848+0000 7f45dad9d640 1 -- 172.21.15.89:0/3775658291 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45c4005190 msgr2=0x7f45dc158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:41.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.848+0000 7f45dad9d640 1 --2- 172.21.15.89:0/3775658291 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45c4005190 0x7f45dc158ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:41.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.848+0000 7f45dad9d640 1 -- 172.21.15.89:0/3775658291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45dc158590 msgr2=0x7f45dc0c6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:41.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.848+0000 7f45dad9d640 1 --2- 172.21.15.89:0/3775658291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45dc158590 0x7f45dc0c6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:41.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.848+0000 7f45dad9d640 1 -- 172.21.15.89:0/3775658291 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45dc0c51f0 con 0x7f45dc0c6540 2023-10-18T10:46:41.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.848+0000 7f45dad9d640 1 --2- 172.21.15.89:0/3775658291 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45dc0c6540 0x7f45dc0c69c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f45d401fb90 tx=0x7f45c4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:41.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.848+0000 7f45caffd640 1 -- 172.21.15.89:0/3775658291 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d4005ea0 con 0x7f45dc0c6540 2023-10-18T10:46:41.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.848+0000 7f45caffd640 1 -- 172.21.15.89:0/3775658291 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45d4001c00 con 0x7f45dc0c6540 2023-10-18T10:46:41.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.848+0000 7f45e0cbd640 1 -- 172.21.15.89:0/3775658291 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45dc0c54d0 con 0x7f45dc0c6540 2023-10-18T10:46:41.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.848+0000 7f45caffd640 1 -- 172.21.15.89:0/3775658291 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d40016a0 con 0x7f45dc0c6540 2023-10-18T10:46:41.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.848+0000 7f45e0cbd640 1 -- 172.21.15.89:0/3775658291 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45dc15ce10 con 0x7f45dc0c6540 2023-10-18T10:46:41.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.848+0000 7f45e0cbd640 1 -- 172.21.15.89:0/3775658291 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45dc04dbe0 con 0x7f45dc0c6540 2023-10-18T10:46:41.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.848+0000 7f45caffd640 1 -- 172.21.15.89:0/3775658291 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f45d40ad020 con 0x7f45dc0c6540 2023-10-18T10:46:41.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.852+0000 7f45caffd640 1 --2- 172.21.15.89:0/3775658291 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f45bc065680 0x7f45bc067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:41.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.852+0000 7f45caffd640 1 -- 172.21.15.89:0/3775658291 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f45d411ff70 con 0x7f45dc0c6540 2023-10-18T10:46:41.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.852+0000 7f45da59c640 1 --2- 172.21.15.89:0/3775658291 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f45bc065680 0x7f45bc067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:41.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.852+0000 7f45da59c640 1 --2- 172.21.15.89:0/3775658291 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f45bc065680 0x7f45bc067b40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f45dc0b5eb0 tx=0x7f45cc00b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:41.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:41.856+0000 7f45caffd640 1 -- 172.21.15.89:0/3775658291 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f45d40ed7a0 con 0x7f45dc0c6540 2023-10-18T10:46:42.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:42.056+0000 7f45e0cbd640 1 -- 172.21.15.89:0/3775658291 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientNetworkRecovery.test_network_death"]} v 0) v1 -- 0x7f45dc0b6170 con 0x7f45dc0c6540 2023-10-18T10:46:42.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:42.992+0000 7f45caffd640 1 -- 172.21.15.89:0/3775658291 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientNetworkRecovery.test_network_death"]}]=0 v70) v1 ==== 160+0+0 (secure 0 0 0) 0x7f45d40be020 con 0x7f45dc0c6540 2023-10-18T10:46:42.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:42.992+0000 7f45e0cbd640 1 -- 172.21.15.89:0/3775658291 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f45bc065680 msgr2=0x7f45bc067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:42.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:42.992+0000 7f45e0cbd640 1 --2- 172.21.15.89:0/3775658291 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f45bc065680 0x7f45bc067b40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f45dc0b5eb0 tx=0x7f45cc00b040 comp rx=0 tx=0).stop 2023-10-18T10:46:42.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:42.992+0000 7f45e0cbd640 1 -- 172.21.15.89:0/3775658291 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45dc0c6540 msgr2=0x7f45dc0c69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:43.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:42.992+0000 7f45e0cbd640 1 --2- 172.21.15.89:0/3775658291 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45dc0c6540 0x7f45dc0c69c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f45d401fb90 tx=0x7f45c4005800 comp rx=0 tx=0).stop 2023-10-18T10:46:43.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:42.996+0000 7f45e0cbd640 1 -- 172.21.15.89:0/3775658291 shutdown_connections 2023-10-18T10:46:43.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:42.996+0000 7f45e0cbd640 1 --2- 172.21.15.89:0/3775658291 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f45bc065680 0x7f45bc067b40 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:43.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:42.996+0000 7f45e0cbd640 1 --2- 172.21.15.89:0/3775658291 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45dc0c6540 0x7f45dc0c69c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:43.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:42.996+0000 7f45e0cbd640 1 --2- 172.21.15.89:0/3775658291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45dc158590 0x7f45dc0c6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:43.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:42.996+0000 7f45e0cbd640 1 --2- 172.21.15.89:0/3775658291 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45c4005190 0x7f45dc158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:43.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:42.996+0000 7f45e0cbd640 1 -- 172.21.15.89:0/3775658291 >> 172.21.15.89:0/3775658291 conn(0x7f45dc06dad0 msgr2=0x7f45dc15ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:43.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:42.996+0000 7f45e0cbd640 1 -- 172.21.15.89:0/3775658291 shutdown_connections 2023-10-18T10:46:43.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:42.996+0000 7f45e0cbd640 1 -- 172.21.15.89:0/3775658291 wait complete. 2023-10-18T10:46:43.020 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:46:43.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.156+0000 7fb609f2c640 1 Processor -- start 2023-10-18T10:46:43.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.156+0000 7fb609f2c640 1 -- start start 2023-10-18T10:46:43.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.156+0000 7fb609f2c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb604159490 0x7fb60415d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:43.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.156+0000 7fb609f2c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb604070600 0x7fb60406b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:43.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.156+0000 7fb609f2c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb60406bd10 0x7fb6040cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:43.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.156+0000 7fb609f2c640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb60406c1a0 con 0x7fb604070600 2023-10-18T10:46:43.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.156+0000 7fb609f2c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb60415ddf0 con 0x7fb60406bd10 2023-10-18T10:46:43.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.156+0000 7fb609f2c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb604070b90 con 0x7fb604159490 2023-10-18T10:46:43.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.156+0000 7fb602ffd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb604070600 0x7fb60406b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:43.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.156+0000 7fb602ffd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb604070600 0x7fb60406b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56028/0 (socket says 172.21.15.89:56028) 2023-10-18T10:46:43.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.156+0000 7fb602ffd640 1 -- 172.21.15.89:0/844639451 learned_addr learned my addr 172.21.15.89:0/844639451 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:43.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.156+0000 7fb602ffd640 1 -- 172.21.15.89:0/844639451 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb604159490 msgr2=0x7fb60415d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:43.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.156+0000 7fb602ffd640 1 --2- 172.21.15.89:0/844639451 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb604159490 0x7fb60415d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:43.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.156+0000 7fb602ffd640 1 -- 172.21.15.89:0/844639451 >> v2:172.21.15.154:3300/0 conn(0x7fb60406bd10 msgr2=0x7fb6040cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:43.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.156+0000 7fb603fff640 1 --2- 172.21.15.89:0/844639451 >> v2:172.21.15.154:3300/0 conn(0x7fb60406bd10 0x7fb6040cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:43.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.156+0000 7fb602ffd640 1 --2- 172.21.15.89:0/844639451 >> v2:172.21.15.154:3300/0 conn(0x7fb60406bd10 0x7fb6040cbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:43.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.156+0000 7fb602ffd640 1 -- 172.21.15.89:0/844639451 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb604070e70 con 0x7fb604070600 2023-10-18T10:46:43.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.156+0000 7fb603fff640 1 --2- 172.21.15.89:0/844639451 >> v2:172.21.15.154:3300/0 conn(0x7fb60406bd10 0x7fb6040cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:43.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.160+0000 7fb602ffd640 1 --2- 172.21.15.89:0/844639451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb604070600 0x7fb60406b7d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb6040add20 tx=0x7fb5f801c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a7ccd25c50f84df server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:43.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.160+0000 7fb6027fc640 1 -- 172.21.15.89:0/844639451 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f80ab600 con 0x7fb604070600 2023-10-18T10:46:43.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.160+0000 7fb6027fc640 1 -- 172.21.15.89:0/844639451 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f80a7050 con 0x7fb604070600 2023-10-18T10:46:43.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.160+0000 7fb6027fc640 1 -- 172.21.15.89:0/844639451 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f80ab980 con 0x7fb604070600 2023-10-18T10:46:43.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.160+0000 7fb609f2c640 1 -- 172.21.15.89:0/844639451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb604070600 msgr2=0x7fb60406b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:43.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.160+0000 7fb609f2c640 1 --2- 172.21.15.89:0/844639451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb604070600 0x7fb60406b7d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb6040add20 tx=0x7fb5f801c180 comp rx=0 tx=0).stop 2023-10-18T10:46:43.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.160+0000 7fb609f2c640 1 -- 172.21.15.89:0/844639451 shutdown_connections 2023-10-18T10:46:43.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.160+0000 7fb609f2c640 1 --2- 172.21.15.89:0/844639451 >> v2:172.21.15.154:3300/0 conn(0x7fb60406bd10 0x7fb6040cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:43.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.160+0000 7fb609f2c640 1 --2- 172.21.15.89:0/844639451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb604070600 0x7fb60406b7d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:43.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.160+0000 7fb609f2c640 1 --2- 172.21.15.89:0/844639451 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb604159490 0x7fb60415d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:43.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.160+0000 7fb609f2c640 1 -- 172.21.15.89:0/844639451 >> 172.21.15.89:0/844639451 conn(0x7fb604069f00 msgr2=0x7fb6040cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:43.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.160+0000 7fb609f2c640 1 -- 172.21.15.89:0/844639451 shutdown_connections 2023-10-18T10:46:43.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.160+0000 7fb609f2c640 1 -- 172.21.15.89:0/844639451 wait complete. 2023-10-18T10:46:43.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.164+0000 7fb609f2c640 1 Processor -- start 2023-10-18T10:46:43.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.164+0000 7fb609f2c640 1 -- start start 2023-10-18T10:46:43.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.164+0000 7fb609f2c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb60406bd10 0x7fb6040c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:43.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.164+0000 7fb609f2c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb604070600 0x7fb604151740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:43.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.164+0000 7fb609f2c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb604159490 0x7fb6040c1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:43.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.164+0000 7fb609f2c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb604154040 con 0x7fb60406bd10 2023-10-18T10:46:43.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.164+0000 7fb609f2c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6040c2c90 con 0x7fb604159490 2023-10-18T10:46:43.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.164+0000 7fb609f2c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb604153a10 con 0x7fb604070600 2023-10-18T10:46:43.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.164+0000 7fb6037fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb60406bd10 0x7fb6040c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:43.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.164+0000 7fb6037fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb60406bd10 0x7fb6040c53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56044/0 (socket says 172.21.15.89:56044) 2023-10-18T10:46:43.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.164+0000 7fb6037fe640 1 -- 172.21.15.89:0/2051786975 learned_addr learned my addr 172.21.15.89:0/2051786975 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:43.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.164+0000 7fb602ffd640 1 --2- 172.21.15.89:0/2051786975 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb604070600 0x7fb604151740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:43.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.164+0000 7fb603fff640 1 --2- 172.21.15.89:0/2051786975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb604159490 0x7fb6040c1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:43.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.164+0000 7fb6037fe640 1 -- 172.21.15.89:0/2051786975 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb604070600 msgr2=0x7fb604151740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:43.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.164+0000 7fb6037fe640 1 --2- 172.21.15.89:0/2051786975 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb604070600 0x7fb604151740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:43.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.164+0000 7fb6037fe640 1 -- 172.21.15.89:0/2051786975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb604159490 msgr2=0x7fb6040c1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:43.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.164+0000 7fb6037fe640 1 --2- 172.21.15.89:0/2051786975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb604159490 0x7fb6040c1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:43.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.164+0000 7fb6037fe640 1 -- 172.21.15.89:0/2051786975 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f80094d0 con 0x7fb60406bd10 2023-10-18T10:46:43.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.164+0000 7fb6037fe640 1 --2- 172.21.15.89:0/2051786975 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb60406bd10 0x7fb6040c53e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb5f400b410 tx=0x7fb5f400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:43.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.164+0000 7fb5ebfff640 1 -- 172.21.15.89:0/2051786975 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4011830 con 0x7fb60406bd10 2023-10-18T10:46:43.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.164+0000 7fb5ebfff640 1 -- 172.21.15.89:0/2051786975 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f4011990 con 0x7fb60406bd10 2023-10-18T10:46:43.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.164+0000 7fb5ebfff640 1 -- 172.21.15.89:0/2051786975 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4011d40 con 0x7fb60406bd10 2023-10-18T10:46:43.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.164+0000 7fb609f2c640 1 -- 172.21.15.89:0/2051786975 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6040c29b0 con 0x7fb60406bd10 2023-10-18T10:46:43.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.164+0000 7fb609f2c640 1 -- 172.21.15.89:0/2051786975 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb604151c80 con 0x7fb60406bd10 2023-10-18T10:46:43.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.168+0000 7fb609f2c640 1 -- 172.21.15.89:0/2051786975 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb60404dbe0 con 0x7fb60406bd10 2023-10-18T10:46:43.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.168+0000 7fb5ebfff640 1 -- 172.21.15.89:0/2051786975 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fb5f401a400 con 0x7fb60406bd10 2023-10-18T10:46:43.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.168+0000 7fb5ebfff640 1 --2- 172.21.15.89:0/2051786975 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb5f00656b0 0x7fb5f0067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:43.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.168+0000 7fb5ebfff640 1 -- 172.21.15.89:0/2051786975 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb5f40830b0 con 0x7fb60406bd10 2023-10-18T10:46:43.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.168+0000 7fb602ffd640 1 --2- 172.21.15.89:0/2051786975 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb5f00656b0 0x7fb5f0067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:43.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.168+0000 7fb602ffd640 1 --2- 172.21.15.89:0/2051786975 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb5f00656b0 0x7fb5f0067b70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80ad000 tx=0x7fb5f80a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:43.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.172+0000 7fb5ebfff640 1 -- 172.21.15.89:0/2051786975 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fb5f40508e0 con 0x7fb60406bd10 2023-10-18T10:46:43.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.324+0000 7fb609f2c640 1 -- 172.21.15.89:0/2051786975 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6040cbb60 con 0x7fb60406bd10 2023-10-18T10:46:43.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.324+0000 7fb5ebfff640 1 -- 172.21.15.89:0/2051786975 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20192 (secure 0 0 0) 0x7fb5f4053f00 con 0x7fb60406bd10 2023-10-18T10:46:43.330 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:43.330 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":29,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:46:36.958604+0000","last_up_change":"2023-10-18T10:45:42.164688+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:45:53.309122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-18T10:45:54.659763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:45:57.796514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6824","nonce":773078364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6827","nonce":773078364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6831","nonce":773078364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6829","nonce":773078364}]},"public_addr":"172.21.15.154:6824/773078364","cluster_addr":"172.21.15.154:6827/773078364","heartbeat_back_addr":"172.21.15.154:6831/773078364","heartbeat_front_addr":"172.21.15.154:6829/773078364","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":1,"to":5}]},{"pgid":"4.1a","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:46:43.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.328+0000 7fb609f2c640 1 -- 172.21.15.89:0/2051786975 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb5f00656b0 msgr2=0x7fb5f0067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:43.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.328+0000 7fb609f2c640 1 --2- 172.21.15.89:0/2051786975 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb5f00656b0 0x7fb5f0067b70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80ad000 tx=0x7fb5f80a6000 comp rx=0 tx=0).stop 2023-10-18T10:46:43.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.328+0000 7fb609f2c640 1 -- 172.21.15.89:0/2051786975 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb60406bd10 msgr2=0x7fb6040c53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:43.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.328+0000 7fb609f2c640 1 --2- 172.21.15.89:0/2051786975 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb60406bd10 0x7fb6040c53e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb5f400b410 tx=0x7fb5f400b8d0 comp rx=0 tx=0).stop 2023-10-18T10:46:43.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.328+0000 7fb609f2c640 1 -- 172.21.15.89:0/2051786975 shutdown_connections 2023-10-18T10:46:43.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.328+0000 7fb609f2c640 1 --2- 172.21.15.89:0/2051786975 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb5f00656b0 0x7fb5f0067b70 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:43.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.328+0000 7fb609f2c640 1 --2- 172.21.15.89:0/2051786975 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb604070600 0x7fb604151740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:43.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.328+0000 7fb609f2c640 1 --2- 172.21.15.89:0/2051786975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb604159490 0x7fb6040c1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:43.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.328+0000 7fb609f2c640 1 --2- 172.21.15.89:0/2051786975 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb60406bd10 0x7fb6040c53e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:43.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.328+0000 7fb609f2c640 1 -- 172.21.15.89:0/2051786975 >> 172.21.15.89:0/2051786975 conn(0x7fb604069f00 msgr2=0x7fb6040adf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:43.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.328+0000 7fb609f2c640 1 -- 172.21.15.89:0/2051786975 shutdown_connections 2023-10-18T10:46:43.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.328+0000 7fb609f2c640 1 -- 172.21.15.89:0/2051786975 wait complete. 2023-10-18T10:46:43.351 INFO:tasks.cephfs.cephfs_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:45:43.114184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-18T10:45:48.730968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:45:53.309122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:45:54.659763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T10:45:57.796514+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T10:46:43.352 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:46:43.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.488+0000 7fa528093640 1 Processor -- start 2023-10-18T10:46:43.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.488+0000 7fa528093640 1 -- start start 2023-10-18T10:46:43.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.488+0000 7fa528093640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa5200cd4f0 0x7fa520143470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:43.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.488+0000 7fa528093640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa520145b70 0x7fa520160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:43.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.488+0000 7fa528093640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5200cd8d0 con 0x7fa5200ce070 2023-10-18T10:46:43.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.488+0000 7fa528093640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa520069900 con 0x7fa520145b70 2023-10-18T10:46:43.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.488+0000 7fa528093640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa52015cbf0 con 0x7fa5200cd4f0 2023-10-18T10:46:43.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.488+0000 7fa52562e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa520145b70 0x7fa520160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:43.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.488+0000 7fa526630640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa5200ce070 0x7fa52006abb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48606/0 (socket says 172.21.15.89:48606) 2023-10-18T10:46:43.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.488+0000 7fa526630640 1 -- 172.21.15.89:0/664399557 learned_addr learned my addr 172.21.15.89:0/664399557 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:43.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.488+0000 7fa525e2f640 1 --2- 172.21.15.89:0/664399557 >> v2:172.21.15.154:3300/0 conn(0x7fa5200cd4f0 0x7fa520143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:43.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.488+0000 7fa52562e640 1 -- 172.21.15.89:0/664399557 >> v2:172.21.15.154:3300/0 conn(0x7fa5200cd4f0 msgr2=0x7fa520143470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:43.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.488+0000 7fa52562e640 1 --2- 172.21.15.89:0/664399557 >> v2:172.21.15.154:3300/0 conn(0x7fa5200cd4f0 0x7fa520143470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:43.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.488+0000 7fa52562e640 1 -- 172.21.15.89:0/664399557 >> v1:172.21.15.89:6789/0 conn(0x7fa5200ce070 legacy=0x7fa52006abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:43.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.488+0000 7fa52562e640 1 -- 172.21.15.89:0/664399557 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa520161230 con 0x7fa520145b70 2023-10-18T10:46:43.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.488+0000 7fa524e2d640 1 -- 172.21.15.89:0/664399557 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3187338345 0 0) 0x7fa5200cd8d0 con 0x7fa5200ce070 2023-10-18T10:46:43.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.492+0000 7fa52562e640 1 --2- 172.21.15.89:0/664399557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa520145b70 0x7fa520160cf0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa5200716d0 tx=0x7fa51001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a7d0ca7a59ffd63 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:43.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.492+0000 7fa524e2d640 1 -- 172.21.15.89:0/664399557 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5100a7070 con 0x7fa520145b70 2023-10-18T10:46:43.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.492+0000 7fa524e2d640 1 -- 172.21.15.89:0/664399557 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5100ab430 con 0x7fa520145b70 2023-10-18T10:46:43.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.492+0000 7fa524e2d640 1 -- 172.21.15.89:0/664399557 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5100ab7c0 con 0x7fa520145b70 2023-10-18T10:46:43.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.492+0000 7fa528093640 1 -- 172.21.15.89:0/664399557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa520145b70 msgr2=0x7fa520160cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:43.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.492+0000 7fa528093640 1 --2- 172.21.15.89:0/664399557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa520145b70 0x7fa520160cf0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa5200716d0 tx=0x7fa51001c430 comp rx=0 tx=0).stop 2023-10-18T10:46:43.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.492+0000 7fa528093640 1 -- 172.21.15.89:0/664399557 shutdown_connections 2023-10-18T10:46:43.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.492+0000 7fa528093640 1 --2- 172.21.15.89:0/664399557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa520145b70 0x7fa520160cf0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:43.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.492+0000 7fa528093640 1 --2- 172.21.15.89:0/664399557 >> v2:172.21.15.154:3300/0 conn(0x7fa5200cd4f0 0x7fa520143470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:43.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.492+0000 7fa528093640 1 -- 172.21.15.89:0/664399557 >> 172.21.15.89:0/664399557 conn(0x7fa52006dad0 msgr2=0x7fa52006dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:43.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.492+0000 7fa528093640 1 -- 172.21.15.89:0/664399557 shutdown_connections 2023-10-18T10:46:43.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.492+0000 7fa528093640 1 -- 172.21.15.89:0/664399557 wait complete. 2023-10-18T10:46:43.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.496+0000 7fa528093640 1 Processor -- start 2023-10-18T10:46:43.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.496+0000 7fa528093640 1 -- start start 2023-10-18T10:46:43.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.496+0000 7fa528093640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5200cd4f0 0x7fa5200c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:43.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.496+0000 7fa528093640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa5200ce070 0x7fa5200b5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:43.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.496+0000 7fa528093640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa520145b70 0x7fa5200c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:43.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.496+0000 7fa528093640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5200b5e80 con 0x7fa5200cd4f0 2023-10-18T10:46:43.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.496+0000 7fa528093640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5200c5950 con 0x7fa520145b70 2023-10-18T10:46:43.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.496+0000 7fa528093640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5200c5c30 con 0x7fa5200ce070 2023-10-18T10:46:43.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.496+0000 7fa525e2f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5200cd4f0 0x7fa5200c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:43.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.496+0000 7fa525e2f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5200cd4f0 0x7fa5200c6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56060/0 (socket says 172.21.15.89:56060) 2023-10-18T10:46:43.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.496+0000 7fa525e2f640 1 -- 172.21.15.89:0/3243458713 learned_addr learned my addr 172.21.15.89:0/3243458713 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:43.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.496+0000 7fa526630640 1 --2- 172.21.15.89:0/3243458713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa520145b70 0x7fa5200c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:43.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.496+0000 7fa52562e640 1 --2- 172.21.15.89:0/3243458713 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa5200ce070 0x7fa5200b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:43.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.496+0000 7fa525e2f640 1 -- 172.21.15.89:0/3243458713 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa5200ce070 msgr2=0x7fa5200b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:43.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.496+0000 7fa525e2f640 1 --2- 172.21.15.89:0/3243458713 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa5200ce070 0x7fa5200b5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:43.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.496+0000 7fa525e2f640 1 -- 172.21.15.89:0/3243458713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa520145b70 msgr2=0x7fa5200c7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:43.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.496+0000 7fa525e2f640 1 --2- 172.21.15.89:0/3243458713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa520145b70 0x7fa5200c7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:43.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.496+0000 7fa525e2f640 1 -- 172.21.15.89:0/3243458713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa510009420 con 0x7fa5200cd4f0 2023-10-18T10:46:43.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.496+0000 7fa52562e640 1 --2- 172.21.15.89:0/3243458713 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa5200ce070 0x7fa5200b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:43.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.496+0000 7fa525e2f640 1 --2- 172.21.15.89:0/3243458713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5200cd4f0 0x7fa5200c6cf0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa51c00e870 tx=0x7fa51c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:43.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.496+0000 7fa5167fc640 1 -- 172.21.15.89:0/3243458713 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c009510 con 0x7fa5200cd4f0 2023-10-18T10:46:43.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.496+0000 7fa5167fc640 1 -- 172.21.15.89:0/3243458713 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51c00eec0 con 0x7fa5200cd4f0 2023-10-18T10:46:43.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.496+0000 7fa5167fc640 1 -- 172.21.15.89:0/3243458713 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c00cb40 con 0x7fa5200cd4f0 2023-10-18T10:46:43.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.496+0000 7fa528093640 1 -- 172.21.15.89:0/3243458713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5200c5f10 con 0x7fa5200cd4f0 2023-10-18T10:46:43.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.496+0000 7fa528093640 1 -- 172.21.15.89:0/3243458713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa52015fc60 con 0x7fa5200cd4f0 2023-10-18T10:46:43.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.496+0000 7fa528093640 1 -- 172.21.15.89:0/3243458713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa52006a670 con 0x7fa5200cd4f0 2023-10-18T10:46:43.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.500+0000 7fa5167fc640 1 -- 172.21.15.89:0/3243458713 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fa51c00cca0 con 0x7fa5200cd4f0 2023-10-18T10:46:43.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.500+0000 7fa5167fc640 1 --2- 172.21.15.89:0/3243458713 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa5040656b0 0x7fa504067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:43.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.500+0000 7fa5167fc640 1 -- 172.21.15.89:0/3243458713 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa51c0831c0 con 0x7fa5200cd4f0 2023-10-18T10:46:43.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.500+0000 7fa52562e640 1 --2- 172.21.15.89:0/3243458713 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa5040656b0 0x7fa504067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:43.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.500+0000 7fa52562e640 1 --2- 172.21.15.89:0/3243458713 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa5040656b0 0x7fa504067b70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa5100af000 tx=0x7fa5100a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:43.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.504+0000 7fa5167fc640 1 -- 172.21.15.89:0/3243458713 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fa52006a670 con 0x7fa5200cd4f0 2023-10-18T10:46:43.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.652+0000 7fa528093640 1 -- 172.21.15.89:0/3243458713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa52006a670 con 0x7fa5200cd4f0 2023-10-18T10:46:43.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.652+0000 7fa5167fc640 1 -- 172.21.15.89:0/3243458713 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa52006a670 con 0x7fa5200cd4f0 2023-10-18T10:46:43.656 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-18T10:46:43.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.652+0000 7fa528093640 1 -- 172.21.15.89:0/3243458713 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa5040656b0 msgr2=0x7fa504067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:43.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.652+0000 7fa528093640 1 --2- 172.21.15.89:0/3243458713 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa5040656b0 0x7fa504067b70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa5100af000 tx=0x7fa5100a7000 comp rx=0 tx=0).stop 2023-10-18T10:46:43.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.652+0000 7fa528093640 1 -- 172.21.15.89:0/3243458713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5200cd4f0 msgr2=0x7fa5200c6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:43.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.652+0000 7fa528093640 1 --2- 172.21.15.89:0/3243458713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5200cd4f0 0x7fa5200c6cf0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa51c00e870 tx=0x7fa51c00ec70 comp rx=0 tx=0).stop 2023-10-18T10:46:43.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.652+0000 7fa528093640 1 -- 172.21.15.89:0/3243458713 shutdown_connections 2023-10-18T10:46:43.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.652+0000 7fa528093640 1 --2- 172.21.15.89:0/3243458713 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa5040656b0 0x7fa504067b70 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:43.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.652+0000 7fa528093640 1 --2- 172.21.15.89:0/3243458713 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa5200ce070 0x7fa5200b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:43.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.652+0000 7fa528093640 1 --2- 172.21.15.89:0/3243458713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa520145b70 0x7fa5200c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:43.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.652+0000 7fa528093640 1 --2- 172.21.15.89:0/3243458713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5200cd4f0 0x7fa5200c6cf0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:43.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.652+0000 7fa528093640 1 -- 172.21.15.89:0/3243458713 >> 172.21.15.89:0/3243458713 conn(0x7fa52006dad0 msgr2=0x7fa520155b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:43.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.652+0000 7fa528093640 1 -- 172.21.15.89:0/3243458713 shutdown_connections 2023-10-18T10:46:43.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.652+0000 7fa528093640 1 -- 172.21.15.89:0/3243458713 wait complete. 2023-10-18T10:46:43.677 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-18T10:46:43.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.816+0000 7f6c35e34640 1 Processor -- start 2023-10-18T10:46:43.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.816+0000 7f6c35e34640 1 -- start start 2023-10-18T10:46:43.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.816+0000 7f6c35e34640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6c3015a480 0x7f6c3015e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:43.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.816+0000 7f6c35e34640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6c3006ff60 0x7f6c3015a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:43.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.816+0000 7f6c35e34640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c3015ba30 con 0x7f6c300ca650 2023-10-18T10:46:43.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.816+0000 7f6c35e34640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c3015b470 con 0x7f6c3006ff60 2023-10-18T10:46:43.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.816+0000 7f6c35e34640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c3015b750 con 0x7f6c3015a480 2023-10-18T10:46:43.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.820+0000 7f6c2ffff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6c3006ff60 0x7f6c3015a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:43.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.820+0000 7f6c2ffff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6c3006ff60 0x7f6c3015a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56064/0 (socket says 172.21.15.89:56064) 2023-10-18T10:46:43.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.820+0000 7f6c2ffff640 1 -- 172.21.15.89:0/2546470178 learned_addr learned my addr 172.21.15.89:0/2546470178 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:43.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.820+0000 7f6c2effd640 1 --1- 172.21.15.89:0/2546470178 >> v1:172.21.15.89:6789/0 conn(0x7f6c300ca650 0x7f6c3006f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48612/0 (socket says 172.21.15.89:48612) 2023-10-18T10:46:43.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.820+0000 7f6c2f7fe640 1 --2- 172.21.15.89:0/2546470178 >> v2:172.21.15.154:3300/0 conn(0x7f6c3015a480 0x7f6c3015e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:43.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.820+0000 7f6c2ffff640 1 -- 172.21.15.89:0/2546470178 >> v2:172.21.15.154:3300/0 conn(0x7f6c3015a480 msgr2=0x7f6c3015e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:43.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.820+0000 7f6c2ffff640 1 --2- 172.21.15.89:0/2546470178 >> v2:172.21.15.154:3300/0 conn(0x7f6c3015a480 0x7f6c3015e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:43.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.820+0000 7f6c2ffff640 1 -- 172.21.15.89:0/2546470178 >> v1:172.21.15.89:6789/0 conn(0x7f6c300ca650 legacy=0x7f6c3006f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:43.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.820+0000 7f6c2ffff640 1 -- 172.21.15.89:0/2546470178 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3015e200 con 0x7f6c3006ff60 2023-10-18T10:46:43.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.820+0000 7f6c2e7fc640 1 -- 172.21.15.89:0/2546470178 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3556785356 0 0) 0x7f6c3015ba30 con 0x7f6c300ca650 2023-10-18T10:46:43.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.820+0000 7f6c2ffff640 1 --2- 172.21.15.89:0/2546470178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c3006ff60 0x7f6c3015a860 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6c300b0ec0 tx=0x7f6c2801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e099e1f7ab430d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:43.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.820+0000 7f6c2e7fc640 1 -- 172.21.15.89:0/2546470178 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c280a57f0 con 0x7f6c3006ff60 2023-10-18T10:46:43.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.820+0000 7f6c2e7fc640 1 -- 172.21.15.89:0/2546470178 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c280a5950 con 0x7f6c3006ff60 2023-10-18T10:46:43.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.820+0000 7f6c2e7fc640 1 -- 172.21.15.89:0/2546470178 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c280a5ce0 con 0x7f6c3006ff60 2023-10-18T10:46:43.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.824+0000 7f6c35e34640 1 -- 172.21.15.89:0/2546470178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c3006ff60 msgr2=0x7f6c3015a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:43.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.824+0000 7f6c35e34640 1 --2- 172.21.15.89:0/2546470178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c3006ff60 0x7f6c3015a860 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6c300b0ec0 tx=0x7f6c2801c910 comp rx=0 tx=0).stop 2023-10-18T10:46:43.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.824+0000 7f6c35e34640 1 -- 172.21.15.89:0/2546470178 shutdown_connections 2023-10-18T10:46:43.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.824+0000 7f6c35e34640 1 --2- 172.21.15.89:0/2546470178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c3006ff60 0x7f6c3015a860 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:43.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.824+0000 7f6c35e34640 1 --2- 172.21.15.89:0/2546470178 >> v2:172.21.15.154:3300/0 conn(0x7f6c3015a480 0x7f6c3015e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:43.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.824+0000 7f6c35e34640 1 -- 172.21.15.89:0/2546470178 >> 172.21.15.89:0/2546470178 conn(0x7f6c30069f00 msgr2=0x7f6c300cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:43.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.824+0000 7f6c35e34640 1 -- 172.21.15.89:0/2546470178 shutdown_connections 2023-10-18T10:46:43.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.824+0000 7f6c35e34640 1 -- 172.21.15.89:0/2546470178 wait complete. 2023-10-18T10:46:43.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.824+0000 7f6c35e34640 1 Processor -- start 2023-10-18T10:46:43.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.824+0000 7f6c35e34640 1 -- start start 2023-10-18T10:46:43.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.824+0000 7f6c35e34640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c3006ff60 0x7f6c300cd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:43.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.824+0000 7f6c35e34640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c300ca650 0x7f6c300b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:43.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.824+0000 7f6c35e34640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c3015a480 0x7f6c300c1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:43.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.824+0000 7f6c35e34640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c30158560 con 0x7f6c3015a480 2023-10-18T10:46:43.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.824+0000 7f6c35e34640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c300c2500 con 0x7f6c3006ff60 2023-10-18T10:46:43.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.824+0000 7f6c35e34640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c300c27e0 con 0x7f6c300ca650 2023-10-18T10:46:43.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.824+0000 7f6c2ffff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c3015a480 0x7f6c300c1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:43.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.824+0000 7f6c2ffff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c3015a480 0x7f6c300c1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56072/0 (socket says 172.21.15.89:56072) 2023-10-18T10:46:43.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.824+0000 7f6c2ffff640 1 -- 172.21.15.89:0/2615739158 learned_addr learned my addr 172.21.15.89:0/2615739158 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:43.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.824+0000 7f6c2f7fe640 1 --2- 172.21.15.89:0/2615739158 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c3006ff60 0x7f6c300cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:43.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.824+0000 7f6c2effd640 1 --2- 172.21.15.89:0/2615739158 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c300ca650 0x7f6c300b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:43.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.824+0000 7f6c2ffff640 1 -- 172.21.15.89:0/2615739158 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c300ca650 msgr2=0x7f6c300b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:43.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.824+0000 7f6c2ffff640 1 --2- 172.21.15.89:0/2615739158 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c300ca650 0x7f6c300b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:43.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.824+0000 7f6c2ffff640 1 -- 172.21.15.89:0/2615739158 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c3006ff60 msgr2=0x7f6c300cd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:43.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.824+0000 7f6c2ffff640 1 --2- 172.21.15.89:0/2615739158 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c3006ff60 0x7f6c300cd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:43.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.824+0000 7f6c2ffff640 1 -- 172.21.15.89:0/2615739158 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c28009b30 con 0x7f6c3015a480 2023-10-18T10:46:43.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.824+0000 7f6c2effd640 1 --2- 172.21.15.89:0/2615739158 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c300ca650 0x7f6c300b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:43.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.828+0000 7f6c2ffff640 1 --2- 172.21.15.89:0/2615739158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c3015a480 0x7f6c300c1e40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6c28019520 tx=0x7f6c280196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:43.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.828+0000 7f6c17fff640 1 -- 172.21.15.89:0/2615739158 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c280a57f0 con 0x7f6c3015a480 2023-10-18T10:46:43.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.828+0000 7f6c17fff640 1 -- 172.21.15.89:0/2615739158 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c28009510 con 0x7f6c3015a480 2023-10-18T10:46:43.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.828+0000 7f6c17fff640 1 -- 172.21.15.89:0/2615739158 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c280b8880 con 0x7f6c3015a480 2023-10-18T10:46:43.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.828+0000 7f6c35e34640 1 -- 172.21.15.89:0/2615739158 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c300c2ac0 con 0x7f6c3015a480 2023-10-18T10:46:43.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.828+0000 7f6c35e34640 1 -- 172.21.15.89:0/2615739158 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c300c3320 con 0x7f6c3015a480 2023-10-18T10:46:43.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.828+0000 7f6c35e34640 1 -- 172.21.15.89:0/2615739158 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c30158710 con 0x7f6c3015a480 2023-10-18T10:46:43.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.828+0000 7f6c17fff640 1 -- 172.21.15.89:0/2615739158 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f6c280b89e0 con 0x7f6c3015a480 2023-10-18T10:46:43.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.828+0000 7f6c17fff640 1 --2- 172.21.15.89:0/2615739158 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6c1c065680 0x7f6c1c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:43.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.828+0000 7f6c17fff640 1 -- 172.21.15.89:0/2615739158 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6c2811e660 con 0x7f6c3015a480 2023-10-18T10:46:43.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.828+0000 7f6c2f7fe640 1 --2- 172.21.15.89:0/2615739158 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6c1c065680 0x7f6c1c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:43.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.832+0000 7f6c2f7fe640 1 --2- 172.21.15.89:0/2615739158 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6c1c065680 0x7f6c1c067b40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009b60 tx=0x7f6c20009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:43.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.832+0000 7f6c17fff640 1 -- 172.21.15.89:0/2615739158 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f6c280ebe70 con 0x7f6c3015a480 2023-10-18T10:46:43.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.984+0000 7f6c35e34640 1 -- 172.21.15.89:0/2615739158 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6c30073b50 con 0x7f6c3015a480 2023-10-18T10:46:43.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.984+0000 7f6c17fff640 1 -- 172.21.15.89:0/2615739158 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6c280ef490 con 0x7f6c3015a480 2023-10-18T10:46:43.988 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-10-18T10:46:43.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.984+0000 7f6c35e34640 1 -- 172.21.15.89:0/2615739158 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6c1c065680 msgr2=0x7f6c1c067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:43.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.984+0000 7f6c35e34640 1 --2- 172.21.15.89:0/2615739158 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6c1c065680 0x7f6c1c067b40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009b60 tx=0x7f6c20009190 comp rx=0 tx=0).stop 2023-10-18T10:46:43.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.984+0000 7f6c35e34640 1 -- 172.21.15.89:0/2615739158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c3015a480 msgr2=0x7f6c300c1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:43.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.984+0000 7f6c35e34640 1 --2- 172.21.15.89:0/2615739158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c3015a480 0x7f6c300c1e40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6c28019520 tx=0x7f6c280196b0 comp rx=0 tx=0).stop 2023-10-18T10:46:43.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.984+0000 7f6c35e34640 1 -- 172.21.15.89:0/2615739158 shutdown_connections 2023-10-18T10:46:43.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.984+0000 7f6c35e34640 1 --2- 172.21.15.89:0/2615739158 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6c1c065680 0x7f6c1c067b40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:43.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.984+0000 7f6c35e34640 1 --2- 172.21.15.89:0/2615739158 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c300ca650 0x7f6c300b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:43.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.984+0000 7f6c35e34640 1 --2- 172.21.15.89:0/2615739158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c3015a480 0x7f6c300c1e40 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:43.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.984+0000 7f6c35e34640 1 --2- 172.21.15.89:0/2615739158 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c3006ff60 0x7f6c300cd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:43.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.984+0000 7f6c35e34640 1 -- 172.21.15.89:0/2615739158 >> 172.21.15.89:0/2615739158 conn(0x7f6c30069f00 msgr2=0x7f6c300aeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:43.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.984+0000 7f6c35e34640 1 -- 172.21.15.89:0/2615739158 shutdown_connections 2023-10-18T10:46:43.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:43.984+0000 7f6c35e34640 1 -- 172.21.15.89:0/2615739158 wait complete. 2023-10-18T10:46:44.010 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:46:44.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.144+0000 7fe8d69db640 1 Processor -- start 2023-10-18T10:46:44.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.148+0000 7fe8d69db640 1 -- start start 2023-10-18T10:46:44.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.148+0000 7fe8d69db640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8d00c9970 0x7fe8d0164230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:44.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.148+0000 7fe8d69db640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d0070b90 con 0x7fe8d0070600 2023-10-18T10:46:44.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.148+0000 7fe8d69db640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d0070e70 con 0x7fe8d0159490 2023-10-18T10:46:44.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.148+0000 7fe8d69db640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d0071150 con 0x7fe8d00c9970 2023-10-18T10:46:44.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.148+0000 7fe8cf7fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe8d0070600 0x7fe8d006b7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48628/0 (socket says 172.21.15.89:48628) 2023-10-18T10:46:44.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.148+0000 7fe8cf7fe640 1 -- 172.21.15.89:0/1692618556 learned_addr learned my addr 172.21.15.89:0/1692618556 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:44.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.148+0000 7fe8d4f78640 1 --2- 172.21.15.89:0/1692618556 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8d00c9970 0x7fe8d0164230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:44.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.148+0000 7fe8ceffd640 1 -- 172.21.15.89:0/1692618556 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2713151165 0 0) 0x7fe8d0070b90 con 0x7fe8d0070600 2023-10-18T10:46:44.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.148+0000 7fe8ceffd640 1 -- 172.21.15.89:0/1692618556 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8b8003530 con 0x7fe8d0070600 2023-10-18T10:46:44.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.148+0000 7fe8ceffd640 1 -- 172.21.15.89:0/1692618556 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1264514729 0 0) 0x7fe8d0070e70 con 0x7fe8d0159490 2023-10-18T10:46:44.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.148+0000 7fe8ceffd640 1 -- 172.21.15.89:0/1692618556 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8d0070b90 con 0x7fe8d0159490 2023-10-18T10:46:44.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.148+0000 7fe8ceffd640 1 -- 172.21.15.89:0/1692618556 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2658280994 0 0) 0x7fe8b8003530 con 0x7fe8d0070600 2023-10-18T10:46:44.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.148+0000 7fe8d4f78640 1 -- 172.21.15.89:0/1692618556 >> v1:172.21.15.154:6789/0 conn(0x7fe8d0159490 legacy=0x7fe8d015d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:44.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.148+0000 7fe8d4f78640 1 -- 172.21.15.89:0/1692618556 >> v1:172.21.15.89:6789/0 conn(0x7fe8d0070600 legacy=0x7fe8d006b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:44.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.148+0000 7fe8d4f78640 1 -- 172.21.15.89:0/1692618556 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d0071430 con 0x7fe8d00c9970 2023-10-18T10:46:44.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.152+0000 7fe8d4f78640 1 --2- 172.21.15.89:0/1692618556 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8d00c9970 0x7fe8d0164230 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8009dc0 tx=0x7fe8c801c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adb9155bed70fffb server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:44.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.152+0000 7fe8ceffd640 1 -- 172.21.15.89:0/1692618556 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4001680 con 0x7fe8d00c9970 2023-10-18T10:46:44.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.152+0000 7fe8ceffd640 1 -- 172.21.15.89:0/1692618556 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c80a55f0 con 0x7fe8d00c9970 2023-10-18T10:46:44.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.152+0000 7fe8ceffd640 1 -- 172.21.15.89:0/1692618556 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c80a5980 con 0x7fe8d00c9970 2023-10-18T10:46:44.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.152+0000 7fe8d69db640 1 -- 172.21.15.89:0/1692618556 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8d00c9970 msgr2=0x7fe8d0164230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:44.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.152+0000 7fe8d69db640 1 --2- 172.21.15.89:0/1692618556 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8d00c9970 0x7fe8d0164230 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8009dc0 tx=0x7fe8c801c7d0 comp rx=0 tx=0).stop 2023-10-18T10:46:44.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.152+0000 7fe8d69db640 1 -- 172.21.15.89:0/1692618556 shutdown_connections 2023-10-18T10:46:44.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.152+0000 7fe8d69db640 1 --2- 172.21.15.89:0/1692618556 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8d00c9970 0x7fe8d0164230 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:44.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.152+0000 7fe8d69db640 1 -- 172.21.15.89:0/1692618556 >> 172.21.15.89:0/1692618556 conn(0x7fe8d0069f00 msgr2=0x7fe8d00cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:44.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.152+0000 7fe8d69db640 1 -- 172.21.15.89:0/1692618556 shutdown_connections 2023-10-18T10:46:44.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.152+0000 7fe8d69db640 1 -- 172.21.15.89:0/1692618556 wait complete. 2023-10-18T10:46:44.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.156+0000 7fe8d69db640 1 Processor -- start 2023-10-18T10:46:44.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.156+0000 7fe8d69db640 1 -- start start 2023-10-18T10:46:44.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.156+0000 7fe8d69db640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8d0070600 0x7fe8d00c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:44.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.156+0000 7fe8d69db640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8d00c9970 0x7fe8d0162a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:44.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.156+0000 7fe8d69db640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8d0159490 0x7fe8d00b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:44.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.156+0000 7fe8d69db640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d00ae150 con 0x7fe8d00c9970 2023-10-18T10:46:44.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.156+0000 7fe8d69db640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d00c7e30 con 0x7fe8d0159490 2023-10-18T10:46:44.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.156+0000 7fe8d69db640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d0163380 con 0x7fe8d0070600 2023-10-18T10:46:44.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.156+0000 7fe8cf7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8d00c9970 0x7fe8d0162a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:44.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.156+0000 7fe8cf7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8d00c9970 0x7fe8d0162a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56074/0 (socket says 172.21.15.89:56074) 2023-10-18T10:46:44.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.156+0000 7fe8cf7fe640 1 -- 172.21.15.89:0/1773907317 learned_addr learned my addr 172.21.15.89:0/1773907317 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:44.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.156+0000 7fe8d4f78640 1 --2- 172.21.15.89:0/1773907317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8d0159490 0x7fe8d00b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:44.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.156+0000 7fe8cffff640 1 --2- 172.21.15.89:0/1773907317 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8d0070600 0x7fe8d00c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:44.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.156+0000 7fe8cf7fe640 1 -- 172.21.15.89:0/1773907317 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8d0070600 msgr2=0x7fe8d00c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:44.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.156+0000 7fe8cf7fe640 1 --2- 172.21.15.89:0/1773907317 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8d0070600 0x7fe8d00c78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:44.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.156+0000 7fe8cf7fe640 1 -- 172.21.15.89:0/1773907317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8d0159490 msgr2=0x7fe8d00b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:44.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.156+0000 7fe8cf7fe640 1 --2- 172.21.15.89:0/1773907317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8d0159490 0x7fe8d00b1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:44.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.156+0000 7fe8cf7fe640 1 -- 172.21.15.89:0/1773907317 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c8009a70 con 0x7fe8d00c9970 2023-10-18T10:46:44.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.156+0000 7fe8d4f78640 1 --2- 172.21.15.89:0/1773907317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8d0159490 0x7fe8d00b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:44.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.156+0000 7fe8cffff640 1 --2- 172.21.15.89:0/1773907317 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8d0070600 0x7fe8d00c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:44.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.156+0000 7fe8cf7fe640 1 --2- 172.21.15.89:0/1773907317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8d00c9970 0x7fe8d0162a80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe8c400c780 tx=0x7fe8c400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:44.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.156+0000 7fe8ccff9640 1 -- 172.21.15.89:0/1773907317 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4005cf0 con 0x7fe8d00c9970 2023-10-18T10:46:44.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.156+0000 7fe8ccff9640 1 -- 172.21.15.89:0/1773907317 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c4005e70 con 0x7fe8d00c9970 2023-10-18T10:46:44.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.156+0000 7fe8ccff9640 1 -- 172.21.15.89:0/1773907317 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4007290 con 0x7fe8d00c9970 2023-10-18T10:46:44.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.156+0000 7fe8d69db640 1 -- 172.21.15.89:0/1773907317 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d0163660 con 0x7fe8d00c9970 2023-10-18T10:46:44.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.156+0000 7fe8d69db640 1 -- 172.21.15.89:0/1773907317 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d00ae680 con 0x7fe8d00c9970 2023-10-18T10:46:44.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.160+0000 7fe8d69db640 1 -- 172.21.15.89:0/1773907317 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d004dbe0 con 0x7fe8d00c9970 2023-10-18T10:46:44.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.160+0000 7fe8ccff9640 1 -- 172.21.15.89:0/1773907317 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fe8c40073f0 con 0x7fe8d00c9970 2023-10-18T10:46:44.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.160+0000 7fe8ccff9640 1 --2- 172.21.15.89:0/1773907317 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe8bc0656b0 0x7fe8bc067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:44.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.160+0000 7fe8ccff9640 1 -- 172.21.15.89:0/1773907317 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe8c4082eb0 con 0x7fe8d00c9970 2023-10-18T10:46:44.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.160+0000 7fe8cffff640 1 --2- 172.21.15.89:0/1773907317 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe8bc0656b0 0x7fe8bc067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:44.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.160+0000 7fe8cffff640 1 --2- 172.21.15.89:0/1773907317 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe8bc0656b0 0x7fe8bc067b70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe8c00011f0 tx=0x7fe8c0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:44.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.164+0000 7fe8ccff9640 1 -- 172.21.15.89:0/1773907317 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fe8c4050640 con 0x7fe8d00c9970 2023-10-18T10:46:44.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.312+0000 7fe8d69db640 1 -- 172.21.15.89:0/1773907317 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe8d00cbb60 con 0x7fe8d00c9970 2023-10-18T10:46:44.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.316+0000 7fe8ccff9640 1 -- 172.21.15.89:0/1773907317 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe8c4053c60 con 0x7fe8d00c9970 2023-10-18T10:46:44.319 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-18T10:46:44.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.316+0000 7fe8d69db640 1 -- 172.21.15.89:0/1773907317 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe8bc0656b0 msgr2=0x7fe8bc067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:44.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.316+0000 7fe8d69db640 1 --2- 172.21.15.89:0/1773907317 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe8bc0656b0 0x7fe8bc067b70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe8c00011f0 tx=0x7fe8c0009040 comp rx=0 tx=0).stop 2023-10-18T10:46:44.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.316+0000 7fe8d69db640 1 -- 172.21.15.89:0/1773907317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8d00c9970 msgr2=0x7fe8d0162a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:44.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.316+0000 7fe8d69db640 1 --2- 172.21.15.89:0/1773907317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8d00c9970 0x7fe8d0162a80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe8c400c780 tx=0x7fe8c400cc40 comp rx=0 tx=0).stop 2023-10-18T10:46:44.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.316+0000 7fe8d69db640 1 -- 172.21.15.89:0/1773907317 shutdown_connections 2023-10-18T10:46:44.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.316+0000 7fe8d69db640 1 --2- 172.21.15.89:0/1773907317 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe8bc0656b0 0x7fe8bc067b70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:44.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.316+0000 7fe8d69db640 1 --2- 172.21.15.89:0/1773907317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8d0159490 0x7fe8d00b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:44.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.316+0000 7fe8d69db640 1 --2- 172.21.15.89:0/1773907317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8d00c9970 0x7fe8d0162a80 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:44.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.316+0000 7fe8d69db640 1 --2- 172.21.15.89:0/1773907317 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8d0070600 0x7fe8d00c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:44.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.316+0000 7fe8d69db640 1 -- 172.21.15.89:0/1773907317 >> 172.21.15.89:0/1773907317 conn(0x7fe8d0069f00 msgr2=0x7fe8d00cc400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:44.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.316+0000 7fe8d69db640 1 -- 172.21.15.89:0/1773907317 shutdown_connections 2023-10-18T10:46:44.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.316+0000 7fe8d69db640 1 -- 172.21.15.89:0/1773907317 wait complete. 2023-10-18T10:46:44.340 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:46:44.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.476+0000 7f6f648d8640 1 Processor -- start 2023-10-18T10:46:44.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.476+0000 7f6f648d8640 1 -- start start 2023-10-18T10:46:44.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.476+0000 7f6f648d8640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6f600b1950 0x7f6f600d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:44.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.476+0000 7f6f648d8640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f600b1430 con 0x7f6f6006a010 2023-10-18T10:46:44.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.476+0000 7f6f648d8640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f600d3080 con 0x7f6f6015ff30 2023-10-18T10:46:44.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.476+0000 7f6f648d8640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f600d3690 con 0x7f6f600b1950 2023-10-18T10:46:44.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.476+0000 7f6f5ed9d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6f600b1950 0x7f6f600d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:44.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.476+0000 7f6f5ed9d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6f600b1950 0x7f6f600d0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56088/0 (socket says 172.21.15.89:56088) 2023-10-18T10:46:44.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.476+0000 7f6f5ed9d640 1 -- 172.21.15.89:0/3625321140 learned_addr learned my addr 172.21.15.89:0/3625321140 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:44.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.480+0000 7f6f5ed9d640 1 -- 172.21.15.89:0/3625321140 >> v1:172.21.15.154:6789/0 conn(0x7f6f6015ff30 legacy=0x7f6f60069900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:44.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.480+0000 7f6f5ed9d640 1 -- 172.21.15.89:0/3625321140 >> v1:172.21.15.89:6789/0 conn(0x7f6f6006a010 legacy=0x7f6f600b0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:44.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.480+0000 7f6f5ed9d640 1 -- 172.21.15.89:0/3625321140 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f600d3320 con 0x7f6f600b1950 2023-10-18T10:46:44.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.480+0000 7f6f5d59a640 1 -- 172.21.15.89:0/3625321140 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 297036859 0 0) 0x7f6f600b1430 con 0x7f6f6006a010 2023-10-18T10:46:44.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.480+0000 7f6f5ed9d640 1 --2- 172.21.15.89:0/3625321140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f600b1950 0x7f6f600d0fd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6f58009870 tx=0x7f6f5801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dccee0ca30d80d05 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:44.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.480+0000 7f6f5d59a640 1 -- 172.21.15.89:0/3625321140 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f580ab640 con 0x7f6f600b1950 2023-10-18T10:46:44.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.480+0000 7f6f5d59a640 1 -- 172.21.15.89:0/3625321140 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f580a7070 con 0x7f6f600b1950 2023-10-18T10:46:44.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.480+0000 7f6f5d59a640 1 -- 172.21.15.89:0/3625321140 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f580ab9c0 con 0x7f6f600b1950 2023-10-18T10:46:44.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.480+0000 7f6f648d8640 1 -- 172.21.15.89:0/3625321140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f600b1950 msgr2=0x7f6f600d0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:44.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.480+0000 7f6f648d8640 1 --2- 172.21.15.89:0/3625321140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f600b1950 0x7f6f600d0fd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6f58009870 tx=0x7f6f5801c420 comp rx=0 tx=0).stop 2023-10-18T10:46:44.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.480+0000 7f6f648d8640 1 -- 172.21.15.89:0/3625321140 shutdown_connections 2023-10-18T10:46:44.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.480+0000 7f6f648d8640 1 --2- 172.21.15.89:0/3625321140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f600b1950 0x7f6f600d0fd0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:44.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.480+0000 7f6f648d8640 1 -- 172.21.15.89:0/3625321140 >> 172.21.15.89:0/3625321140 conn(0x7f6f6009b9d0 msgr2=0x7f6f600d26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:44.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.480+0000 7f6f648d8640 1 -- 172.21.15.89:0/3625321140 shutdown_connections 2023-10-18T10:46:44.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.484+0000 7f6f648d8640 1 -- 172.21.15.89:0/3625321140 wait complete. 2023-10-18T10:46:44.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.484+0000 7f6f648d8640 1 Processor -- start 2023-10-18T10:46:44.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.484+0000 7f6f648d8640 1 -- start start 2023-10-18T10:46:44.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.484+0000 7f6f648d8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f6006a010 0x7f6f600ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:44.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.484+0000 7f6f648d8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f600b1950 0x7f6f600caf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:44.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.484+0000 7f6f648d8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f6015ff30 0x7f6f600b2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:44.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.484+0000 7f6f648d8640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f600c9790 con 0x7f6f6006a010 2023-10-18T10:46:44.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.484+0000 7f6f648d8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f600cba50 con 0x7f6f600b1950 2023-10-18T10:46:44.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.484+0000 7f6f648d8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f600cbd30 con 0x7f6f6015ff30 2023-10-18T10:46:44.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.484+0000 7f6f5e59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f6006a010 0x7f6f600ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:44.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.484+0000 7f6f5e59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f6006a010 0x7f6f600ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56096/0 (socket says 172.21.15.89:56096) 2023-10-18T10:46:44.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.484+0000 7f6f5e59c640 1 -- 172.21.15.89:0/1585930894 learned_addr learned my addr 172.21.15.89:0/1585930894 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:44.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.484+0000 7f6f5dd9b640 1 --2- 172.21.15.89:0/1585930894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f600b1950 0x7f6f600caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:44.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.484+0000 7f6f5ed9d640 1 --2- 172.21.15.89:0/1585930894 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f6015ff30 0x7f6f600b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:44.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.484+0000 7f6f5e59c640 1 -- 172.21.15.89:0/1585930894 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f6015ff30 msgr2=0x7f6f600b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:44.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.484+0000 7f6f5e59c640 1 --2- 172.21.15.89:0/1585930894 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f6015ff30 0x7f6f600b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:44.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.484+0000 7f6f5e59c640 1 -- 172.21.15.89:0/1585930894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f600b1950 msgr2=0x7f6f600caf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:44.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.484+0000 7f6f5e59c640 1 --2- 172.21.15.89:0/1585930894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f600b1950 0x7f6f600caf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:44.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.484+0000 7f6f5e59c640 1 -- 172.21.15.89:0/1585930894 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f580094d0 con 0x7f6f6006a010 2023-10-18T10:46:44.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.484+0000 7f6f5e59c640 1 --2- 172.21.15.89:0/1585930894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f6006a010 0x7f6f600ca1e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6f5000c730 tx=0x7f6f5000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:44.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.488+0000 7f6f4effd640 1 -- 172.21.15.89:0/1585930894 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f50003bd0 con 0x7f6f6006a010 2023-10-18T10:46:44.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.488+0000 7f6f4effd640 1 -- 172.21.15.89:0/1585930894 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f50003d50 con 0x7f6f6006a010 2023-10-18T10:46:44.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.488+0000 7f6f648d8640 1 -- 172.21.15.89:0/1585930894 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f600c99e0 con 0x7f6f6006a010 2023-10-18T10:46:44.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.488+0000 7f6f4effd640 1 -- 172.21.15.89:0/1585930894 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5000aa90 con 0x7f6f6006a010 2023-10-18T10:46:44.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.488+0000 7f6f648d8640 1 -- 172.21.15.89:0/1585930894 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f600ca720 con 0x7f6f6006a010 2023-10-18T10:46:44.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.488+0000 7f6f648d8640 1 -- 172.21.15.89:0/1585930894 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f600cbef0 con 0x7f6f6006a010 2023-10-18T10:46:44.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.488+0000 7f6f4effd640 1 -- 172.21.15.89:0/1585930894 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f6f5000abf0 con 0x7f6f6006a010 2023-10-18T10:46:44.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.488+0000 7f6f4effd640 1 --2- 172.21.15.89:0/1585930894 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6f40065680 0x7f6f40067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:44.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.488+0000 7f6f4effd640 1 -- 172.21.15.89:0/1585930894 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6f50083050 con 0x7f6f6006a010 2023-10-18T10:46:44.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.488+0000 7f6f5dd9b640 1 --2- 172.21.15.89:0/1585930894 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6f40065680 0x7f6f40067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:44.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.492+0000 7f6f5dd9b640 1 --2- 172.21.15.89:0/1585930894 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6f40065680 0x7f6f40067b40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6f600d0020 tx=0x7f6f54009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:44.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.492+0000 7f6f4effd640 1 -- 172.21.15.89:0/1585930894 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f6f500b3d40 con 0x7f6f6006a010 2023-10-18T10:46:44.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.640+0000 7f6f648d8640 1 -- 172.21.15.89:0/1585930894 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6f600c9f00 con 0x7f6f6006a010 2023-10-18T10:46:44.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.640+0000 7f6f4effd640 1 -- 172.21.15.89:0/1585930894 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6f600c9f00 con 0x7f6f6006a010 2023-10-18T10:46:44.647 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-18T10:46:44.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.644+0000 7f6f648d8640 1 -- 172.21.15.89:0/1585930894 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6f40065680 msgr2=0x7f6f40067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:44.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.644+0000 7f6f648d8640 1 --2- 172.21.15.89:0/1585930894 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6f40065680 0x7f6f40067b40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6f600d0020 tx=0x7f6f54009040 comp rx=0 tx=0).stop 2023-10-18T10:46:44.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.644+0000 7f6f648d8640 1 -- 172.21.15.89:0/1585930894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f6006a010 msgr2=0x7f6f600ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:44.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.644+0000 7f6f648d8640 1 --2- 172.21.15.89:0/1585930894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f6006a010 0x7f6f600ca1e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6f5000c730 tx=0x7f6f5000cbf0 comp rx=0 tx=0).stop 2023-10-18T10:46:44.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.644+0000 7f6f648d8640 1 -- 172.21.15.89:0/1585930894 shutdown_connections 2023-10-18T10:46:44.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.644+0000 7f6f648d8640 1 --2- 172.21.15.89:0/1585930894 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6f40065680 0x7f6f40067b40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:44.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.644+0000 7f6f648d8640 1 --2- 172.21.15.89:0/1585930894 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f6015ff30 0x7f6f600b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:44.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.644+0000 7f6f648d8640 1 --2- 172.21.15.89:0/1585930894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f600b1950 0x7f6f600caf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:44.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.644+0000 7f6f648d8640 1 --2- 172.21.15.89:0/1585930894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f6006a010 0x7f6f600ca1e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:44.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.644+0000 7f6f648d8640 1 -- 172.21.15.89:0/1585930894 >> 172.21.15.89:0/1585930894 conn(0x7f6f6009b9d0 msgr2=0x7f6f600b3540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:44.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.644+0000 7f6f648d8640 1 -- 172.21.15.89:0/1585930894 shutdown_connections 2023-10-18T10:46:44.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.644+0000 7f6f648d8640 1 -- 172.21.15.89:0/1585930894 wait complete. 2023-10-18T10:46:44.667 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T10:46:44.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.804+0000 7f09f7493640 1 Processor -- start 2023-10-18T10:46:44.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.804+0000 7f09f7493640 1 -- start start 2023-10-18T10:46:44.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.804+0000 7f09f7493640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f09f015c940 0x7f09f00d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:44.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.804+0000 7f09f7493640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f09f00d2f40 0x7f09f00d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:44.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.804+0000 7f09f7493640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09f00d10f0 con 0x7f09f00d0ca0 2023-10-18T10:46:44.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.804+0000 7f09f7493640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09f015da50 con 0x7f09f015c940 2023-10-18T10:46:44.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.804+0000 7f09f7493640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09f015ce40 con 0x7f09f00d2f40 2023-10-18T10:46:44.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.804+0000 7f09f522f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f09f015c940 0x7f09f00d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:44.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.804+0000 7f09f522f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f09f015c940 0x7f09f00d2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56100/0 (socket says 172.21.15.89:56100) 2023-10-18T10:46:44.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.804+0000 7f09f522f640 1 -- 172.21.15.89:0/1752315360 learned_addr learned my addr 172.21.15.89:0/1752315360 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:44.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.804+0000 7f09f4a2e640 1 --2- 172.21.15.89:0/1752315360 >> v2:172.21.15.154:3300/0 conn(0x7f09f00d2f40 0x7f09f00d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:44.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.804+0000 7f09f522f640 1 -- 172.21.15.89:0/1752315360 >> v2:172.21.15.154:3300/0 conn(0x7f09f00d2f40 msgr2=0x7f09f00d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:44.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.804+0000 7f09f522f640 1 --2- 172.21.15.89:0/1752315360 >> v2:172.21.15.154:3300/0 conn(0x7f09f00d2f40 0x7f09f00d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:44.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.804+0000 7f09f522f640 1 -- 172.21.15.89:0/1752315360 >> v1:172.21.15.154:6789/0 conn(0x7f09f00d0ca0 legacy=0x7f09f00cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:44.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.804+0000 7f09f522f640 1 -- 172.21.15.89:0/1752315360 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f015d120 con 0x7f09f015c940 2023-10-18T10:46:44.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.804+0000 7f09f4a2e640 1 --2- 172.21.15.89:0/1752315360 >> v2:172.21.15.154:3300/0 conn(0x7f09f00d2f40 0x7f09f00d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:44.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.808+0000 7f09f522f640 1 --2- 172.21.15.89:0/1752315360 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09f015c940 0x7f09f00d2a00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f09f00724f0 tx=0x7f09ec01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=108624a16b858dc9 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:44.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.808+0000 7f09e7fff640 1 -- 172.21.15.89:0/1752315360 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec0ac640 con 0x7f09f015c940 2023-10-18T10:46:44.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.808+0000 7f09e7fff640 1 -- 172.21.15.89:0/1752315360 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ec0ac7a0 con 0x7f09f015c940 2023-10-18T10:46:44.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.808+0000 7f09e7fff640 1 -- 172.21.15.89:0/1752315360 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec0acb30 con 0x7f09f015c940 2023-10-18T10:46:44.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.808+0000 7f09f7493640 1 -- 172.21.15.89:0/1752315360 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09f015c940 msgr2=0x7f09f00d2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:44.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.808+0000 7f09f7493640 1 --2- 172.21.15.89:0/1752315360 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09f015c940 0x7f09f00d2a00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f09f00724f0 tx=0x7f09ec01c4d0 comp rx=0 tx=0).stop 2023-10-18T10:46:44.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.808+0000 7f09f7493640 1 -- 172.21.15.89:0/1752315360 shutdown_connections 2023-10-18T10:46:44.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.808+0000 7f09f7493640 1 --2- 172.21.15.89:0/1752315360 >> v2:172.21.15.154:3300/0 conn(0x7f09f00d2f40 0x7f09f00d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:44.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.808+0000 7f09f7493640 1 --2- 172.21.15.89:0/1752315360 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09f015c940 0x7f09f00d2a00 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:44.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.808+0000 7f09f7493640 1 -- 172.21.15.89:0/1752315360 >> 172.21.15.89:0/1752315360 conn(0x7f09f0089fd0 msgr2=0x7f09f009b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:44.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.808+0000 7f09f7493640 1 -- 172.21.15.89:0/1752315360 shutdown_connections 2023-10-18T10:46:44.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.808+0000 7f09f7493640 1 -- 172.21.15.89:0/1752315360 wait complete. 2023-10-18T10:46:44.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.812+0000 7f09f7493640 1 Processor -- start 2023-10-18T10:46:44.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.812+0000 7f09f7493640 1 -- start start 2023-10-18T10:46:44.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.812+0000 7f09f7493640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09f00d0ca0 0x7f09f0154710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:44.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.812+0000 7f09f7493640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09f0156420 0x7f09f00c6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:44.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.812+0000 7f09f7493640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09f00cbf70 0x7f09f015e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:44.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.812+0000 7f09f7493640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f0154c50 con 0x7f09f00d0ca0 2023-10-18T10:46:44.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.812+0000 7f09f7493640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f00cab90 con 0x7f09f00cbf70 2023-10-18T10:46:44.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.812+0000 7f09f7493640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09f00cb520 con 0x7f09f0156420 2023-10-18T10:46:44.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.812+0000 7f09f522f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09f00d0ca0 0x7f09f0154710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:44.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.812+0000 7f09f522f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09f00d0ca0 0x7f09f0154710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56108/0 (socket says 172.21.15.89:56108) 2023-10-18T10:46:44.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.812+0000 7f09f522f640 1 -- 172.21.15.89:0/3583924722 learned_addr learned my addr 172.21.15.89:0/3583924722 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:44.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.812+0000 7f09f5a30640 1 --2- 172.21.15.89:0/3583924722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09f00cbf70 0x7f09f015e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:44.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.812+0000 7f09f4a2e640 1 --2- 172.21.15.89:0/3583924722 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09f0156420 0x7f09f00c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:44.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.812+0000 7f09f522f640 1 -- 172.21.15.89:0/3583924722 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09f0156420 msgr2=0x7f09f00c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:44.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.812+0000 7f09f522f640 1 --2- 172.21.15.89:0/3583924722 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09f0156420 0x7f09f00c6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:44.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.812+0000 7f09f522f640 1 -- 172.21.15.89:0/3583924722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09f00cbf70 msgr2=0x7f09f015e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:44.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.812+0000 7f09f522f640 1 --2- 172.21.15.89:0/3583924722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09f00cbf70 0x7f09f015e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:44.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.812+0000 7f09f522f640 1 -- 172.21.15.89:0/3583924722 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ec0094d0 con 0x7f09f00d0ca0 2023-10-18T10:46:44.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.812+0000 7f09f522f640 1 --2- 172.21.15.89:0/3583924722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09f00d0ca0 0x7f09f0154710 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f09ec01a100 tx=0x7f09ec01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:44.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.812+0000 7f09e5ffb640 1 -- 172.21.15.89:0/3583924722 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec0b7830 con 0x7f09f00d0ca0 2023-10-18T10:46:44.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.812+0000 7f09e5ffb640 1 -- 172.21.15.89:0/3583924722 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ec0b7990 con 0x7f09f00d0ca0 2023-10-18T10:46:44.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.812+0000 7f09e5ffb640 1 -- 172.21.15.89:0/3583924722 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec0b7d00 con 0x7f09f00d0ca0 2023-10-18T10:46:44.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.812+0000 7f09f7493640 1 -- 172.21.15.89:0/3583924722 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09f00c66c0 con 0x7f09f00d0ca0 2023-10-18T10:46:44.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.812+0000 7f09f7493640 1 -- 172.21.15.89:0/3583924722 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09f00cb0c0 con 0x7f09f00d0ca0 2023-10-18T10:46:44.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.816+0000 7f09f7493640 1 -- 172.21.15.89:0/3583924722 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09f004dbe0 con 0x7f09f00d0ca0 2023-10-18T10:46:44.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.816+0000 7f09e5ffb640 1 -- 172.21.15.89:0/3583924722 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f09ec0b3040 con 0x7f09f00d0ca0 2023-10-18T10:46:44.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.816+0000 7f09e5ffb640 1 --2- 172.21.15.89:0/3583924722 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f09d40656b0 0x7f09d4067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:44.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.816+0000 7f09e5ffb640 1 -- 172.21.15.89:0/3583924722 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f09ec11d7e0 con 0x7f09f00d0ca0 2023-10-18T10:46:44.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.816+0000 7f09f4a2e640 1 --2- 172.21.15.89:0/3583924722 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f09d40656b0 0x7f09d4067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:44.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.816+0000 7f09f4a2e640 1 --2- 172.21.15.89:0/3583924722 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f09d40656b0 0x7f09d4067b70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f09f00b7fe0 tx=0x7f09e0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:44.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.820+0000 7f09e5ffb640 1 -- 172.21.15.89:0/3583924722 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f09ec0eaed0 con 0x7f09f00d0ca0 2023-10-18T10:46:44.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.968+0000 7f09f7493640 1 -- 172.21.15.89:0/3583924722 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f09f00b85d0 con 0x7f09f00d0ca0 2023-10-18T10:46:44.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.968+0000 7f09e5ffb640 1 -- 172.21.15.89:0/3583924722 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f09ec0b7e60 con 0x7f09f00d0ca0 2023-10-18T10:46:44.975 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-18T10:46:44.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.972+0000 7f09f7493640 1 -- 172.21.15.89:0/3583924722 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f09d40656b0 msgr2=0x7f09d4067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:44.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.972+0000 7f09f7493640 1 --2- 172.21.15.89:0/3583924722 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f09d40656b0 0x7f09d4067b70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f09f00b7fe0 tx=0x7f09e0008040 comp rx=0 tx=0).stop 2023-10-18T10:46:44.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.972+0000 7f09f7493640 1 -- 172.21.15.89:0/3583924722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09f00d0ca0 msgr2=0x7f09f0154710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:44.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.972+0000 7f09f7493640 1 --2- 172.21.15.89:0/3583924722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09f00d0ca0 0x7f09f0154710 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f09ec01a100 tx=0x7f09ec01ad10 comp rx=0 tx=0).stop 2023-10-18T10:46:44.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.972+0000 7f09f7493640 1 -- 172.21.15.89:0/3583924722 shutdown_connections 2023-10-18T10:46:44.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.972+0000 7f09f7493640 1 --2- 172.21.15.89:0/3583924722 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f09d40656b0 0x7f09d4067b70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:44.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.972+0000 7f09f7493640 1 --2- 172.21.15.89:0/3583924722 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09f0156420 0x7f09f00c6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:44.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.972+0000 7f09f7493640 1 --2- 172.21.15.89:0/3583924722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09f00cbf70 0x7f09f015e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:44.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.972+0000 7f09f7493640 1 --2- 172.21.15.89:0/3583924722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09f00d0ca0 0x7f09f0154710 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:44.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.972+0000 7f09f7493640 1 -- 172.21.15.89:0/3583924722 >> 172.21.15.89:0/3583924722 conn(0x7f09f0089fd0 msgr2=0x7f09f006af80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:44.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.972+0000 7f09f7493640 1 -- 172.21.15.89:0/3583924722 shutdown_connections 2023-10-18T10:46:44.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:44.972+0000 7f09f7493640 1 -- 172.21.15.89:0/3583924722 wait complete. 2023-10-18T10:46:44.996 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-18T10:46:45.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.132+0000 7faba965b640 1 Processor -- start 2023-10-18T10:46:45.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.132+0000 7faba965b640 1 -- start start 2023-10-18T10:46:45.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.136+0000 7faba965b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7faba415c5f0 0x7faba40d0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:45.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.136+0000 7faba965b640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faba40cd040 con 0x7faba40ce1a0 2023-10-18T10:46:45.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.136+0000 7faba965b640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faba40d0af0 con 0x7faba40b81c0 2023-10-18T10:46:45.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.136+0000 7faba965b640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faba40b78c0 con 0x7faba415c5f0 2023-10-18T10:46:45.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.136+0000 7faba8e5a640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7faba40ce1a0 0x7faba40b5460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48650/0 (socket says 172.21.15.89:48650) 2023-10-18T10:46:45.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.136+0000 7faba8e5a640 1 -- 172.21.15.89:0/4083646452 learned_addr learned my addr 172.21.15.89:0/4083646452 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:45.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.136+0000 7faba2ffd640 1 -- 172.21.15.89:0/4083646452 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3042653177 0 0) 0x7faba40cd040 con 0x7faba40ce1a0 2023-10-18T10:46:45.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.136+0000 7faba2ffd640 1 -- 172.21.15.89:0/4083646452 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab8c003530 con 0x7faba40ce1a0 2023-10-18T10:46:45.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.136+0000 7faba3fff640 1 --2- 172.21.15.89:0/4083646452 >> v2:172.21.15.154:3300/0 conn(0x7faba415c5f0 0x7faba40d0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:45.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.136+0000 7faba2ffd640 1 -- 172.21.15.89:0/4083646452 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1241708023 0 0) 0x7fab8c003530 con 0x7faba40ce1a0 2023-10-18T10:46:45.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.136+0000 7faba2ffd640 1 -- 172.21.15.89:0/4083646452 >> v2:172.21.15.154:3300/0 conn(0x7faba415c5f0 msgr2=0x7faba40d0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:45.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.136+0000 7faba2ffd640 1 --2- 172.21.15.89:0/4083646452 >> v2:172.21.15.154:3300/0 conn(0x7faba415c5f0 0x7faba40d0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:45.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.136+0000 7faba2ffd640 1 -- 172.21.15.89:0/4083646452 >> v1:172.21.15.154:6789/0 conn(0x7faba40b81c0 legacy=0x7faba40cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:45.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.136+0000 7faba2ffd640 1 -- 172.21.15.89:0/4083646452 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba40d0dd0 con 0x7faba40ce1a0 2023-10-18T10:46:45.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.136+0000 7faba2ffd640 1 -- 172.21.15.89:0/4083646452 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7fab9c001a00 con 0x7faba40ce1a0 2023-10-18T10:46:45.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.136+0000 7faba2ffd640 1 -- 172.21.15.89:0/4083646452 >> v1:172.21.15.89:6789/0 conn(0x7faba40ce1a0 legacy=0x7faba40b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:45.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.136+0000 7faba2ffd640 1 --2- 172.21.15.89:0/4083646452 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab8c0039f0 0x7fab8c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:45.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.136+0000 7faba2ffd640 1 --2- 172.21.15.89:0/4083646452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab8c0047d0 0x7fab8c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:45.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.136+0000 7faba2ffd640 1 --2- 172.21.15.89:0/4083646452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab8c0052b0 0x7fab8c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:45.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.136+0000 7faba2ffd640 1 -- 172.21.15.89:0/4083646452 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba40b78c0 con 0x7fab8c0052b0 2023-10-18T10:46:45.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.136+0000 7faba2ffd640 1 -- 172.21.15.89:0/4083646452 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c00c200 con 0x7fab8c0047d0 2023-10-18T10:46:45.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.136+0000 7faba2ffd640 1 -- 172.21.15.89:0/4083646452 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c00c3c0 con 0x7fab8c0039f0 2023-10-18T10:46:45.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.136+0000 7faba8e5a640 1 --2- 172.21.15.89:0/4083646452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab8c0052b0 0x7fab8c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:45.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.136+0000 7faba3fff640 1 --2- 172.21.15.89:0/4083646452 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab8c0039f0 0x7fab8c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:45.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.136+0000 7faba37fe640 1 --2- 172.21.15.89:0/4083646452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab8c0047d0 0x7fab8c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:45.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.136+0000 7faba8e5a640 1 -- 172.21.15.89:0/4083646452 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab8c0039f0 msgr2=0x7fab8c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:45.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.136+0000 7faba8e5a640 1 --2- 172.21.15.89:0/4083646452 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab8c0039f0 0x7fab8c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:45.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.136+0000 7faba8e5a640 1 -- 172.21.15.89:0/4083646452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab8c0047d0 msgr2=0x7fab8c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:45.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.136+0000 7faba8e5a640 1 --2- 172.21.15.89:0/4083646452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab8c0047d0 0x7fab8c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:45.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.136+0000 7faba8e5a640 1 -- 172.21.15.89:0/4083646452 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab8c003530 con 0x7fab8c0052b0 2023-10-18T10:46:45.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.136+0000 7faba37fe640 1 --2- 172.21.15.89:0/4083646452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab8c0047d0 0x7fab8c004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:45.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.140+0000 7faba8e5a640 1 -- 172.21.15.89:0/4083646452 reap_dead start 2023-10-18T10:46:45.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.140+0000 7faba8e5a640 1 --2- 172.21.15.89:0/4083646452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab8c0052b0 0x7fab8c00b990 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fab9c00a800 tx=0x7fab9c01fa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0f56f2e1eda662f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:45.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.140+0000 7faba2ffd640 1 -- 172.21.15.89:0/4083646452 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c0072d0 con 0x7fab8c0052b0 2023-10-18T10:46:45.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.140+0000 7faba2ffd640 1 -- 172.21.15.89:0/4083646452 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9c007430 con 0x7fab8c0052b0 2023-10-18T10:46:45.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.140+0000 7faba2ffd640 1 -- 172.21.15.89:0/4083646452 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c0077e0 con 0x7fab8c0052b0 2023-10-18T10:46:45.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.140+0000 7faba965b640 1 -- 172.21.15.89:0/4083646452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab8c0052b0 msgr2=0x7fab8c00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:45.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.140+0000 7faba965b640 1 --2- 172.21.15.89:0/4083646452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab8c0052b0 0x7fab8c00b990 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fab9c00a800 tx=0x7fab9c01fa60 comp rx=0 tx=0).stop 2023-10-18T10:46:45.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.140+0000 7faba965b640 1 -- 172.21.15.89:0/4083646452 shutdown_connections 2023-10-18T10:46:45.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.140+0000 7faba965b640 1 --2- 172.21.15.89:0/4083646452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab8c0052b0 0x7fab8c00b990 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:45.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.140+0000 7faba965b640 1 -- 172.21.15.89:0/4083646452 >> 172.21.15.89:0/4083646452 conn(0x7faba4070320 msgr2=0x7faba4070730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:45.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.140+0000 7faba965b640 1 -- 172.21.15.89:0/4083646452 shutdown_connections 2023-10-18T10:46:45.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.140+0000 7faba965b640 1 -- 172.21.15.89:0/4083646452 wait complete. 2023-10-18T10:46:45.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.140+0000 7faba965b640 1 Processor -- start 2023-10-18T10:46:45.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.144+0000 7faba965b640 1 -- start start 2023-10-18T10:46:45.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.144+0000 7faba965b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab8c0052b0 0x7faba4154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:45.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.144+0000 7faba965b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faba4155920 0x7faba4153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:45.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.144+0000 7faba965b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faba41f59e0 0x7faba41f5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:45.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.144+0000 7faba965b640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba4156180 con 0x7faba4155920 2023-10-18T10:46:45.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.144+0000 7faba965b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba41f5550 con 0x7fab8c0052b0 2023-10-18T10:46:45.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.144+0000 7faba965b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba41551c0 con 0x7faba41f59e0 2023-10-18T10:46:45.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.144+0000 7faba37fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faba4155920 0x7faba4153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:45.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.144+0000 7faba37fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faba4155920 0x7faba4153250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56126/0 (socket says 172.21.15.89:56126) 2023-10-18T10:46:45.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.144+0000 7faba37fe640 1 -- 172.21.15.89:0/2205037134 learned_addr learned my addr 172.21.15.89:0/2205037134 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:45.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.144+0000 7faba3fff640 1 --2- 172.21.15.89:0/2205037134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab8c0052b0 0x7faba4154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:45.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.144+0000 7faba8e5a640 1 --2- 172.21.15.89:0/2205037134 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faba41f59e0 0x7faba41f5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:45.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.144+0000 7faba37fe640 1 -- 172.21.15.89:0/2205037134 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faba41f59e0 msgr2=0x7faba41f5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:45.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.144+0000 7faba37fe640 1 --2- 172.21.15.89:0/2205037134 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faba41f59e0 0x7faba41f5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:45.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.144+0000 7faba37fe640 1 -- 172.21.15.89:0/2205037134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab8c0052b0 msgr2=0x7faba4154c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:45.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.144+0000 7faba37fe640 1 --2- 172.21.15.89:0/2205037134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab8c0052b0 0x7faba4154c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:45.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.144+0000 7faba37fe640 1 -- 172.21.15.89:0/2205037134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba4156390 con 0x7faba4155920 2023-10-18T10:46:45.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.144+0000 7faba37fe640 1 --2- 172.21.15.89:0/2205037134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faba4155920 0x7faba4153250 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fab9800e440 tx=0x7fab98001ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:45.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.144+0000 7faba0ff9640 1 -- 172.21.15.89:0/2205037134 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab980052e0 con 0x7faba4155920 2023-10-18T10:46:45.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.144+0000 7faba0ff9640 1 -- 172.21.15.89:0/2205037134 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab98005440 con 0x7faba4155920 2023-10-18T10:46:45.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.144+0000 7faba0ff9640 1 -- 172.21.15.89:0/2205037134 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab98003870 con 0x7faba4155920 2023-10-18T10:46:45.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.144+0000 7faba965b640 1 -- 172.21.15.89:0/2205037134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba4156670 con 0x7faba4155920 2023-10-18T10:46:45.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.144+0000 7faba965b640 1 -- 172.21.15.89:0/2205037134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faba4156c30 con 0x7faba4155920 2023-10-18T10:46:45.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.144+0000 7faba965b640 1 -- 172.21.15.89:0/2205037134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faba4073b50 con 0x7faba4155920 2023-10-18T10:46:45.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.148+0000 7faba0ff9640 1 -- 172.21.15.89:0/2205037134 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fab980039d0 con 0x7faba4155920 2023-10-18T10:46:45.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.148+0000 7faba0ff9640 1 --2- 172.21.15.89:0/2205037134 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fab90065680 0x7fab90067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:45.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.148+0000 7faba0ff9640 1 -- 172.21.15.89:0/2205037134 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fab980827e0 con 0x7faba4155920 2023-10-18T10:46:45.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.148+0000 7faba3fff640 1 --2- 172.21.15.89:0/2205037134 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fab90065680 0x7fab90067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:45.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.148+0000 7faba3fff640 1 --2- 172.21.15.89:0/2205037134 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fab90065680 0x7fab90067b40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7faba41531e0 tx=0x7fab94008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:45.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.152+0000 7faba0ff9640 1 -- 172.21.15.89:0/2205037134 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fab98050010 con 0x7faba4155920 2023-10-18T10:46:45.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.300+0000 7faba965b640 1 -- 172.21.15.89:0/2205037134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7faba41f5690 con 0x7faba4155920 2023-10-18T10:46:45.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.304+0000 7faba0ff9640 1 -- 172.21.15.89:0/2205037134 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fab98053630 con 0x7faba4155920 2023-10-18T10:46:45.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.308+0000 7faba965b640 1 -- 172.21.15.89:0/2205037134 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fab90065680 msgr2=0x7fab90067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:45.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.308+0000 7faba965b640 1 --2- 172.21.15.89:0/2205037134 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fab90065680 0x7fab90067b40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7faba41531e0 tx=0x7fab94008040 comp rx=0 tx=0).stop 2023-10-18T10:46:45.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.308+0000 7faba965b640 1 -- 172.21.15.89:0/2205037134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faba4155920 msgr2=0x7faba4153250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:45.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.308+0000 7faba965b640 1 --2- 172.21.15.89:0/2205037134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faba4155920 0x7faba4153250 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fab9800e440 tx=0x7fab98001ed0 comp rx=0 tx=0).stop 2023-10-18T10:46:45.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.308+0000 7faba965b640 1 -- 172.21.15.89:0/2205037134 shutdown_connections 2023-10-18T10:46:45.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.308+0000 7faba965b640 1 --2- 172.21.15.89:0/2205037134 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fab90065680 0x7fab90067b40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:45.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.308+0000 7faba965b640 1 --2- 172.21.15.89:0/2205037134 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faba41f59e0 0x7faba41f5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:45.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.308+0000 7faba965b640 1 --2- 172.21.15.89:0/2205037134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faba4155920 0x7faba4153250 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:45.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.308+0000 7faba965b640 1 --2- 172.21.15.89:0/2205037134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab8c0052b0 0x7faba4154c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:45.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.308+0000 7faba965b640 1 -- 172.21.15.89:0/2205037134 >> 172.21.15.89:0/2205037134 conn(0x7faba4070320 msgr2=0x7faba4070700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:45.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.308+0000 7faba965b640 1 -- 172.21.15.89:0/2205037134 shutdown_connections 2023-10-18T10:46:45.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.308+0000 7faba965b640 1 -- 172.21.15.89:0/2205037134 wait complete. 2023-10-18T10:46:45.331 INFO:tasks.cephfs_test_runner:Simulate software freeze or temporary network failure. ... skipped 'Require first client to be on separate server from MDSs' 2023-10-18T10:46:45.331 INFO:tasks.cephfs_test_runner:Starting test: test_client_eviction_if_config_is_set (tasks.cephfs.test_client_recovery.TestClientOnLaggyOSD) If any client gets unresponsive/it's session get idle due to lagginess 2023-10-18T10:46:45.332 INFO:tasks.cephfs_test_runner:test_client_eviction_if_config_is_set (tasks.cephfs.test_client_recovery.TestClientOnLaggyOSD) 2023-10-18T10:46:45.332 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_recovery.TestClientOnLaggyOSD.test_client_eviction_if_config_is_set' 2023-10-18T10:46:45.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.464+0000 7f51c4a56640 1 Processor -- start 2023-10-18T10:46:45.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.464+0000 7f51c4a56640 1 -- start start 2023-10-18T10:46:45.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.468+0000 7f51c4a56640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f51c0070600 0x7f51c006b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:45.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.468+0000 7f51c4a56640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f51c006bd10 0x7f51c00c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:45.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.468+0000 7f51c4a56640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c0070b90 con 0x7f51c0159490 2023-10-18T10:46:45.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.468+0000 7f51c4a56640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51c006c1a0 con 0x7f51c0070600 2023-10-18T10:46:45.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.468+0000 7f51c4a56640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51c0070e70 con 0x7f51c006bd10 2023-10-18T10:46:45.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.468+0000 7f51be59c640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f51c0159490 0x7f51c015d8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48662/0 (socket says 172.21.15.89:48662) 2023-10-18T10:46:45.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.468+0000 7f51be59c640 1 -- 172.21.15.89:0/303980616 learned_addr learned my addr 172.21.15.89:0/303980616 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:45.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.468+0000 7f51bdd9b640 1 --2- 172.21.15.89:0/303980616 >> v2:172.21.15.89:3300/0 conn(0x7f51c0070600 0x7f51c006b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:45.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.468+0000 7f51bed9d640 1 --2- 172.21.15.89:0/303980616 >> v2:172.21.15.154:3300/0 conn(0x7f51c006bd10 0x7f51c00c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:45.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.468+0000 7f51bdd9b640 1 -- 172.21.15.89:0/303980616 >> v2:172.21.15.154:3300/0 conn(0x7f51c006bd10 msgr2=0x7f51c00c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:45.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.468+0000 7f51bdd9b640 1 --2- 172.21.15.89:0/303980616 >> v2:172.21.15.154:3300/0 conn(0x7f51c006bd10 0x7f51c00c8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:45.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.468+0000 7f51bdd9b640 1 -- 172.21.15.89:0/303980616 >> v1:172.21.15.89:6789/0 conn(0x7f51c0159490 legacy=0x7f51c015d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:45.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.468+0000 7f51bdd9b640 1 -- 172.21.15.89:0/303980616 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c0071150 con 0x7f51c0070600 2023-10-18T10:46:45.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.468+0000 7f51bd59a640 1 -- 172.21.15.89:0/303980616 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1788260940 0 0) 0x7f51c0070b90 con 0x7f51c0159490 2023-10-18T10:46:45.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.472+0000 7f51bdd9b640 1 --2- 172.21.15.89:0/303980616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51c0070600 0x7f51c006b7d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f51c00c88f0 tx=0x7f51b401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b29096279ad1c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:45.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.472+0000 7f51bd59a640 1 -- 172.21.15.89:0/303980616 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b40ac600 con 0x7f51c0070600 2023-10-18T10:46:45.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.472+0000 7f51bd59a640 1 -- 172.21.15.89:0/303980616 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b40a8050 con 0x7f51c0070600 2023-10-18T10:46:45.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.472+0000 7f51bd59a640 1 -- 172.21.15.89:0/303980616 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b40ac980 con 0x7f51c0070600 2023-10-18T10:46:45.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.472+0000 7f51c4a56640 1 -- 172.21.15.89:0/303980616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51c0070600 msgr2=0x7f51c006b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:45.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.472+0000 7f51c4a56640 1 --2- 172.21.15.89:0/303980616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51c0070600 0x7f51c006b7d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f51c00c88f0 tx=0x7f51b401c3a0 comp rx=0 tx=0).stop 2023-10-18T10:46:45.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.472+0000 7f51c4a56640 1 -- 172.21.15.89:0/303980616 shutdown_connections 2023-10-18T10:46:45.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.472+0000 7f51c4a56640 1 --2- 172.21.15.89:0/303980616 >> v2:172.21.15.154:3300/0 conn(0x7f51c006bd10 0x7f51c00c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:45.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.472+0000 7f51c4a56640 1 --2- 172.21.15.89:0/303980616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51c0070600 0x7f51c006b7d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:45.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.472+0000 7f51c4a56640 1 -- 172.21.15.89:0/303980616 >> 172.21.15.89:0/303980616 conn(0x7f51c0069f00 msgr2=0x7f51c00cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:45.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.472+0000 7f51c4a56640 1 -- 172.21.15.89:0/303980616 shutdown_connections 2023-10-18T10:46:45.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.472+0000 7f51c4a56640 1 -- 172.21.15.89:0/303980616 wait complete. 2023-10-18T10:46:45.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.472+0000 7f51c4a56640 1 Processor -- start 2023-10-18T10:46:45.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.472+0000 7f51c4a56640 1 -- start start 2023-10-18T10:46:45.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.476+0000 7f51c4a56640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51c006bd10 0x7f51c00c3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:45.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.476+0000 7f51c4a56640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51c0070600 0x7f51c00ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:45.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.476+0000 7f51c4a56640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51c0159490 0x7f51c00c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:45.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.476+0000 7f51c4a56640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c00c4360 con 0x7f51c0070600 2023-10-18T10:46:45.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.476+0000 7f51c4a56640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c01f5b50 con 0x7f51c0159490 2023-10-18T10:46:45.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.476+0000 7f51c4a56640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c00c3060 con 0x7f51c006bd10 2023-10-18T10:46:45.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.476+0000 7f51bdd9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51c0070600 0x7f51c00ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:45.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.476+0000 7f51bdd9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51c0070600 0x7f51c00ae060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56146/0 (socket says 172.21.15.89:56146) 2023-10-18T10:46:45.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.476+0000 7f51bdd9b640 1 -- 172.21.15.89:0/1107453169 learned_addr learned my addr 172.21.15.89:0/1107453169 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:45.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.476+0000 7f51bed9d640 1 --2- 172.21.15.89:0/1107453169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51c0159490 0x7f51c00c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:45.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.476+0000 7f51be59c640 1 --2- 172.21.15.89:0/1107453169 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51c006bd10 0x7f51c00c3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:45.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.476+0000 7f51bdd9b640 1 -- 172.21.15.89:0/1107453169 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51c006bd10 msgr2=0x7f51c00c3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:45.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.476+0000 7f51bdd9b640 1 --2- 172.21.15.89:0/1107453169 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51c006bd10 0x7f51c00c3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:45.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.476+0000 7f51bdd9b640 1 -- 172.21.15.89:0/1107453169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51c0159490 msgr2=0x7f51c00c7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:45.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.476+0000 7f51bdd9b640 1 --2- 172.21.15.89:0/1107453169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51c0159490 0x7f51c00c7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:45.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.476+0000 7f51bdd9b640 1 -- 172.21.15.89:0/1107453169 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b4009420 con 0x7f51c0070600 2023-10-18T10:46:45.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.476+0000 7f51be59c640 1 --2- 172.21.15.89:0/1107453169 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51c006bd10 0x7f51c00c3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:45.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.476+0000 7f51bdd9b640 1 --2- 172.21.15.89:0/1107453169 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51c0070600 0x7f51c00ae060 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f51b4019fd0 tx=0x7f51b401abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:45.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.476+0000 7f51aeffd640 1 -- 172.21.15.89:0/1107453169 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b40b7620 con 0x7f51c0070600 2023-10-18T10:46:45.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.476+0000 7f51aeffd640 1 -- 172.21.15.89:0/1107453169 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b40b77a0 con 0x7f51c0070600 2023-10-18T10:46:45.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.476+0000 7f51c4a56640 1 -- 172.21.15.89:0/1107453169 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c00c3340 con 0x7f51c0070600 2023-10-18T10:46:45.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.476+0000 7f51aeffd640 1 -- 172.21.15.89:0/1107453169 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b40b7b50 con 0x7f51c0070600 2023-10-18T10:46:45.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.476+0000 7f51c4a56640 1 -- 172.21.15.89:0/1107453169 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c00c3900 con 0x7f51c0070600 2023-10-18T10:46:45.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.476+0000 7f51c4a56640 1 -- 172.21.15.89:0/1107453169 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c00c3af0 con 0x7f51c0070600 2023-10-18T10:46:45.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.476+0000 7f51aeffd640 1 -- 172.21.15.89:0/1107453169 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f51b40b3040 con 0x7f51c0070600 2023-10-18T10:46:45.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.480+0000 7f51aeffd640 1 --2- 172.21.15.89:0/1107453169 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f51a0065680 0x7f51a0067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:45.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.480+0000 7f51aeffd640 1 -- 172.21.15.89:0/1107453169 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f51b411d670 con 0x7f51c0070600 2023-10-18T10:46:45.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.480+0000 7f51be59c640 1 --2- 172.21.15.89:0/1107453169 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f51a0065680 0x7f51a0067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:45.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.480+0000 7f51be59c640 1 --2- 172.21.15.89:0/1107453169 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f51a0065680 0x7f51a0067b40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f51b0000f70 tx=0x7f51b0007cb0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:45.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.484+0000 7f51aeffd640 1 -- 172.21.15.89:0/1107453169 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f51b40eaea0 con 0x7f51c0070600 2023-10-18T10:46:45.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:45.684+0000 7f51c4a56640 1 -- 172.21.15.89:0/1107453169 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientOnLaggyOSD.test_client_eviction_if_config_is_set"]} v 0) v1 -- 0x7f51c004dbe0 con 0x7f51c0070600 2023-10-18T10:46:46.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.308+0000 7f51aeffd640 1 -- 172.21.15.89:0/1107453169 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_recovery.TestClientOnLaggyOSD.test_client_eviction_if_config_is_set"]}]=0 v73) v1 ==== 174+0+0 (secure 0 0 0) 0x7f51b40ee4c0 con 0x7f51c0070600 2023-10-18T10:46:46.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.308+0000 7f51c4a56640 1 -- 172.21.15.89:0/1107453169 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f51a0065680 msgr2=0x7f51a0067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:46.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.308+0000 7f51c4a56640 1 --2- 172.21.15.89:0/1107453169 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f51a0065680 0x7f51a0067b40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f51b0000f70 tx=0x7f51b0007cb0 comp rx=0 tx=0).stop 2023-10-18T10:46:46.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.308+0000 7f51c4a56640 1 -- 172.21.15.89:0/1107453169 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51c0070600 msgr2=0x7f51c00ae060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:46.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.308+0000 7f51c4a56640 1 --2- 172.21.15.89:0/1107453169 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51c0070600 0x7f51c00ae060 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f51b4019fd0 tx=0x7f51b401abe0 comp rx=0 tx=0).stop 2023-10-18T10:46:46.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.308+0000 7f51c4a56640 1 -- 172.21.15.89:0/1107453169 shutdown_connections 2023-10-18T10:46:46.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.308+0000 7f51c4a56640 1 --2- 172.21.15.89:0/1107453169 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f51a0065680 0x7f51a0067b40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:46.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.308+0000 7f51c4a56640 1 --2- 172.21.15.89:0/1107453169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51c0159490 0x7f51c00c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:46.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.308+0000 7f51c4a56640 1 --2- 172.21.15.89:0/1107453169 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51c0070600 0x7f51c00ae060 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:46.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.308+0000 7f51c4a56640 1 --2- 172.21.15.89:0/1107453169 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51c006bd10 0x7f51c00c3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:46.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.308+0000 7f51c4a56640 1 -- 172.21.15.89:0/1107453169 >> 172.21.15.89:0/1107453169 conn(0x7f51c0069f00 msgr2=0x7f51c00acf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:46.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.308+0000 7f51c4a56640 1 -- 172.21.15.89:0/1107453169 shutdown_connections 2023-10-18T10:46:46.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.308+0000 7f51c4a56640 1 -- 172.21.15.89:0/1107453169 wait complete. 2023-10-18T10:46:46.337 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:46:46.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.476+0000 7f3713048640 1 Processor -- start 2023-10-18T10:46:46.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.476+0000 7f3713048640 1 -- start start 2023-10-18T10:46:46.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.476+0000 7f3713048640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f370c15c940 0x7f370c0d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:46.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.476+0000 7f3713048640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f370c15da50 con 0x7f370c0d2f40 2023-10-18T10:46:46.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.476+0000 7f3713048640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f370c15ce40 con 0x7f370c0d0e70 2023-10-18T10:46:46.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.476+0000 7f3713048640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f370c15d120 con 0x7f370c15c940 2023-10-18T10:46:46.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.476+0000 7f3710de4640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f370c15c940 0x7f370c0d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:46.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.476+0000 7f370bfff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f370c0d2f40 0x7f370c0d0760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48668/0 (socket says 172.21.15.89:48668) 2023-10-18T10:46:46.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.476+0000 7f370bfff640 1 -- 172.21.15.89:0/1070994052 learned_addr learned my addr 172.21.15.89:0/1070994052 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:46.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.476+0000 7f370b7fe640 1 -- 172.21.15.89:0/1070994052 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4048559858 0 0) 0x7f370c15da50 con 0x7f370c0d2f40 2023-10-18T10:46:46.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.476+0000 7f370b7fe640 1 -- 172.21.15.89:0/1070994052 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36f4003530 con 0x7f370c0d2f40 2023-10-18T10:46:46.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.476+0000 7f3710de4640 1 -- 172.21.15.89:0/1070994052 >> v1:172.21.15.154:6789/0 conn(0x7f370c0d0e70 legacy=0x7f370c0cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:46.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.476+0000 7f3710de4640 1 -- 172.21.15.89:0/1070994052 >> v1:172.21.15.89:6789/0 conn(0x7f370c0d2f40 legacy=0x7f370c0d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:46.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.476+0000 7f3710de4640 1 -- 172.21.15.89:0/1070994052 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370c15d400 con 0x7f370c15c940 2023-10-18T10:46:46.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.480+0000 7f3710de4640 1 --2- 172.21.15.89:0/1070994052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f370c15c940 0x7f370c0d2a00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f36fc0097c0 tx=0x7f36fc01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a5c51eb3cb9bf6 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:46.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.480+0000 7f370b7fe640 1 -- 172.21.15.89:0/1070994052 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc0a7070 con 0x7f370c15c940 2023-10-18T10:46:46.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.480+0000 7f370b7fe640 1 -- 172.21.15.89:0/1070994052 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36fc0ab430 con 0x7f370c15c940 2023-10-18T10:46:46.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.480+0000 7f370b7fe640 1 -- 172.21.15.89:0/1070994052 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc0ab7a0 con 0x7f370c15c940 2023-10-18T10:46:46.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.480+0000 7f3713048640 1 -- 172.21.15.89:0/1070994052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f370c15c940 msgr2=0x7f370c0d2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:46.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.480+0000 7f3713048640 1 --2- 172.21.15.89:0/1070994052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f370c15c940 0x7f370c0d2a00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f36fc0097c0 tx=0x7f36fc01c410 comp rx=0 tx=0).stop 2023-10-18T10:46:46.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.480+0000 7f3713048640 1 -- 172.21.15.89:0/1070994052 shutdown_connections 2023-10-18T10:46:46.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.480+0000 7f3713048640 1 --2- 172.21.15.89:0/1070994052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f370c15c940 0x7f370c0d2a00 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:46.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.480+0000 7f3713048640 1 -- 172.21.15.89:0/1070994052 >> 172.21.15.89:0/1070994052 conn(0x7f370c089fd0 msgr2=0x7f370c09b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:46.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.480+0000 7f3713048640 1 -- 172.21.15.89:0/1070994052 shutdown_connections 2023-10-18T10:46:46.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.480+0000 7f3713048640 1 -- 172.21.15.89:0/1070994052 wait complete. 2023-10-18T10:46:46.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.484+0000 7f3713048640 1 Processor -- start 2023-10-18T10:46:46.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.484+0000 7f3713048640 1 -- start start 2023-10-18T10:46:46.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.484+0000 7f3713048640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f370c0d0e70 0x7f370c150c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:46.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.484+0000 7f3713048640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f370c153a50 0x7f370c151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:46.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.484+0000 7f3713048640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f370c1519b0 0x7f370c0ca960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:46.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.484+0000 7f3713048640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370c0ca4d0 con 0x7f370c153a50 2023-10-18T10:46:46.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.484+0000 7f3713048640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370c0ca1d0 con 0x7f370c1519b0 2023-10-18T10:46:46.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.484+0000 7f3713048640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f370c0cbf70 con 0x7f370c0d0e70 2023-10-18T10:46:46.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.484+0000 7f370bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f370c153a50 0x7f370c151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:46.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.484+0000 7f370bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f370c153a50 0x7f370c151470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56162/0 (socket says 172.21.15.89:56162) 2023-10-18T10:46:46.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.484+0000 7f370bfff640 1 -- 172.21.15.89:0/1447170413 learned_addr learned my addr 172.21.15.89:0/1447170413 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:46.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.484+0000 7f370bfff640 1 -- 172.21.15.89:0/1447170413 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f370c0d0e70 msgr2=0x7f370c150c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:46.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.484+0000 7f370bfff640 1 --2- 172.21.15.89:0/1447170413 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f370c0d0e70 0x7f370c150c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:46.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.484+0000 7f370bfff640 1 -- 172.21.15.89:0/1447170413 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f370c1519b0 msgr2=0x7f370c0ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:46.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.484+0000 7f37115e5640 1 --2- 172.21.15.89:0/1447170413 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f370c1519b0 0x7f370c0ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:46.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.484+0000 7f370bfff640 1 --2- 172.21.15.89:0/1447170413 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f370c1519b0 0x7f370c0ca960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:46.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.484+0000 7f370bfff640 1 -- 172.21.15.89:0/1447170413 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36fc009420 con 0x7f370c153a50 2023-10-18T10:46:46.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.484+0000 7f37115e5640 1 --2- 172.21.15.89:0/1447170413 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f370c1519b0 0x7f370c0ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:46.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.484+0000 7f370bfff640 1 --2- 172.21.15.89:0/1447170413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f370c153a50 0x7f370c151470 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f370000c450 tx=0x7f370000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:46.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.484+0000 7f37097fa640 1 -- 172.21.15.89:0/1447170413 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3700005cf0 con 0x7f370c153a50 2023-10-18T10:46:46.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.484+0000 7f37097fa640 1 -- 172.21.15.89:0/1447170413 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3700005e50 con 0x7f370c153a50 2023-10-18T10:46:46.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.484+0000 7f37097fa640 1 -- 172.21.15.89:0/1447170413 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3700007290 con 0x7f370c153a50 2023-10-18T10:46:46.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.484+0000 7f3713048640 1 -- 172.21.15.89:0/1447170413 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f370c0cc250 con 0x7f370c153a50 2023-10-18T10:46:46.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.484+0000 7f3713048640 1 -- 172.21.15.89:0/1447170413 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f370c151180 con 0x7f370c153a50 2023-10-18T10:46:46.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.488+0000 7f3713048640 1 -- 172.21.15.89:0/1447170413 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f370c04dbe0 con 0x7f370c153a50 2023-10-18T10:46:46.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.488+0000 7f37097fa640 1 -- 172.21.15.89:0/1447170413 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f37000073f0 con 0x7f370c153a50 2023-10-18T10:46:46.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.488+0000 7f37097fa640 1 --2- 172.21.15.89:0/1447170413 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f36f8065680 0x7f36f8067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:46.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.488+0000 7f37097fa640 1 -- 172.21.15.89:0/1447170413 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3700082780 con 0x7f370c153a50 2023-10-18T10:46:46.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.488+0000 7f3710de4640 1 --2- 172.21.15.89:0/1447170413 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f36f8065680 0x7f36f8067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:46.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.488+0000 7f3710de4640 1 --2- 172.21.15.89:0/1447170413 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f36f8065680 0x7f36f8067b40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f36fc0af000 tx=0x7f36fc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:46.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.492+0000 7f37097fa640 1 -- 172.21.15.89:0/1447170413 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f370004fe70 con 0x7f370c153a50 2023-10-18T10:46:46.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.644+0000 7f3713048640 1 -- 172.21.15.89:0/1447170413 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f370c0ca610 con 0x7f370c153a50 2023-10-18T10:46:46.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.644+0000 7f37097fa640 1 -- 172.21.15.89:0/1447170413 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20192 (secure 0 0 0) 0x7f3700053490 con 0x7f370c153a50 2023-10-18T10:46:46.648 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:46.649 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":29,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:46:36.958604+0000","last_up_change":"2023-10-18T10:45:42.164688+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:45:53.309122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-18T10:45:54.659763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:45:57.796514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6824","nonce":773078364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6827","nonce":773078364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6831","nonce":773078364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6829","nonce":773078364}]},"public_addr":"172.21.15.154:6824/773078364","cluster_addr":"172.21.15.154:6827/773078364","heartbeat_back_addr":"172.21.15.154:6831/773078364","heartbeat_front_addr":"172.21.15.154:6829/773078364","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":1,"to":5}]},{"pgid":"4.1a","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:46:46.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.644+0000 7f3713048640 1 -- 172.21.15.89:0/1447170413 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f36f8065680 msgr2=0x7f36f8067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:46.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.644+0000 7f3713048640 1 --2- 172.21.15.89:0/1447170413 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f36f8065680 0x7f36f8067b40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f36fc0af000 tx=0x7f36fc008040 comp rx=0 tx=0).stop 2023-10-18T10:46:46.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.644+0000 7f3713048640 1 -- 172.21.15.89:0/1447170413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f370c153a50 msgr2=0x7f370c151470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:46.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.644+0000 7f3713048640 1 --2- 172.21.15.89:0/1447170413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f370c153a50 0x7f370c151470 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f370000c450 tx=0x7f370000c910 comp rx=0 tx=0).stop 2023-10-18T10:46:46.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.644+0000 7f3713048640 1 -- 172.21.15.89:0/1447170413 shutdown_connections 2023-10-18T10:46:46.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.644+0000 7f3713048640 1 --2- 172.21.15.89:0/1447170413 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f36f8065680 0x7f36f8067b40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:46.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.644+0000 7f3713048640 1 --2- 172.21.15.89:0/1447170413 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f370c1519b0 0x7f370c0ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:46.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.644+0000 7f3713048640 1 --2- 172.21.15.89:0/1447170413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f370c153a50 0x7f370c151470 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:46.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.644+0000 7f3713048640 1 --2- 172.21.15.89:0/1447170413 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f370c0d0e70 0x7f370c150c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:46.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.644+0000 7f3713048640 1 -- 172.21.15.89:0/1447170413 >> 172.21.15.89:0/1447170413 conn(0x7f370c089fd0 msgr2=0x7f370c15fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:46.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.644+0000 7f3713048640 1 -- 172.21.15.89:0/1447170413 shutdown_connections 2023-10-18T10:46:46.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.644+0000 7f3713048640 1 -- 172.21.15.89:0/1447170413 wait complete. 2023-10-18T10:46:46.671 INFO:tasks.cephfs.cephfs_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:45:43.114184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-18T10:45:48.730968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:45:53.309122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:45:54.659763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T10:45:57.796514+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T10:46:46.672 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:46:46.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.808+0000 7fe220226640 1 Processor -- start 2023-10-18T10:46:46.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.808+0000 7fe220226640 1 -- start start 2023-10-18T10:46:46.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.808+0000 7fe220226640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe218070600 0x7fe21806b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:46.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.808+0000 7fe220226640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe21806bd10 0x7fe2180c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:46.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.808+0000 7fe220226640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe218070b90 con 0x7fe218159490 2023-10-18T10:46:46.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.808+0000 7fe220226640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe21806c1a0 con 0x7fe21806bd10 2023-10-18T10:46:46.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.808+0000 7fe220226640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe218070e70 con 0x7fe218070600 2023-10-18T10:46:46.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.808+0000 7fe21dfc2640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe218159490 0x7fe21815d8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48670/0 (socket says 172.21.15.89:48670) 2023-10-18T10:46:46.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.808+0000 7fe21dfc2640 1 -- 172.21.15.89:0/3983687883 learned_addr learned my addr 172.21.15.89:0/3983687883 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:46.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.808+0000 7fe21e7c3640 1 --2- 172.21.15.89:0/3983687883 >> v2:172.21.15.89:3300/0 conn(0x7fe21806bd10 0x7fe2180c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:46.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.808+0000 7fe21d7c1640 1 --2- 172.21.15.89:0/3983687883 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe218070600 0x7fe21806b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:46.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.808+0000 7fe21e7c3640 1 -- 172.21.15.89:0/3983687883 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe218070600 msgr2=0x7fe21806b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:46.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.808+0000 7fe21e7c3640 1 --2- 172.21.15.89:0/3983687883 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe218070600 0x7fe21806b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:46.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.808+0000 7fe21e7c3640 1 -- 172.21.15.89:0/3983687883 >> v1:172.21.15.89:6789/0 conn(0x7fe218159490 legacy=0x7fe21815d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:46.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.808+0000 7fe21e7c3640 1 -- 172.21.15.89:0/3983687883 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe218071150 con 0x7fe21806bd10 2023-10-18T10:46:46.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.808+0000 7fe21cfc0640 1 -- 172.21.15.89:0/3983687883 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 689607857 0 0) 0x7fe218070b90 con 0x7fe218159490 2023-10-18T10:46:46.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.812+0000 7fe21e7c3640 1 --2- 172.21.15.89:0/3983687883 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe21806bd10 0x7fe2180c8960 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe210009d90 tx=0x7fe21001c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f406498141725061 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:46.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.812+0000 7fe21cfc0640 1 -- 172.21.15.89:0/3983687883 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2100a5950 con 0x7fe21806bd10 2023-10-18T10:46:46.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.812+0000 7fe21cfc0640 1 -- 172.21.15.89:0/3983687883 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2100a5ab0 con 0x7fe21806bd10 2023-10-18T10:46:46.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.812+0000 7fe21cfc0640 1 -- 172.21.15.89:0/3983687883 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2100a5e40 con 0x7fe21806bd10 2023-10-18T10:46:46.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.812+0000 7fe220226640 1 -- 172.21.15.89:0/3983687883 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe21806bd10 msgr2=0x7fe2180c8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:46.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.812+0000 7fe220226640 1 --2- 172.21.15.89:0/3983687883 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe21806bd10 0x7fe2180c8960 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe210009d90 tx=0x7fe21001c890 comp rx=0 tx=0).stop 2023-10-18T10:46:46.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.812+0000 7fe220226640 1 -- 172.21.15.89:0/3983687883 shutdown_connections 2023-10-18T10:46:46.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.812+0000 7fe220226640 1 --2- 172.21.15.89:0/3983687883 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe21806bd10 0x7fe2180c8960 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:46.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.812+0000 7fe220226640 1 --2- 172.21.15.89:0/3983687883 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe218070600 0x7fe21806b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:46.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.812+0000 7fe220226640 1 -- 172.21.15.89:0/3983687883 >> 172.21.15.89:0/3983687883 conn(0x7fe218069f00 msgr2=0x7fe2180cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:46.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.812+0000 7fe220226640 1 -- 172.21.15.89:0/3983687883 shutdown_connections 2023-10-18T10:46:46.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.812+0000 7fe220226640 1 -- 172.21.15.89:0/3983687883 wait complete. 2023-10-18T10:46:46.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.816+0000 7fe220226640 1 Processor -- start 2023-10-18T10:46:46.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.816+0000 7fe220226640 1 -- start start 2023-10-18T10:46:46.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.816+0000 7fe220226640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe21806bd10 0x7fe2181568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:46.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.816+0000 7fe220226640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe218070600 0x7fe2181518f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:46.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.816+0000 7fe220226640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe218159490 0x7fe2180c3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:46.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.816+0000 7fe220226640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe218156e20 con 0x7fe21806bd10 2023-10-18T10:46:46.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.816+0000 7fe220226640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2180c3a40 con 0x7fe218159490 2023-10-18T10:46:46.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.816+0000 7fe220226640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2181533c0 con 0x7fe218070600 2023-10-18T10:46:46.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.816+0000 7fe21dfc2640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe21806bd10 0x7fe2181568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:46.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.816+0000 7fe21dfc2640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe21806bd10 0x7fe2181568e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56180/0 (socket says 172.21.15.89:56180) 2023-10-18T10:46:46.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.816+0000 7fe21dfc2640 1 -- 172.21.15.89:0/197243938 learned_addr learned my addr 172.21.15.89:0/197243938 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:46.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.816+0000 7fe21d7c1640 1 --2- 172.21.15.89:0/197243938 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe218070600 0x7fe2181518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:46.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.816+0000 7fe21e7c3640 1 --2- 172.21.15.89:0/197243938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe218159490 0x7fe2180c3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:46.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.816+0000 7fe21dfc2640 1 -- 172.21.15.89:0/197243938 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe218070600 msgr2=0x7fe2181518f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:46.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.816+0000 7fe21dfc2640 1 --2- 172.21.15.89:0/197243938 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe218070600 0x7fe2181518f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:46.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.816+0000 7fe21dfc2640 1 -- 172.21.15.89:0/197243938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe218159490 msgr2=0x7fe2180c3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:46.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.816+0000 7fe21dfc2640 1 --2- 172.21.15.89:0/197243938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe218159490 0x7fe2180c3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:46.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.816+0000 7fe21dfc2640 1 -- 172.21.15.89:0/197243938 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe210009a70 con 0x7fe21806bd10 2023-10-18T10:46:46.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.816+0000 7fe21d7c1640 1 --2- 172.21.15.89:0/197243938 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe218070600 0x7fe2181518f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:46.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.816+0000 7fe21dfc2640 1 --2- 172.21.15.89:0/197243938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe21806bd10 0x7fe2181568e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe21400c4b0 tx=0x7fe21400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:46.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.816+0000 7fe20e7fc640 1 -- 172.21.15.89:0/197243938 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2140074e0 con 0x7fe21806bd10 2023-10-18T10:46:46.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.816+0000 7fe20e7fc640 1 -- 172.21.15.89:0/197243938 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe214007660 con 0x7fe21806bd10 2023-10-18T10:46:46.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.816+0000 7fe220226640 1 -- 172.21.15.89:0/197243938 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe218151e30 con 0x7fe21806bd10 2023-10-18T10:46:46.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.816+0000 7fe20e7fc640 1 -- 172.21.15.89:0/197243938 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe214004a30 con 0x7fe21806bd10 2023-10-18T10:46:46.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.816+0000 7fe220226640 1 -- 172.21.15.89:0/197243938 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe218152e00 con 0x7fe21806bd10 2023-10-18T10:46:46.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.820+0000 7fe220226640 1 -- 172.21.15.89:0/197243938 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe21804dbe0 con 0x7fe21806bd10 2023-10-18T10:46:46.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.820+0000 7fe20e7fc640 1 -- 172.21.15.89:0/197243938 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fe214004b90 con 0x7fe21806bd10 2023-10-18T10:46:46.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.820+0000 7fe20e7fc640 1 --2- 172.21.15.89:0/197243938 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe1fc065680 0x7fe1fc067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:46.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.820+0000 7fe20e7fc640 1 -- 172.21.15.89:0/197243938 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe2140825b0 con 0x7fe21806bd10 2023-10-18T10:46:46.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.820+0000 7fe21d7c1640 1 --2- 172.21.15.89:0/197243938 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe1fc065680 0x7fe1fc067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:46.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.820+0000 7fe21d7c1640 1 --2- 172.21.15.89:0/197243938 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe1fc065680 0x7fe1fc067b40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe21806fd30 tx=0x7fe208008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:46.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.824+0000 7fe20e7fc640 1 -- 172.21.15.89:0/197243938 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fe214053860 con 0x7fe21806bd10 2023-10-18T10:46:46.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.976+0000 7fe220226640 1 -- 172.21.15.89:0/197243938 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe2180cbb60 con 0x7fe21806bd10 2023-10-18T10:46:46.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.976+0000 7fe20e7fc640 1 -- 172.21.15.89:0/197243938 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe2140533f0 con 0x7fe21806bd10 2023-10-18T10:46:46.980 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-18T10:46:46.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.976+0000 7fe220226640 1 -- 172.21.15.89:0/197243938 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe1fc065680 msgr2=0x7fe1fc067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:46.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.976+0000 7fe220226640 1 --2- 172.21.15.89:0/197243938 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe1fc065680 0x7fe1fc067b40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe21806fd30 tx=0x7fe208008040 comp rx=0 tx=0).stop 2023-10-18T10:46:46.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.976+0000 7fe220226640 1 -- 172.21.15.89:0/197243938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe21806bd10 msgr2=0x7fe2181568e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:46.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.976+0000 7fe220226640 1 --2- 172.21.15.89:0/197243938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe21806bd10 0x7fe2181568e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe21400c4b0 tx=0x7fe21400c970 comp rx=0 tx=0).stop 2023-10-18T10:46:46.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.976+0000 7fe220226640 1 -- 172.21.15.89:0/197243938 shutdown_connections 2023-10-18T10:46:46.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.976+0000 7fe220226640 1 --2- 172.21.15.89:0/197243938 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe1fc065680 0x7fe1fc067b40 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:46.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.976+0000 7fe220226640 1 --2- 172.21.15.89:0/197243938 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe218070600 0x7fe2181518f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:46.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.976+0000 7fe220226640 1 --2- 172.21.15.89:0/197243938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe218159490 0x7fe2180c3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:46.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.976+0000 7fe220226640 1 --2- 172.21.15.89:0/197243938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe21806bd10 0x7fe2181568e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:46.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.976+0000 7fe220226640 1 -- 172.21.15.89:0/197243938 >> 172.21.15.89:0/197243938 conn(0x7fe218069f00 msgr2=0x7fe2180afe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:46.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.976+0000 7fe220226640 1 -- 172.21.15.89:0/197243938 shutdown_connections 2023-10-18T10:46:46.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:46.976+0000 7fe220226640 1 -- 172.21.15.89:0/197243938 wait complete. 2023-10-18T10:46:47.001 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-18T10:46:47.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.136+0000 7f51c4774640 1 Processor -- start 2023-10-18T10:46:47.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.136+0000 7f51c4774640 1 -- start start 2023-10-18T10:46:47.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.136+0000 7f51c4774640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51bc06ff00 0x7f51bc15e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:47.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.140+0000 7f51c4774640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f51bc0c8960 0x7f51bc0c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:47.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.140+0000 7f51c4774640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51bc15acb0 con 0x7f51bc0ca340 2023-10-18T10:46:47.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.140+0000 7f51c4774640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51bc15ef30 con 0x7f51bc0c8960 2023-10-18T10:46:47.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.140+0000 7f51c4774640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc15aa10 con 0x7f51bc06ff00 2023-10-18T10:46:47.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.140+0000 7f51c2d11640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f51bc0c8960 0x7f51bc0c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:47.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.140+0000 7f51c2d11640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f51bc0c8960 0x7f51bc0c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56188/0 (socket says 172.21.15.89:56188) 2023-10-18T10:46:47.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.140+0000 7f51c2d11640 1 -- 172.21.15.89:0/3913210249 learned_addr learned my addr 172.21.15.89:0/3913210249 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:47.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.140+0000 7f51c1d0f640 1 --1- 172.21.15.89:0/3913210249 >> v1:172.21.15.89:6789/0 conn(0x7f51bc0ca340 0x7f51bc15b140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48676/0 (socket says 172.21.15.89:48676) 2023-10-18T10:46:47.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.140+0000 7f51c2510640 1 --2- 172.21.15.89:0/3913210249 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51bc06ff00 0x7f51bc15e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:47.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.140+0000 7f51c2d11640 1 -- 172.21.15.89:0/3913210249 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51bc06ff00 msgr2=0x7f51bc15e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:47.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.140+0000 7f51c2d11640 1 --2- 172.21.15.89:0/3913210249 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51bc06ff00 0x7f51bc15e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:47.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.140+0000 7f51c2d11640 1 -- 172.21.15.89:0/3913210249 >> v1:172.21.15.89:6789/0 conn(0x7f51bc0ca340 legacy=0x7f51bc15b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:47.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.140+0000 7f51c2d11640 1 -- 172.21.15.89:0/3913210249 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51bc0702e0 con 0x7f51bc0c8960 2023-10-18T10:46:47.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.140+0000 7f51c150e640 1 -- 172.21.15.89:0/3913210249 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3319429918 0 0) 0x7f51bc15acb0 con 0x7f51bc0ca340 2023-10-18T10:46:47.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.140+0000 7f51c2510640 1 --2- 172.21.15.89:0/3913210249 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51bc06ff00 0x7f51bc15e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:47.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.140+0000 7f51c2d11640 1 --2- 172.21.15.89:0/3913210249 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51bc0c8960 0x7f51bc0c8db0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f51b4009e50 tx=0x7f51b401c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c23b72249244f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:47.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.140+0000 7f51c150e640 1 -- 172.21.15.89:0/3913210249 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b40a5730 con 0x7f51bc0c8960 2023-10-18T10:46:47.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.140+0000 7f51c150e640 1 -- 172.21.15.89:0/3913210249 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b40a5890 con 0x7f51bc0c8960 2023-10-18T10:46:47.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.140+0000 7f51c150e640 1 -- 172.21.15.89:0/3913210249 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b40a5c20 con 0x7f51bc0c8960 2023-10-18T10:46:47.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.144+0000 7f51c4774640 1 -- 172.21.15.89:0/3913210249 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51bc0c8960 msgr2=0x7f51bc0c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:47.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.144+0000 7f51c4774640 1 --2- 172.21.15.89:0/3913210249 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51bc0c8960 0x7f51bc0c8db0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f51b4009e50 tx=0x7f51b401c980 comp rx=0 tx=0).stop 2023-10-18T10:46:47.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.144+0000 7f51c4774640 1 -- 172.21.15.89:0/3913210249 shutdown_connections 2023-10-18T10:46:47.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.144+0000 7f51c4774640 1 --2- 172.21.15.89:0/3913210249 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51bc0c8960 0x7f51bc0c8db0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:47.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.144+0000 7f51c4774640 1 --2- 172.21.15.89:0/3913210249 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51bc06ff00 0x7f51bc15e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:47.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.144+0000 7f51c4774640 1 -- 172.21.15.89:0/3913210249 >> 172.21.15.89:0/3913210249 conn(0x7f51bc069f00 msgr2=0x7f51bc0cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:47.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.144+0000 7f51c4774640 1 -- 172.21.15.89:0/3913210249 shutdown_connections 2023-10-18T10:46:47.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.144+0000 7f51c4774640 1 -- 172.21.15.89:0/3913210249 wait complete. 2023-10-18T10:46:47.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.144+0000 7f51c4774640 1 Processor -- start 2023-10-18T10:46:47.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.144+0000 7f51c4774640 1 -- start start 2023-10-18T10:46:47.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.144+0000 7f51c4774640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51bc06ff00 0x7f51bc0b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:47.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.144+0000 7f51c4774640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51bc0c8960 0x7f51bc0c2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:47.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.144+0000 7f51c4774640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51bc0ca340 0x7f51bc0c13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:47.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.144+0000 7f51c4774640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc0c7410 con 0x7f51bc06ff00 2023-10-18T10:46:47.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.144+0000 7f51c4774640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc0b1890 con 0x7f51bc0ca340 2023-10-18T10:46:47.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.144+0000 7f51c4774640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc1f8760 con 0x7f51bc0c8960 2023-10-18T10:46:47.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.144+0000 7f51c2510640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51bc06ff00 0x7f51bc0b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:47.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.144+0000 7f51c2510640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51bc06ff00 0x7f51bc0b1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56202/0 (socket says 172.21.15.89:56202) 2023-10-18T10:46:47.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.144+0000 7f51c2510640 1 -- 172.21.15.89:0/2078138459 learned_addr learned my addr 172.21.15.89:0/2078138459 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:47.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.148+0000 7f51c1d0f640 1 --2- 172.21.15.89:0/2078138459 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51bc0c8960 0x7f51bc0c2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:47.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.148+0000 7f51c2510640 1 -- 172.21.15.89:0/2078138459 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51bc0c8960 msgr2=0x7f51bc0c2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:47.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.148+0000 7f51c2d11640 1 --2- 172.21.15.89:0/2078138459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51bc0ca340 0x7f51bc0c13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:47.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.148+0000 7f51c2510640 1 --2- 172.21.15.89:0/2078138459 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51bc0c8960 0x7f51bc0c2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:47.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.148+0000 7f51c2510640 1 -- 172.21.15.89:0/2078138459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51bc0ca340 msgr2=0x7f51bc0c13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:47.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.148+0000 7f51c2510640 1 --2- 172.21.15.89:0/2078138459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51bc0ca340 0x7f51bc0c13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:47.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.148+0000 7f51c2510640 1 -- 172.21.15.89:0/2078138459 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b4009b30 con 0x7f51bc06ff00 2023-10-18T10:46:47.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.148+0000 7f51c2d11640 1 --2- 172.21.15.89:0/2078138459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51bc0ca340 0x7f51bc0c13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:47.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.148+0000 7f51c1d0f640 1 --2- 172.21.15.89:0/2078138459 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51bc0c8960 0x7f51bc0c2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:47.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.148+0000 7f51c2510640 1 --2- 172.21.15.89:0/2078138459 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51bc06ff00 0x7f51bc0b1350 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f51b800e810 tx=0x7f51b800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:47.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.148+0000 7f51b2ffd640 1 -- 172.21.15.89:0/2078138459 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b800c8d0 con 0x7f51bc06ff00 2023-10-18T10:46:47.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.148+0000 7f51b2ffd640 1 -- 172.21.15.89:0/2078138459 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b800ca30 con 0x7f51bc06ff00 2023-10-18T10:46:47.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.148+0000 7f51c4774640 1 -- 172.21.15.89:0/2078138459 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51bc1f8a40 con 0x7f51bc06ff00 2023-10-18T10:46:47.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.148+0000 7f51b2ffd640 1 -- 172.21.15.89:0/2078138459 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b800cdc0 con 0x7f51bc06ff00 2023-10-18T10:46:47.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.148+0000 7f51c4774640 1 -- 172.21.15.89:0/2078138459 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51bc1f9000 con 0x7f51bc06ff00 2023-10-18T10:46:47.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.148+0000 7f51c4774640 1 -- 172.21.15.89:0/2078138459 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51bc0c7170 con 0x7f51bc06ff00 2023-10-18T10:46:47.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.148+0000 7f51b2ffd640 1 -- 172.21.15.89:0/2078138459 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f51b8013410 con 0x7f51bc06ff00 2023-10-18T10:46:47.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.148+0000 7f51b2ffd640 1 --2- 172.21.15.89:0/2078138459 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f51a0065680 0x7f51a0067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:47.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.148+0000 7f51b2ffd640 1 -- 172.21.15.89:0/2078138459 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f51b8082fb0 con 0x7f51bc06ff00 2023-10-18T10:46:47.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.152+0000 7f51c1d0f640 1 --2- 172.21.15.89:0/2078138459 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f51a0065680 0x7f51a0067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:47.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.152+0000 7f51c1d0f640 1 --2- 172.21.15.89:0/2078138459 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f51a0065680 0x7f51a0067b40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f51ac007fd0 tx=0x7f51ac009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:47.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.152+0000 7f51b2ffd640 1 -- 172.21.15.89:0/2078138459 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f51bc0c7170 con 0x7f51bc06ff00 2023-10-18T10:46:47.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.300+0000 7f51c4774640 1 -- 172.21.15.89:0/2078138459 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f51bc0c7170 con 0x7f51bc06ff00 2023-10-18T10:46:47.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.304+0000 7f51b2ffd640 1 -- 172.21.15.89:0/2078138459 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f51bc0c7170 con 0x7f51bc06ff00 2023-10-18T10:46:47.308 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-10-18T10:46:47.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.304+0000 7f51c4774640 1 -- 172.21.15.89:0/2078138459 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f51a0065680 msgr2=0x7f51a0067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:47.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.304+0000 7f51c4774640 1 --2- 172.21.15.89:0/2078138459 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f51a0065680 0x7f51a0067b40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f51ac007fd0 tx=0x7f51ac009040 comp rx=0 tx=0).stop 2023-10-18T10:46:47.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.304+0000 7f51c4774640 1 -- 172.21.15.89:0/2078138459 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51bc06ff00 msgr2=0x7f51bc0b1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:47.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.304+0000 7f51c4774640 1 --2- 172.21.15.89:0/2078138459 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51bc06ff00 0x7f51bc0b1350 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f51b800e810 tx=0x7f51b800ecd0 comp rx=0 tx=0).stop 2023-10-18T10:46:47.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.304+0000 7f51c4774640 1 -- 172.21.15.89:0/2078138459 shutdown_connections 2023-10-18T10:46:47.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.304+0000 7f51c4774640 1 --2- 172.21.15.89:0/2078138459 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f51a0065680 0x7f51a0067b40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:47.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.304+0000 7f51c4774640 1 --2- 172.21.15.89:0/2078138459 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f51bc0c8960 0x7f51bc0c2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:47.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.304+0000 7f51c4774640 1 --2- 172.21.15.89:0/2078138459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51bc0ca340 0x7f51bc0c13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:47.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.304+0000 7f51c4774640 1 --2- 172.21.15.89:0/2078138459 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51bc06ff00 0x7f51bc0b1350 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:47.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.304+0000 7f51c4774640 1 -- 172.21.15.89:0/2078138459 >> 172.21.15.89:0/2078138459 conn(0x7f51bc069f00 msgr2=0x7f51bc15b140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:47.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.304+0000 7f51c4774640 1 -- 172.21.15.89:0/2078138459 shutdown_connections 2023-10-18T10:46:47.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.304+0000 7f51c4774640 1 -- 172.21.15.89:0/2078138459 wait complete. 2023-10-18T10:46:47.331 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:46:47.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.468+0000 7fe12a567640 1 Processor -- start 2023-10-18T10:46:47.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.468+0000 7fe12a567640 1 -- start start 2023-10-18T10:46:47.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.468+0000 7fe12a567640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe12415a4a0 0x7fe1240cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:47.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.468+0000 7fe12a567640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe1240cfbd0 0x7fe12415e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:47.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.468+0000 7fe12a567640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe12415de80 con 0x7fe12415ccc0 2023-10-18T10:46:47.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.468+0000 7fe12a567640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe12415d510 con 0x7fe12415a4a0 2023-10-18T10:46:47.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.468+0000 7fe12a567640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe12415d7f0 con 0x7fe1240cfbd0 2023-10-18T10:46:47.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.468+0000 7fe123fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe12415a4a0 0x7fe1240cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:47.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.468+0000 7fe123fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe12415a4a0 0x7fe1240cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56206/0 (socket says 172.21.15.89:56206) 2023-10-18T10:46:47.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.468+0000 7fe123fff640 1 -- 172.21.15.89:0/1873437016 learned_addr learned my addr 172.21.15.89:0/1873437016 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:47.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.468+0000 7fe128b04640 1 --2- 172.21.15.89:0/1873437016 >> v2:172.21.15.154:3300/0 conn(0x7fe1240cfbd0 0x7fe12415e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:47.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.468+0000 7fe123fff640 1 -- 172.21.15.89:0/1873437016 >> v2:172.21.15.154:3300/0 conn(0x7fe1240cfbd0 msgr2=0x7fe12415e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:47.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.468+0000 7fe123fff640 1 --2- 172.21.15.89:0/1873437016 >> v2:172.21.15.154:3300/0 conn(0x7fe1240cfbd0 0x7fe12415e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:47.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.468+0000 7fe123fff640 1 -- 172.21.15.89:0/1873437016 >> v1:172.21.15.154:6789/0 conn(0x7fe12415ccc0 legacy=0x7fe1240cf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:47.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.468+0000 7fe123fff640 1 -- 172.21.15.89:0/1873437016 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe12415dad0 con 0x7fe12415a4a0 2023-10-18T10:46:47.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.472+0000 7fe123fff640 1 --2- 172.21.15.89:0/1873437016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe12415a4a0 0x7fe1240cd910 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe1240ccc80 tx=0x7fe11401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753ad38efd486a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:47.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.472+0000 7fe122ffd640 1 -- 172.21.15.89:0/1873437016 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1140aa610 con 0x7fe12415a4a0 2023-10-18T10:46:47.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.472+0000 7fe122ffd640 1 -- 172.21.15.89:0/1873437016 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1140a6040 con 0x7fe12415a4a0 2023-10-18T10:46:47.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.472+0000 7fe122ffd640 1 -- 172.21.15.89:0/1873437016 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1140aa990 con 0x7fe12415a4a0 2023-10-18T10:46:47.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.472+0000 7fe12a567640 1 -- 172.21.15.89:0/1873437016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe12415a4a0 msgr2=0x7fe1240cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:47.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.472+0000 7fe12a567640 1 --2- 172.21.15.89:0/1873437016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe12415a4a0 0x7fe1240cd910 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe1240ccc80 tx=0x7fe11401c490 comp rx=0 tx=0).stop 2023-10-18T10:46:47.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.472+0000 7fe12a567640 1 -- 172.21.15.89:0/1873437016 shutdown_connections 2023-10-18T10:46:47.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.472+0000 7fe12a567640 1 --2- 172.21.15.89:0/1873437016 >> v2:172.21.15.154:3300/0 conn(0x7fe1240cfbd0 0x7fe12415e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:47.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.472+0000 7fe12a567640 1 --2- 172.21.15.89:0/1873437016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe12415a4a0 0x7fe1240cd910 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:47.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.472+0000 7fe12a567640 1 -- 172.21.15.89:0/1873437016 >> 172.21.15.89:0/1873437016 conn(0x7fe124069ad0 msgr2=0x7fe124069ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:47.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.472+0000 7fe12a567640 1 -- 172.21.15.89:0/1873437016 shutdown_connections 2023-10-18T10:46:47.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.472+0000 7fe12a567640 1 -- 172.21.15.89:0/1873437016 wait complete. 2023-10-18T10:46:47.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.476+0000 7fe12a567640 1 Processor -- start 2023-10-18T10:46:47.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.476+0000 7fe12a567640 1 -- start start 2023-10-18T10:46:47.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.476+0000 7fe12a567640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1240cfbd0 0x7fe1240d0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:47.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.476+0000 7fe12a567640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe12415a4a0 0x7fe1240c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:47.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.476+0000 7fe12a567640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe12415ccc0 0x7fe124153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:47.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.476+0000 7fe12a567640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1240cb520 con 0x7fe1240cfbd0 2023-10-18T10:46:47.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.476+0000 7fe12a567640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1240ca910 con 0x7fe12415ccc0 2023-10-18T10:46:47.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.476+0000 7fe12a567640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1240cabf0 con 0x7fe12415a4a0 2023-10-18T10:46:47.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.476+0000 7fe123fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1240cfbd0 0x7fe1240d0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:47.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.476+0000 7fe123fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1240cfbd0 0x7fe1240d0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56214/0 (socket says 172.21.15.89:56214) 2023-10-18T10:46:47.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.476+0000 7fe123fff640 1 -- 172.21.15.89:0/1139730563 learned_addr learned my addr 172.21.15.89:0/1139730563 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:47.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.476+0000 7fe128b04640 1 --2- 172.21.15.89:0/1139730563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe12415ccc0 0x7fe124153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:47.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.476+0000 7fe1237fe640 1 --2- 172.21.15.89:0/1139730563 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe12415a4a0 0x7fe1240c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:47.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.476+0000 7fe123fff640 1 -- 172.21.15.89:0/1139730563 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe12415a4a0 msgr2=0x7fe1240c6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:47.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.476+0000 7fe123fff640 1 --2- 172.21.15.89:0/1139730563 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe12415a4a0 0x7fe1240c6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:47.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.476+0000 7fe123fff640 1 -- 172.21.15.89:0/1139730563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe12415ccc0 msgr2=0x7fe124153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:47.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.476+0000 7fe123fff640 1 --2- 172.21.15.89:0/1139730563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe12415ccc0 0x7fe124153730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:47.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.476+0000 7fe123fff640 1 -- 172.21.15.89:0/1139730563 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1140094d0 con 0x7fe1240cfbd0 2023-10-18T10:46:47.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.476+0000 7fe123fff640 1 --2- 172.21.15.89:0/1139730563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1240cfbd0 0x7fe1240d0250 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe11401a0c0 tx=0x7fe11401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:47.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.476+0000 7fe120ff9640 1 -- 172.21.15.89:0/1139730563 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1140bd5f0 con 0x7fe1240cfbd0 2023-10-18T10:46:47.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.476+0000 7fe120ff9640 1 -- 172.21.15.89:0/1139730563 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1140bd750 con 0x7fe1240cfbd0 2023-10-18T10:46:47.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.476+0000 7fe120ff9640 1 -- 172.21.15.89:0/1139730563 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1140bdae0 con 0x7fe1240cfbd0 2023-10-18T10:46:47.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.476+0000 7fe12a567640 1 -- 172.21.15.89:0/1139730563 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1240caed0 con 0x7fe1240cfbd0 2023-10-18T10:46:47.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.476+0000 7fe12a567640 1 -- 172.21.15.89:0/1139730563 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1240c7100 con 0x7fe1240cfbd0 2023-10-18T10:46:47.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.476+0000 7fe12a567640 1 -- 172.21.15.89:0/1139730563 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1240c72f0 con 0x7fe1240cfbd0 2023-10-18T10:46:47.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.480+0000 7fe120ff9640 1 -- 172.21.15.89:0/1139730563 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fe1140b33f0 con 0x7fe1240cfbd0 2023-10-18T10:46:47.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.480+0000 7fe120ff9640 1 --2- 172.21.15.89:0/1139730563 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe1100656b0 0x7fe110067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:47.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.480+0000 7fe120ff9640 1 -- 172.21.15.89:0/1139730563 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe11411de00 con 0x7fe1240cfbd0 2023-10-18T10:46:47.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.480+0000 7fe1237fe640 1 --2- 172.21.15.89:0/1139730563 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe1100656b0 0x7fe110067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:47.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.480+0000 7fe1237fe640 1 --2- 172.21.15.89:0/1139730563 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe1100656b0 0x7fe110067b70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe1240c6740 tx=0x7fe118008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:47.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.484+0000 7fe120ff9640 1 -- 172.21.15.89:0/1139730563 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fe1140eb630 con 0x7fe1240cfbd0 2023-10-18T10:46:47.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.632+0000 7fe12a567640 1 -- 172.21.15.89:0/1139730563 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe124073b50 con 0x7fe1240cfbd0 2023-10-18T10:46:47.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.632+0000 7fe120ff9640 1 -- 172.21.15.89:0/1139730563 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe1140c30a0 con 0x7fe1240cfbd0 2023-10-18T10:46:47.637 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-18T10:46:47.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.632+0000 7fe12a567640 1 -- 172.21.15.89:0/1139730563 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe1100656b0 msgr2=0x7fe110067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:47.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.632+0000 7fe12a567640 1 --2- 172.21.15.89:0/1139730563 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe1100656b0 0x7fe110067b70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe1240c6740 tx=0x7fe118008040 comp rx=0 tx=0).stop 2023-10-18T10:46:47.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.632+0000 7fe12a567640 1 -- 172.21.15.89:0/1139730563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1240cfbd0 msgr2=0x7fe1240d0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:47.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.632+0000 7fe12a567640 1 --2- 172.21.15.89:0/1139730563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1240cfbd0 0x7fe1240d0250 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe11401a0c0 tx=0x7fe11401acd0 comp rx=0 tx=0).stop 2023-10-18T10:46:47.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.632+0000 7fe12a567640 1 -- 172.21.15.89:0/1139730563 shutdown_connections 2023-10-18T10:46:47.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.632+0000 7fe12a567640 1 --2- 172.21.15.89:0/1139730563 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe1100656b0 0x7fe110067b70 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:47.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.632+0000 7fe12a567640 1 --2- 172.21.15.89:0/1139730563 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe12415a4a0 0x7fe1240c6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:47.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.632+0000 7fe12a567640 1 --2- 172.21.15.89:0/1139730563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe12415ccc0 0x7fe124153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:47.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.632+0000 7fe12a567640 1 --2- 172.21.15.89:0/1139730563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1240cfbd0 0x7fe1240d0250 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:47.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.632+0000 7fe12a567640 1 -- 172.21.15.89:0/1139730563 >> 172.21.15.89:0/1139730563 conn(0x7fe124069ad0 msgr2=0x7fe12415c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:47.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.636+0000 7fe12a567640 1 -- 172.21.15.89:0/1139730563 shutdown_connections 2023-10-18T10:46:47.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.636+0000 7fe12a567640 1 -- 172.21.15.89:0/1139730563 wait complete. 2023-10-18T10:46:47.658 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:46:47.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.796+0000 7ffb41d37640 1 Processor -- start 2023-10-18T10:46:47.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.796+0000 7ffb41d37640 1 -- start start 2023-10-18T10:46:47.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.800+0000 7ffb41d37640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ffb3c15c940 0x7ffb3c0d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:47.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.800+0000 7ffb41d37640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ffb3c0d2f40 0x7ffb3c0d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:47.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.800+0000 7ffb41d37640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb3c0d10f0 con 0x7ffb3c0d0ca0 2023-10-18T10:46:47.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.800+0000 7ffb41d37640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb3c15da50 con 0x7ffb3c0d2f40 2023-10-18T10:46:47.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.800+0000 7ffb41d37640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb3c15ce40 con 0x7ffb3c15c940 2023-10-18T10:46:47.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.800+0000 7ffb3affd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ffb3c0d2f40 0x7ffb3c0d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:47.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.800+0000 7ffb3affd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ffb3c0d2f40 0x7ffb3c0d0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56222/0 (socket says 172.21.15.89:56222) 2023-10-18T10:46:47.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.800+0000 7ffb3affd640 1 -- 172.21.15.89:0/2111090517 learned_addr learned my addr 172.21.15.89:0/2111090517 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:47.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.800+0000 7ffb3b7fe640 1 --2- 172.21.15.89:0/2111090517 >> v2:172.21.15.154:3300/0 conn(0x7ffb3c15c940 0x7ffb3c0d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:47.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.800+0000 7ffb3affd640 1 -- 172.21.15.89:0/2111090517 >> v2:172.21.15.154:3300/0 conn(0x7ffb3c15c940 msgr2=0x7ffb3c0d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:47.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.800+0000 7ffb3affd640 1 --2- 172.21.15.89:0/2111090517 >> v2:172.21.15.154:3300/0 conn(0x7ffb3c15c940 0x7ffb3c0d2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:47.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.800+0000 7ffb3affd640 1 -- 172.21.15.89:0/2111090517 >> v1:172.21.15.154:6789/0 conn(0x7ffb3c0d0ca0 legacy=0x7ffb3c0cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:47.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.800+0000 7ffb3affd640 1 -- 172.21.15.89:0/2111090517 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3c15d120 con 0x7ffb3c0d2f40 2023-10-18T10:46:47.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.800+0000 7ffb3affd640 1 --2- 172.21.15.89:0/2111090517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb3c0d2f40 0x7ffb3c0d0760 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c0725b0 tx=0x7ffb3001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe372351332dfd3 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:47.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.804+0000 7ffb3a7fc640 1 -- 172.21.15.89:0/2111090517 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb300ac640 con 0x7ffb3c0d2f40 2023-10-18T10:46:47.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.804+0000 7ffb3a7fc640 1 -- 172.21.15.89:0/2111090517 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb300ac7a0 con 0x7ffb3c0d2f40 2023-10-18T10:46:47.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.804+0000 7ffb3a7fc640 1 -- 172.21.15.89:0/2111090517 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb300acb30 con 0x7ffb3c0d2f40 2023-10-18T10:46:47.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.804+0000 7ffb41d37640 1 -- 172.21.15.89:0/2111090517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb3c0d2f40 msgr2=0x7ffb3c0d0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:47.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.804+0000 7ffb41d37640 1 --2- 172.21.15.89:0/2111090517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb3c0d2f40 0x7ffb3c0d0760 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c0725b0 tx=0x7ffb3001c4d0 comp rx=0 tx=0).stop 2023-10-18T10:46:47.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.804+0000 7ffb41d37640 1 -- 172.21.15.89:0/2111090517 shutdown_connections 2023-10-18T10:46:47.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.804+0000 7ffb41d37640 1 --2- 172.21.15.89:0/2111090517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb3c0d2f40 0x7ffb3c0d0760 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:47.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.804+0000 7ffb41d37640 1 --2- 172.21.15.89:0/2111090517 >> v2:172.21.15.154:3300/0 conn(0x7ffb3c15c940 0x7ffb3c0d2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:47.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.804+0000 7ffb41d37640 1 -- 172.21.15.89:0/2111090517 >> 172.21.15.89:0/2111090517 conn(0x7ffb3c089fd0 msgr2=0x7ffb3c09b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:47.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.804+0000 7ffb41d37640 1 -- 172.21.15.89:0/2111090517 shutdown_connections 2023-10-18T10:46:47.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.804+0000 7ffb41d37640 1 -- 172.21.15.89:0/2111090517 wait complete. 2023-10-18T10:46:47.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.804+0000 7ffb41d37640 1 Processor -- start 2023-10-18T10:46:47.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.804+0000 7ffb41d37640 1 -- start start 2023-10-18T10:46:47.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.804+0000 7ffb41d37640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb3c0d0ca0 0x7ffb3c159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:47.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.804+0000 7ffb41d37640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffb3c0c6b90 0x7ffb3c0c7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:47.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.804+0000 7ffb41d37640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb3c15b840 0x7ffb3c15bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:47.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.804+0000 7ffb41d37640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c15c230 con 0x7ffb3c15b840 2023-10-18T10:46:47.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.804+0000 7ffb41d37640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c0c7700 con 0x7ffb3c0d0ca0 2023-10-18T10:46:47.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.804+0000 7ffb41d37640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c0c79e0 con 0x7ffb3c0c6b90 2023-10-18T10:46:47.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.808+0000 7ffb3bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb3c15b840 0x7ffb3c15bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:47.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.808+0000 7ffb3b7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb3c0d0ca0 0x7ffb3c159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:47.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.808+0000 7ffb3bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb3c15b840 0x7ffb3c15bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56226/0 (socket says 172.21.15.89:56226) 2023-10-18T10:46:47.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.808+0000 7ffb3bfff640 1 -- 172.21.15.89:0/1507941041 learned_addr learned my addr 172.21.15.89:0/1507941041 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:47.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.808+0000 7ffb3affd640 1 --2- 172.21.15.89:0/1507941041 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffb3c0c6b90 0x7ffb3c0c7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:47.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.808+0000 7ffb3bfff640 1 -- 172.21.15.89:0/1507941041 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffb3c0c6b90 msgr2=0x7ffb3c0c7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:47.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.808+0000 7ffb3bfff640 1 --2- 172.21.15.89:0/1507941041 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffb3c0c6b90 0x7ffb3c0c7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:47.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.808+0000 7ffb3bfff640 1 -- 172.21.15.89:0/1507941041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb3c0d0ca0 msgr2=0x7ffb3c159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:47.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.808+0000 7ffb3bfff640 1 --2- 172.21.15.89:0/1507941041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb3c0d0ca0 0x7ffb3c159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:47.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.808+0000 7ffb3bfff640 1 -- 172.21.15.89:0/1507941041 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb300094d0 con 0x7ffb3c15b840 2023-10-18T10:46:47.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.808+0000 7ffb3b7fe640 1 --2- 172.21.15.89:0/1507941041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb3c0d0ca0 0x7ffb3c159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:46:47.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.808+0000 7ffb3bfff640 1 --2- 172.21.15.89:0/1507941041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb3c15b840 0x7ffb3c15bcf0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ffb3400c770 tx=0x7ffb3400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:47.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.808+0000 7ffb23fff640 1 -- 172.21.15.89:0/1507941041 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb34003b80 con 0x7ffb3c15b840 2023-10-18T10:46:47.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.808+0000 7ffb23fff640 1 -- 172.21.15.89:0/1507941041 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb34003ce0 con 0x7ffb3c15b840 2023-10-18T10:46:47.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.808+0000 7ffb23fff640 1 -- 172.21.15.89:0/1507941041 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3400aa90 con 0x7ffb3c15b840 2023-10-18T10:46:47.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.808+0000 7ffb41d37640 1 -- 172.21.15.89:0/1507941041 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb3c0c7cc0 con 0x7ffb3c15b840 2023-10-18T10:46:47.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.808+0000 7ffb41d37640 1 -- 172.21.15.89:0/1507941041 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb3c0c8280 con 0x7ffb3c15b840 2023-10-18T10:46:47.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.808+0000 7ffb41d37640 1 -- 172.21.15.89:0/1507941041 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb3c1f5680 con 0x7ffb3c15b840 2023-10-18T10:46:47.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.808+0000 7ffb23fff640 1 -- 172.21.15.89:0/1507941041 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7ffb3400abf0 con 0x7ffb3c15b840 2023-10-18T10:46:47.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.812+0000 7ffb23fff640 1 --2- 172.21.15.89:0/1507941041 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ffb28065680 0x7ffb28067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:47.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.812+0000 7ffb23fff640 1 -- 172.21.15.89:0/1507941041 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ffb34082c40 con 0x7ffb3c15b840 2023-10-18T10:46:47.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.812+0000 7ffb3b7fe640 1 --2- 172.21.15.89:0/1507941041 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ffb28065680 0x7ffb28067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:47.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.812+0000 7ffb3b7fe640 1 --2- 172.21.15.89:0/1507941041 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ffb28065680 0x7ffb28067b40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c009eb0 tx=0x7ffb2c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:47.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.816+0000 7ffb23fff640 1 -- 172.21.15.89:0/1507941041 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7ffb340504f0 con 0x7ffb3c15b840 2023-10-18T10:46:47.960 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:46:47.964 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:46:47.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.964+0000 7ffb41d37640 1 -- 172.21.15.89:0/1507941041 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffb3c0d10f0 con 0x7ffb3c15b840 2023-10-18T10:46:47.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.964+0000 7ffb23fff640 1 -- 172.21.15.89:0/1507941041 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffb3c0d10f0 con 0x7ffb3c15b840 2023-10-18T10:46:47.969 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-18T10:46:47.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.968+0000 7ffb41d37640 1 -- 172.21.15.89:0/1507941041 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ffb28065680 msgr2=0x7ffb28067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:47.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.968+0000 7ffb41d37640 1 --2- 172.21.15.89:0/1507941041 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ffb28065680 0x7ffb28067b40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c009eb0 tx=0x7ffb2c009190 comp rx=0 tx=0).stop 2023-10-18T10:46:47.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.968+0000 7ffb41d37640 1 -- 172.21.15.89:0/1507941041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb3c15b840 msgr2=0x7ffb3c15bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:47.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.968+0000 7ffb41d37640 1 --2- 172.21.15.89:0/1507941041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb3c15b840 0x7ffb3c15bcf0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ffb3400c770 tx=0x7ffb3400cc30 comp rx=0 tx=0).stop 2023-10-18T10:46:47.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.968+0000 7ffb41d37640 1 -- 172.21.15.89:0/1507941041 shutdown_connections 2023-10-18T10:46:47.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.968+0000 7ffb41d37640 1 --2- 172.21.15.89:0/1507941041 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ffb28065680 0x7ffb28067b40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:47.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.968+0000 7ffb41d37640 1 --2- 172.21.15.89:0/1507941041 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffb3c0c6b90 0x7ffb3c0c7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:47.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.968+0000 7ffb41d37640 1 --2- 172.21.15.89:0/1507941041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb3c15b840 0x7ffb3c15bcf0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:47.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.968+0000 7ffb41d37640 1 --2- 172.21.15.89:0/1507941041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb3c0d0ca0 0x7ffb3c159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:47.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.968+0000 7ffb41d37640 1 -- 172.21.15.89:0/1507941041 >> 172.21.15.89:0/1507941041 conn(0x7ffb3c089fd0 msgr2=0x7ffb3c0cf040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:47.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.968+0000 7ffb41d37640 1 -- 172.21.15.89:0/1507941041 shutdown_connections 2023-10-18T10:46:47.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:47.968+0000 7ffb41d37640 1 -- 172.21.15.89:0/1507941041 wait complete. 2023-10-18T10:46:47.991 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T10:46:48.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.172+0000 7fc17cf37640 1 Processor -- start 2023-10-18T10:46:48.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.172+0000 7fc17cf37640 1 -- start start 2023-10-18T10:46:48.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.172+0000 7fc17cf37640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc1780cd4f0 0x7fc178143470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:48.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.172+0000 7fc17cf37640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc1780ce070 0x7fc1780ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:48.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.172+0000 7fc17cf37640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1780cd8d0 con 0x7fc178145b70 2023-10-18T10:46:48.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.172+0000 7fc17cf37640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc178069900 con 0x7fc1780cd4f0 2023-10-18T10:46:48.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.172+0000 7fc17cf37640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1780ce9e0 con 0x7fc1780ce070 2023-10-18T10:46:48.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.172+0000 7fc17659c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc1780cd4f0 0x7fc178143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:48.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.172+0000 7fc175d9b640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc178145b70 0x7fc178160cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48682/0 (socket says 172.21.15.89:48682) 2023-10-18T10:46:48.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.172+0000 7fc175d9b640 1 -- 172.21.15.89:0/1862443614 learned_addr learned my addr 172.21.15.89:0/1862443614 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:48.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.172+0000 7fc176d9d640 1 --2- 172.21.15.89:0/1862443614 >> v2:172.21.15.154:3300/0 conn(0x7fc1780ce070 0x7fc1780ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:48.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.172+0000 7fc17659c640 1 -- 172.21.15.89:0/1862443614 >> v2:172.21.15.154:3300/0 conn(0x7fc1780ce070 msgr2=0x7fc1780ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:48.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.172+0000 7fc17659c640 1 --2- 172.21.15.89:0/1862443614 >> v2:172.21.15.154:3300/0 conn(0x7fc1780ce070 0x7fc1780ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:48.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.172+0000 7fc17659c640 1 -- 172.21.15.89:0/1862443614 >> v1:172.21.15.89:6789/0 conn(0x7fc178145b70 legacy=0x7fc178160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:48.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.172+0000 7fc17659c640 1 -- 172.21.15.89:0/1862443614 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17815cbf0 con 0x7fc1780cd4f0 2023-10-18T10:46:48.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.172+0000 7fc17559a640 1 -- 172.21.15.89:0/1862443614 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1882661299 0 0) 0x7fc1780cd8d0 con 0x7fc178145b70 2023-10-18T10:46:48.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.172+0000 7fc176d9d640 1 --2- 172.21.15.89:0/1862443614 >> v2:172.21.15.154:3300/0 conn(0x7fc1780ce070 0x7fc1780ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:48.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.176+0000 7fc17659c640 1 --2- 172.21.15.89:0/1862443614 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1780cd4f0 0x7fc178143470 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc1780b63d0 tx=0x7fc16801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf011d5cc85b61c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:48.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.176+0000 7fc17559a640 1 -- 172.21.15.89:0/1862443614 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1680a7070 con 0x7fc1780cd4f0 2023-10-18T10:46:48.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.176+0000 7fc17559a640 1 -- 172.21.15.89:0/1862443614 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1680ab430 con 0x7fc1780cd4f0 2023-10-18T10:46:48.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.176+0000 7fc17559a640 1 -- 172.21.15.89:0/1862443614 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1680ab7c0 con 0x7fc1780cd4f0 2023-10-18T10:46:48.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.176+0000 7fc17cf37640 1 -- 172.21.15.89:0/1862443614 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1780cd4f0 msgr2=0x7fc178143470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:48.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.176+0000 7fc17cf37640 1 --2- 172.21.15.89:0/1862443614 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1780cd4f0 0x7fc178143470 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc1780b63d0 tx=0x7fc16801c430 comp rx=0 tx=0).stop 2023-10-18T10:46:48.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.176+0000 7fc17cf37640 1 -- 172.21.15.89:0/1862443614 shutdown_connections 2023-10-18T10:46:48.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.176+0000 7fc17cf37640 1 --2- 172.21.15.89:0/1862443614 >> v2:172.21.15.154:3300/0 conn(0x7fc1780ce070 0x7fc1780ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:48.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.176+0000 7fc17cf37640 1 --2- 172.21.15.89:0/1862443614 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1780cd4f0 0x7fc178143470 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:48.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.176+0000 7fc17cf37640 1 -- 172.21.15.89:0/1862443614 >> 172.21.15.89:0/1862443614 conn(0x7fc17806dad0 msgr2=0x7fc17806dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:48.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.176+0000 7fc17cf37640 1 -- 172.21.15.89:0/1862443614 shutdown_connections 2023-10-18T10:46:48.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.176+0000 7fc17cf37640 1 -- 172.21.15.89:0/1862443614 wait complete. 2023-10-18T10:46:48.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.180+0000 7fc17cf37640 1 Processor -- start 2023-10-18T10:46:48.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.180+0000 7fc17cf37640 1 -- start start 2023-10-18T10:46:48.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.180+0000 7fc17cf37640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1780cd4f0 0x7fc178152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:48.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.180+0000 7fc17cf37640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1780ce070 0x7fc17815bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:48.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.180+0000 7fc17cf37640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc178145b70 0x7fc17815c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:48.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.180+0000 7fc17cf37640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc178155710 con 0x7fc1780ce070 2023-10-18T10:46:48.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.180+0000 7fc17cf37640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1781559f0 con 0x7fc1780cd4f0 2023-10-18T10:46:48.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.180+0000 7fc17cf37640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc178155cd0 con 0x7fc178145b70 2023-10-18T10:46:48.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.180+0000 7fc175d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1780ce070 0x7fc17815bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:48.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.180+0000 7fc175d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1780ce070 0x7fc17815bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56242/0 (socket says 172.21.15.89:56242) 2023-10-18T10:46:48.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.180+0000 7fc175d9b640 1 -- 172.21.15.89:0/3254170998 learned_addr learned my addr 172.21.15.89:0/3254170998 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:48.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.180+0000 7fc17659c640 1 --2- 172.21.15.89:0/3254170998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1780cd4f0 0x7fc178152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:48.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.180+0000 7fc176d9d640 1 --2- 172.21.15.89:0/3254170998 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc178145b70 0x7fc17815c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:48.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.180+0000 7fc175d9b640 1 -- 172.21.15.89:0/3254170998 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc178145b70 msgr2=0x7fc17815c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:48.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.180+0000 7fc175d9b640 1 --2- 172.21.15.89:0/3254170998 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc178145b70 0x7fc17815c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:48.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.180+0000 7fc175d9b640 1 -- 172.21.15.89:0/3254170998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1780cd4f0 msgr2=0x7fc178152680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:48.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.180+0000 7fc175d9b640 1 --2- 172.21.15.89:0/3254170998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1780cd4f0 0x7fc178152680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:48.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.180+0000 7fc175d9b640 1 -- 172.21.15.89:0/3254170998 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc168009420 con 0x7fc1780ce070 2023-10-18T10:46:48.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.180+0000 7fc17659c640 1 --2- 172.21.15.89:0/3254170998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1780cd4f0 0x7fc178152680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:48.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.180+0000 7fc176d9d640 1 --2- 172.21.15.89:0/3254170998 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc178145b70 0x7fc17815c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:48.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.180+0000 7fc175d9b640 1 --2- 172.21.15.89:0/3254170998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1780ce070 0x7fc17815bcf0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc16c007fd0 tx=0x7fc16c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:48.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.180+0000 7fc166ffd640 1 -- 172.21.15.89:0/3254170998 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c005cf0 con 0x7fc1780ce070 2023-10-18T10:46:48.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.180+0000 7fc166ffd640 1 -- 172.21.15.89:0/3254170998 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16c005e70 con 0x7fc1780ce070 2023-10-18T10:46:48.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.180+0000 7fc166ffd640 1 -- 172.21.15.89:0/3254170998 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c0072f0 con 0x7fc1780ce070 2023-10-18T10:46:48.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.180+0000 7fc17cf37640 1 -- 172.21.15.89:0/3254170998 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc178155fb0 con 0x7fc1780ce070 2023-10-18T10:46:48.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.180+0000 7fc17cf37640 1 -- 172.21.15.89:0/3254170998 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc178152bc0 con 0x7fc1780ce070 2023-10-18T10:46:48.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.180+0000 7fc17cf37640 1 -- 172.21.15.89:0/3254170998 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc17804dbe0 con 0x7fc1780ce070 2023-10-18T10:46:48.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.184+0000 7fc166ffd640 1 -- 172.21.15.89:0/3254170998 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fc16c007450 con 0x7fc1780ce070 2023-10-18T10:46:48.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.184+0000 7fc166ffd640 1 --2- 172.21.15.89:0/3254170998 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc158065680 0x7fc158067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:48.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.184+0000 7fc166ffd640 1 -- 172.21.15.89:0/3254170998 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc16c0827c0 con 0x7fc1780ce070 2023-10-18T10:46:48.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.184+0000 7fc17659c640 1 --2- 172.21.15.89:0/3254170998 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc158065680 0x7fc158067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:48.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.184+0000 7fc17659c640 1 --2- 172.21.15.89:0/3254170998 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc158065680 0x7fc158067b40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc1680af000 tx=0x7fc168008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:48.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.188+0000 7fc166ffd640 1 -- 172.21.15.89:0/3254170998 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fc16c04feb0 con 0x7fc1780ce070 2023-10-18T10:46:48.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.340+0000 7fc17cf37640 1 -- 172.21.15.89:0/3254170998 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc178156460 con 0x7fc1780ce070 2023-10-18T10:46:48.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.340+0000 7fc166ffd640 1 -- 172.21.15.89:0/3254170998 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc16c0534d0 con 0x7fc1780ce070 2023-10-18T10:46:48.344 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-18T10:46:48.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.340+0000 7fc17cf37640 1 -- 172.21.15.89:0/3254170998 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc158065680 msgr2=0x7fc158067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:48.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.340+0000 7fc17cf37640 1 --2- 172.21.15.89:0/3254170998 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc158065680 0x7fc158067b40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc1680af000 tx=0x7fc168008040 comp rx=0 tx=0).stop 2023-10-18T10:46:48.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.340+0000 7fc17cf37640 1 -- 172.21.15.89:0/3254170998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1780ce070 msgr2=0x7fc17815bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:48.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.340+0000 7fc17cf37640 1 --2- 172.21.15.89:0/3254170998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1780ce070 0x7fc17815bcf0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc16c007fd0 tx=0x7fc16c00c8e0 comp rx=0 tx=0).stop 2023-10-18T10:46:48.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.340+0000 7fc17cf37640 1 -- 172.21.15.89:0/3254170998 shutdown_connections 2023-10-18T10:46:48.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.340+0000 7fc17cf37640 1 --2- 172.21.15.89:0/3254170998 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc158065680 0x7fc158067b40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:48.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.340+0000 7fc17cf37640 1 --2- 172.21.15.89:0/3254170998 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc178145b70 0x7fc17815c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:48.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.340+0000 7fc17cf37640 1 --2- 172.21.15.89:0/3254170998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1780ce070 0x7fc17815bcf0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:48.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.340+0000 7fc17cf37640 1 --2- 172.21.15.89:0/3254170998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1780cd4f0 0x7fc178152680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:48.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.340+0000 7fc17cf37640 1 -- 172.21.15.89:0/3254170998 >> 172.21.15.89:0/3254170998 conn(0x7fc17806dad0 msgr2=0x7fc1781453d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:48.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.340+0000 7fc17cf37640 1 -- 172.21.15.89:0/3254170998 shutdown_connections 2023-10-18T10:46:48.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.340+0000 7fc17cf37640 1 -- 172.21.15.89:0/3254170998 wait complete. 2023-10-18T10:46:48.365 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-18T10:46:48.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.500+0000 7f2003974640 1 Processor -- start 2023-10-18T10:46:48.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.500+0000 7f2003974640 1 -- start start 2023-10-18T10:46:48.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.500+0000 7f2003974640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1ffc0cd4f0 0x7f1ffc143470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:48.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.500+0000 7f2003974640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ffc0ce070 0x7f1ffc0ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:48.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.500+0000 7f2003974640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ffc0cd8d0 con 0x7f1ffc145b70 2023-10-18T10:46:48.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.500+0000 7f2003974640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ffc069900 con 0x7f1ffc0cd4f0 2023-10-18T10:46:48.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.500+0000 7f2003974640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc0cea20 con 0x7f1ffc0ce070 2023-10-18T10:46:48.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.500+0000 7f2001710640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1ffc0cd4f0 0x7f1ffc143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:48.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.500+0000 7f2000f0f640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1ffc145b70 0x7f1ffc160cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48698/0 (socket says 172.21.15.89:48698) 2023-10-18T10:46:48.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.504+0000 7f2001710640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1ffc0cd4f0 0x7f1ffc143470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56244/0 (socket says 172.21.15.89:56244) 2023-10-18T10:46:48.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.504+0000 7f2000f0f640 1 -- 172.21.15.89:0/1591000438 learned_addr learned my addr 172.21.15.89:0/1591000438 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:48.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.504+0000 7f2001f11640 1 --2- 172.21.15.89:0/1591000438 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ffc0ce070 0x7f1ffc0ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:48.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.504+0000 7f2001710640 1 -- 172.21.15.89:0/1591000438 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ffc0ce070 msgr2=0x7f1ffc0ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:48.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.504+0000 7f2001710640 1 --2- 172.21.15.89:0/1591000438 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ffc0ce070 0x7f1ffc0ce4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:48.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.504+0000 7f2001710640 1 -- 172.21.15.89:0/1591000438 >> v1:172.21.15.89:6789/0 conn(0x7f1ffc145b70 legacy=0x7f1ffc160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:48.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.504+0000 7f2001710640 1 -- 172.21.15.89:0/1591000438 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ffc15cbf0 con 0x7f1ffc0cd4f0 2023-10-18T10:46:48.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.504+0000 7f1ff3fff640 1 -- 172.21.15.89:0/1591000438 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2945558625 0 0) 0x7f1ffc0cd8d0 con 0x7f1ffc145b70 2023-10-18T10:46:48.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.504+0000 7f2001710640 1 --2- 172.21.15.89:0/1591000438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ffc0cd4f0 0x7f1ffc143470 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc069a40 tx=0x7f1ff801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8dbd2b56c6093c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:48.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.504+0000 7f1ff3fff640 1 -- 172.21.15.89:0/1591000438 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff80ab650 con 0x7f1ffc0cd4f0 2023-10-18T10:46:48.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.504+0000 7f1ff3fff640 1 -- 172.21.15.89:0/1591000438 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff80ab7b0 con 0x7f1ffc0cd4f0 2023-10-18T10:46:48.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.504+0000 7f1ff3fff640 1 -- 172.21.15.89:0/1591000438 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff80abb40 con 0x7f1ffc0cd4f0 2023-10-18T10:46:48.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.508+0000 7f2003974640 1 -- 172.21.15.89:0/1591000438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ffc0cd4f0 msgr2=0x7f1ffc143470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:48.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.508+0000 7f2003974640 1 --2- 172.21.15.89:0/1591000438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ffc0cd4f0 0x7f1ffc143470 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc069a40 tx=0x7f1ff801c450 comp rx=0 tx=0).stop 2023-10-18T10:46:48.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.508+0000 7f2003974640 1 -- 172.21.15.89:0/1591000438 shutdown_connections 2023-10-18T10:46:48.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.508+0000 7f2003974640 1 --2- 172.21.15.89:0/1591000438 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ffc0ce070 0x7f1ffc0ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:48.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.508+0000 7f2003974640 1 --2- 172.21.15.89:0/1591000438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ffc0cd4f0 0x7f1ffc143470 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:48.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.508+0000 7f2003974640 1 -- 172.21.15.89:0/1591000438 >> 172.21.15.89:0/1591000438 conn(0x7f1ffc06dad0 msgr2=0x7f1ffc06dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:48.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.508+0000 7f2003974640 1 -- 172.21.15.89:0/1591000438 shutdown_connections 2023-10-18T10:46:48.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.508+0000 7f2003974640 1 -- 172.21.15.89:0/1591000438 wait complete. 2023-10-18T10:46:48.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.508+0000 7f2003974640 1 Processor -- start 2023-10-18T10:46:48.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.508+0000 7f2003974640 1 -- start start 2023-10-18T10:46:48.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.508+0000 7f2003974640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ffc0cd4f0 0x7f1ffc158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:48.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.508+0000 7f2003974640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ffc0ce070 0x7f1ffc0c49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:48.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.508+0000 7f2003974640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ffc145b70 0x7f1ffc0c3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:48.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.508+0000 7f2003974640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc0c4f20 con 0x7f1ffc0ce070 2023-10-18T10:46:48.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.508+0000 7f2003974640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc0c51c0 con 0x7f1ffc0cd4f0 2023-10-18T10:46:48.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.508+0000 7f2003974640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc0c54a0 con 0x7f1ffc145b70 2023-10-18T10:46:48.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.508+0000 7f2000f0f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ffc0ce070 0x7f1ffc0c49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:48.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.508+0000 7f2000f0f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ffc0ce070 0x7f1ffc0c49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56260/0 (socket says 172.21.15.89:56260) 2023-10-18T10:46:48.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.508+0000 7f2000f0f640 1 -- 172.21.15.89:0/2220062268 learned_addr learned my addr 172.21.15.89:0/2220062268 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:48.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.508+0000 7f2001710640 1 --2- 172.21.15.89:0/2220062268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ffc0cd4f0 0x7f1ffc158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:48.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.508+0000 7f2001f11640 1 --2- 172.21.15.89:0/2220062268 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ffc145b70 0x7f1ffc0c3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:48.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.508+0000 7f2000f0f640 1 -- 172.21.15.89:0/2220062268 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ffc145b70 msgr2=0x7f1ffc0c3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:48.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.508+0000 7f2000f0f640 1 --2- 172.21.15.89:0/2220062268 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ffc145b70 0x7f1ffc0c3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:48.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.508+0000 7f2000f0f640 1 -- 172.21.15.89:0/2220062268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ffc0cd4f0 msgr2=0x7f1ffc158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:48.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.508+0000 7f2000f0f640 1 --2- 172.21.15.89:0/2220062268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ffc0cd4f0 0x7f1ffc158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:48.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.508+0000 7f2000f0f640 1 -- 172.21.15.89:0/2220062268 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff8009420 con 0x7f1ffc0ce070 2023-10-18T10:46:48.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.508+0000 7f2001710640 1 --2- 172.21.15.89:0/2220062268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ffc0cd4f0 0x7f1ffc158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:48.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.508+0000 7f2000f0f640 1 --2- 172.21.15.89:0/2220062268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ffc0ce070 0x7f1ffc0c49e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1fec0011f0 tx=0x7f1fec00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:48.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.512+0000 7f1ff1ffb640 1 -- 172.21.15.89:0/2220062268 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fec0074e0 con 0x7f1ffc0ce070 2023-10-18T10:46:48.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.512+0000 7f1ff1ffb640 1 -- 172.21.15.89:0/2220062268 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fec007640 con 0x7f1ffc0ce070 2023-10-18T10:46:48.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.512+0000 7f2003974640 1 -- 172.21.15.89:0/2220062268 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ffc0c5780 con 0x7f1ffc0ce070 2023-10-18T10:46:48.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.512+0000 7f1ff1ffb640 1 -- 172.21.15.89:0/2220062268 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fec004a10 con 0x7f1ffc0ce070 2023-10-18T10:46:48.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.512+0000 7f2003974640 1 -- 172.21.15.89:0/2220062268 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ffc0c5dd0 con 0x7f1ffc0ce070 2023-10-18T10:46:48.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.512+0000 7f2003974640 1 -- 172.21.15.89:0/2220062268 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ffc04dbe0 con 0x7f1ffc0ce070 2023-10-18T10:46:48.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.512+0000 7f1ff1ffb640 1 -- 172.21.15.89:0/2220062268 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f1fec004b70 con 0x7f1ffc0ce070 2023-10-18T10:46:48.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.512+0000 7f1ff1ffb640 1 --2- 172.21.15.89:0/2220062268 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1fe0065680 0x7f1fe0067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:48.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.512+0000 7f1ff1ffb640 1 -- 172.21.15.89:0/2220062268 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1fec082730 con 0x7f1ffc0ce070 2023-10-18T10:46:48.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.512+0000 7f2001710640 1 --2- 172.21.15.89:0/2220062268 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1fe0065680 0x7f1fe0067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:48.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.516+0000 7f2001710640 1 --2- 172.21.15.89:0/2220062268 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1fe0065680 0x7f1fe0067b40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80af000 tx=0x7f1ff80a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:48.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.516+0000 7f1ff1ffb640 1 -- 172.21.15.89:0/2220062268 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f1fec04fe20 con 0x7f1ffc0ce070 2023-10-18T10:46:48.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.668+0000 7f2003974640 1 -- 172.21.15.89:0/2220062268 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1ffc15eab0 con 0x7f1ffc0ce070 2023-10-18T10:46:48.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.668+0000 7f1ff1ffb640 1 -- 172.21.15.89:0/2220062268 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1fec053440 con 0x7f1ffc0ce070 2023-10-18T10:46:48.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.668+0000 7f2003974640 1 -- 172.21.15.89:0/2220062268 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1fe0065680 msgr2=0x7f1fe0067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:48.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.668+0000 7f2003974640 1 --2- 172.21.15.89:0/2220062268 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1fe0065680 0x7f1fe0067b40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80af000 tx=0x7f1ff80a7000 comp rx=0 tx=0).stop 2023-10-18T10:46:48.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.668+0000 7f2003974640 1 -- 172.21.15.89:0/2220062268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ffc0ce070 msgr2=0x7f1ffc0c49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:48.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.668+0000 7f2003974640 1 --2- 172.21.15.89:0/2220062268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ffc0ce070 0x7f1ffc0c49e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1fec0011f0 tx=0x7f1fec00c9b0 comp rx=0 tx=0).stop 2023-10-18T10:46:48.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.668+0000 7f2003974640 1 -- 172.21.15.89:0/2220062268 shutdown_connections 2023-10-18T10:46:48.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.668+0000 7f2003974640 1 --2- 172.21.15.89:0/2220062268 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1fe0065680 0x7f1fe0067b40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:48.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.668+0000 7f2003974640 1 --2- 172.21.15.89:0/2220062268 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ffc145b70 0x7f1ffc0c3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:48.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.668+0000 7f2003974640 1 --2- 172.21.15.89:0/2220062268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ffc0ce070 0x7f1ffc0c49e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:48.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.668+0000 7f2003974640 1 --2- 172.21.15.89:0/2220062268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ffc0cd4f0 0x7f1ffc158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:48.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.668+0000 7f2003974640 1 -- 172.21.15.89:0/2220062268 >> 172.21.15.89:0/2220062268 conn(0x7f1ffc06dad0 msgr2=0x7f1ffc144c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:48.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.668+0000 7f2003974640 1 -- 172.21.15.89:0/2220062268 shutdown_connections 2023-10-18T10:46:48.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:48.668+0000 7f2003974640 1 -- 172.21.15.89:0/2220062268 wait complete. 2023-10-18T10:46:48.694 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T10:46:48.694 DEBUG:teuthology.orchestra.run.smithi089:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T10:46:48.781 DEBUG:teuthology.orchestra.run.smithi154:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T10:46:48.851 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T10:46:48.852 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:46:48.852 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:46:48.861 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:46:48.862 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:46:48.915 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:46:48.915 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:46:48.967 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:46:49.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.140+0000 7f7cecb40640 1 Processor -- start 2023-10-18T10:46:49.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.140+0000 7f7cecb40640 1 -- start start 2023-10-18T10:46:49.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.144+0000 7f7cecb40640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7ce806ff00 0x7f7ce815e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:49.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.144+0000 7f7cecb40640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ce815acb0 con 0x7f7ce80ca340 2023-10-18T10:46:49.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.144+0000 7f7cecb40640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ce80702e0 con 0x7f7ce80c8960 2023-10-18T10:46:49.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.144+0000 7f7cecb40640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ce815ef30 con 0x7f7ce806ff00 2023-10-18T10:46:49.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.144+0000 7f7cddd9b640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7ce80ca340 0x7f7ce815b140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48708/0 (socket says 172.21.15.89:48708) 2023-10-18T10:46:49.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.144+0000 7f7cddd9b640 1 -- 172.21.15.89:0/3795403134 learned_addr learned my addr 172.21.15.89:0/3795403134 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:49.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.144+0000 7f7ce5d9b640 1 -- 172.21.15.89:0/3795403134 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3856391277 0 0) 0x7f7ce815acb0 con 0x7f7ce80ca340 2023-10-18T10:46:49.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.144+0000 7f7ce5d9b640 1 -- 172.21.15.89:0/3795403134 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cd4003530 con 0x7f7ce80ca340 2023-10-18T10:46:49.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.144+0000 7f7ce659c640 1 --2- 172.21.15.89:0/3795403134 >> v2:172.21.15.154:3300/0 conn(0x7f7ce806ff00 0x7f7ce815e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:49.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.144+0000 7f7ce5d9b640 1 -- 172.21.15.89:0/3795403134 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1914800032 0 0) 0x7f7cd4003530 con 0x7f7ce80ca340 2023-10-18T10:46:49.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.144+0000 7f7ce5d9b640 1 -- 172.21.15.89:0/3795403134 >> v2:172.21.15.154:3300/0 conn(0x7f7ce806ff00 msgr2=0x7f7ce815e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:49.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.144+0000 7f7ce5d9b640 1 --2- 172.21.15.89:0/3795403134 >> v2:172.21.15.154:3300/0 conn(0x7f7ce806ff00 0x7f7ce815e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:49.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.144+0000 7f7ce5d9b640 1 -- 172.21.15.89:0/3795403134 >> v1:172.21.15.154:6789/0 conn(0x7f7ce80c8960 legacy=0x7f7ce80afd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:49.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.144+0000 7f7ce5d9b640 1 -- 172.21.15.89:0/3795403134 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce815b850 con 0x7f7ce80ca340 2023-10-18T10:46:49.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.144+0000 7f7ce5d9b640 1 -- 172.21.15.89:0/3795403134 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7f7cd0001680 con 0x7f7ce80ca340 2023-10-18T10:46:49.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.144+0000 7f7ce5d9b640 1 -- 172.21.15.89:0/3795403134 >> v1:172.21.15.89:6789/0 conn(0x7f7ce80ca340 legacy=0x7f7ce815b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:49.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.144+0000 7f7ce5d9b640 1 --2- 172.21.15.89:0/3795403134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cd4003980 0x7f7cd4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:49.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.144+0000 7f7ce5d9b640 1 --2- 172.21.15.89:0/3795403134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cd40047d0 0x7f7cd4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:49.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.144+0000 7f7ce5d9b640 1 --2- 172.21.15.89:0/3795403134 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7cd40052b0 0x7f7cd400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:49.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.144+0000 7f7ce5d9b640 1 -- 172.21.15.89:0/3795403134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce815ef30 con 0x7f7cd40047d0 2023-10-18T10:46:49.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.144+0000 7f7ce5d9b640 1 -- 172.21.15.89:0/3795403134 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd400c180 con 0x7f7cd4003980 2023-10-18T10:46:49.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.144+0000 7f7ce5d9b640 1 -- 172.21.15.89:0/3795403134 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd400c340 con 0x7f7cd40052b0 2023-10-18T10:46:49.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.144+0000 7f7cddd9b640 1 --2- 172.21.15.89:0/3795403134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cd40047d0 0x7f7cd4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:49.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.144+0000 7f7ce659c640 1 --2- 172.21.15.89:0/3795403134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cd4003980 0x7f7cd4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:49.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.144+0000 7f7ce6d9d640 1 --2- 172.21.15.89:0/3795403134 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7cd40052b0 0x7f7cd400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:49.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.144+0000 7f7cddd9b640 1 -- 172.21.15.89:0/3795403134 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7cd40052b0 msgr2=0x7f7cd400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:49.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.144+0000 7f7cddd9b640 1 --2- 172.21.15.89:0/3795403134 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7cd40052b0 0x7f7cd400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:49.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.144+0000 7f7cddd9b640 1 -- 172.21.15.89:0/3795403134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cd4003980 msgr2=0x7f7cd4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:49.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.144+0000 7f7cddd9b640 1 --2- 172.21.15.89:0/3795403134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cd4003980 0x7f7cd4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:49.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.144+0000 7f7cddd9b640 1 -- 172.21.15.89:0/3795403134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd4003530 con 0x7f7cd40047d0 2023-10-18T10:46:49.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.144+0000 7f7ce6d9d640 1 -- 172.21.15.89:0/3795403134 reap_dead start 2023-10-18T10:46:49.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.148+0000 7f7cddd9b640 1 --2- 172.21.15.89:0/3795403134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cd40047d0 0x7f7cd4004c50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000c210 tx=0x7f7cd001cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99cf37037f0f1ad3 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:49.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.148+0000 7f7ce5d9b640 1 -- 172.21.15.89:0/3795403134 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0005ae0 con 0x7f7cd40047d0 2023-10-18T10:46:49.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.148+0000 7f7ce5d9b640 1 -- 172.21.15.89:0/3795403134 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd0005c40 con 0x7f7cd40047d0 2023-10-18T10:46:49.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.148+0000 7f7ce5d9b640 1 -- 172.21.15.89:0/3795403134 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd00a5a30 con 0x7f7cd40047d0 2023-10-18T10:46:49.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.148+0000 7f7cecb40640 1 -- 172.21.15.89:0/3795403134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cd40047d0 msgr2=0x7f7cd4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:49.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.148+0000 7f7cecb40640 1 --2- 172.21.15.89:0/3795403134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cd40047d0 0x7f7cd4004c50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000c210 tx=0x7f7cd001cc00 comp rx=0 tx=0).stop 2023-10-18T10:46:49.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.148+0000 7f7cecb40640 1 -- 172.21.15.89:0/3795403134 shutdown_connections 2023-10-18T10:46:49.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.148+0000 7f7cecb40640 1 --2- 172.21.15.89:0/3795403134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cd40047d0 0x7f7cd4004c50 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:49.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.148+0000 7f7cecb40640 1 -- 172.21.15.89:0/3795403134 >> 172.21.15.89:0/3795403134 conn(0x7f7ce8069f00 msgr2=0x7f7ce80cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:49.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.148+0000 7f7cecb40640 1 -- 172.21.15.89:0/3795403134 shutdown_connections 2023-10-18T10:46:49.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.148+0000 7f7cecb40640 1 -- 172.21.15.89:0/3795403134 wait complete. 2023-10-18T10:46:49.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.152+0000 7f7cecb40640 1 Processor -- start 2023-10-18T10:46:49.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.152+0000 7f7cecb40640 1 -- start start 2023-10-18T10:46:49.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.152+0000 7f7cecb40640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cd40047d0 0x7f7ce80cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:49.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.152+0000 7f7cecb40640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ce80b2ea0 0x7f7ce8159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:49.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.152+0000 7f7cecb40640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ce80b16f0 0x7f7ce814f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:49.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.152+0000 7f7cecb40640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce80cd8c0 con 0x7f7ce80b2ea0 2023-10-18T10:46:49.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.152+0000 7f7cecb40640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce80cdba0 con 0x7f7cd40047d0 2023-10-18T10:46:49.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.152+0000 7f7cecb40640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce80cde80 con 0x7f7ce80b16f0 2023-10-18T10:46:49.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.152+0000 7f7cddd9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ce80b2ea0 0x7f7ce8159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:49.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.152+0000 7f7cddd9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ce80b2ea0 0x7f7ce8159690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56282/0 (socket says 172.21.15.89:56282) 2023-10-18T10:46:49.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.152+0000 7f7cddd9b640 1 -- 172.21.15.89:0/1076178434 learned_addr learned my addr 172.21.15.89:0/1076178434 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:49.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.152+0000 7f7ce659c640 1 --2- 172.21.15.89:0/1076178434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cd40047d0 0x7f7ce80cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:49.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.152+0000 7f7ce6d9d640 1 --2- 172.21.15.89:0/1076178434 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ce80b16f0 0x7f7ce814f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:49.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.152+0000 7f7cddd9b640 1 -- 172.21.15.89:0/1076178434 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ce80b16f0 msgr2=0x7f7ce814f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:49.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.152+0000 7f7cddd9b640 1 --2- 172.21.15.89:0/1076178434 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ce80b16f0 0x7f7ce814f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:49.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.152+0000 7f7cddd9b640 1 -- 172.21.15.89:0/1076178434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cd40047d0 msgr2=0x7f7ce80cd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:49.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.152+0000 7f7cddd9b640 1 --2- 172.21.15.89:0/1076178434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cd40047d0 0x7f7ce80cd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:49.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.152+0000 7f7cddd9b640 1 -- 172.21.15.89:0/1076178434 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce80b0ef0 con 0x7f7ce80b2ea0 2023-10-18T10:46:49.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.152+0000 7f7ce659c640 1 --2- 172.21.15.89:0/1076178434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cd40047d0 0x7f7ce80cd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:46:49.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.152+0000 7f7cddd9b640 1 --2- 172.21.15.89:0/1076178434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ce80b2ea0 0x7f7ce8159690 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7cd001ca30 tx=0x7f7cd4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:49.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.152+0000 7f7cdf7fe640 1 -- 172.21.15.89:0/1076178434 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd00b4070 con 0x7f7ce80b2ea0 2023-10-18T10:46:49.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.152+0000 7f7cdf7fe640 1 -- 172.21.15.89:0/1076178434 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd0001440 con 0x7f7ce80b2ea0 2023-10-18T10:46:49.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.152+0000 7f7cdf7fe640 1 -- 172.21.15.89:0/1076178434 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd00a57f0 con 0x7f7ce80b2ea0 2023-10-18T10:46:49.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.152+0000 7f7cecb40640 1 -- 172.21.15.89:0/1076178434 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce80b11d0 con 0x7f7ce80b2ea0 2023-10-18T10:46:49.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.152+0000 7f7cecb40640 1 -- 172.21.15.89:0/1076178434 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce80b34f0 con 0x7f7ce80b2ea0 2023-10-18T10:46:49.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.156+0000 7f7cecb40640 1 -- 172.21.15.89:0/1076178434 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce804dbe0 con 0x7f7ce80b2ea0 2023-10-18T10:46:49.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.156+0000 7f7cdf7fe640 1 -- 172.21.15.89:0/1076178434 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f7cd00a5970 con 0x7f7ce80b2ea0 2023-10-18T10:46:49.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.156+0000 7f7cdf7fe640 1 --2- 172.21.15.89:0/1076178434 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7cc80656b0 0x7f7cc8067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:49.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.156+0000 7f7cdf7fe640 1 -- 172.21.15.89:0/1076178434 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7cd011ca30 con 0x7f7ce80b2ea0 2023-10-18T10:46:49.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.156+0000 7f7ce659c640 1 --2- 172.21.15.89:0/1076178434 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7cc80656b0 0x7f7cc8067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:49.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.156+0000 7f7ce659c640 1 --2- 172.21.15.89:0/1076178434 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7cc80656b0 0x7f7cc8067b70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8003b10 tx=0x7f7cd8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:49.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.160+0000 7f7cdf7fe640 1 -- 172.21.15.89:0/1076178434 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f7cd00ea1d0 con 0x7f7ce80b2ea0 2023-10-18T10:46:49.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.308+0000 7f7cecb40640 1 -- 172.21.15.89:0/1076178434 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ce80b34f0 con 0x7f7ce80b2ea0 2023-10-18T10:46:49.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.308+0000 7f7cdf7fe640 1 -- 172.21.15.89:0/1076178434 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20192 (secure 0 0 0) 0x7f7cd010aef0 con 0x7f7ce80b2ea0 2023-10-18T10:46:49.315 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:49.315 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":29,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:46:36.958604+0000","last_up_change":"2023-10-18T10:45:42.164688+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:45:53.309122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-18T10:45:54.659763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:45:57.796514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6824","nonce":773078364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6827","nonce":773078364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6831","nonce":773078364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6829","nonce":773078364}]},"public_addr":"172.21.15.154:6824/773078364","cluster_addr":"172.21.15.154:6827/773078364","heartbeat_back_addr":"172.21.15.154:6831/773078364","heartbeat_front_addr":"172.21.15.154:6829/773078364","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":1,"to":5}]},{"pgid":"4.1a","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:46:49.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.312+0000 7f7cecb40640 1 -- 172.21.15.89:0/1076178434 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7cc80656b0 msgr2=0x7f7cc8067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:49.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.312+0000 7f7cecb40640 1 --2- 172.21.15.89:0/1076178434 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7cc80656b0 0x7f7cc8067b70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8003b10 tx=0x7f7cd8008040 comp rx=0 tx=0).stop 2023-10-18T10:46:49.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.312+0000 7f7cecb40640 1 -- 172.21.15.89:0/1076178434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ce80b2ea0 msgr2=0x7f7ce8159690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:49.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.312+0000 7f7cecb40640 1 --2- 172.21.15.89:0/1076178434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ce80b2ea0 0x7f7ce8159690 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7cd001ca30 tx=0x7f7cd4005800 comp rx=0 tx=0).stop 2023-10-18T10:46:49.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.312+0000 7f7cecb40640 1 -- 172.21.15.89:0/1076178434 shutdown_connections 2023-10-18T10:46:49.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.312+0000 7f7cecb40640 1 --2- 172.21.15.89:0/1076178434 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7cc80656b0 0x7f7cc8067b70 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:49.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.312+0000 7f7cecb40640 1 --2- 172.21.15.89:0/1076178434 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ce80b16f0 0x7f7ce814f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:49.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.312+0000 7f7cecb40640 1 --2- 172.21.15.89:0/1076178434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ce80b2ea0 0x7f7ce8159690 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:49.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.312+0000 7f7cecb40640 1 --2- 172.21.15.89:0/1076178434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cd40047d0 0x7f7ce80cd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:49.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.312+0000 7f7cecb40640 1 -- 172.21.15.89:0/1076178434 >> 172.21.15.89:0/1076178434 conn(0x7f7ce8069f00 msgr2=0x7f7ce80c9180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:49.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.312+0000 7f7cecb40640 1 -- 172.21.15.89:0/1076178434 shutdown_connections 2023-10-18T10:46:49.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.312+0000 7f7cecb40640 1 -- 172.21.15.89:0/1076178434 wait complete. 2023-10-18T10:46:49.337 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:45:43.114184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-18T10:45:48.730968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:45:53.309122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:45:54.659763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T10:45:57.796514+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T10:46:49.338 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:46:49.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.476+0000 7fa8d1648640 1 Processor -- start 2023-10-18T10:46:49.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.476+0000 7fa8d1648640 1 -- start start 2023-10-18T10:46:49.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.476+0000 7fa8d1648640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa8cc0d0760 0x7fa8cc0d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:49.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.476+0000 7fa8d1648640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8cc0cd160 0x7fa8cc0cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:49.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.476+0000 7fa8d1648640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8cc0d1100 con 0x7fa8cc15c940 2023-10-18T10:46:49.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.476+0000 7fa8d1648640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8cc15da50 con 0x7fa8cc0d0760 2023-10-18T10:46:49.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.476+0000 7fa8d1648640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc0d3110 con 0x7fa8cc0cd160 2023-10-18T10:46:49.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.476+0000 7fa8cb7fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8cc0cd160 0x7fa8cc0cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:49.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.476+0000 7fa8c3fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa8cc0d0760 0x7fa8cc0d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:49.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.476+0000 7fa8caffd640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fa8cc15c940 0x7fa8cc0d2a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.89:57118/0 (socket says 172.21.15.89:57118) 2023-10-18T10:46:49.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.476+0000 7fa8caffd640 1 -- 172.21.15.89:0/275761858 learned_addr learned my addr 172.21.15.89:0/275761858 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:49.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.476+0000 7fa8ca7fc640 1 -- 172.21.15.89:0/275761858 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1572009103 0 0) 0x7fa8cc0d1100 con 0x7fa8cc15c940 2023-10-18T10:46:49.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.476+0000 7fa8ca7fc640 1 -- 172.21.15.89:0/275761858 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8b8003530 con 0x7fa8cc15c940 2023-10-18T10:46:49.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.476+0000 7fa8c3fff640 1 -- 172.21.15.89:0/275761858 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8cc0cd160 msgr2=0x7fa8cc0cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:49.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.476+0000 7fa8c3fff640 1 --2- 172.21.15.89:0/275761858 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8cc0cd160 0x7fa8cc0cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:49.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.476+0000 7fa8c3fff640 1 -- 172.21.15.89:0/275761858 >> v1:172.21.15.154:6789/0 conn(0x7fa8cc15c940 legacy=0x7fa8cc0d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:49.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.476+0000 7fa8c3fff640 1 -- 172.21.15.89:0/275761858 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8cc15ce40 con 0x7fa8cc0d0760 2023-10-18T10:46:49.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.476+0000 7fa8cb7fe640 1 --2- 172.21.15.89:0/275761858 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8cc0cd160 0x7fa8cc0cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:46:49.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.480+0000 7fa8c3fff640 1 --2- 172.21.15.89:0/275761858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8cc0d0760 0x7fa8cc0d0bc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa8b40097c0 tx=0x7fa8b401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=978f930347bdb085 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:49.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.480+0000 7fa8ca7fc640 1 -- 172.21.15.89:0/275761858 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b40ab650 con 0x7fa8cc0d0760 2023-10-18T10:46:49.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.480+0000 7fa8ca7fc640 1 -- 172.21.15.89:0/275761858 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b40ab7b0 con 0x7fa8cc0d0760 2023-10-18T10:46:49.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.480+0000 7fa8ca7fc640 1 -- 172.21.15.89:0/275761858 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b40abb40 con 0x7fa8cc0d0760 2023-10-18T10:46:49.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.480+0000 7fa8d1648640 1 -- 172.21.15.89:0/275761858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8cc0d0760 msgr2=0x7fa8cc0d0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:49.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.480+0000 7fa8d1648640 1 --2- 172.21.15.89:0/275761858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8cc0d0760 0x7fa8cc0d0bc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa8b40097c0 tx=0x7fa8b401c450 comp rx=0 tx=0).stop 2023-10-18T10:46:49.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.480+0000 7fa8d1648640 1 -- 172.21.15.89:0/275761858 shutdown_connections 2023-10-18T10:46:49.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.480+0000 7fa8d1648640 1 --2- 172.21.15.89:0/275761858 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8cc0cd160 0x7fa8cc0cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:49.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.480+0000 7fa8d1648640 1 --2- 172.21.15.89:0/275761858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8cc0d0760 0x7fa8cc0d0bc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:49.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.480+0000 7fa8d1648640 1 -- 172.21.15.89:0/275761858 >> 172.21.15.89:0/275761858 conn(0x7fa8cc089fd0 msgr2=0x7fa8cc09b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:49.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.480+0000 7fa8d1648640 1 -- 172.21.15.89:0/275761858 shutdown_connections 2023-10-18T10:46:49.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.480+0000 7fa8d1648640 1 -- 172.21.15.89:0/275761858 wait complete. 2023-10-18T10:46:49.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.484+0000 7fa8d1648640 1 Processor -- start 2023-10-18T10:46:49.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.484+0000 7fa8d1648640 1 -- start start 2023-10-18T10:46:49.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.484+0000 7fa8d1648640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8cc0cd160 0x7fa8cc159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:49.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.484+0000 7fa8d1648640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8cc0d0760 0x7fa8cc0cb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:49.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.484+0000 7fa8d1648640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8cc0ca3f0 0x7fa8cc1f4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:49.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.484+0000 7fa8d1648640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc0c6e80 con 0x7fa8cc0d0760 2023-10-18T10:46:49.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.484+0000 7fa8d1648640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc0c62b0 con 0x7fa8cc0ca3f0 2023-10-18T10:46:49.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.484+0000 7fa8d1648640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc0c6550 con 0x7fa8cc0cd160 2023-10-18T10:46:49.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.484+0000 7fa8c3fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8cc0d0760 0x7fa8cc0cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:49.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.484+0000 7fa8c3fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8cc0d0760 0x7fa8cc0cb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56286/0 (socket says 172.21.15.89:56286) 2023-10-18T10:46:49.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.484+0000 7fa8c3fff640 1 -- 172.21.15.89:0/2114662567 learned_addr learned my addr 172.21.15.89:0/2114662567 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:49.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.484+0000 7fa8caffd640 1 --2- 172.21.15.89:0/2114662567 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8cc0cd160 0x7fa8cc159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:49.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.484+0000 7fa8cb7fe640 1 --2- 172.21.15.89:0/2114662567 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8cc0ca3f0 0x7fa8cc1f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:49.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.484+0000 7fa8c3fff640 1 -- 172.21.15.89:0/2114662567 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8cc0cd160 msgr2=0x7fa8cc159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:49.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.484+0000 7fa8c3fff640 1 --2- 172.21.15.89:0/2114662567 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8cc0cd160 0x7fa8cc159b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:49.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.484+0000 7fa8c3fff640 1 -- 172.21.15.89:0/2114662567 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8cc0ca3f0 msgr2=0x7fa8cc1f4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:49.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.484+0000 7fa8c3fff640 1 --2- 172.21.15.89:0/2114662567 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8cc0ca3f0 0x7fa8cc1f4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:49.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.484+0000 7fa8c3fff640 1 -- 172.21.15.89:0/2114662567 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c4009a70 con 0x7fa8cc0d0760 2023-10-18T10:46:49.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.484+0000 7fa8c3fff640 1 --2- 172.21.15.89:0/2114662567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8cc0d0760 0x7fa8cc0cb260 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa8b40ad040 tx=0x7fa8b401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:49.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.484+0000 7fa8c37fe640 1 -- 172.21.15.89:0/2114662567 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b40bf610 con 0x7fa8cc0d0760 2023-10-18T10:46:49.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.484+0000 7fa8c37fe640 1 -- 172.21.15.89:0/2114662567 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b40bf7b0 con 0x7fa8cc0d0760 2023-10-18T10:46:49.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.484+0000 7fa8c37fe640 1 -- 172.21.15.89:0/2114662567 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b40bfb60 con 0x7fa8cc0d0760 2023-10-18T10:46:49.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.484+0000 7fa8d1648640 1 -- 172.21.15.89:0/2114662567 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8b4009420 con 0x7fa8cc0d0760 2023-10-18T10:46:49.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.484+0000 7fa8d1648640 1 -- 172.21.15.89:0/2114662567 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8cc0c6b10 con 0x7fa8cc0d0760 2023-10-18T10:46:49.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.484+0000 7fa8d1648640 1 -- 172.21.15.89:0/2114662567 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8cc04dbe0 con 0x7fa8cc0d0760 2023-10-18T10:46:49.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.488+0000 7fa8c37fe640 1 -- 172.21.15.89:0/2114662567 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fa8b40b5400 con 0x7fa8cc0d0760 2023-10-18T10:46:49.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.488+0000 7fa8c37fe640 1 --2- 172.21.15.89:0/2114662567 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa8ac065680 0x7fa8ac067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:49.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.488+0000 7fa8c37fe640 1 -- 172.21.15.89:0/2114662567 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa8b411db90 con 0x7fa8cc0d0760 2023-10-18T10:46:49.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.488+0000 7fa8caffd640 1 --2- 172.21.15.89:0/2114662567 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa8ac065680 0x7fa8ac067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:49.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.488+0000 7fa8caffd640 1 --2- 172.21.15.89:0/2114662567 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa8ac065680 0x7fa8ac067b40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0011f0 tx=0x7fa8bc007d40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:49.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.492+0000 7fa8c37fe640 1 -- 172.21.15.89:0/2114662567 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fa8b40eb3c0 con 0x7fa8cc0d0760 2023-10-18T10:46:49.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.644+0000 7fa8d1648640 1 -- 172.21.15.89:0/2114662567 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa8cc0d3330 con 0x7fa8cc0d0760 2023-10-18T10:46:49.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.644+0000 7fa8c37fe640 1 -- 172.21.15.89:0/2114662567 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa8b40ee9e0 con 0x7fa8cc0d0760 2023-10-18T10:46:49.649 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-18T10:46:49.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.644+0000 7fa8d1648640 1 -- 172.21.15.89:0/2114662567 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa8ac065680 msgr2=0x7fa8ac067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:49.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.644+0000 7fa8d1648640 1 --2- 172.21.15.89:0/2114662567 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa8ac065680 0x7fa8ac067b40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0011f0 tx=0x7fa8bc007d40 comp rx=0 tx=0).stop 2023-10-18T10:46:49.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.644+0000 7fa8d1648640 1 -- 172.21.15.89:0/2114662567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8cc0d0760 msgr2=0x7fa8cc0cb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:49.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.644+0000 7fa8d1648640 1 --2- 172.21.15.89:0/2114662567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8cc0d0760 0x7fa8cc0cb260 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa8b40ad040 tx=0x7fa8b401ac90 comp rx=0 tx=0).stop 2023-10-18T10:46:49.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.644+0000 7fa8d1648640 1 -- 172.21.15.89:0/2114662567 shutdown_connections 2023-10-18T10:46:49.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.644+0000 7fa8d1648640 1 --2- 172.21.15.89:0/2114662567 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa8ac065680 0x7fa8ac067b40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:49.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.644+0000 7fa8d1648640 1 --2- 172.21.15.89:0/2114662567 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8cc0ca3f0 0x7fa8cc1f4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:49.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.644+0000 7fa8d1648640 1 --2- 172.21.15.89:0/2114662567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8cc0d0760 0x7fa8cc0cb260 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:49.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.644+0000 7fa8d1648640 1 --2- 172.21.15.89:0/2114662567 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8cc0cd160 0x7fa8cc159b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:49.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.644+0000 7fa8d1648640 1 -- 172.21.15.89:0/2114662567 >> 172.21.15.89:0/2114662567 conn(0x7fa8cc089fd0 msgr2=0x7fa8cc15f610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:49.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.644+0000 7fa8d1648640 1 -- 172.21.15.89:0/2114662567 shutdown_connections 2023-10-18T10:46:49.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.644+0000 7fa8d1648640 1 -- 172.21.15.89:0/2114662567 wait complete. 2023-10-18T10:46:49.669 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-18T10:46:49.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.804+0000 7efda7656640 1 Processor -- start 2023-10-18T10:46:49.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.804+0000 7efda7656640 1 -- start start 2023-10-18T10:46:49.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.804+0000 7efda7656640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efda0159490 0x7efda015d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:49.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.804+0000 7efda7656640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efda006bee0 0x7efda00c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:49.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.804+0000 7efda7656640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efda015ddf0 con 0x7efda0070600 2023-10-18T10:46:49.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.804+0000 7efda7656640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efda0070b90 con 0x7efda006bee0 2023-10-18T10:46:49.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.804+0000 7efda7656640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda0070e70 con 0x7efda0159490 2023-10-18T10:46:49.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.804+0000 7efda4bf1640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7efda0070600 0x7efda006b7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48722/0 (socket says 172.21.15.89:48722) 2023-10-18T10:46:49.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.804+0000 7efda4bf1640 1 -- 172.21.15.89:0/136183748 learned_addr learned my addr 172.21.15.89:0/136183748 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:49.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.808+0000 7efda5bf3640 1 --2- 172.21.15.89:0/136183748 >> v2:172.21.15.154:3300/0 conn(0x7efda006bee0 0x7efda00c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:49.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.808+0000 7efda53f2640 1 --2- 172.21.15.89:0/136183748 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efda0159490 0x7efda015d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:49.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.808+0000 7efd97fff640 1 -- 172.21.15.89:0/136183748 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2784912143 0 0) 0x7efda015ddf0 con 0x7efda0070600 2023-10-18T10:46:49.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.808+0000 7efd97fff640 1 -- 172.21.15.89:0/136183748 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd8c003530 con 0x7efda0070600 2023-10-18T10:46:49.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.808+0000 7efda5bf3640 1 -- 172.21.15.89:0/136183748 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efda0159490 msgr2=0x7efda015d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:49.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.808+0000 7efda5bf3640 1 --2- 172.21.15.89:0/136183748 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efda0159490 0x7efda015d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:49.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.808+0000 7efda5bf3640 1 -- 172.21.15.89:0/136183748 >> v1:172.21.15.89:6789/0 conn(0x7efda0070600 legacy=0x7efda006b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:49.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.808+0000 7efda5bf3640 1 -- 172.21.15.89:0/136183748 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda0071150 con 0x7efda006bee0 2023-10-18T10:46:49.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.808+0000 7efda5bf3640 1 --2- 172.21.15.89:0/136183748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efda006bee0 0x7efda00c8960 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efd980097c0 tx=0x7efd9801c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46a4c6b23d4fd483 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:49.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.808+0000 7efd97fff640 1 -- 172.21.15.89:0/136183748 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd980ac620 con 0x7efda006bee0 2023-10-18T10:46:49.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.808+0000 7efd97fff640 1 -- 172.21.15.89:0/136183748 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd980ac780 con 0x7efda006bee0 2023-10-18T10:46:49.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.808+0000 7efd97fff640 1 -- 172.21.15.89:0/136183748 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd980acb10 con 0x7efda006bee0 2023-10-18T10:46:49.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.812+0000 7efda7656640 1 -- 172.21.15.89:0/136183748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efda006bee0 msgr2=0x7efda00c8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:49.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.812+0000 7efda7656640 1 --2- 172.21.15.89:0/136183748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efda006bee0 0x7efda00c8960 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efd980097c0 tx=0x7efd9801c3c0 comp rx=0 tx=0).stop 2023-10-18T10:46:49.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.812+0000 7efda7656640 1 -- 172.21.15.89:0/136183748 shutdown_connections 2023-10-18T10:46:49.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.812+0000 7efda7656640 1 --2- 172.21.15.89:0/136183748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efda006bee0 0x7efda00c8960 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:49.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.812+0000 7efda7656640 1 --2- 172.21.15.89:0/136183748 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efda0159490 0x7efda015d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:49.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.812+0000 7efda7656640 1 -- 172.21.15.89:0/136183748 >> 172.21.15.89:0/136183748 conn(0x7efda0069f00 msgr2=0x7efda00cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:49.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.812+0000 7efda7656640 1 -- 172.21.15.89:0/136183748 shutdown_connections 2023-10-18T10:46:49.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.812+0000 7efda7656640 1 -- 172.21.15.89:0/136183748 wait complete. 2023-10-18T10:46:49.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.812+0000 7efda7656640 1 Processor -- start 2023-10-18T10:46:49.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.812+0000 7efda7656640 1 -- start start 2023-10-18T10:46:49.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.812+0000 7efda7656640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efda006bee0 0x7efda00c1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:49.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.812+0000 7efda7656640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efda0070600 0x7efda01518e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:49.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.812+0000 7efda7656640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efda0159490 0x7efda00c4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:49.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.812+0000 7efda7656640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda0154480 con 0x7efda0070600 2023-10-18T10:46:49.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.812+0000 7efda7656640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda0151e20 con 0x7efda0159490 2023-10-18T10:46:49.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.812+0000 7efda7656640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda0153ec0 con 0x7efda006bee0 2023-10-18T10:46:49.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.812+0000 7efda4bf1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efda0070600 0x7efda01518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:49.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.812+0000 7efda4bf1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efda0070600 0x7efda01518e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56302/0 (socket says 172.21.15.89:56302) 2023-10-18T10:46:49.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.812+0000 7efda5bf3640 1 --2- 172.21.15.89:0/2441389448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efda0159490 0x7efda00c4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:49.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.812+0000 7efda4bf1640 1 -- 172.21.15.89:0/2441389448 learned_addr learned my addr 172.21.15.89:0/2441389448 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:49.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.812+0000 7efda53f2640 1 --2- 172.21.15.89:0/2441389448 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efda006bee0 0x7efda00c1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:49.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.816+0000 7efda5bf3640 1 -- 172.21.15.89:0/2441389448 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efda006bee0 msgr2=0x7efda00c1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:49.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.816+0000 7efda5bf3640 1 --2- 172.21.15.89:0/2441389448 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efda006bee0 0x7efda00c1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:49.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.816+0000 7efda5bf3640 1 -- 172.21.15.89:0/2441389448 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efda0070600 msgr2=0x7efda01518e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:49.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.816+0000 7efda5bf3640 1 --2- 172.21.15.89:0/2441389448 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efda0070600 0x7efda01518e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:49.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.816+0000 7efda5bf3640 1 -- 172.21.15.89:0/2441389448 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd98009420 con 0x7efda0159490 2023-10-18T10:46:49.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.816+0000 7efda4bf1640 1 --2- 172.21.15.89:0/2441389448 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efda0070600 0x7efda01518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:46:49.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.816+0000 7efda5bf3640 1 --2- 172.21.15.89:0/2441389448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efda0159490 0x7efda00c4a90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efd980ac000 tx=0x7efd9801ac00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:49.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.816+0000 7efd95ffb640 1 -- 172.21.15.89:0/2441389448 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd980ac620 con 0x7efda0159490 2023-10-18T10:46:49.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.816+0000 7efd95ffb640 1 -- 172.21.15.89:0/2441389448 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd980b8450 con 0x7efda0159490 2023-10-18T10:46:49.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.816+0000 7efd95ffb640 1 -- 172.21.15.89:0/2441389448 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd980b8800 con 0x7efda0159490 2023-10-18T10:46:49.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.816+0000 7efda7656640 1 -- 172.21.15.89:0/2441389448 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda01541a0 con 0x7efda0159490 2023-10-18T10:46:49.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.816+0000 7efda7656640 1 -- 172.21.15.89:0/2441389448 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda00c4fd0 con 0x7efda0159490 2023-10-18T10:46:49.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.816+0000 7efda7656640 1 -- 172.21.15.89:0/2441389448 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda00c43d0 con 0x7efda0159490 2023-10-18T10:46:49.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.816+0000 7efd95ffb640 1 -- 172.21.15.89:0/2441389448 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7efd980aa070 con 0x7efda0159490 2023-10-18T10:46:49.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.820+0000 7efd95ffb640 1 --2- 172.21.15.89:0/2441389448 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efd84065700 0x7efd84067bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:49.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.820+0000 7efd95ffb640 1 -- 172.21.15.89:0/2441389448 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7efd9811d560 con 0x7efda0159490 2023-10-18T10:46:49.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.820+0000 7efda53f2640 1 --2- 172.21.15.89:0/2441389448 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efd84065700 0x7efd84067bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:49.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.820+0000 7efda53f2640 1 --2- 172.21.15.89:0/2441389448 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efd84065700 0x7efd84067bc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efd9c011340 tx=0x7efd9c00b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:49.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.820+0000 7efd95ffb640 1 -- 172.21.15.89:0/2441389448 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7efd980eae10 con 0x7efda0159490 2023-10-18T10:46:49.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.972+0000 7efda7656640 1 -- 172.21.15.89:0/2441389448 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7efda00c43d0 con 0x7efda0159490 2023-10-18T10:46:49.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.972+0000 7efd95ffb640 1 -- 172.21.15.89:0/2441389448 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7efda00c43d0 con 0x7efda0159490 2023-10-18T10:46:49.977 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-10-18T10:46:49.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.972+0000 7efda7656640 1 -- 172.21.15.89:0/2441389448 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efd84065700 msgr2=0x7efd84067bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:49.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.972+0000 7efda7656640 1 --2- 172.21.15.89:0/2441389448 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efd84065700 0x7efd84067bc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efd9c011340 tx=0x7efd9c00b040 comp rx=0 tx=0).stop 2023-10-18T10:46:49.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.972+0000 7efda7656640 1 -- 172.21.15.89:0/2441389448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efda0159490 msgr2=0x7efda00c4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:49.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.972+0000 7efda7656640 1 --2- 172.21.15.89:0/2441389448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efda0159490 0x7efda00c4a90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efd980ac000 tx=0x7efd9801ac00 comp rx=0 tx=0).stop 2023-10-18T10:46:49.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.972+0000 7efda7656640 1 -- 172.21.15.89:0/2441389448 shutdown_connections 2023-10-18T10:46:49.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.972+0000 7efda7656640 1 --2- 172.21.15.89:0/2441389448 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efd84065700 0x7efd84067bc0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:49.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.972+0000 7efda7656640 1 --2- 172.21.15.89:0/2441389448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efda0159490 0x7efda00c4a90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:49.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.972+0000 7efda7656640 1 --2- 172.21.15.89:0/2441389448 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efda0070600 0x7efda01518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:49.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.972+0000 7efda7656640 1 --2- 172.21.15.89:0/2441389448 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efda006bee0 0x7efda00c1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:49.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.972+0000 7efda7656640 1 -- 172.21.15.89:0/2441389448 >> 172.21.15.89:0/2441389448 conn(0x7efda0069f00 msgr2=0x7efda00cbd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:49.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.976+0000 7efda7656640 1 -- 172.21.15.89:0/2441389448 shutdown_connections 2023-10-18T10:46:49.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:49.976+0000 7efda7656640 1 -- 172.21.15.89:0/2441389448 wait complete. 2023-10-18T10:46:49.998 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:46:50.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.132+0000 7f06a200b640 1 Processor -- start 2023-10-18T10:46:50.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.132+0000 7f06a200b640 1 -- start start 2023-10-18T10:46:50.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.136+0000 7f06a200b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f069c15c940 0x7f069c0d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:50.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.136+0000 7f06a200b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f069c0d2f40 0x7f069c0d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:50.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.136+0000 7f06a200b640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f069c15da50 con 0x7f069c0d0ca0 2023-10-18T10:46:50.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.136+0000 7f06a200b640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f069c0d1130 con 0x7f069c0d2f40 2023-10-18T10:46:50.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.136+0000 7f06a200b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069c15ce40 con 0x7f069c15c940 2023-10-18T10:46:50.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.136+0000 7f069b7fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f069c15c940 0x7f069c0d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:50.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.136+0000 7f069bfff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f069c0d0ca0 0x7f069c0cf060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.89:57130/0 (socket says 172.21.15.89:57130) 2023-10-18T10:46:50.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.136+0000 7f069b7fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f069c15c940 0x7f069c0d2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:39886/0 (socket says 172.21.15.89:39886) 2023-10-18T10:46:50.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.136+0000 7f069bfff640 1 -- 172.21.15.89:0/3740616910 learned_addr learned my addr 172.21.15.89:0/3740616910 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:50.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.136+0000 7f069affd640 1 --2- 172.21.15.89:0/3740616910 >> v2:172.21.15.154:3300/0 conn(0x7f069c0d2f40 0x7f069c0d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:50.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.136+0000 7f069a7fc640 1 -- 172.21.15.89:0/3740616910 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 409394338 0 0) 0x7f069c15da50 con 0x7f069c0d0ca0 2023-10-18T10:46:50.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.136+0000 7f069b7fe640 1 -- 172.21.15.89:0/3740616910 >> v2:172.21.15.154:3300/0 conn(0x7f069c0d2f40 msgr2=0x7f069c0d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:50.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.136+0000 7f069b7fe640 1 --2- 172.21.15.89:0/3740616910 >> v2:172.21.15.154:3300/0 conn(0x7f069c0d2f40 0x7f069c0d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:50.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.136+0000 7f069b7fe640 1 -- 172.21.15.89:0/3740616910 >> v1:172.21.15.154:6789/0 conn(0x7f069c0d0ca0 legacy=0x7f069c0cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:50.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.136+0000 7f069b7fe640 1 -- 172.21.15.89:0/3740616910 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069c15d120 con 0x7f069c15c940 2023-10-18T10:46:50.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.136+0000 7f069affd640 1 --2- 172.21.15.89:0/3740616910 >> v2:172.21.15.154:3300/0 conn(0x7f069c0d2f40 0x7f069c0d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:46:50.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.140+0000 7f069b7fe640 1 --2- 172.21.15.89:0/3740616910 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f069c15c940 0x7f069c0d2a00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f068c0097c0 tx=0x7f068c01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb128f3b12ac6bba server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:50.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.140+0000 7f069a7fc640 1 -- 172.21.15.89:0/3740616910 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068c0ad660 con 0x7f069c15c940 2023-10-18T10:46:50.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.140+0000 7f069a7fc640 1 -- 172.21.15.89:0/3740616910 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f068c0ad7c0 con 0x7f069c15c940 2023-10-18T10:46:50.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.140+0000 7f069a7fc640 1 -- 172.21.15.89:0/3740616910 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068c0adb50 con 0x7f069c15c940 2023-10-18T10:46:50.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.140+0000 7f06a200b640 1 -- 172.21.15.89:0/3740616910 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f069c15c940 msgr2=0x7f069c0d2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:50.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.140+0000 7f06a200b640 1 --2- 172.21.15.89:0/3740616910 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f069c15c940 0x7f069c0d2a00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f068c0097c0 tx=0x7f068c01c450 comp rx=0 tx=0).stop 2023-10-18T10:46:50.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.140+0000 7f06a200b640 1 -- 172.21.15.89:0/3740616910 shutdown_connections 2023-10-18T10:46:50.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.140+0000 7f06a200b640 1 --2- 172.21.15.89:0/3740616910 >> v2:172.21.15.154:3300/0 conn(0x7f069c0d2f40 0x7f069c0d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:50.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.140+0000 7f06a200b640 1 --2- 172.21.15.89:0/3740616910 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f069c15c940 0x7f069c0d2a00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:50.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.140+0000 7f06a200b640 1 -- 172.21.15.89:0/3740616910 >> 172.21.15.89:0/3740616910 conn(0x7f069c089fd0 msgr2=0x7f069c09b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:50.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.140+0000 7f06a200b640 1 -- 172.21.15.89:0/3740616910 shutdown_connections 2023-10-18T10:46:50.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.140+0000 7f06a200b640 1 -- 172.21.15.89:0/3740616910 wait complete. 2023-10-18T10:46:50.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.144+0000 7f06a200b640 1 Processor -- start 2023-10-18T10:46:50.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.144+0000 7f06a200b640 1 -- start start 2023-10-18T10:46:50.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.144+0000 7f06a200b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f069c0d0ca0 0x7f069c0cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:50.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.144+0000 7f06a200b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f069c0c9720 0x7f069c1534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:50.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.144+0000 7f06a200b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f069c152b60 0x7f069c1539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:50.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.144+0000 7f06a200b640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c15a690 con 0x7f069c152b60 2023-10-18T10:46:50.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.144+0000 7f06a200b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c15a970 con 0x7f069c0d0ca0 2023-10-18T10:46:50.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.144+0000 7f06a200b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069c15ac50 con 0x7f069c0c9720 2023-10-18T10:46:50.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.144+0000 7f069bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f069c152b60 0x7f069c1539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:50.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.144+0000 7f069bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f069c152b60 0x7f069c1539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56306/0 (socket says 172.21.15.89:56306) 2023-10-18T10:46:50.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.144+0000 7f069bfff640 1 -- 172.21.15.89:0/332129350 learned_addr learned my addr 172.21.15.89:0/332129350 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:50.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.144+0000 7f069affd640 1 --2- 172.21.15.89:0/332129350 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f069c0c9720 0x7f069c1534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:50.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.144+0000 7f069b7fe640 1 --2- 172.21.15.89:0/332129350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f069c0d0ca0 0x7f069c0cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:50.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.144+0000 7f069bfff640 1 -- 172.21.15.89:0/332129350 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f069c0c9720 msgr2=0x7f069c1534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:50.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.144+0000 7f069bfff640 1 --2- 172.21.15.89:0/332129350 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f069c0c9720 0x7f069c1534b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:50.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.144+0000 7f069bfff640 1 -- 172.21.15.89:0/332129350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f069c0d0ca0 msgr2=0x7f069c0cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:50.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.144+0000 7f069bfff640 1 --2- 172.21.15.89:0/332129350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f069c0d0ca0 0x7f069c0cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:50.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.144+0000 7f069bfff640 1 -- 172.21.15.89:0/332129350 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0690009420 con 0x7f069c152b60 2023-10-18T10:46:50.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.144+0000 7f069bfff640 1 --2- 172.21.15.89:0/332129350 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f069c152b60 0x7f069c1539f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f069400c7a0 tx=0x7f069400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:50.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.144+0000 7f0683fff640 1 -- 172.21.15.89:0/332129350 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069400a640 con 0x7f069c152b60 2023-10-18T10:46:50.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.144+0000 7f0683fff640 1 -- 172.21.15.89:0/332129350 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069400a7a0 con 0x7f069c152b60 2023-10-18T10:46:50.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.144+0000 7f0683fff640 1 -- 172.21.15.89:0/332129350 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069400abd0 con 0x7f069c152b60 2023-10-18T10:46:50.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.144+0000 7f06a200b640 1 -- 172.21.15.89:0/332129350 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f068c009420 con 0x7f069c152b60 2023-10-18T10:46:50.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.144+0000 7f06a200b640 1 -- 172.21.15.89:0/332129350 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f069c0b6560 con 0x7f069c152b60 2023-10-18T10:46:50.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.148+0000 7f06a200b640 1 -- 172.21.15.89:0/332129350 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f069c04dbe0 con 0x7f069c152b60 2023-10-18T10:46:50.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.148+0000 7f0683fff640 1 -- 172.21.15.89:0/332129350 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f069400ad30 con 0x7f069c152b60 2023-10-18T10:46:50.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.148+0000 7f0683fff640 1 --2- 172.21.15.89:0/332129350 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0688065680 0x7f0688067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:50.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.148+0000 7f0683fff640 1 -- 172.21.15.89:0/332129350 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0694082830 con 0x7f069c152b60 2023-10-18T10:46:50.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.148+0000 7f069b7fe640 1 --2- 172.21.15.89:0/332129350 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0688065680 0x7f0688067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:50.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.148+0000 7f069b7fe640 1 --2- 172.21.15.89:0/332129350 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0688065680 0x7f0688067b40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f068c0af000 tx=0x7f068c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:50.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.152+0000 7f0683fff640 1 -- 172.21.15.89:0/332129350 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f069404fe90 con 0x7f069c152b60 2023-10-18T10:46:50.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.300+0000 7f06a200b640 1 -- 172.21.15.89:0/332129350 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f069c15a7d0 con 0x7f069c152b60 2023-10-18T10:46:50.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.300+0000 7f0683fff640 1 -- 172.21.15.89:0/332129350 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f069c15a7d0 con 0x7f069c152b60 2023-10-18T10:46:50.306 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-18T10:46:50.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.304+0000 7f06a200b640 1 -- 172.21.15.89:0/332129350 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0688065680 msgr2=0x7f0688067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:50.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.304+0000 7f06a200b640 1 --2- 172.21.15.89:0/332129350 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0688065680 0x7f0688067b40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f068c0af000 tx=0x7f068c008040 comp rx=0 tx=0).stop 2023-10-18T10:46:50.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.304+0000 7f06a200b640 1 -- 172.21.15.89:0/332129350 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f069c152b60 msgr2=0x7f069c1539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:50.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.304+0000 7f06a200b640 1 --2- 172.21.15.89:0/332129350 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f069c152b60 0x7f069c1539f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f069400c7a0 tx=0x7f069400cc60 comp rx=0 tx=0).stop 2023-10-18T10:46:50.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.304+0000 7f06a200b640 1 -- 172.21.15.89:0/332129350 shutdown_connections 2023-10-18T10:46:50.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.304+0000 7f06a200b640 1 --2- 172.21.15.89:0/332129350 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0688065680 0x7f0688067b40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:50.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.304+0000 7f06a200b640 1 --2- 172.21.15.89:0/332129350 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f069c0c9720 0x7f069c1534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:50.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.304+0000 7f06a200b640 1 --2- 172.21.15.89:0/332129350 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f069c152b60 0x7f069c1539f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:50.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.304+0000 7f06a200b640 1 --2- 172.21.15.89:0/332129350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f069c0d0ca0 0x7f069c0cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:50.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.304+0000 7f06a200b640 1 -- 172.21.15.89:0/332129350 >> 172.21.15.89:0/332129350 conn(0x7f069c089fd0 msgr2=0x7f069c0b4140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:50.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.304+0000 7f06a200b640 1 -- 172.21.15.89:0/332129350 shutdown_connections 2023-10-18T10:46:50.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.304+0000 7f06a200b640 1 -- 172.21.15.89:0/332129350 wait complete. 2023-10-18T10:46:50.327 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:46:50.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.464+0000 7fd392fba640 1 Processor -- start 2023-10-18T10:46:50.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.464+0000 7fd392fba640 1 -- start start 2023-10-18T10:46:50.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.464+0000 7fd392fba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd38c159490 0x7fd38c15d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:50.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.464+0000 7fd392fba640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd38c15ddf0 con 0x7fd38c06bee0 2023-10-18T10:46:50.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.464+0000 7fd392fba640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd38c070b90 con 0x7fd38c070600 2023-10-18T10:46:50.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.464+0000 7fd392fba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c070e70 con 0x7fd38c159490 2023-10-18T10:46:50.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.464+0000 7fd391557640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd38c06bee0 0x7fd38c0c8960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48734/0 (socket says 172.21.15.89:48734) 2023-10-18T10:46:50.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.464+0000 7fd391557640 1 -- 172.21.15.89:0/1395518656 learned_addr learned my addr 172.21.15.89:0/1395518656 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:50.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.464+0000 7fd390d56640 1 --2- 172.21.15.89:0/1395518656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd38c159490 0x7fd38c15d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:50.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.464+0000 7fd38b7fe640 1 -- 172.21.15.89:0/1395518656 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 501680466 0 0) 0x7fd38c15ddf0 con 0x7fd38c06bee0 2023-10-18T10:46:50.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.464+0000 7fd38b7fe640 1 -- 172.21.15.89:0/1395518656 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd374003530 con 0x7fd38c06bee0 2023-10-18T10:46:50.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.464+0000 7fd38b7fe640 1 -- 172.21.15.89:0/1395518656 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2863854124 0 0) 0x7fd38c070b90 con 0x7fd38c070600 2023-10-18T10:46:50.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.464+0000 7fd38b7fe640 1 -- 172.21.15.89:0/1395518656 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd38c15ddf0 con 0x7fd38c070600 2023-10-18T10:46:50.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.464+0000 7fd390d56640 1 -- 172.21.15.89:0/1395518656 >> v1:172.21.15.154:6789/0 conn(0x7fd38c070600 legacy=0x7fd38c06b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:50.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.464+0000 7fd390d56640 1 -- 172.21.15.89:0/1395518656 >> v1:172.21.15.89:6789/0 conn(0x7fd38c06bee0 legacy=0x7fd38c0c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:50.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.464+0000 7fd38b7fe640 1 -- 172.21.15.89:0/1395518656 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3086317080 0 0) 0x7fd374003530 con 0x7fd38c06bee0 2023-10-18T10:46:50.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.464+0000 7fd390d56640 1 -- 172.21.15.89:0/1395518656 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38c071150 con 0x7fd38c159490 2023-10-18T10:46:50.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.468+0000 7fd390d56640 1 --2- 172.21.15.89:0/1395518656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd38c159490 0x7fd38c15d8b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd37c009dc0 tx=0x7fd37c01c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1c1341cdf86fcf8 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:50.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.468+0000 7fd38b7fe640 1 -- 172.21.15.89:0/1395518656 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd384001680 con 0x7fd38c159490 2023-10-18T10:46:50.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.468+0000 7fd38b7fe640 1 -- 172.21.15.89:0/1395518656 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37c0a5610 con 0x7fd38c159490 2023-10-18T10:46:50.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.468+0000 7fd38b7fe640 1 -- 172.21.15.89:0/1395518656 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0a59a0 con 0x7fd38c159490 2023-10-18T10:46:50.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.468+0000 7fd392fba640 1 -- 172.21.15.89:0/1395518656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd38c159490 msgr2=0x7fd38c15d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:50.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.468+0000 7fd392fba640 1 --2- 172.21.15.89:0/1395518656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd38c159490 0x7fd38c15d8b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd37c009dc0 tx=0x7fd37c01c7d0 comp rx=0 tx=0).stop 2023-10-18T10:46:50.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.468+0000 7fd392fba640 1 -- 172.21.15.89:0/1395518656 shutdown_connections 2023-10-18T10:46:50.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.468+0000 7fd392fba640 1 --2- 172.21.15.89:0/1395518656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd38c159490 0x7fd38c15d8b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:50.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.468+0000 7fd392fba640 1 -- 172.21.15.89:0/1395518656 >> 172.21.15.89:0/1395518656 conn(0x7fd38c069f00 msgr2=0x7fd38c0cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:50.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.472+0000 7fd392fba640 1 -- 172.21.15.89:0/1395518656 shutdown_connections 2023-10-18T10:46:50.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.472+0000 7fd392fba640 1 -- 172.21.15.89:0/1395518656 wait complete. 2023-10-18T10:46:50.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.472+0000 7fd392fba640 1 Processor -- start 2023-10-18T10:46:50.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.472+0000 7fd392fba640 1 -- start start 2023-10-18T10:46:50.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.472+0000 7fd392fba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd38c06bee0 0x7fd38c0b0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:50.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.472+0000 7fd392fba640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd38c070600 0x7fd38c0c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:50.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.472+0000 7fd392fba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd38c159490 0x7fd38c0c5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:50.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.472+0000 7fd392fba640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c0b38f0 con 0x7fd38c070600 2023-10-18T10:46:50.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.472+0000 7fd392fba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c0b0a40 con 0x7fd38c06bee0 2023-10-18T10:46:50.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.472+0000 7fd392fba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c0b2ce0 con 0x7fd38c159490 2023-10-18T10:46:50.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.472+0000 7fd38bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd38c070600 0x7fd38c0c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:50.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.472+0000 7fd38bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd38c070600 0x7fd38c0c53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56310/0 (socket says 172.21.15.89:56310) 2023-10-18T10:46:50.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.472+0000 7fd38bfff640 1 -- 172.21.15.89:0/3021128281 learned_addr learned my addr 172.21.15.89:0/3021128281 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:50.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.472+0000 7fd390d56640 1 --2- 172.21.15.89:0/3021128281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd38c06bee0 0x7fd38c0b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:50.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.472+0000 7fd391557640 1 --2- 172.21.15.89:0/3021128281 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd38c159490 0x7fd38c0c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:50.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.472+0000 7fd38bfff640 1 -- 172.21.15.89:0/3021128281 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd38c159490 msgr2=0x7fd38c0c5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:50.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.472+0000 7fd38bfff640 1 --2- 172.21.15.89:0/3021128281 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd38c159490 0x7fd38c0c5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:50.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.472+0000 7fd38bfff640 1 -- 172.21.15.89:0/3021128281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd38c06bee0 msgr2=0x7fd38c0b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:50.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.472+0000 7fd38bfff640 1 --2- 172.21.15.89:0/3021128281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd38c06bee0 0x7fd38c0b0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:50.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.472+0000 7fd38bfff640 1 -- 172.21.15.89:0/3021128281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37c009a70 con 0x7fd38c070600 2023-10-18T10:46:50.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.476+0000 7fd38bfff640 1 --2- 172.21.15.89:0/3021128281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd38c070600 0x7fd38c0c53e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd38000c490 tx=0x7fd38000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:50.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.476+0000 7fd3897fa640 1 -- 172.21.15.89:0/3021128281 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3800052d0 con 0x7fd38c070600 2023-10-18T10:46:50.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.476+0000 7fd3897fa640 1 -- 172.21.15.89:0/3021128281 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd380005430 con 0x7fd38c070600 2023-10-18T10:46:50.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.476+0000 7fd392fba640 1 -- 172.21.15.89:0/3021128281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd38c0b2fc0 con 0x7fd38c070600 2023-10-18T10:46:50.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.476+0000 7fd3897fa640 1 -- 172.21.15.89:0/3021128281 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd380012870 con 0x7fd38c070600 2023-10-18T10:46:50.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.476+0000 7fd392fba640 1 -- 172.21.15.89:0/3021128281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd38c0b3580 con 0x7fd38c070600 2023-10-18T10:46:50.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.476+0000 7fd392fba640 1 -- 172.21.15.89:0/3021128281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd38c0b3aa0 con 0x7fd38c070600 2023-10-18T10:46:50.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.476+0000 7fd3897fa640 1 -- 172.21.15.89:0/3021128281 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fd3800129d0 con 0x7fd38c070600 2023-10-18T10:46:50.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.476+0000 7fd3897fa640 1 --2- 172.21.15.89:0/3021128281 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd3780656b0 0x7fd378067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:50.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.476+0000 7fd3897fa640 1 -- 172.21.15.89:0/3021128281 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd380082b60 con 0x7fd38c070600 2023-10-18T10:46:50.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.476+0000 7fd390d56640 1 --2- 172.21.15.89:0/3021128281 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd3780656b0 0x7fd378067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:50.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.480+0000 7fd390d56640 1 --2- 172.21.15.89:0/3021128281 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd3780656b0 0x7fd378067b70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0af000 tx=0x7fd37c0a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:50.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.480+0000 7fd3897fa640 1 -- 172.21.15.89:0/3021128281 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fd3800503a0 con 0x7fd38c070600 2023-10-18T10:46:50.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.628+0000 7fd392fba640 1 -- 172.21.15.89:0/3021128281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd38c04dbe0 con 0x7fd38c070600 2023-10-18T10:46:50.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.632+0000 7fd3897fa640 1 -- 172.21.15.89:0/3021128281 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd3800539c0 con 0x7fd38c070600 2023-10-18T10:46:50.637 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-18T10:46:50.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.632+0000 7fd392fba640 1 -- 172.21.15.89:0/3021128281 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd3780656b0 msgr2=0x7fd378067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:50.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.632+0000 7fd392fba640 1 --2- 172.21.15.89:0/3021128281 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd3780656b0 0x7fd378067b70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0af000 tx=0x7fd37c0a7070 comp rx=0 tx=0).stop 2023-10-18T10:46:50.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.632+0000 7fd392fba640 1 -- 172.21.15.89:0/3021128281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd38c070600 msgr2=0x7fd38c0c53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:50.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.632+0000 7fd392fba640 1 --2- 172.21.15.89:0/3021128281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd38c070600 0x7fd38c0c53e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd38000c490 tx=0x7fd38000c950 comp rx=0 tx=0).stop 2023-10-18T10:46:50.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.632+0000 7fd392fba640 1 -- 172.21.15.89:0/3021128281 shutdown_connections 2023-10-18T10:46:50.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.632+0000 7fd392fba640 1 --2- 172.21.15.89:0/3021128281 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd3780656b0 0x7fd378067b70 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:50.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.632+0000 7fd392fba640 1 --2- 172.21.15.89:0/3021128281 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd38c159490 0x7fd38c0c5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:50.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.632+0000 7fd392fba640 1 --2- 172.21.15.89:0/3021128281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd38c070600 0x7fd38c0c53e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:50.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.632+0000 7fd392fba640 1 --2- 172.21.15.89:0/3021128281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd38c06bee0 0x7fd38c0b0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:50.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.632+0000 7fd392fba640 1 -- 172.21.15.89:0/3021128281 >> 172.21.15.89:0/3021128281 conn(0x7fd38c069f00 msgr2=0x7fd38c0c8940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:50.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.632+0000 7fd392fba640 1 -- 172.21.15.89:0/3021128281 shutdown_connections 2023-10-18T10:46:50.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.632+0000 7fd392fba640 1 -- 172.21.15.89:0/3021128281 wait complete. 2023-10-18T10:46:50.656 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T10:46:50.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.792+0000 7f4edfb67640 1 Processor -- start 2023-10-18T10:46:50.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.792+0000 7f4edfb67640 1 -- start start 2023-10-18T10:46:50.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.792+0000 7f4edfb67640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ed80cfc50 0x7f4ed80ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:50.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.792+0000 7f4edfb67640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4ed8143820 0x7f4ed80cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:50.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.792+0000 7f4edfb67640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ed815e0d0 con 0x7f4ed8143d70 2023-10-18T10:46:50.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.792+0000 7f4edfb67640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ed815e320 con 0x7f4ed8143820 2023-10-18T10:46:50.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.792+0000 7f4edfb67640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed815e600 con 0x7f4ed80cfc50 2023-10-18T10:46:50.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.792+0000 7f4edd102640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4ed8143820 0x7f4ed80cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:50.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.796+0000 7f4edd102640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4ed8143820 0x7f4ed80cd250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56326/0 (socket says 172.21.15.89:56326) 2023-10-18T10:46:50.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.796+0000 7f4edd102640 1 -- 172.21.15.89:0/1208147560 learned_addr learned my addr 172.21.15.89:0/1208147560 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:50.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.796+0000 7f4edd903640 1 --2- 172.21.15.89:0/1208147560 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ed80cfc50 0x7f4ed80ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:50.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.796+0000 7f4edd102640 1 -- 172.21.15.89:0/1208147560 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ed80cfc50 msgr2=0x7f4ed80ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:50.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.796+0000 7f4edd102640 1 --2- 172.21.15.89:0/1208147560 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ed80cfc50 0x7f4ed80ccd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:50.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.796+0000 7f4edd102640 1 -- 172.21.15.89:0/1208147560 >> v1:172.21.15.154:6789/0 conn(0x7f4ed8143d70 legacy=0x7f4ed80b1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:50.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.796+0000 7f4edd102640 1 -- 172.21.15.89:0/1208147560 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed815e8e0 con 0x7f4ed8143820 2023-10-18T10:46:50.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.796+0000 7f4edc901640 1 -- 172.21.15.89:0/1208147560 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1289222721 0 0) 0x7f4ed815e0d0 con 0x7f4ed8143d70 2023-10-18T10:46:50.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.796+0000 7f4edd102640 1 --2- 172.21.15.89:0/1208147560 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ed8143820 0x7f4ed80cd250 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8009870 tx=0x7f4ec801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc05601399bbd66e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:50.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.796+0000 7f4edc901640 1 -- 172.21.15.89:0/1208147560 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec80aa5f0 con 0x7f4ed8143820 2023-10-18T10:46:50.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.796+0000 7f4edc901640 1 -- 172.21.15.89:0/1208147560 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec80aa750 con 0x7f4ed8143820 2023-10-18T10:46:50.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.796+0000 7f4edc901640 1 -- 172.21.15.89:0/1208147560 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec80aaae0 con 0x7f4ed8143820 2023-10-18T10:46:50.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.800+0000 7f4edfb67640 1 -- 172.21.15.89:0/1208147560 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ed8143820 msgr2=0x7f4ed80cd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:50.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.800+0000 7f4edfb67640 1 --2- 172.21.15.89:0/1208147560 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ed8143820 0x7f4ed80cd250 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8009870 tx=0x7f4ec801c4b0 comp rx=0 tx=0).stop 2023-10-18T10:46:50.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.800+0000 7f4edfb67640 1 -- 172.21.15.89:0/1208147560 shutdown_connections 2023-10-18T10:46:50.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.800+0000 7f4edfb67640 1 --2- 172.21.15.89:0/1208147560 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ed8143820 0x7f4ed80cd250 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:50.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.800+0000 7f4edfb67640 1 --2- 172.21.15.89:0/1208147560 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ed80cfc50 0x7f4ed80ccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:50.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.800+0000 7f4edfb67640 1 -- 172.21.15.89:0/1208147560 >> 172.21.15.89:0/1208147560 conn(0x7f4ed806dad0 msgr2=0x7f4ed806dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:50.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.800+0000 7f4edfb67640 1 -- 172.21.15.89:0/1208147560 shutdown_connections 2023-10-18T10:46:50.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.800+0000 7f4edfb67640 1 -- 172.21.15.89:0/1208147560 wait complete. 2023-10-18T10:46:50.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.800+0000 7f4edfb67640 1 Processor -- start 2023-10-18T10:46:50.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.800+0000 7f4edfb67640 1 -- start start 2023-10-18T10:46:50.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.800+0000 7f4edfb67640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ed80cfc50 0x7f4ed80b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:50.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.800+0000 7f4edfb67640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ed8143820 0x7f4ed8149830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:50.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.800+0000 7f4edfb67640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ed8143d70 0x7f4ed81544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:50.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.800+0000 7f4edfb67640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed8148790 con 0x7f4ed80cfc50 2023-10-18T10:46:50.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.800+0000 7f4edfb67640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed8148a70 con 0x7f4ed8143d70 2023-10-18T10:46:50.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.800+0000 7f4edfb67640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed8148d50 con 0x7f4ed8143820 2023-10-18T10:46:50.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.800+0000 7f4edd903640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ed80cfc50 0x7f4ed80b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:50.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.800+0000 7f4edd903640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ed80cfc50 0x7f4ed80b4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56330/0 (socket says 172.21.15.89:56330) 2023-10-18T10:46:50.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.800+0000 7f4edd903640 1 -- 172.21.15.89:0/3767692915 learned_addr learned my addr 172.21.15.89:0/3767692915 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:50.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.800+0000 7f4ede104640 1 --2- 172.21.15.89:0/3767692915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ed8143d70 0x7f4ed81544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:50.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.800+0000 7f4edd102640 1 --2- 172.21.15.89:0/3767692915 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ed8143820 0x7f4ed8149830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:50.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.800+0000 7f4edd903640 1 -- 172.21.15.89:0/3767692915 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ed8143820 msgr2=0x7f4ed8149830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:50.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.800+0000 7f4edd903640 1 --2- 172.21.15.89:0/3767692915 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ed8143820 0x7f4ed8149830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:50.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.800+0000 7f4edd903640 1 -- 172.21.15.89:0/3767692915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ed8143d70 msgr2=0x7f4ed81544c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:50.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.800+0000 7f4edd903640 1 --2- 172.21.15.89:0/3767692915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ed8143d70 0x7f4ed81544c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:50.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.800+0000 7f4edd903640 1 -- 172.21.15.89:0/3767692915 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec80094d0 con 0x7f4ed80cfc50 2023-10-18T10:46:50.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.800+0000 7f4edd903640 1 --2- 172.21.15.89:0/3767692915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ed80cfc50 0x7f4ed80b4240 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400efc0 tx=0x7f4ed400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:50.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.804+0000 7f4ece7fc640 1 -- 172.21.15.89:0/3767692915 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4009b80 con 0x7f4ed80cfc50 2023-10-18T10:46:50.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.804+0000 7f4ece7fc640 1 -- 172.21.15.89:0/3767692915 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed400f040 con 0x7f4ed80cfc50 2023-10-18T10:46:50.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.804+0000 7f4ece7fc640 1 -- 172.21.15.89:0/3767692915 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed40146a0 con 0x7f4ed80cfc50 2023-10-18T10:46:50.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.804+0000 7f4edfb67640 1 -- 172.21.15.89:0/3767692915 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ed8149030 con 0x7f4ed80cfc50 2023-10-18T10:46:50.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.804+0000 7f4edfb67640 1 -- 172.21.15.89:0/3767692915 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed80b4780 con 0x7f4ed80cfc50 2023-10-18T10:46:50.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.804+0000 7f4edfb67640 1 -- 172.21.15.89:0/3767692915 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ed804dbe0 con 0x7f4ed80cfc50 2023-10-18T10:46:50.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.804+0000 7f4ece7fc640 1 -- 172.21.15.89:0/3767692915 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f4ed4014800 con 0x7f4ed80cfc50 2023-10-18T10:46:50.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.804+0000 7f4ece7fc640 1 --2- 172.21.15.89:0/3767692915 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4ebc0656b0 0x7f4ebc067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:50.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.804+0000 7f4ece7fc640 1 -- 172.21.15.89:0/3767692915 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4ed4083ce0 con 0x7f4ed80cfc50 2023-10-18T10:46:50.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.804+0000 7f4edd102640 1 --2- 172.21.15.89:0/3767692915 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4ebc0656b0 0x7f4ebc067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:50.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.804+0000 7f4edd102640 1 --2- 172.21.15.89:0/3767692915 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4ebc0656b0 0x7f4ebc067b70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80ae000 tx=0x7f4ec80a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:50.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.808+0000 7f4ece7fc640 1 -- 172.21.15.89:0/3767692915 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f4ed40513d0 con 0x7f4ed80cfc50 2023-10-18T10:46:50.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.956+0000 7f4edfb67640 1 -- 172.21.15.89:0/3767692915 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4ed806e5f0 con 0x7f4ed80cfc50 2023-10-18T10:46:50.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.956+0000 7f4ece7fc640 1 -- 172.21.15.89:0/3767692915 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4ed40549f0 con 0x7f4ed80cfc50 2023-10-18T10:46:50.963 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-18T10:46:50.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.960+0000 7f4edfb67640 1 -- 172.21.15.89:0/3767692915 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4ebc0656b0 msgr2=0x7f4ebc067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:50.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.960+0000 7f4edfb67640 1 --2- 172.21.15.89:0/3767692915 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4ebc0656b0 0x7f4ebc067b70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80ae000 tx=0x7f4ec80a7000 comp rx=0 tx=0).stop 2023-10-18T10:46:50.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.960+0000 7f4edfb67640 1 -- 172.21.15.89:0/3767692915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ed80cfc50 msgr2=0x7f4ed80b4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:50.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.960+0000 7f4edfb67640 1 --2- 172.21.15.89:0/3767692915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ed80cfc50 0x7f4ed80b4240 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400efc0 tx=0x7f4ed400c490 comp rx=0 tx=0).stop 2023-10-18T10:46:50.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.960+0000 7f4edfb67640 1 -- 172.21.15.89:0/3767692915 shutdown_connections 2023-10-18T10:46:50.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.960+0000 7f4edfb67640 1 --2- 172.21.15.89:0/3767692915 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4ebc0656b0 0x7f4ebc067b70 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:50.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.960+0000 7f4edfb67640 1 --2- 172.21.15.89:0/3767692915 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ed8143820 0x7f4ed8149830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:50.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.960+0000 7f4edfb67640 1 --2- 172.21.15.89:0/3767692915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ed8143d70 0x7f4ed81544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:50.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.960+0000 7f4edfb67640 1 --2- 172.21.15.89:0/3767692915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ed80cfc50 0x7f4ed80b4240 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:50.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.960+0000 7f4edfb67640 1 -- 172.21.15.89:0/3767692915 >> 172.21.15.89:0/3767692915 conn(0x7f4ed806dad0 msgr2=0x7f4ed815ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:50.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.960+0000 7f4edfb67640 1 -- 172.21.15.89:0/3767692915 shutdown_connections 2023-10-18T10:46:50.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:50.960+0000 7f4edfb67640 1 -- 172.21.15.89:0/3767692915 wait complete. 2023-10-18T10:46:50.983 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T10:46:51.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.116+0000 7fbf044ec640 1 Processor -- start 2023-10-18T10:46:51.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.116+0000 7fbf044ec640 1 -- start start 2023-10-18T10:46:51.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.120+0000 7fbf044ec640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbefc0c9070 0x7fbefc15e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:51.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.120+0000 7fbf044ec640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbefc15acb0 con 0x7fbefc06ff00 2023-10-18T10:46:51.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.120+0000 7fbf044ec640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbefc0702e0 con 0x7fbefc0ca340 2023-10-18T10:46:51.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.120+0000 7fbf044ec640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbefc15ef30 con 0x7fbefc0c9070 2023-10-18T10:46:51.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.120+0000 7fbf02288640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fbefc06ff00 0x7fbefc15b140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48746/0 (socket says 172.21.15.89:48746) 2023-10-18T10:46:51.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.120+0000 7fbf02288640 1 -- 172.21.15.89:0/2371153580 learned_addr learned my addr 172.21.15.89:0/2371153580 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:51.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.120+0000 7fbf02a89640 1 --2- 172.21.15.89:0/2371153580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbefc0c9070 0x7fbefc15e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:51.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.120+0000 7fbf01286640 1 -- 172.21.15.89:0/2371153580 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1854257570 0 0) 0x7fbefc15acb0 con 0x7fbefc06ff00 2023-10-18T10:46:51.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.120+0000 7fbf01286640 1 -- 172.21.15.89:0/2371153580 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbee8003530 con 0x7fbefc06ff00 2023-10-18T10:46:51.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.120+0000 7fbf01286640 1 -- 172.21.15.89:0/2371153580 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 304595594 0 0) 0x7fbee8003530 con 0x7fbefc06ff00 2023-10-18T10:46:51.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.120+0000 7fbf01286640 1 -- 172.21.15.89:0/2371153580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbefc0c9070 msgr2=0x7fbefc15e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:51.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.120+0000 7fbf01286640 1 --2- 172.21.15.89:0/2371153580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbefc0c9070 0x7fbefc15e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:51.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.120+0000 7fbf01286640 1 -- 172.21.15.89:0/2371153580 >> v1:172.21.15.154:6789/0 conn(0x7fbefc0ca340 legacy=0x7fbefc0c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:51.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.120+0000 7fbf01286640 1 -- 172.21.15.89:0/2371153580 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbefc15b850 con 0x7fbefc06ff00 2023-10-18T10:46:51.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.120+0000 7fbf01286640 1 -- 172.21.15.89:0/2371153580 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7fbef8001680 con 0x7fbefc06ff00 2023-10-18T10:46:51.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.120+0000 7fbf01286640 1 -- 172.21.15.89:0/2371153580 >> v1:172.21.15.89:6789/0 conn(0x7fbefc06ff00 legacy=0x7fbefc15b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:51.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.120+0000 7fbf01286640 1 --2- 172.21.15.89:0/2371153580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee80039b0 0x7fbee8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:51.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.120+0000 7fbf01286640 1 --2- 172.21.15.89:0/2371153580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbee8004360 0x7fbee8004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:51.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.120+0000 7fbf01286640 1 --2- 172.21.15.89:0/2371153580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbee80050d0 0x7fbee800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:51.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.120+0000 7fbf01286640 1 -- 172.21.15.89:0/2371153580 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbefc15ef30 con 0x7fbee80039b0 2023-10-18T10:46:51.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.120+0000 7fbf01286640 1 -- 172.21.15.89:0/2371153580 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee800bed0 con 0x7fbee8004360 2023-10-18T10:46:51.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.120+0000 7fbf01286640 1 -- 172.21.15.89:0/2371153580 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee800c010 con 0x7fbee80050d0 2023-10-18T10:46:51.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.120+0000 7fbf02288640 1 --2- 172.21.15.89:0/2371153580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee80039b0 0x7fbee8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:51.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.120+0000 7fbf01a87640 1 --2- 172.21.15.89:0/2371153580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbee8004360 0x7fbee8004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:51.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.120+0000 7fbf02a89640 1 --2- 172.21.15.89:0/2371153580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbee80050d0 0x7fbee800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:51.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.120+0000 7fbf02288640 1 -- 172.21.15.89:0/2371153580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbee80050d0 msgr2=0x7fbee800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:51.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.120+0000 7fbf02288640 1 --2- 172.21.15.89:0/2371153580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbee80050d0 0x7fbee800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:51.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.120+0000 7fbf02288640 1 -- 172.21.15.89:0/2371153580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbee8004360 msgr2=0x7fbee8004830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:51.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.120+0000 7fbf02288640 1 --2- 172.21.15.89:0/2371153580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbee8004360 0x7fbee8004830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:51.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.120+0000 7fbf02288640 1 -- 172.21.15.89:0/2371153580 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee8003530 con 0x7fbee80039b0 2023-10-18T10:46:51.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.124+0000 7fbf02288640 1 --2- 172.21.15.89:0/2371153580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee80039b0 0x7fbee8003d90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fbef800c190 tx=0x7fbef801cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df44c71fbbca64c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:51.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.124+0000 7fbf01286640 1 -- 172.21.15.89:0/2371153580 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef8005ae0 con 0x7fbee80039b0 2023-10-18T10:46:51.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.124+0000 7fbf01286640 1 -- 172.21.15.89:0/2371153580 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef8005c60 con 0x7fbee80039b0 2023-10-18T10:46:51.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.124+0000 7fbf01286640 1 -- 172.21.15.89:0/2371153580 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef8007070 con 0x7fbee80039b0 2023-10-18T10:46:51.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.124+0000 7fbf044ec640 1 -- 172.21.15.89:0/2371153580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee80039b0 msgr2=0x7fbee8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:51.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.124+0000 7fbf044ec640 1 --2- 172.21.15.89:0/2371153580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee80039b0 0x7fbee8003d90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fbef800c190 tx=0x7fbef801cbe0 comp rx=0 tx=0).stop 2023-10-18T10:46:51.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.124+0000 7fbf02a89640 1 -- 172.21.15.89:0/2371153580 reap_dead start 2023-10-18T10:46:51.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.124+0000 7fbf044ec640 1 -- 172.21.15.89:0/2371153580 shutdown_connections 2023-10-18T10:46:51.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.124+0000 7fbf044ec640 1 -- 172.21.15.89:0/2371153580 >> 172.21.15.89:0/2371153580 conn(0x7fbefc069f00 msgr2=0x7fbefc0cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:51.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.124+0000 7fbf044ec640 1 -- 172.21.15.89:0/2371153580 shutdown_connections 2023-10-18T10:46:51.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.124+0000 7fbf044ec640 1 -- 172.21.15.89:0/2371153580 wait complete. 2023-10-18T10:46:51.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.124+0000 7fbf044ec640 1 Processor -- start 2023-10-18T10:46:51.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.124+0000 7fbf044ec640 1 -- start start 2023-10-18T10:46:51.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.128+0000 7fbf044ec640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee80039b0 0x7fbefc13fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:51.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.128+0000 7fbf044ec640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbefc13ffa0 0x7fbefc145130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:51.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.128+0000 7fbf044ec640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbefc145670 0x7fbefc141890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:51.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.128+0000 7fbf044ec640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbefc1412e0 con 0x7fbee80039b0 2023-10-18T10:46:51.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.128+0000 7fbf044ec640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbefc140400 con 0x7fbefc145670 2023-10-18T10:46:51.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.128+0000 7fbf044ec640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbefc1406d0 con 0x7fbefc13ffa0 2023-10-18T10:46:51.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.128+0000 7fbf02288640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee80039b0 0x7fbefc13fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:51.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.128+0000 7fbf02288640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee80039b0 0x7fbefc13fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56352/0 (socket says 172.21.15.89:56352) 2023-10-18T10:46:51.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.128+0000 7fbf02288640 1 -- 172.21.15.89:0/501981341 learned_addr learned my addr 172.21.15.89:0/501981341 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:51.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.128+0000 7fbf02a89640 1 --2- 172.21.15.89:0/501981341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbefc145670 0x7fbefc141890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:51.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.128+0000 7fbf01a87640 1 --2- 172.21.15.89:0/501981341 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbefc13ffa0 0x7fbefc145130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:51.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.128+0000 7fbf02288640 1 -- 172.21.15.89:0/501981341 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbefc13ffa0 msgr2=0x7fbefc145130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:51.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.128+0000 7fbf02288640 1 --2- 172.21.15.89:0/501981341 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbefc13ffa0 0x7fbefc145130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:51.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.128+0000 7fbf02288640 1 -- 172.21.15.89:0/501981341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbefc145670 msgr2=0x7fbefc141890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:51.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.128+0000 7fbf02288640 1 --2- 172.21.15.89:0/501981341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbefc145670 0x7fbefc141890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:51.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.128+0000 7fbf02288640 1 -- 172.21.15.89:0/501981341 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbefc1409b0 con 0x7fbee80039b0 2023-10-18T10:46:51.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.128+0000 7fbf02288640 1 --2- 172.21.15.89:0/501981341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee80039b0 0x7fbefc13fa60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbef800c190 tx=0x7fbee8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:51.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.128+0000 7fbef2ffd640 1 -- 172.21.15.89:0/501981341 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef80b4070 con 0x7fbee80039b0 2023-10-18T10:46:51.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.128+0000 7fbef2ffd640 1 -- 172.21.15.89:0/501981341 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef8005dd0 con 0x7fbee80039b0 2023-10-18T10:46:51.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.128+0000 7fbf044ec640 1 -- 172.21.15.89:0/501981341 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbefc140c90 con 0x7fbee80039b0 2023-10-18T10:46:51.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.128+0000 7fbef2ffd640 1 -- 172.21.15.89:0/501981341 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef8001640 con 0x7fbee80039b0 2023-10-18T10:46:51.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.128+0000 7fbf044ec640 1 -- 172.21.15.89:0/501981341 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbefc15c900 con 0x7fbee80039b0 2023-10-18T10:46:51.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.128+0000 7fbf044ec640 1 -- 172.21.15.89:0/501981341 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbefc04dbe0 con 0x7fbee80039b0 2023-10-18T10:46:51.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.128+0000 7fbef2ffd640 1 -- 172.21.15.89:0/501981341 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fbef8007040 con 0x7fbee80039b0 2023-10-18T10:46:51.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.132+0000 7fbef2ffd640 1 --2- 172.21.15.89:0/501981341 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fbee0065680 0x7fbee0067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:51.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.132+0000 7fbef2ffd640 1 -- 172.21.15.89:0/501981341 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fbef811c530 con 0x7fbee80039b0 2023-10-18T10:46:51.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.132+0000 7fbf01a87640 1 --2- 172.21.15.89:0/501981341 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fbee0065680 0x7fbee0067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:51.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.132+0000 7fbf01a87640 1 --2- 172.21.15.89:0/501981341 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fbee0065680 0x7fbee0067b40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbefc1f5ec0 tx=0x7fbeec009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:51.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.136+0000 7fbef2ffd640 1 -- 172.21.15.89:0/501981341 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fbef80e9c30 con 0x7fbee80039b0 2023-10-18T10:46:51.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.296+0000 7fbf044ec640 1 -- 172.21.15.89:0/501981341 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbefc070920 con 0x7fbee80039b0 2023-10-18T10:46:51.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.300+0000 7fbef2ffd640 1 -- 172.21.15.89:0/501981341 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbef80ed250 con 0x7fbee80039b0 2023-10-18T10:46:51.303 INFO:teuthology.orchestra.run.smithi089.stderr:listed 0 entries 2023-10-18T10:46:51.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.300+0000 7fbf044ec640 1 -- 172.21.15.89:0/501981341 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fbee0065680 msgr2=0x7fbee0067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:51.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.300+0000 7fbf044ec640 1 --2- 172.21.15.89:0/501981341 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fbee0065680 0x7fbee0067b40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbefc1f5ec0 tx=0x7fbeec009040 comp rx=0 tx=0).stop 2023-10-18T10:46:51.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.300+0000 7fbf044ec640 1 -- 172.21.15.89:0/501981341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee80039b0 msgr2=0x7fbefc13fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:51.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.300+0000 7fbf044ec640 1 --2- 172.21.15.89:0/501981341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee80039b0 0x7fbefc13fa60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbef800c190 tx=0x7fbee8005800 comp rx=0 tx=0).stop 2023-10-18T10:46:51.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.300+0000 7fbf044ec640 1 -- 172.21.15.89:0/501981341 shutdown_connections 2023-10-18T10:46:51.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.300+0000 7fbf044ec640 1 --2- 172.21.15.89:0/501981341 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fbee0065680 0x7fbee0067b40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:51.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.300+0000 7fbf044ec640 1 --2- 172.21.15.89:0/501981341 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbefc13ffa0 0x7fbefc145130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:51.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.300+0000 7fbf044ec640 1 --2- 172.21.15.89:0/501981341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbefc145670 0x7fbefc141890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:51.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.300+0000 7fbf044ec640 1 --2- 172.21.15.89:0/501981341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee80039b0 0x7fbefc13fa60 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:51.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.300+0000 7fbf044ec640 1 -- 172.21.15.89:0/501981341 >> 172.21.15.89:0/501981341 conn(0x7fbefc069f00 msgr2=0x7fbefc0c0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:51.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.300+0000 7fbf044ec640 1 -- 172.21.15.89:0/501981341 shutdown_connections 2023-10-18T10:46:51.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.300+0000 7fbf044ec640 1 -- 172.21.15.89:0/501981341 wait complete. 2023-10-18T10:46:51.324 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-18T10:46:51.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:51.325 DEBUG:teuthology.orchestra.run.smithi089:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-10-18T10:46:51.600 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi089.front.sepia.ceph.com 2023-10-18T10:46:51.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:51.601 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-18T10:46:51.611 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T10:46:51.611 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:46:51.621 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T10:46:51.622 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:46:51.674 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T10:46:51.674 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:46:51.730 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:46:51.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.872+0000 7f5c52b37640 1 Processor -- start 2023-10-18T10:46:51.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.872+0000 7f5c52b37640 1 -- start start 2023-10-18T10:46:51.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.872+0000 7f5c52b37640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5c4c15c940 0x7f5c4c0d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:51.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.872+0000 7f5c52b37640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c4c0d2f40 0x7f5c4c0d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:51.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.872+0000 7f5c52b37640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c4c15da50 con 0x7f5c4c0d0ca0 2023-10-18T10:46:51.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.872+0000 7f5c52b37640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c4c0d1130 con 0x7f5c4c15c940 2023-10-18T10:46:51.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.872+0000 7f5c52b37640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c15ce40 con 0x7f5c4c0d2f40 2023-10-18T10:46:51.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.872+0000 7f5c43fff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c4c0d2f40 0x7f5c4c0d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:51.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.872+0000 7f5c43fff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c4c0d2f40 0x7f5c4c0d0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:39970/0 (socket says 172.21.15.89:39970) 2023-10-18T10:46:51.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.872+0000 7f5c43fff640 1 -- 172.21.15.89:0/1432592789 learned_addr learned my addr 172.21.15.89:0/1432592789 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:51.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.872+0000 7f5c508d3640 1 --2- 172.21.15.89:0/1432592789 >> v2:172.21.15.154:3300/0 conn(0x7f5c4c15c940 0x7f5c4c0d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:51.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.872+0000 7f5c437fe640 1 -- 172.21.15.89:0/1432592789 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4135643408 0 0) 0x7f5c4c15da50 con 0x7f5c4c0d0ca0 2023-10-18T10:46:51.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.872+0000 7f5c437fe640 1 -- 172.21.15.89:0/1432592789 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c34003530 con 0x7f5c4c0d0ca0 2023-10-18T10:46:51.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.872+0000 7f5c508d3640 1 -- 172.21.15.89:0/1432592789 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c4c0d2f40 msgr2=0x7f5c4c0d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:51.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.872+0000 7f5c508d3640 1 --2- 172.21.15.89:0/1432592789 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c4c0d2f40 0x7f5c4c0d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:51.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.872+0000 7f5c508d3640 1 -- 172.21.15.89:0/1432592789 >> v1:172.21.15.154:6789/0 conn(0x7f5c4c0d0ca0 legacy=0x7f5c4c0cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:51.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.872+0000 7f5c508d3640 1 -- 172.21.15.89:0/1432592789 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c4c15d120 con 0x7f5c4c15c940 2023-10-18T10:46:51.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.872+0000 7f5c43fff640 1 --2- 172.21.15.89:0/1432592789 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c4c0d2f40 0x7f5c4c0d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:46:51.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.876+0000 7f5c508d3640 1 --2- 172.21.15.89:0/1432592789 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c4c15c940 0x7f5c4c0d2a00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5c480097c0 tx=0x7f5c4801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=999f3f6065791c84 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:51.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.876+0000 7f5c437fe640 1 -- 172.21.15.89:0/1432592789 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c480ad660 con 0x7f5c4c15c940 2023-10-18T10:46:51.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.876+0000 7f5c437fe640 1 -- 172.21.15.89:0/1432592789 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c480ad7c0 con 0x7f5c4c15c940 2023-10-18T10:46:51.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.876+0000 7f5c437fe640 1 -- 172.21.15.89:0/1432592789 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c480adb50 con 0x7f5c4c15c940 2023-10-18T10:46:51.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.876+0000 7f5c52b37640 1 -- 172.21.15.89:0/1432592789 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c4c15c940 msgr2=0x7f5c4c0d2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:51.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.876+0000 7f5c52b37640 1 --2- 172.21.15.89:0/1432592789 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c4c15c940 0x7f5c4c0d2a00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5c480097c0 tx=0x7f5c4801c450 comp rx=0 tx=0).stop 2023-10-18T10:46:51.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.876+0000 7f5c52b37640 1 -- 172.21.15.89:0/1432592789 shutdown_connections 2023-10-18T10:46:51.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.876+0000 7f5c52b37640 1 --2- 172.21.15.89:0/1432592789 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c4c0d2f40 0x7f5c4c0d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:51.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.876+0000 7f5c52b37640 1 --2- 172.21.15.89:0/1432592789 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c4c15c940 0x7f5c4c0d2a00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:51.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.876+0000 7f5c52b37640 1 -- 172.21.15.89:0/1432592789 >> 172.21.15.89:0/1432592789 conn(0x7f5c4c089fd0 msgr2=0x7f5c4c09b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:51.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.876+0000 7f5c52b37640 1 -- 172.21.15.89:0/1432592789 shutdown_connections 2023-10-18T10:46:51.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.876+0000 7f5c52b37640 1 -- 172.21.15.89:0/1432592789 wait complete. 2023-10-18T10:46:51.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.880+0000 7f5c52b37640 1 Processor -- start 2023-10-18T10:46:51.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.880+0000 7f5c52b37640 1 -- start start 2023-10-18T10:46:51.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.880+0000 7f5c52b37640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c4c0d0ca0 0x7f5c4c069ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:51.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.880+0000 7f5c52b37640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c4c15b340 0x7f5c4c15b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:51.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.880+0000 7f5c52b37640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c4c151ca0 0x7f5c4c152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:51.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.880+0000 7f5c52b37640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c153220 con 0x7f5c4c151ca0 2023-10-18T10:46:51.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.880+0000 7f5c52b37640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c0b5580 con 0x7f5c4c0d0ca0 2023-10-18T10:46:51.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.880+0000 7f5c52b37640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c0b5270 con 0x7f5c4c15b340 2023-10-18T10:46:51.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.880+0000 7f5c510d4640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c4c151ca0 0x7f5c4c152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:51.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.880+0000 7f5c508d3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c4c0d0ca0 0x7f5c4c069ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:51.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.880+0000 7f5c43fff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c4c15b340 0x7f5c4c15b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:51.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.880+0000 7f5c508d3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c4c0d0ca0 0x7f5c4c069ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:45412/0 (socket says 172.21.15.89:45412) 2023-10-18T10:46:51.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.880+0000 7f5c508d3640 1 -- 172.21.15.89:0/2948482856 learned_addr learned my addr 172.21.15.89:0/2948482856 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:51.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.880+0000 7f5c508d3640 1 -- 172.21.15.89:0/2948482856 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c4c15b340 msgr2=0x7f5c4c15b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:51.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.880+0000 7f5c508d3640 1 --2- 172.21.15.89:0/2948482856 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c4c15b340 0x7f5c4c15b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:51.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.880+0000 7f5c508d3640 1 -- 172.21.15.89:0/2948482856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c4c151ca0 msgr2=0x7f5c4c152120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:51.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.880+0000 7f5c508d3640 1 --2- 172.21.15.89:0/2948482856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c4c151ca0 0x7f5c4c152120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:51.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.880+0000 7f5c508d3640 1 -- 172.21.15.89:0/2948482856 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c3c0094d0 con 0x7f5c4c0d0ca0 2023-10-18T10:46:51.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.880+0000 7f5c510d4640 1 --2- 172.21.15.89:0/2948482856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c4c151ca0 0x7f5c4c152120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:46:51.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.880+0000 7f5c508d3640 1 --2- 172.21.15.89:0/2948482856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c4c0d0ca0 0x7f5c4c069ff0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5c480b0040 tx=0x7f5c4801ac90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:51.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.880+0000 7f5c417fa640 1 -- 172.21.15.89:0/2948482856 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c480b9650 con 0x7f5c4c0d0ca0 2023-10-18T10:46:51.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.880+0000 7f5c417fa640 1 -- 172.21.15.89:0/2948482856 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c480b97f0 con 0x7f5c4c0d0ca0 2023-10-18T10:46:51.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.880+0000 7f5c52b37640 1 -- 172.21.15.89:0/2948482856 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c48009420 con 0x7f5c4c0d0ca0 2023-10-18T10:46:51.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.880+0000 7f5c417fa640 1 -- 172.21.15.89:0/2948482856 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c480b9ba0 con 0x7f5c4c0d0ca0 2023-10-18T10:46:51.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.880+0000 7f5c52b37640 1 -- 172.21.15.89:0/2948482856 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c4c152660 con 0x7f5c4c0d0ca0 2023-10-18T10:46:51.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.880+0000 7f5c52b37640 1 -- 172.21.15.89:0/2948482856 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c4c0b56c0 con 0x7f5c4c0d0ca0 2023-10-18T10:46:51.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.884+0000 7f5c417fa640 1 -- 172.21.15.89:0/2948482856 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f5c480b9d00 con 0x7f5c4c0d0ca0 2023-10-18T10:46:51.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.884+0000 7f5c417fa640 1 --2- 172.21.15.89:0/2948482856 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5c38065680 0x7f5c38067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:51.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.884+0000 7f5c417fa640 1 -- 172.21.15.89:0/2948482856 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5c4811dc40 con 0x7f5c4c0d0ca0 2023-10-18T10:46:51.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.884+0000 7f5c43fff640 1 --2- 172.21.15.89:0/2948482856 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5c38065680 0x7f5c38067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:51.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.884+0000 7f5c43fff640 1 --2- 172.21.15.89:0/2948482856 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5c38065680 0x7f5c38067b40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c0a0450 tx=0x7f5c3c009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:51.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:51.888+0000 7f5c417fa640 1 -- 172.21.15.89:0/2948482856 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f5c480eb5f0 con 0x7f5c4c0d0ca0 2023-10-18T10:46:52.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.040+0000 7f5c52b37640 1 -- 172.21.15.89:0/2948482856 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c4c0b56c0 con 0x7f5c4c0d0ca0 2023-10-18T10:46:52.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.040+0000 7f5c417fa640 1 -- 172.21.15.89:0/2948482856 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20192 (secure 0 0 0) 0x7f5c4c0b56c0 con 0x7f5c4c0d0ca0 2023-10-18T10:46:52.042 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:52.042 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":29,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:46:36.958604+0000","last_up_change":"2023-10-18T10:45:42.164688+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:45:53.309122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-18T10:45:54.659763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:45:57.796514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6824","nonce":773078364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6827","nonce":773078364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6831","nonce":773078364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6829","nonce":773078364}]},"public_addr":"172.21.15.154:6824/773078364","cluster_addr":"172.21.15.154:6827/773078364","heartbeat_back_addr":"172.21.15.154:6831/773078364","heartbeat_front_addr":"172.21.15.154:6829/773078364","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":1,"to":5}]},{"pgid":"4.1a","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:46:52.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.044+0000 7f5c52b37640 1 -- 172.21.15.89:0/2948482856 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5c38065680 msgr2=0x7f5c38067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:52.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.044+0000 7f5c52b37640 1 --2- 172.21.15.89:0/2948482856 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5c38065680 0x7f5c38067b40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c0a0450 tx=0x7f5c3c009240 comp rx=0 tx=0).stop 2023-10-18T10:46:52.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.044+0000 7f5c52b37640 1 -- 172.21.15.89:0/2948482856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c4c0d0ca0 msgr2=0x7f5c4c069ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:52.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.044+0000 7f5c52b37640 1 --2- 172.21.15.89:0/2948482856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c4c0d0ca0 0x7f5c4c069ff0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5c480b0040 tx=0x7f5c4801ac90 comp rx=0 tx=0).stop 2023-10-18T10:46:52.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.044+0000 7f5c52b37640 1 -- 172.21.15.89:0/2948482856 shutdown_connections 2023-10-18T10:46:52.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.044+0000 7f5c52b37640 1 --2- 172.21.15.89:0/2948482856 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5c38065680 0x7f5c38067b40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:52.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.044+0000 7f5c52b37640 1 --2- 172.21.15.89:0/2948482856 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c4c15b340 0x7f5c4c15b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:52.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.044+0000 7f5c52b37640 1 --2- 172.21.15.89:0/2948482856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c4c151ca0 0x7f5c4c152120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:52.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.044+0000 7f5c52b37640 1 --2- 172.21.15.89:0/2948482856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c4c0d0ca0 0x7f5c4c069ff0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:52.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.044+0000 7f5c52b37640 1 -- 172.21.15.89:0/2948482856 >> 172.21.15.89:0/2948482856 conn(0x7f5c4c089fd0 msgr2=0x7f5c4c15fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:52.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.044+0000 7f5c52b37640 1 -- 172.21.15.89:0/2948482856 shutdown_connections 2023-10-18T10:46:52.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.044+0000 7f5c52b37640 1 -- 172.21.15.89:0/2948482856 wait complete. 2023-10-18T10:46:52.063 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:45:43.114184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-18T10:45:48.730968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:45:53.309122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:45:54.659763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T10:45:57.796514+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T10:46:52.064 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:46:52.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.204+0000 7f1800e5f640 1 Processor -- start 2023-10-18T10:46:52.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.204+0000 7f1800e5f640 1 -- start start 2023-10-18T10:46:52.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.204+0000 7f1800e5f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f17fc159490 0x7f17fc15d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:52.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.204+0000 7f1800e5f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17fc06bee0 0x7f17fc0c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:52.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.204+0000 7f1800e5f640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17fc15ddf0 con 0x7f17fc070600 2023-10-18T10:46:52.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.204+0000 7f1800e5f640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17fc070b90 con 0x7f17fc159490 2023-10-18T10:46:52.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.204+0000 7f1800e5f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17fc070e70 con 0x7f17fc06bee0 2023-10-18T10:46:52.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.204+0000 7f17fa59c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f17fc159490 0x7f17fc15d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:52.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.204+0000 7f17f9d9b640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f17fc070600 0x7f17fc06b7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48752/0 (socket says 172.21.15.89:48752) 2023-10-18T10:46:52.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.204+0000 7f17f9d9b640 1 -- 172.21.15.89:0/1199994854 learned_addr learned my addr 172.21.15.89:0/1199994854 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:52.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.204+0000 7f17fa59c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f17fc159490 0x7f17fc15d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56372/0 (socket says 172.21.15.89:56372) 2023-10-18T10:46:52.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.204+0000 7f17fa59c640 1 -- 172.21.15.89:0/1199994854 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17fc06bee0 msgr2=0x7f17fc0c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:52.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.204+0000 7f17fad9d640 1 --2- 172.21.15.89:0/1199994854 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17fc06bee0 0x7f17fc0c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:52.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.204+0000 7f17f959a640 1 -- 172.21.15.89:0/1199994854 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2933499827 0 0) 0x7f17fc15ddf0 con 0x7f17fc070600 2023-10-18T10:46:52.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.204+0000 7f17fa59c640 1 --2- 172.21.15.89:0/1199994854 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17fc06bee0 0x7f17fc0c8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:52.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.204+0000 7f17fa59c640 1 -- 172.21.15.89:0/1199994854 >> v1:172.21.15.89:6789/0 conn(0x7f17fc070600 legacy=0x7f17fc06b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:52.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.204+0000 7f17fa59c640 1 -- 172.21.15.89:0/1199994854 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17fc071150 con 0x7f17fc159490 2023-10-18T10:46:52.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.204+0000 7f17fad9d640 1 --2- 172.21.15.89:0/1199994854 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17fc06bee0 0x7f17fc0c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:52.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.208+0000 7f17fa59c640 1 --2- 172.21.15.89:0/1199994854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17fc159490 0x7f17fc15d8b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f17ec0097c0 tx=0x7f17ec01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c879b5023ae7c48 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:52.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.208+0000 7f17f959a640 1 -- 172.21.15.89:0/1199994854 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ec0ac620 con 0x7f17fc159490 2023-10-18T10:46:52.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.208+0000 7f17f959a640 1 -- 172.21.15.89:0/1199994854 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17ec0ac780 con 0x7f17fc159490 2023-10-18T10:46:52.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.208+0000 7f17f959a640 1 -- 172.21.15.89:0/1199994854 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ec0acb10 con 0x7f17fc159490 2023-10-18T10:46:52.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.208+0000 7f1800e5f640 1 -- 172.21.15.89:0/1199994854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17fc159490 msgr2=0x7f17fc15d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:52.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.208+0000 7f1800e5f640 1 --2- 172.21.15.89:0/1199994854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17fc159490 0x7f17fc15d8b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f17ec0097c0 tx=0x7f17ec01c3c0 comp rx=0 tx=0).stop 2023-10-18T10:46:52.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.208+0000 7f1800e5f640 1 -- 172.21.15.89:0/1199994854 shutdown_connections 2023-10-18T10:46:52.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.208+0000 7f1800e5f640 1 --2- 172.21.15.89:0/1199994854 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17fc06bee0 0x7f17fc0c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:52.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.208+0000 7f1800e5f640 1 --2- 172.21.15.89:0/1199994854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17fc159490 0x7f17fc15d8b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:52.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.208+0000 7f1800e5f640 1 -- 172.21.15.89:0/1199994854 >> 172.21.15.89:0/1199994854 conn(0x7f17fc069f00 msgr2=0x7f17fc0cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:52.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.208+0000 7f1800e5f640 1 -- 172.21.15.89:0/1199994854 shutdown_connections 2023-10-18T10:46:52.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.208+0000 7f1800e5f640 1 -- 172.21.15.89:0/1199994854 wait complete. 2023-10-18T10:46:52.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.212+0000 7f1800e5f640 1 Processor -- start 2023-10-18T10:46:52.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.212+0000 7f1800e5f640 1 -- start start 2023-10-18T10:46:52.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.212+0000 7f1800e5f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17fc06bee0 0x7f17fc0c48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:52.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.212+0000 7f1800e5f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17fc070600 0x7f17fc0c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:52.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.212+0000 7f1800e5f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17fc159490 0x7f17fc14a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:52.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.212+0000 7f1800e5f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17fc0c4e20 con 0x7f17fc159490 2023-10-18T10:46:52.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.212+0000 7f1800e5f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17fc1502a0 con 0x7f17fc070600 2023-10-18T10:46:52.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.212+0000 7f1800e5f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17fc153950 con 0x7f17fc06bee0 2023-10-18T10:46:52.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.212+0000 7f17fad9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17fc159490 0x7f17fc14a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:52.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.212+0000 7f17fad9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17fc159490 0x7f17fc14a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56380/0 (socket says 172.21.15.89:56380) 2023-10-18T10:46:52.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.212+0000 7f17fad9d640 1 -- 172.21.15.89:0/3721586245 learned_addr learned my addr 172.21.15.89:0/3721586245 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:52.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.212+0000 7f17f9d9b640 1 --2- 172.21.15.89:0/3721586245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17fc070600 0x7f17fc0c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:52.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.212+0000 7f17fa59c640 1 --2- 172.21.15.89:0/3721586245 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17fc06bee0 0x7f17fc0c48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:52.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.212+0000 7f17fad9d640 1 -- 172.21.15.89:0/3721586245 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17fc06bee0 msgr2=0x7f17fc0c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:52.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.212+0000 7f17fad9d640 1 --2- 172.21.15.89:0/3721586245 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17fc06bee0 0x7f17fc0c48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:52.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.212+0000 7f17fad9d640 1 -- 172.21.15.89:0/3721586245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17fc070600 msgr2=0x7f17fc0c3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:52.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.212+0000 7f17fad9d640 1 --2- 172.21.15.89:0/3721586245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17fc070600 0x7f17fc0c3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:52.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.212+0000 7f17fad9d640 1 -- 172.21.15.89:0/3721586245 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17ec009420 con 0x7f17fc159490 2023-10-18T10:46:52.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.212+0000 7f17fad9d640 1 --2- 172.21.15.89:0/3721586245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17fc159490 0x7f17fc14a360 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f17f400ce40 tx=0x7f17f400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:52.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.212+0000 7f17eaffd640 1 -- 172.21.15.89:0/3721586245 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f400d840 con 0x7f17fc159490 2023-10-18T10:46:52.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.212+0000 7f17eaffd640 1 -- 172.21.15.89:0/3721586245 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17f400d9a0 con 0x7f17fc159490 2023-10-18T10:46:52.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.212+0000 7f17eaffd640 1 -- 172.21.15.89:0/3721586245 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f400dd50 con 0x7f17fc159490 2023-10-18T10:46:52.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.212+0000 7f1800e5f640 1 -- 172.21.15.89:0/3721586245 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17fc153c30 con 0x7f17fc159490 2023-10-18T10:46:52.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.212+0000 7f1800e5f640 1 -- 172.21.15.89:0/3721586245 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17fc1507d0 con 0x7f17fc159490 2023-10-18T10:46:52.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.212+0000 7f1800e5f640 1 -- 172.21.15.89:0/3721586245 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17fc1509f0 con 0x7f17fc159490 2023-10-18T10:46:52.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.216+0000 7f17eaffd640 1 -- 172.21.15.89:0/3721586245 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f17f401e3b0 con 0x7f17fc159490 2023-10-18T10:46:52.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.216+0000 7f17eaffd640 1 --2- 172.21.15.89:0/3721586245 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f17dc065680 0x7f17dc067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:52.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.216+0000 7f17eaffd640 1 -- 172.21.15.89:0/3721586245 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f17f4083b40 con 0x7f17fc159490 2023-10-18T10:46:52.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.216+0000 7f17fa59c640 1 --2- 172.21.15.89:0/3721586245 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f17dc065680 0x7f17dc067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:52.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.216+0000 7f17fa59c640 1 --2- 172.21.15.89:0/3721586245 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f17dc065680 0x7f17dc067b40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f17ec0ae000 tx=0x7f17ec0a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:52.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.220+0000 7f17eaffd640 1 -- 172.21.15.89:0/3721586245 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f17f40512d0 con 0x7f17fc159490 2023-10-18T10:46:52.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.368+0000 7f1800e5f640 1 -- 172.21.15.89:0/3721586245 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f17fc04dbe0 con 0x7f17fc159490 2023-10-18T10:46:52.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.372+0000 7f17eaffd640 1 -- 172.21.15.89:0/3721586245 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f17f401a020 con 0x7f17fc159490 2023-10-18T10:46:52.372 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-18T10:46:52.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.372+0000 7f1800e5f640 1 -- 172.21.15.89:0/3721586245 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f17dc065680 msgr2=0x7f17dc067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:52.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.372+0000 7f1800e5f640 1 --2- 172.21.15.89:0/3721586245 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f17dc065680 0x7f17dc067b40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f17ec0ae000 tx=0x7f17ec0a7000 comp rx=0 tx=0).stop 2023-10-18T10:46:52.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.372+0000 7f1800e5f640 1 -- 172.21.15.89:0/3721586245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17fc159490 msgr2=0x7f17fc14a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:52.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.372+0000 7f1800e5f640 1 --2- 172.21.15.89:0/3721586245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17fc159490 0x7f17fc14a360 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f17f400ce40 tx=0x7f17f400a710 comp rx=0 tx=0).stop 2023-10-18T10:46:52.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.372+0000 7f1800e5f640 1 -- 172.21.15.89:0/3721586245 shutdown_connections 2023-10-18T10:46:52.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.372+0000 7f1800e5f640 1 --2- 172.21.15.89:0/3721586245 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f17dc065680 0x7f17dc067b40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:52.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.372+0000 7f1800e5f640 1 --2- 172.21.15.89:0/3721586245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17fc159490 0x7f17fc14a360 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:52.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.372+0000 7f1800e5f640 1 --2- 172.21.15.89:0/3721586245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17fc070600 0x7f17fc0c3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:52.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.372+0000 7f1800e5f640 1 --2- 172.21.15.89:0/3721586245 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17fc06bee0 0x7f17fc0c48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:52.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.372+0000 7f1800e5f640 1 -- 172.21.15.89:0/3721586245 >> 172.21.15.89:0/3721586245 conn(0x7f17fc069f00 msgr2=0x7f17fc0c80e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:52.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.372+0000 7f1800e5f640 1 -- 172.21.15.89:0/3721586245 shutdown_connections 2023-10-18T10:46:52.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.372+0000 7f1800e5f640 1 -- 172.21.15.89:0/3721586245 wait complete. 2023-10-18T10:46:52.392 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-18T10:46:52.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.532+0000 7f341098f640 1 Processor -- start 2023-10-18T10:46:52.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.532+0000 7f341098f640 1 -- start start 2023-10-18T10:46:52.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.532+0000 7f341098f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f340c145b70 0x7f340c143470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:52.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.532+0000 7f341098f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f340c0ce070 0x7f340c0ce500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:52.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.532+0000 7f341098f640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f340c0cd8d0 con 0x7f340c0cd4f0 2023-10-18T10:46:52.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.532+0000 7f341098f640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f340c0cea40 con 0x7f340c0ce070 2023-10-18T10:46:52.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.532+0000 7f341098f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340c069900 con 0x7f340c145b70 2023-10-18T10:46:52.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.532+0000 7f340ad9d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f340c0ce070 0x7f340c0ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:52.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.532+0000 7f340ad9d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f340c0ce070 0x7f340c0ce500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57204/0 (socket says 172.21.15.89:57204) 2023-10-18T10:46:52.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.532+0000 7f340ad9d640 1 -- 172.21.15.89:0/678002002 learned_addr learned my addr 172.21.15.89:0/678002002 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:52.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.536+0000 7f3401d9b640 1 --2- 172.21.15.89:0/678002002 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f340c145b70 0x7f340c143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:52.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.536+0000 7f340ad9d640 1 -- 172.21.15.89:0/678002002 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f340c145b70 msgr2=0x7f340c143470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:52.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.536+0000 7f340ad9d640 1 --2- 172.21.15.89:0/678002002 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f340c145b70 0x7f340c143470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:52.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.536+0000 7f340ad9d640 1 -- 172.21.15.89:0/678002002 >> v1:172.21.15.154:6789/0 conn(0x7f340c0cd4f0 legacy=0x7f340c160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:52.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.536+0000 7f340ad9d640 1 -- 172.21.15.89:0/678002002 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340c15cbf0 con 0x7f340c0ce070 2023-10-18T10:46:52.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.536+0000 7f340ad9d640 1 --2- 172.21.15.89:0/678002002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f340c0ce070 0x7f340c0ce500 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3404009e50 tx=0x7f340401c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab8ec7fba7bf531 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:52.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.536+0000 7f3409d9b640 1 -- 172.21.15.89:0/678002002 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34040a58a0 con 0x7f340c0ce070 2023-10-18T10:46:52.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.536+0000 7f3409d9b640 1 -- 172.21.15.89:0/678002002 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34040a5a00 con 0x7f340c0ce070 2023-10-18T10:46:52.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.536+0000 7f3409d9b640 1 -- 172.21.15.89:0/678002002 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34040a5d90 con 0x7f340c0ce070 2023-10-18T10:46:52.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.536+0000 7f341098f640 1 -- 172.21.15.89:0/678002002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f340c0ce070 msgr2=0x7f340c0ce500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:52.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.536+0000 7f341098f640 1 --2- 172.21.15.89:0/678002002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f340c0ce070 0x7f340c0ce500 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3404009e50 tx=0x7f340401c9e0 comp rx=0 tx=0).stop 2023-10-18T10:46:52.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.540+0000 7f341098f640 1 -- 172.21.15.89:0/678002002 shutdown_connections 2023-10-18T10:46:52.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.540+0000 7f341098f640 1 --2- 172.21.15.89:0/678002002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f340c0ce070 0x7f340c0ce500 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:52.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.540+0000 7f341098f640 1 --2- 172.21.15.89:0/678002002 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f340c145b70 0x7f340c143470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:52.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.540+0000 7f341098f640 1 -- 172.21.15.89:0/678002002 >> 172.21.15.89:0/678002002 conn(0x7f340c06dad0 msgr2=0x7f340c06dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:52.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.540+0000 7f341098f640 1 -- 172.21.15.89:0/678002002 shutdown_connections 2023-10-18T10:46:52.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.540+0000 7f341098f640 1 -- 172.21.15.89:0/678002002 wait complete. 2023-10-18T10:46:52.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.540+0000 7f341098f640 1 Processor -- start 2023-10-18T10:46:52.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.540+0000 7f341098f640 1 -- start start 2023-10-18T10:46:52.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.540+0000 7f341098f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f340c0cd4f0 0x7f340c15aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:52.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.540+0000 7f341098f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f340c0ce070 0x7f340c156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:52.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.540+0000 7f341098f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f340c145b70 0x7f340c1565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:52.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.540+0000 7f341098f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340c153700 con 0x7f340c0ce070 2023-10-18T10:46:52.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.540+0000 7f341098f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340c15b460 con 0x7f340c145b70 2023-10-18T10:46:52.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.540+0000 7f341098f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340c15b740 con 0x7f340c0cd4f0 2023-10-18T10:46:52.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.540+0000 7f3401d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f340c0ce070 0x7f340c156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:52.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.540+0000 7f3401d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f340c0ce070 0x7f340c156060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57216/0 (socket says 172.21.15.89:57216) 2023-10-18T10:46:52.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.540+0000 7f3401d9b640 1 -- 172.21.15.89:0/3466112363 learned_addr learned my addr 172.21.15.89:0/3466112363 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:52.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.540+0000 7f340ad9d640 1 --2- 172.21.15.89:0/3466112363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f340c145b70 0x7f340c1565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:52.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.540+0000 7f340a59c640 1 --2- 172.21.15.89:0/3466112363 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f340c0cd4f0 0x7f340c15aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:52.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.544+0000 7f3401d9b640 1 -- 172.21.15.89:0/3466112363 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f340c0cd4f0 msgr2=0x7f340c15aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:52.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.544+0000 7f3401d9b640 1 --2- 172.21.15.89:0/3466112363 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f340c0cd4f0 0x7f340c15aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:52.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.544+0000 7f3401d9b640 1 -- 172.21.15.89:0/3466112363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f340c145b70 msgr2=0x7f340c1565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:52.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.544+0000 7f3401d9b640 1 --2- 172.21.15.89:0/3466112363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f340c145b70 0x7f340c1565a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:52.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.544+0000 7f3401d9b640 1 -- 172.21.15.89:0/3466112363 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3404009b30 con 0x7f340c0ce070 2023-10-18T10:46:52.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.544+0000 7f3401d9b640 1 --2- 172.21.15.89:0/3466112363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f340c0ce070 0x7f340c156060 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f33f400d800 tx=0x7f33f400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:52.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.544+0000 7f34037fe640 1 -- 172.21.15.89:0/3466112363 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f400b6c0 con 0x7f340c0ce070 2023-10-18T10:46:52.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.544+0000 7f34037fe640 1 -- 172.21.15.89:0/3466112363 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f400b820 con 0x7f340c0ce070 2023-10-18T10:46:52.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.544+0000 7f34037fe640 1 -- 172.21.15.89:0/3466112363 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f400bb90 con 0x7f340c0ce070 2023-10-18T10:46:52.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.544+0000 7f341098f640 1 -- 172.21.15.89:0/3466112363 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f340c15ba20 con 0x7f340c0ce070 2023-10-18T10:46:52.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.544+0000 7f341098f640 1 -- 172.21.15.89:0/3466112363 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f340c15af70 con 0x7f340c0ce070 2023-10-18T10:46:52.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.544+0000 7f341098f640 1 -- 172.21.15.89:0/3466112363 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f340c15b1f0 con 0x7f340c0ce070 2023-10-18T10:46:52.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.544+0000 7f34037fe640 1 -- 172.21.15.89:0/3466112363 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f33f400bcf0 con 0x7f340c0ce070 2023-10-18T10:46:52.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.544+0000 7f34037fe640 1 --2- 172.21.15.89:0/3466112363 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f33ec065680 0x7f33ec067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:52.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.548+0000 7f34037fe640 1 -- 172.21.15.89:0/3466112363 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f33f40834c0 con 0x7f340c0ce070 2023-10-18T10:46:52.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.548+0000 7f340a59c640 1 --2- 172.21.15.89:0/3466112363 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f33ec065680 0x7f33ec067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:52.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.548+0000 7f340a59c640 1 --2- 172.21.15.89:0/3466112363 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f33ec065680 0x7f33ec067b40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f33fc007fd0 tx=0x7f33fc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:52.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.552+0000 7f34037fe640 1 -- 172.21.15.89:0/3466112363 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f33f4054820 con 0x7f340c0ce070 2023-10-18T10:46:52.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.704+0000 7f341098f640 1 -- 172.21.15.89:0/3466112363 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f340c04dbe0 con 0x7f340c0ce070 2023-10-18T10:46:52.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.704+0000 7f34037fe640 1 -- 172.21.15.89:0/3466112363 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f33f40543b0 con 0x7f340c0ce070 2023-10-18T10:46:52.706 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-10-18T10:46:52.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.704+0000 7f341098f640 1 -- 172.21.15.89:0/3466112363 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f33ec065680 msgr2=0x7f33ec067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:52.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.704+0000 7f341098f640 1 --2- 172.21.15.89:0/3466112363 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f33ec065680 0x7f33ec067b40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f33fc007fd0 tx=0x7f33fc009040 comp rx=0 tx=0).stop 2023-10-18T10:46:52.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.704+0000 7f341098f640 1 -- 172.21.15.89:0/3466112363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f340c0ce070 msgr2=0x7f340c156060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:52.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.704+0000 7f341098f640 1 --2- 172.21.15.89:0/3466112363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f340c0ce070 0x7f340c156060 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f33f400d800 tx=0x7f33f400dcc0 comp rx=0 tx=0).stop 2023-10-18T10:46:52.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.704+0000 7f341098f640 1 -- 172.21.15.89:0/3466112363 shutdown_connections 2023-10-18T10:46:52.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.704+0000 7f341098f640 1 --2- 172.21.15.89:0/3466112363 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f33ec065680 0x7f33ec067b40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:52.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.704+0000 7f341098f640 1 --2- 172.21.15.89:0/3466112363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f340c145b70 0x7f340c1565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:52.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.704+0000 7f341098f640 1 --2- 172.21.15.89:0/3466112363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f340c0ce070 0x7f340c156060 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:52.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.704+0000 7f341098f640 1 --2- 172.21.15.89:0/3466112363 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f340c0cd4f0 0x7f340c15aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:52.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.704+0000 7f341098f640 1 -- 172.21.15.89:0/3466112363 >> 172.21.15.89:0/3466112363 conn(0x7f340c06dad0 msgr2=0x7f340c15d250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:52.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.704+0000 7f341098f640 1 -- 172.21.15.89:0/3466112363 shutdown_connections 2023-10-18T10:46:52.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.704+0000 7f341098f640 1 -- 172.21.15.89:0/3466112363 wait complete. 2023-10-18T10:46:52.726 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:46:52.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.864+0000 7f362f767640 1 Processor -- start 2023-10-18T10:46:52.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.864+0000 7f362f767640 1 -- start start 2023-10-18T10:46:52.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.868+0000 7f362f767640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f362806ff00 0x7f362815e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:52.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.868+0000 7f362f767640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36280ca340 0x7f362815b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:52.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.868+0000 7f362f767640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f362815acb0 con 0x7f362815b680 2023-10-18T10:46:52.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.868+0000 7f362f767640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f362815ef30 con 0x7f362806ff00 2023-10-18T10:46:52.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.868+0000 7f362f767640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362815aa10 con 0x7f36280ca340 2023-10-18T10:46:52.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.868+0000 7f362dd04640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f362815b680 0x7f36280c8960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58112/0 (socket says 172.21.15.89:58112) 2023-10-18T10:46:52.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.868+0000 7f362dd04640 1 -- 172.21.15.89:0/3753472725 learned_addr learned my addr 172.21.15.89:0/3753472725 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:52.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.868+0000 7f362cd02640 1 --2- 172.21.15.89:0/3753472725 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36280ca340 0x7f362815b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:52.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.868+0000 7f362d503640 1 --2- 172.21.15.89:0/3753472725 >> v2:172.21.15.154:3300/0 conn(0x7f362806ff00 0x7f362815e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:52.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.868+0000 7f361ffff640 1 -- 172.21.15.89:0/3753472725 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3140678723 0 0) 0x7f362815acb0 con 0x7f362815b680 2023-10-18T10:46:52.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.868+0000 7f361ffff640 1 -- 172.21.15.89:0/3753472725 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3614003530 con 0x7f362815b680 2023-10-18T10:46:52.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.868+0000 7f362d503640 1 -- 172.21.15.89:0/3753472725 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36280ca340 msgr2=0x7f362815b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:52.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.868+0000 7f362d503640 1 --2- 172.21.15.89:0/3753472725 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36280ca340 0x7f362815b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:52.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.868+0000 7f362d503640 1 -- 172.21.15.89:0/3753472725 >> v1:172.21.15.89:6789/0 conn(0x7f362815b680 legacy=0x7f36280c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:52.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.868+0000 7f362d503640 1 -- 172.21.15.89:0/3753472725 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36280702e0 con 0x7f362806ff00 2023-10-18T10:46:52.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.868+0000 7f362cd02640 1 --2- 172.21.15.89:0/3753472725 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36280ca340 0x7f362815b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:46:52.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.872+0000 7f362d503640 1 --2- 172.21.15.89:0/3753472725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f362806ff00 0x7f362815e000 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f36240097c0 tx=0x7f362401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60a8be334a1cbf22 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:52.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.872+0000 7f361ffff640 1 -- 172.21.15.89:0/3753472725 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36240ac620 con 0x7f362806ff00 2023-10-18T10:46:52.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.872+0000 7f361ffff640 1 -- 172.21.15.89:0/3753472725 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36240ac780 con 0x7f362806ff00 2023-10-18T10:46:52.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.872+0000 7f361ffff640 1 -- 172.21.15.89:0/3753472725 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36240acb10 con 0x7f362806ff00 2023-10-18T10:46:52.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.872+0000 7f362f767640 1 -- 172.21.15.89:0/3753472725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f362806ff00 msgr2=0x7f362815e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:52.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.872+0000 7f362f767640 1 --2- 172.21.15.89:0/3753472725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f362806ff00 0x7f362815e000 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f36240097c0 tx=0x7f362401c3c0 comp rx=0 tx=0).stop 2023-10-18T10:46:52.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.872+0000 7f362f767640 1 -- 172.21.15.89:0/3753472725 shutdown_connections 2023-10-18T10:46:52.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.872+0000 7f362f767640 1 --2- 172.21.15.89:0/3753472725 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36280ca340 0x7f362815b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:52.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.872+0000 7f362f767640 1 --2- 172.21.15.89:0/3753472725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f362806ff00 0x7f362815e000 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:52.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.872+0000 7f362f767640 1 -- 172.21.15.89:0/3753472725 >> 172.21.15.89:0/3753472725 conn(0x7f3628069f00 msgr2=0x7f36280cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:52.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.872+0000 7f362f767640 1 -- 172.21.15.89:0/3753472725 shutdown_connections 2023-10-18T10:46:52.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.872+0000 7f362f767640 1 -- 172.21.15.89:0/3753472725 wait complete. 2023-10-18T10:46:52.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.876+0000 7f362f767640 1 Processor -- start 2023-10-18T10:46:52.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.876+0000 7f362f767640 1 -- start start 2023-10-18T10:46:52.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.876+0000 7f362f767640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f362806ff00 0x7f36280c6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:52.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.876+0000 7f362f767640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36280ca340 0x7f36280aefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:52.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.876+0000 7f362f767640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f362814dce0 0x7f362814e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:52.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.876+0000 7f362f767640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36280c6a20 con 0x7f362814dce0 2023-10-18T10:46:52.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.876+0000 7f362f767640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36280c7de0 con 0x7f36280ca340 2023-10-18T10:46:52.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.876+0000 7f362f767640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36280ae820 con 0x7f362806ff00 2023-10-18T10:46:52.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.876+0000 7f362dd04640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f362814dce0 0x7f362814e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:52.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.876+0000 7f362dd04640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f362814dce0 0x7f362814e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57220/0 (socket says 172.21.15.89:57220) 2023-10-18T10:46:52.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.876+0000 7f362dd04640 1 -- 172.21.15.89:0/3129052013 learned_addr learned my addr 172.21.15.89:0/3129052013 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:52.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.876+0000 7f362cd02640 1 --2- 172.21.15.89:0/3129052013 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36280ca340 0x7f36280aefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:52.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.876+0000 7f362d503640 1 --2- 172.21.15.89:0/3129052013 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f362806ff00 0x7f36280c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:52.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.876+0000 7f362dd04640 1 -- 172.21.15.89:0/3129052013 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f362806ff00 msgr2=0x7f36280c6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:52.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.876+0000 7f362dd04640 1 --2- 172.21.15.89:0/3129052013 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f362806ff00 0x7f36280c6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:52.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.876+0000 7f362dd04640 1 -- 172.21.15.89:0/3129052013 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36280ca340 msgr2=0x7f36280aefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:52.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.876+0000 7f362dd04640 1 --2- 172.21.15.89:0/3129052013 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36280ca340 0x7f36280aefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:52.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.876+0000 7f362dd04640 1 -- 172.21.15.89:0/3129052013 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3618009420 con 0x7f362814dce0 2023-10-18T10:46:52.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.876+0000 7f362dd04640 1 --2- 172.21.15.89:0/3129052013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f362814dce0 0x7f362814e190 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f362000c9f0 tx=0x7f362000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:52.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.876+0000 7f361dffb640 1 -- 172.21.15.89:0/3129052013 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362000ab70 con 0x7f362814dce0 2023-10-18T10:46:52.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.876+0000 7f361dffb640 1 -- 172.21.15.89:0/3129052013 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362000acd0 con 0x7f362814dce0 2023-10-18T10:46:52.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.876+0000 7f361dffb640 1 -- 172.21.15.89:0/3129052013 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3620012600 con 0x7f362814dce0 2023-10-18T10:46:52.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.876+0000 7f362f767640 1 -- 172.21.15.89:0/3129052013 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3624009420 con 0x7f362814dce0 2023-10-18T10:46:52.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.876+0000 7f362f767640 1 -- 172.21.15.89:0/3129052013 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36280c5a30 con 0x7f362814dce0 2023-10-18T10:46:52.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.876+0000 7f362f767640 1 -- 172.21.15.89:0/3129052013 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f362804dbe0 con 0x7f362814dce0 2023-10-18T10:46:52.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.880+0000 7f361dffb640 1 -- 172.21.15.89:0/3129052013 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f3620012760 con 0x7f362814dce0 2023-10-18T10:46:52.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.880+0000 7f361dffb640 1 --2- 172.21.15.89:0/3129052013 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f360c0656b0 0x7f360c067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:52.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.880+0000 7f361dffb640 1 -- 172.21.15.89:0/3129052013 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3620082cb0 con 0x7f362814dce0 2023-10-18T10:46:52.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.880+0000 7f362d503640 1 --2- 172.21.15.89:0/3129052013 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f360c0656b0 0x7f360c067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:52.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.880+0000 7f362d503640 1 --2- 172.21.15.89:0/3129052013 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f360c0656b0 0x7f360c067b70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f36240ae000 tx=0x7f36240a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:52.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:52.884+0000 7f361dffb640 1 -- 172.21.15.89:0/3129052013 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f36200504e0 con 0x7f362814dce0 2023-10-18T10:46:53.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.032+0000 7f362f767640 1 -- 172.21.15.89:0/3129052013 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f36280cb820 con 0x7f362814dce0 2023-10-18T10:46:53.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.032+0000 7f361dffb640 1 -- 172.21.15.89:0/3129052013 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3620017020 con 0x7f362814dce0 2023-10-18T10:46:53.034 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-18T10:46:53.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.036+0000 7f362f767640 1 -- 172.21.15.89:0/3129052013 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f360c0656b0 msgr2=0x7f360c067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:53.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.036+0000 7f362f767640 1 --2- 172.21.15.89:0/3129052013 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f360c0656b0 0x7f360c067b70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f36240ae000 tx=0x7f36240a7000 comp rx=0 tx=0).stop 2023-10-18T10:46:53.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.036+0000 7f362f767640 1 -- 172.21.15.89:0/3129052013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f362814dce0 msgr2=0x7f362814e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:53.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.036+0000 7f362f767640 1 --2- 172.21.15.89:0/3129052013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f362814dce0 0x7f362814e190 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f362000c9f0 tx=0x7f362000ceb0 comp rx=0 tx=0).stop 2023-10-18T10:46:53.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.036+0000 7f362f767640 1 -- 172.21.15.89:0/3129052013 shutdown_connections 2023-10-18T10:46:53.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.036+0000 7f362f767640 1 --2- 172.21.15.89:0/3129052013 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f360c0656b0 0x7f360c067b70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:53.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.036+0000 7f362f767640 1 --2- 172.21.15.89:0/3129052013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f362814dce0 0x7f362814e190 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:53.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.036+0000 7f362f767640 1 --2- 172.21.15.89:0/3129052013 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36280ca340 0x7f36280aefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:53.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.036+0000 7f362f767640 1 --2- 172.21.15.89:0/3129052013 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f362806ff00 0x7f36280c6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:53.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.036+0000 7f362f767640 1 -- 172.21.15.89:0/3129052013 >> 172.21.15.89:0/3129052013 conn(0x7f3628069f00 msgr2=0x7f362815bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:53.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.036+0000 7f362f767640 1 -- 172.21.15.89:0/3129052013 shutdown_connections 2023-10-18T10:46:53.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.036+0000 7f362f767640 1 -- 172.21.15.89:0/3129052013 wait complete. 2023-10-18T10:46:53.055 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:46:53.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.196+0000 7f183b18f640 1 Processor -- start 2023-10-18T10:46:53.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.196+0000 7f183b18f640 1 -- start start 2023-10-18T10:46:53.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.196+0000 7f183b18f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f18340cd4f0 0x7f1834143470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:53.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.196+0000 7f183b18f640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18340cd8d0 con 0x7f1834145b70 2023-10-18T10:46:53.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.196+0000 7f183b18f640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f183415cbf0 con 0x7f18340ce070 2023-10-18T10:46:53.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.196+0000 7f183b18f640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1834069900 con 0x7f18340cd4f0 2023-10-18T10:46:53.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.196+0000 7f1838f2b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f18340cd4f0 0x7f1834143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:53.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.196+0000 7f1838f2b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f18340cd4f0 0x7f1834143470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57222/0 (socket says 172.21.15.89:57222) 2023-10-18T10:46:53.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.196+0000 7f1838f2b640 1 -- 172.21.15.89:0/804983755 learned_addr learned my addr 172.21.15.89:0/804983755 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:53.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.196+0000 7f1833fff640 1 --1- 172.21.15.89:0/804983755 >> v1:172.21.15.89:6789/0 conn(0x7f1834145b70 0x7f1834160cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58120/0 (socket says 172.21.15.89:58120) 2023-10-18T10:46:53.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.196+0000 7f1838f2b640 1 -- 172.21.15.89:0/804983755 >> v1:172.21.15.154:6789/0 conn(0x7f18340ce070 legacy=0x7f183406abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:53.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.196+0000 7f1838f2b640 1 -- 172.21.15.89:0/804983755 >> v1:172.21.15.89:6789/0 conn(0x7f1834145b70 legacy=0x7f1834160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:53.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.196+0000 7f1838f2b640 1 -- 172.21.15.89:0/804983755 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1834069b10 con 0x7f18340cd4f0 2023-10-18T10:46:53.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.196+0000 7f18337fe640 1 -- 172.21.15.89:0/804983755 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1758548038 0 0) 0x7f18340cd8d0 con 0x7f1834145b70 2023-10-18T10:46:53.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.200+0000 7f1838f2b640 1 --2- 172.21.15.89:0/804983755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18340cd4f0 0x7f1834143470 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f181c009870 tx=0x7f181c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ffd1b82c43e5f55 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:53.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.200+0000 7f18337fe640 1 -- 172.21.15.89:0/804983755 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c0aa430 con 0x7f18340cd4f0 2023-10-18T10:46:53.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.200+0000 7f18337fe640 1 -- 172.21.15.89:0/804983755 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181c0aa590 con 0x7f18340cd4f0 2023-10-18T10:46:53.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.200+0000 7f18337fe640 1 -- 172.21.15.89:0/804983755 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c0aa920 con 0x7f18340cd4f0 2023-10-18T10:46:53.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.200+0000 7f183b18f640 1 -- 172.21.15.89:0/804983755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18340cd4f0 msgr2=0x7f1834143470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:53.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.200+0000 7f183b18f640 1 --2- 172.21.15.89:0/804983755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18340cd4f0 0x7f1834143470 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f181c009870 tx=0x7f181c01c4d0 comp rx=0 tx=0).stop 2023-10-18T10:46:53.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.200+0000 7f183b18f640 1 -- 172.21.15.89:0/804983755 shutdown_connections 2023-10-18T10:46:53.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.200+0000 7f183b18f640 1 --2- 172.21.15.89:0/804983755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18340cd4f0 0x7f1834143470 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:53.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.200+0000 7f183b18f640 1 -- 172.21.15.89:0/804983755 >> 172.21.15.89:0/804983755 conn(0x7f183406dad0 msgr2=0x7f183406dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:53.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.200+0000 7f183b18f640 1 -- 172.21.15.89:0/804983755 shutdown_connections 2023-10-18T10:46:53.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.200+0000 7f183b18f640 1 -- 172.21.15.89:0/804983755 wait complete. 2023-10-18T10:46:53.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.200+0000 7f183b18f640 1 Processor -- start 2023-10-18T10:46:53.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.200+0000 7f183b18f640 1 -- start start 2023-10-18T10:46:53.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.204+0000 7f183b18f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18340cd4f0 0x7f18341421d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:53.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.204+0000 7f183b18f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18340ce070 0x7f18340c4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:53.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.204+0000 7f183b18f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1834145b70 0x7f18340bff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:53.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.204+0000 7f183b18f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1834142710 con 0x7f1834145b70 2023-10-18T10:46:53.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.204+0000 7f183b18f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18340c47d0 con 0x7f18340cd4f0 2023-10-18T10:46:53.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.204+0000 7f183b18f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18340c7680 con 0x7f18340ce070 2023-10-18T10:46:53.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.204+0000 7f183972c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1834145b70 0x7f18340bff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:53.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.204+0000 7f183972c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1834145b70 0x7f18340bff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57234/0 (socket says 172.21.15.89:57234) 2023-10-18T10:46:53.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.204+0000 7f183972c640 1 -- 172.21.15.89:0/5346093 learned_addr learned my addr 172.21.15.89:0/5346093 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:53.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.204+0000 7f1838f2b640 1 --2- 172.21.15.89:0/5346093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18340cd4f0 0x7f18341421d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:53.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.204+0000 7f1833fff640 1 --2- 172.21.15.89:0/5346093 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18340ce070 0x7f18340c4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:53.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.204+0000 7f183972c640 1 -- 172.21.15.89:0/5346093 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18340ce070 msgr2=0x7f18340c4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:53.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.204+0000 7f183972c640 1 --2- 172.21.15.89:0/5346093 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18340ce070 0x7f18340c4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:53.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.204+0000 7f183972c640 1 -- 172.21.15.89:0/5346093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18340cd4f0 msgr2=0x7f18341421d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:53.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.204+0000 7f183972c640 1 --2- 172.21.15.89:0/5346093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18340cd4f0 0x7f18341421d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:53.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.204+0000 7f183972c640 1 -- 172.21.15.89:0/5346093 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f181c0094d0 con 0x7f1834145b70 2023-10-18T10:46:53.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.204+0000 7f183972c640 1 --2- 172.21.15.89:0/5346093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1834145b70 0x7f18340bff10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f182c00c780 tx=0x7f182c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:53.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.204+0000 7f18317fa640 1 -- 172.21.15.89:0/5346093 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c004b70 con 0x7f1834145b70 2023-10-18T10:46:53.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.204+0000 7f18317fa640 1 -- 172.21.15.89:0/5346093 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182c00ce90 con 0x7f1834145b70 2023-10-18T10:46:53.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.204+0000 7f183b18f640 1 -- 172.21.15.89:0/5346093 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18340c7960 con 0x7f1834145b70 2023-10-18T10:46:53.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.204+0000 7f18317fa640 1 -- 172.21.15.89:0/5346093 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c00a850 con 0x7f1834145b70 2023-10-18T10:46:53.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.204+0000 7f183b18f640 1 -- 172.21.15.89:0/5346093 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18340c7f20 con 0x7f1834145b70 2023-10-18T10:46:53.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.204+0000 7f183b18f640 1 -- 172.21.15.89:0/5346093 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f183404dbe0 con 0x7f1834145b70 2023-10-18T10:46:53.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.208+0000 7f18317fa640 1 -- 172.21.15.89:0/5346093 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f182c00a9b0 con 0x7f1834145b70 2023-10-18T10:46:53.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.208+0000 7f18317fa640 1 --2- 172.21.15.89:0/5346093 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1814065680 0x7f1814067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:53.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.208+0000 7f18317fa640 1 -- 172.21.15.89:0/5346093 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f182c0831b0 con 0x7f1834145b70 2023-10-18T10:46:53.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.208+0000 7f1838f2b640 1 --2- 172.21.15.89:0/5346093 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1814065680 0x7f1814067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:53.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.208+0000 7f1838f2b640 1 --2- 172.21.15.89:0/5346093 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1814065680 0x7f1814067b40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f181c0ae000 tx=0x7f181c0a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:53.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.212+0000 7f18317fa640 1 -- 172.21.15.89:0/5346093 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f182c054480 con 0x7f1834145b70 2023-10-18T10:46:53.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.360+0000 7f183b18f640 1 -- 172.21.15.89:0/5346093 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f18340730b0 con 0x7f1834145b70 2023-10-18T10:46:53.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.360+0000 7f18317fa640 1 -- 172.21.15.89:0/5346093 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f182c054010 con 0x7f1834145b70 2023-10-18T10:46:53.362 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-18T10:46:53.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.364+0000 7f183b18f640 1 -- 172.21.15.89:0/5346093 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1814065680 msgr2=0x7f1814067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:53.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.364+0000 7f183b18f640 1 --2- 172.21.15.89:0/5346093 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1814065680 0x7f1814067b40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f181c0ae000 tx=0x7f181c0a7000 comp rx=0 tx=0).stop 2023-10-18T10:46:53.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.364+0000 7f183b18f640 1 -- 172.21.15.89:0/5346093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1834145b70 msgr2=0x7f18340bff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:53.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.364+0000 7f183b18f640 1 --2- 172.21.15.89:0/5346093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1834145b70 0x7f18340bff10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f182c00c780 tx=0x7f182c00cc40 comp rx=0 tx=0).stop 2023-10-18T10:46:53.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.364+0000 7f183b18f640 1 -- 172.21.15.89:0/5346093 shutdown_connections 2023-10-18T10:46:53.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.364+0000 7f183b18f640 1 --2- 172.21.15.89:0/5346093 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1814065680 0x7f1814067b40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:53.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.364+0000 7f183b18f640 1 --2- 172.21.15.89:0/5346093 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18340ce070 0x7f18340c4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:53.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.364+0000 7f183b18f640 1 --2- 172.21.15.89:0/5346093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1834145b70 0x7f18340bff10 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:53.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.364+0000 7f183b18f640 1 --2- 172.21.15.89:0/5346093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18340cd4f0 0x7f18341421d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:53.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.364+0000 7f183b18f640 1 -- 172.21.15.89:0/5346093 >> 172.21.15.89:0/5346093 conn(0x7f183406dad0 msgr2=0x7f183406abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:53.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.364+0000 7f183b18f640 1 -- 172.21.15.89:0/5346093 shutdown_connections 2023-10-18T10:46:53.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.364+0000 7f183b18f640 1 -- 172.21.15.89:0/5346093 wait complete. 2023-10-18T10:46:53.383 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T10:46:53.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.524+0000 7fe9e2167640 1 Processor -- start 2023-10-18T10:46:53.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.524+0000 7fe9e2167640 1 -- start start 2023-10-18T10:46:53.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.524+0000 7fe9e2167640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9dc0c9970 0x7fe9dc164230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:53.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.524+0000 7fe9e2167640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9dc070b90 con 0x7fe9dc159490 2023-10-18T10:46:53.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.524+0000 7fe9e2167640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9dc070e70 con 0x7fe9dc070600 2023-10-18T10:46:53.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.524+0000 7fe9e2167640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc071150 con 0x7fe9dc0c9970 2023-10-18T10:46:53.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.524+0000 7fe9db7fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe9dc159490 0x7fe9dc15d8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58124/0 (socket says 172.21.15.89:58124) 2023-10-18T10:46:53.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.524+0000 7fe9db7fe640 1 -- 172.21.15.89:0/3063574976 learned_addr learned my addr 172.21.15.89:0/3063574976 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:53.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.524+0000 7fe9dbfff640 1 --2- 172.21.15.89:0/3063574976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9dc0c9970 0x7fe9dc164230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:53.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.524+0000 7fe9da7fc640 1 -- 172.21.15.89:0/3063574976 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3001921492 0 0) 0x7fe9dc070b90 con 0x7fe9dc159490 2023-10-18T10:46:53.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.524+0000 7fe9da7fc640 1 -- 172.21.15.89:0/3063574976 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9c4003530 con 0x7fe9dc159490 2023-10-18T10:46:53.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.524+0000 7fe9da7fc640 1 -- 172.21.15.89:0/3063574976 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2784114918 0 0) 0x7fe9dc070e70 con 0x7fe9dc070600 2023-10-18T10:46:53.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.524+0000 7fe9da7fc640 1 -- 172.21.15.89:0/3063574976 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9dc070b90 con 0x7fe9dc070600 2023-10-18T10:46:53.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.524+0000 7fe9dbfff640 1 -- 172.21.15.89:0/3063574976 >> v1:172.21.15.154:6789/0 conn(0x7fe9dc070600 legacy=0x7fe9dc06b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:53.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.524+0000 7fe9da7fc640 1 -- 172.21.15.89:0/3063574976 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4181727319 0 0) 0x7fe9c4003530 con 0x7fe9dc159490 2023-10-18T10:46:53.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.524+0000 7fe9dbfff640 1 -- 172.21.15.89:0/3063574976 >> v1:172.21.15.89:6789/0 conn(0x7fe9dc159490 legacy=0x7fe9dc15d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:53.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.524+0000 7fe9dbfff640 1 -- 172.21.15.89:0/3063574976 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9dc071430 con 0x7fe9dc0c9970 2023-10-18T10:46:53.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.528+0000 7fe9dbfff640 1 --2- 172.21.15.89:0/3063574976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9dc0c9970 0x7fe9dc164230 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4009dc0 tx=0x7fe9d401c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=259ecebb23b42a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:53.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.528+0000 7fe9da7fc640 1 -- 172.21.15.89:0/3063574976 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc001680 con 0x7fe9dc0c9970 2023-10-18T10:46:53.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.528+0000 7fe9da7fc640 1 -- 172.21.15.89:0/3063574976 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d40a55f0 con 0x7fe9dc0c9970 2023-10-18T10:46:53.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.528+0000 7fe9da7fc640 1 -- 172.21.15.89:0/3063574976 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d40a5980 con 0x7fe9dc0c9970 2023-10-18T10:46:53.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.528+0000 7fe9e2167640 1 -- 172.21.15.89:0/3063574976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9dc0c9970 msgr2=0x7fe9dc164230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:53.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.528+0000 7fe9e2167640 1 --2- 172.21.15.89:0/3063574976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9dc0c9970 0x7fe9dc164230 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4009dc0 tx=0x7fe9d401c7d0 comp rx=0 tx=0).stop 2023-10-18T10:46:53.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.528+0000 7fe9e2167640 1 -- 172.21.15.89:0/3063574976 shutdown_connections 2023-10-18T10:46:53.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.528+0000 7fe9e2167640 1 --2- 172.21.15.89:0/3063574976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9dc0c9970 0x7fe9dc164230 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:53.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.528+0000 7fe9e2167640 1 -- 172.21.15.89:0/3063574976 >> 172.21.15.89:0/3063574976 conn(0x7fe9dc069f00 msgr2=0x7fe9dc0cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:53.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.528+0000 7fe9e2167640 1 -- 172.21.15.89:0/3063574976 shutdown_connections 2023-10-18T10:46:53.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.528+0000 7fe9e2167640 1 -- 172.21.15.89:0/3063574976 wait complete. 2023-10-18T10:46:53.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.532+0000 7fe9e2167640 1 Processor -- start 2023-10-18T10:46:53.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.532+0000 7fe9e2167640 1 -- start start 2023-10-18T10:46:53.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.532+0000 7fe9e2167640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9dc070600 0x7fe9dc0c1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:53.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.532+0000 7fe9e2167640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9dc0c9970 0x7fe9dc14fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:53.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.532+0000 7fe9e2167640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9dc159490 0x7fe9dc14ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:53.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.532+0000 7fe9e2167640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc0b0260 con 0x7fe9dc159490 2023-10-18T10:46:53.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.532+0000 7fe9e2167640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc0c3640 con 0x7fe9dc070600 2023-10-18T10:46:53.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.532+0000 7fe9e2167640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc0c3060 con 0x7fe9dc0c9970 2023-10-18T10:46:53.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.532+0000 7fe9dbfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9dc159490 0x7fe9dc14ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:53.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.532+0000 7fe9dbfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9dc159490 0x7fe9dc14ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57248/0 (socket says 172.21.15.89:57248) 2023-10-18T10:46:53.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.532+0000 7fe9dbfff640 1 -- 172.21.15.89:0/2787241679 learned_addr learned my addr 172.21.15.89:0/2787241679 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:53.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.532+0000 7fe9db7fe640 1 --2- 172.21.15.89:0/2787241679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9dc070600 0x7fe9dc0c1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:53.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.532+0000 7fe9daffd640 1 --2- 172.21.15.89:0/2787241679 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9dc0c9970 0x7fe9dc14fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:53.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.532+0000 7fe9dbfff640 1 -- 172.21.15.89:0/2787241679 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9dc0c9970 msgr2=0x7fe9dc14fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:53.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.532+0000 7fe9dbfff640 1 --2- 172.21.15.89:0/2787241679 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9dc0c9970 0x7fe9dc14fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:53.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.532+0000 7fe9dbfff640 1 -- 172.21.15.89:0/2787241679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9dc070600 msgr2=0x7fe9dc0c1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:53.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.532+0000 7fe9dbfff640 1 --2- 172.21.15.89:0/2787241679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9dc070600 0x7fe9dc0c1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:53.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.532+0000 7fe9dbfff640 1 -- 172.21.15.89:0/2787241679 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d4009a70 con 0x7fe9dc159490 2023-10-18T10:46:53.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.532+0000 7fe9daffd640 1 --2- 172.21.15.89:0/2787241679 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9dc0c9970 0x7fe9dc14fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:53.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.532+0000 7fe9dbfff640 1 --2- 172.21.15.89:0/2787241679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9dc159490 0x7fe9dc14ff80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe9d401ccb0 tx=0x7fe9d4019570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:53.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.532+0000 7fe9c3fff640 1 -- 172.21.15.89:0/2787241679 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4003ba0 con 0x7fe9dc159490 2023-10-18T10:46:53.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.532+0000 7fe9c3fff640 1 -- 172.21.15.89:0/2787241679 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d4003d20 con 0x7fe9dc159490 2023-10-18T10:46:53.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.532+0000 7fe9c3fff640 1 -- 172.21.15.89:0/2787241679 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4004270 con 0x7fe9dc159490 2023-10-18T10:46:53.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.532+0000 7fe9e2167640 1 -- 172.21.15.89:0/2787241679 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9dc166f90 con 0x7fe9dc159490 2023-10-18T10:46:53.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.536+0000 7fe9e2167640 1 -- 172.21.15.89:0/2787241679 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9dc0c32d0 con 0x7fe9dc159490 2023-10-18T10:46:53.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.536+0000 7fe9e2167640 1 -- 172.21.15.89:0/2787241679 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9dc0c37f0 con 0x7fe9dc159490 2023-10-18T10:46:53.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.536+0000 7fe9c3fff640 1 -- 172.21.15.89:0/2787241679 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fe9d40b4050 con 0x7fe9dc159490 2023-10-18T10:46:53.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.536+0000 7fe9c3fff640 1 --2- 172.21.15.89:0/2787241679 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe9c8065680 0x7fe9c8067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:53.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.536+0000 7fe9c3fff640 1 -- 172.21.15.89:0/2787241679 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe9d411dff0 con 0x7fe9dc159490 2023-10-18T10:46:53.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.536+0000 7fe9db7fe640 1 --2- 172.21.15.89:0/2787241679 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe9c8065680 0x7fe9c8067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:53.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.536+0000 7fe9db7fe640 1 --2- 172.21.15.89:0/2787241679 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe9c8065680 0x7fe9c8067b40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc000f70 tx=0x7fe9cc0016c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:53.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.540+0000 7fe9c3fff640 1 -- 172.21.15.89:0/2787241679 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fe9d414ed40 con 0x7fe9dc159490 2023-10-18T10:46:53.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.688+0000 7fe9e2167640 1 -- 172.21.15.89:0/2787241679 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe9dc04dbe0 con 0x7fe9dc159490 2023-10-18T10:46:53.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.688+0000 7fe9c3fff640 1 -- 172.21.15.89:0/2787241679 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe9d40b20a0 con 0x7fe9dc159490 2023-10-18T10:46:53.691 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-18T10:46:53.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.692+0000 7fe9e2167640 1 -- 172.21.15.89:0/2787241679 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe9c8065680 msgr2=0x7fe9c8067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:53.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.692+0000 7fe9e2167640 1 --2- 172.21.15.89:0/2787241679 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe9c8065680 0x7fe9c8067b40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc000f70 tx=0x7fe9cc0016c0 comp rx=0 tx=0).stop 2023-10-18T10:46:53.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.692+0000 7fe9e2167640 1 -- 172.21.15.89:0/2787241679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9dc159490 msgr2=0x7fe9dc14ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:53.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.692+0000 7fe9e2167640 1 --2- 172.21.15.89:0/2787241679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9dc159490 0x7fe9dc14ff80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe9d401ccb0 tx=0x7fe9d4019570 comp rx=0 tx=0).stop 2023-10-18T10:46:53.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.692+0000 7fe9e2167640 1 -- 172.21.15.89:0/2787241679 shutdown_connections 2023-10-18T10:46:53.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.692+0000 7fe9e2167640 1 --2- 172.21.15.89:0/2787241679 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe9c8065680 0x7fe9c8067b40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:53.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.692+0000 7fe9e2167640 1 --2- 172.21.15.89:0/2787241679 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9dc0c9970 0x7fe9dc14fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:53.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.692+0000 7fe9e2167640 1 --2- 172.21.15.89:0/2787241679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9dc159490 0x7fe9dc14ff80 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:53.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.692+0000 7fe9e2167640 1 --2- 172.21.15.89:0/2787241679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9dc070600 0x7fe9dc0c1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:53.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.692+0000 7fe9e2167640 1 -- 172.21.15.89:0/2787241679 >> 172.21.15.89:0/2787241679 conn(0x7fe9dc069f00 msgr2=0x7fe9dc0b3770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:53.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.692+0000 7fe9e2167640 1 -- 172.21.15.89:0/2787241679 shutdown_connections 2023-10-18T10:46:53.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.692+0000 7fe9e2167640 1 -- 172.21.15.89:0/2787241679 wait complete. 2023-10-18T10:46:53.712 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T10:46:53.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.848+0000 7f52dc818640 1 Processor -- start 2023-10-18T10:46:53.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.848+0000 7f52dc818640 1 -- start start 2023-10-18T10:46:53.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.852+0000 7f52dc818640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f52d80d0760 0x7f52d80d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:53.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.852+0000 7f52dc818640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f52d80cd160 0x7f52d80cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:53.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.852+0000 7f52dc818640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d80d1100 con 0x7f52d815c940 2023-10-18T10:46:53.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.852+0000 7f52dc818640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d815da50 con 0x7f52d80cd160 2023-10-18T10:46:53.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.852+0000 7f52dc818640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d80d3110 con 0x7f52d80d0760 2023-10-18T10:46:53.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.852+0000 7f52d659c640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f52d815c940 0x7f52d80d2a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58128/0 (socket says 172.21.15.89:58128) 2023-10-18T10:46:53.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.852+0000 7f52d659c640 1 -- 172.21.15.89:0/1099801052 learned_addr learned my addr 172.21.15.89:0/1099801052 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:53.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.852+0000 7f52d6d9d640 1 --2- 172.21.15.89:0/1099801052 >> v2:172.21.15.89:3300/0 conn(0x7f52d80cd160 0x7f52d80cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:53.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.852+0000 7f52d5d9b640 1 --2- 172.21.15.89:0/1099801052 >> v2:172.21.15.154:3300/0 conn(0x7f52d80d0760 0x7f52d80d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:53.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.852+0000 7f52d559a640 1 -- 172.21.15.89:0/1099801052 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1209407267 0 0) 0x7f52d80d1100 con 0x7f52d815c940 2023-10-18T10:46:53.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.852+0000 7f52d559a640 1 -- 172.21.15.89:0/1099801052 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52c0003530 con 0x7f52d815c940 2023-10-18T10:46:53.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.852+0000 7f52d6d9d640 1 -- 172.21.15.89:0/1099801052 >> v2:172.21.15.154:3300/0 conn(0x7f52d80d0760 msgr2=0x7f52d80d0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:53.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.852+0000 7f52d6d9d640 1 --2- 172.21.15.89:0/1099801052 >> v2:172.21.15.154:3300/0 conn(0x7f52d80d0760 0x7f52d80d0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:53.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.852+0000 7f52d6d9d640 1 -- 172.21.15.89:0/1099801052 >> v1:172.21.15.89:6789/0 conn(0x7f52d815c940 legacy=0x7f52d80d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:53.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.852+0000 7f52d6d9d640 1 -- 172.21.15.89:0/1099801052 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d815ce40 con 0x7f52d80cd160 2023-10-18T10:46:53.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.856+0000 7f52d6d9d640 1 --2- 172.21.15.89:0/1099801052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52d80cd160 0x7f52d80cd5f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f52d80724f0 tx=0x7f52d001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12668e37cc6c18b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:53.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.856+0000 7f52d559a640 1 -- 172.21.15.89:0/1099801052 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d00a59a0 con 0x7f52d80cd160 2023-10-18T10:46:53.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.856+0000 7f52d559a640 1 -- 172.21.15.89:0/1099801052 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d00a5b00 con 0x7f52d80cd160 2023-10-18T10:46:53.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.856+0000 7f52d559a640 1 -- 172.21.15.89:0/1099801052 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d00a5e90 con 0x7f52d80cd160 2023-10-18T10:46:53.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.856+0000 7f52dc818640 1 -- 172.21.15.89:0/1099801052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52d80cd160 msgr2=0x7f52d80cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:53.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.856+0000 7f52dc818640 1 --2- 172.21.15.89:0/1099801052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52d80cd160 0x7f52d80cd5f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f52d80724f0 tx=0x7f52d001c8e0 comp rx=0 tx=0).stop 2023-10-18T10:46:53.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.856+0000 7f52dc818640 1 -- 172.21.15.89:0/1099801052 shutdown_connections 2023-10-18T10:46:53.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.856+0000 7f52dc818640 1 --2- 172.21.15.89:0/1099801052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52d80cd160 0x7f52d80cd5f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:53.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.856+0000 7f52dc818640 1 --2- 172.21.15.89:0/1099801052 >> v2:172.21.15.154:3300/0 conn(0x7f52d80d0760 0x7f52d80d0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:53.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.856+0000 7f52dc818640 1 -- 172.21.15.89:0/1099801052 >> 172.21.15.89:0/1099801052 conn(0x7f52d8089fd0 msgr2=0x7f52d809b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:53.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.856+0000 7f52dc818640 1 -- 172.21.15.89:0/1099801052 shutdown_connections 2023-10-18T10:46:53.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.856+0000 7f52dc818640 1 -- 172.21.15.89:0/1099801052 wait complete. 2023-10-18T10:46:53.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.856+0000 7f52dc818640 1 Processor -- start 2023-10-18T10:46:53.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.856+0000 7f52dc818640 1 -- start start 2023-10-18T10:46:53.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.856+0000 7f52dc818640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52d80cd160 0x7f52d81f5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:53.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.860+0000 7f52dc818640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52d80d0760 0x7f52d814a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:53.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.860+0000 7f52dc818640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52d81f6390 0x7f52d814ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:53.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.860+0000 7f52dc818640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d80b7a10 con 0x7f52d80d0760 2023-10-18T10:46:53.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.860+0000 7f52dc818640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d81f59c0 con 0x7f52d81f6390 2023-10-18T10:46:53.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.860+0000 7f52dc818640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d814beb0 con 0x7f52d80cd160 2023-10-18T10:46:53.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.860+0000 7f52d5d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52d80d0760 0x7f52d814a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:53.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.860+0000 7f52d5d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52d80d0760 0x7f52d814a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57266/0 (socket says 172.21.15.89:57266) 2023-10-18T10:46:53.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.860+0000 7f52d5d9b640 1 -- 172.21.15.89:0/2827372367 learned_addr learned my addr 172.21.15.89:0/2827372367 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:53.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.860+0000 7f52d6d9d640 1 --2- 172.21.15.89:0/2827372367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52d81f6390 0x7f52d814ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:53.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.860+0000 7f52d659c640 1 --2- 172.21.15.89:0/2827372367 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52d80cd160 0x7f52d81f5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:53.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.860+0000 7f52d5d9b640 1 -- 172.21.15.89:0/2827372367 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52d80cd160 msgr2=0x7f52d81f5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:53.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.860+0000 7f52d5d9b640 1 --2- 172.21.15.89:0/2827372367 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52d80cd160 0x7f52d81f5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:53.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.860+0000 7f52d5d9b640 1 -- 172.21.15.89:0/2827372367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52d81f6390 msgr2=0x7f52d814ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:53.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.860+0000 7f52d5d9b640 1 --2- 172.21.15.89:0/2827372367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52d81f6390 0x7f52d814ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:53.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.860+0000 7f52d5d9b640 1 -- 172.21.15.89:0/2827372367 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d0009a70 con 0x7f52d80d0760 2023-10-18T10:46:53.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.860+0000 7f52d6d9d640 1 --2- 172.21.15.89:0/2827372367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52d81f6390 0x7f52d814ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:53.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.860+0000 7f52d5d9b640 1 --2- 172.21.15.89:0/2827372367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52d80d0760 0x7f52d814a620 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f52cc00e960 tx=0x7f52cc00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:53.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.860+0000 7f52c6ffd640 1 -- 172.21.15.89:0/2827372367 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52cc009690 con 0x7f52d80d0760 2023-10-18T10:46:53.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.860+0000 7f52c6ffd640 1 -- 172.21.15.89:0/2827372367 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52cc00c9a0 con 0x7f52d80d0760 2023-10-18T10:46:53.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.860+0000 7f52c6ffd640 1 -- 172.21.15.89:0/2827372367 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52cc00cd10 con 0x7f52d80d0760 2023-10-18T10:46:53.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.860+0000 7f52dc818640 1 -- 172.21.15.89:0/2827372367 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d80b7730 con 0x7f52d80d0760 2023-10-18T10:46:53.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.860+0000 7f52dc818640 1 -- 172.21.15.89:0/2827372367 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d814bb30 con 0x7f52d80d0760 2023-10-18T10:46:53.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.860+0000 7f52dc818640 1 -- 172.21.15.89:0/2827372367 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52d81f54b0 con 0x7f52d80d0760 2023-10-18T10:46:53.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.860+0000 7f52c6ffd640 1 -- 172.21.15.89:0/2827372367 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f52cc013470 con 0x7f52d80d0760 2023-10-18T10:46:53.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.864+0000 7f52c6ffd640 1 --2- 172.21.15.89:0/2827372367 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f52b80656b0 0x7f52b8067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:53.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.864+0000 7f52c6ffd640 1 -- 172.21.15.89:0/2827372367 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f52cc083690 con 0x7f52d80d0760 2023-10-18T10:46:53.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.864+0000 7f52d659c640 1 --2- 172.21.15.89:0/2827372367 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f52b80656b0 0x7f52b8067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:53.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.864+0000 7f52d659c640 1 --2- 172.21.15.89:0/2827372367 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f52b80656b0 0x7f52b8067b70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f52c80011f0 tx=0x7f52c8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:53.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:53.864+0000 7f52c6ffd640 1 -- 172.21.15.89:0/2827372367 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f52d81f54b0 con 0x7f52d80d0760 2023-10-18T10:46:54.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.028+0000 7f52dc818640 1 -- 172.21.15.89:0/2827372367 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f52d81f54b0 con 0x7f52d80d0760 2023-10-18T10:46:54.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.028+0000 7f52c6ffd640 1 -- 172.21.15.89:0/2827372367 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7f52d81f54b0 con 0x7f52d80d0760 2023-10-18T10:46:54.028 INFO:teuthology.orchestra.run.smithi089.stderr:listed 0 entries 2023-10-18T10:46:54.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.028+0000 7f52dc818640 1 -- 172.21.15.89:0/2827372367 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f52b80656b0 msgr2=0x7f52b8067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:54.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.028+0000 7f52dc818640 1 --2- 172.21.15.89:0/2827372367 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f52b80656b0 0x7f52b8067b70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f52c80011f0 tx=0x7f52c8009040 comp rx=0 tx=0).stop 2023-10-18T10:46:54.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.028+0000 7f52dc818640 1 -- 172.21.15.89:0/2827372367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52d80d0760 msgr2=0x7f52d814a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:54.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.028+0000 7f52dc818640 1 --2- 172.21.15.89:0/2827372367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52d80d0760 0x7f52d814a620 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f52cc00e960 tx=0x7f52cc00ee20 comp rx=0 tx=0).stop 2023-10-18T10:46:54.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.028+0000 7f52dc818640 1 -- 172.21.15.89:0/2827372367 shutdown_connections 2023-10-18T10:46:54.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.028+0000 7f52dc818640 1 --2- 172.21.15.89:0/2827372367 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f52b80656b0 0x7f52b8067b70 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:54.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.028+0000 7f52dc818640 1 --2- 172.21.15.89:0/2827372367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52d81f6390 0x7f52d814ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:54.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.028+0000 7f52dc818640 1 --2- 172.21.15.89:0/2827372367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f52d80d0760 0x7f52d814a620 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:54.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.028+0000 7f52dc818640 1 --2- 172.21.15.89:0/2827372367 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52d80cd160 0x7f52d81f5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:54.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.028+0000 7f52dc818640 1 -- 172.21.15.89:0/2827372367 >> 172.21.15.89:0/2827372367 conn(0x7f52d8089fd0 msgr2=0x7f52d815fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:54.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.028+0000 7f52dc818640 1 -- 172.21.15.89:0/2827372367 shutdown_connections 2023-10-18T10:46:54.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.028+0000 7f52dc818640 1 -- 172.21.15.89:0/2827372367 wait complete. 2023-10-18T10:46:54.049 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-18T10:46:54.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:54.049 DEBUG:teuthology.orchestra.run.smithi154:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-10-18T10:46:54.099 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2023-10-18T10:46:54.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:54.100 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-18T10:46:54.118 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:46:54.119 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:46:54.128 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:46:54.129 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:46:54.179 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:46:54.179 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:46:54.231 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:46:54.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.412+0000 7f00409d2640 1 Processor -- start 2023-10-18T10:46:54.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.412+0000 7f00409d2640 1 -- start start 2023-10-18T10:46:54.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.412+0000 7f00409d2640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f003c15c5a0 0x7f003c1602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:54.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.412+0000 7f00409d2640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f003c0b5610 con 0x7f003c0b7f00 2023-10-18T10:46:54.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.412+0000 7f00409d2640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f003c160cc0 con 0x7f003c160820 2023-10-18T10:46:54.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.412+0000 7f00409d2640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f003c161280 con 0x7f003c15c5a0 2023-10-18T10:46:54.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.412+0000 7f003b7fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f003c15c5a0 0x7f003c1602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:54.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.412+0000 7f003affd640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f003c0b7f00 0x7f003c0cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58136/0 (socket says 172.21.15.89:58136) 2023-10-18T10:46:54.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.412+0000 7f003affd640 1 -- 172.21.15.89:0/2755408346 learned_addr learned my addr 172.21.15.89:0/2755408346 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:54.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.412+0000 7f003b7fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f003c15c5a0 0x7f003c1602e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57282/0 (socket says 172.21.15.89:57282) 2023-10-18T10:46:54.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.412+0000 7f003b7fe640 1 -- 172.21.15.89:0/2755408346 >> v1:172.21.15.154:6789/0 conn(0x7f003c160820 legacy=0x7f003c0d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:54.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.412+0000 7f003b7fe640 1 -- 172.21.15.89:0/2755408346 >> v1:172.21.15.89:6789/0 conn(0x7f003c0b7f00 legacy=0x7f003c0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:54.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.412+0000 7f003b7fe640 1 -- 172.21.15.89:0/2755408346 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f003c160fa0 con 0x7f003c15c5a0 2023-10-18T10:46:54.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.412+0000 7f003a7fc640 1 -- 172.21.15.89:0/2755408346 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 978770995 0 0) 0x7f003c0b5610 con 0x7f003c0b7f00 2023-10-18T10:46:54.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.416+0000 7f003b7fe640 1 --2- 172.21.15.89:0/2755408346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f003c15c5a0 0x7f003c1602e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f002c0097c0 tx=0x7f002c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd41ee2471ee535f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:54.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.416+0000 7f003a7fc640 1 -- 172.21.15.89:0/2755408346 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002c0ab610 con 0x7f003c15c5a0 2023-10-18T10:46:54.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.416+0000 7f003a7fc640 1 -- 172.21.15.89:0/2755408346 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f002c0ab770 con 0x7f003c15c5a0 2023-10-18T10:46:54.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.416+0000 7f003a7fc640 1 -- 172.21.15.89:0/2755408346 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002c0abb00 con 0x7f003c15c5a0 2023-10-18T10:46:54.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.416+0000 7f00409d2640 1 -- 172.21.15.89:0/2755408346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f003c15c5a0 msgr2=0x7f003c1602e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:54.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.416+0000 7f00409d2640 1 --2- 172.21.15.89:0/2755408346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f003c15c5a0 0x7f003c1602e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f002c0097c0 tx=0x7f002c01c410 comp rx=0 tx=0).stop 2023-10-18T10:46:54.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.416+0000 7f00409d2640 1 -- 172.21.15.89:0/2755408346 shutdown_connections 2023-10-18T10:46:54.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.416+0000 7f00409d2640 1 --2- 172.21.15.89:0/2755408346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f003c15c5a0 0x7f003c1602e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:54.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.416+0000 7f00409d2640 1 -- 172.21.15.89:0/2755408346 >> 172.21.15.89:0/2755408346 conn(0x7f003c0ceef0 msgr2=0x7f003c0cf300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:54.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.416+0000 7f00409d2640 1 -- 172.21.15.89:0/2755408346 shutdown_connections 2023-10-18T10:46:54.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.416+0000 7f00409d2640 1 -- 172.21.15.89:0/2755408346 wait complete. 2023-10-18T10:46:54.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.416+0000 7f00409d2640 1 Processor -- start 2023-10-18T10:46:54.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.416+0000 7f00409d2640 1 -- start start 2023-10-18T10:46:54.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.420+0000 7f00409d2640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f003c0b7f00 0x7f003c13d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:54.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.420+0000 7f00409d2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f003c15c5a0 0x7f003c14f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:54.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.420+0000 7f00409d2640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f003c160820 0x7f003c0c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:54.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.420+0000 7f00409d2640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003c1f7f20 con 0x7f003c0b7f00 2023-10-18T10:46:54.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.420+0000 7f00409d2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003c14f5c0 con 0x7f003c15c5a0 2023-10-18T10:46:54.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.420+0000 7f00409d2640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f003c0c38d0 con 0x7f003c160820 2023-10-18T10:46:54.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.420+0000 7f003b7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f003c0b7f00 0x7f003c13d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:54.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.420+0000 7f003b7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f003c0b7f00 0x7f003c13d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57296/0 (socket says 172.21.15.89:57296) 2023-10-18T10:46:54.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.420+0000 7f003b7fe640 1 -- 172.21.15.89:0/1030231846 learned_addr learned my addr 172.21.15.89:0/1030231846 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:54.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.420+0000 7f003affd640 1 --2- 172.21.15.89:0/1030231846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f003c15c5a0 0x7f003c14f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:54.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.420+0000 7f003bfff640 1 --2- 172.21.15.89:0/1030231846 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f003c160820 0x7f003c0c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:54.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.420+0000 7f003b7fe640 1 -- 172.21.15.89:0/1030231846 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f003c160820 msgr2=0x7f003c0c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:54.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.420+0000 7f003b7fe640 1 --2- 172.21.15.89:0/1030231846 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f003c160820 0x7f003c0c6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:54.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.420+0000 7f003b7fe640 1 -- 172.21.15.89:0/1030231846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f003c15c5a0 msgr2=0x7f003c14f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:54.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.420+0000 7f003b7fe640 1 --2- 172.21.15.89:0/1030231846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f003c15c5a0 0x7f003c14f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:54.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.420+0000 7f003b7fe640 1 -- 172.21.15.89:0/1030231846 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f002c009420 con 0x7f003c0b7f00 2023-10-18T10:46:54.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.420+0000 7f003b7fe640 1 --2- 172.21.15.89:0/1030231846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f003c0b7f00 0x7f003c13d950 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f002c0ad040 tx=0x7f002c01ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:54.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.420+0000 7f0023fff640 1 -- 172.21.15.89:0/1030231846 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002c0b5810 con 0x7f003c0b7f00 2023-10-18T10:46:54.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.420+0000 7f0023fff640 1 -- 172.21.15.89:0/1030231846 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f002c0b5990 con 0x7f003c0b7f00 2023-10-18T10:46:54.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.420+0000 7f0023fff640 1 -- 172.21.15.89:0/1030231846 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002c0b5d40 con 0x7f003c0b7f00 2023-10-18T10:46:54.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.420+0000 7f00409d2640 1 -- 172.21.15.89:0/1030231846 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f003c0c3b70 con 0x7f003c0b7f00 2023-10-18T10:46:54.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.420+0000 7f00409d2640 1 -- 172.21.15.89:0/1030231846 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f003c0c4130 con 0x7f003c0b7f00 2023-10-18T10:46:54.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.420+0000 7f00409d2640 1 -- 172.21.15.89:0/1030231846 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f003c0cfe90 con 0x7f003c0b7f00 2023-10-18T10:46:54.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.424+0000 7f0023fff640 1 -- 172.21.15.89:0/1030231846 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f002c0bf410 con 0x7f003c0b7f00 2023-10-18T10:46:54.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.424+0000 7f0023fff640 1 --2- 172.21.15.89:0/1030231846 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0028065680 0x7f0028067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:54.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.424+0000 7f0023fff640 1 -- 172.21.15.89:0/1030231846 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f002c11dc30 con 0x7f003c0b7f00 2023-10-18T10:46:54.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.424+0000 7f003affd640 1 --2- 172.21.15.89:0/1030231846 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0028065680 0x7f0028067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:54.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.424+0000 7f003affd640 1 --2- 172.21.15.89:0/1030231846 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0028065680 0x7f0028067b40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f003c0c37a0 tx=0x7f0030009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:54.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.428+0000 7f0023fff640 1 -- 172.21.15.89:0/1030231846 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f002c0eb3d0 con 0x7f003c0b7f00 2023-10-18T10:46:54.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.576+0000 7f00409d2640 1 -- 172.21.15.89:0/1030231846 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f003c073b50 con 0x7f003c0b7f00 2023-10-18T10:46:54.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.576+0000 7f0023fff640 1 -- 172.21.15.89:0/1030231846 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20192 (secure 0 0 0) 0x7f002c0ee9f0 con 0x7f003c0b7f00 2023-10-18T10:46:54.578 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:54.578 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":29,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:46:36.958604+0000","last_up_change":"2023-10-18T10:45:42.164688+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:45:53.309122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-18T10:45:54.659763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:45:57.796514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6824","nonce":773078364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6827","nonce":773078364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6831","nonce":773078364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6829","nonce":773078364}]},"public_addr":"172.21.15.154:6824/773078364","cluster_addr":"172.21.15.154:6827/773078364","heartbeat_back_addr":"172.21.15.154:6831/773078364","heartbeat_front_addr":"172.21.15.154:6829/773078364","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":1,"to":5}]},{"pgid":"4.1a","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:46:54.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.576+0000 7f00409d2640 1 -- 172.21.15.89:0/1030231846 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0028065680 msgr2=0x7f0028067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:54.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.576+0000 7f00409d2640 1 --2- 172.21.15.89:0/1030231846 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0028065680 0x7f0028067b40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f003c0c37a0 tx=0x7f0030009040 comp rx=0 tx=0).stop 2023-10-18T10:46:54.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.576+0000 7f00409d2640 1 -- 172.21.15.89:0/1030231846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f003c0b7f00 msgr2=0x7f003c13d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:54.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.576+0000 7f00409d2640 1 --2- 172.21.15.89:0/1030231846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f003c0b7f00 0x7f003c13d950 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f002c0ad040 tx=0x7f002c01ac50 comp rx=0 tx=0).stop 2023-10-18T10:46:54.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.576+0000 7f00409d2640 1 -- 172.21.15.89:0/1030231846 shutdown_connections 2023-10-18T10:46:54.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.576+0000 7f00409d2640 1 --2- 172.21.15.89:0/1030231846 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0028065680 0x7f0028067b40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:54.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.576+0000 7f00409d2640 1 --2- 172.21.15.89:0/1030231846 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f003c160820 0x7f003c0c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:54.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.576+0000 7f00409d2640 1 --2- 172.21.15.89:0/1030231846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f003c15c5a0 0x7f003c14f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:54.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.576+0000 7f00409d2640 1 --2- 172.21.15.89:0/1030231846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f003c0b7f00 0x7f003c13d950 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:54.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.576+0000 7f00409d2640 1 -- 172.21.15.89:0/1030231846 >> 172.21.15.89:0/1030231846 conn(0x7f003c0ceef0 msgr2=0x7f003c1527b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:54.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.576+0000 7f00409d2640 1 -- 172.21.15.89:0/1030231846 shutdown_connections 2023-10-18T10:46:54.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.576+0000 7f00409d2640 1 -- 172.21.15.89:0/1030231846 wait complete. 2023-10-18T10:46:54.600 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:45:43.114184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-18T10:45:48.730968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:45:53.309122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:45:54.659763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T10:45:57.796514+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T10:46:54.601 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:46:54.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.740+0000 7f301fed8640 1 Processor -- start 2023-10-18T10:46:54.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.740+0000 7f301fed8640 1 -- start start 2023-10-18T10:46:54.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.740+0000 7f301fed8640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f301815c670 0x7f30180cce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:54.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.740+0000 7f301fed8640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f30180cfca0 0x7f3018160280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:54.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.740+0000 7f301fed8640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30180b1ad0 con 0x7f30180cd3c0 2023-10-18T10:46:54.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.740+0000 7f301fed8640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30180cd800 con 0x7f301815c670 2023-10-18T10:46:54.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.740+0000 7f301fed8640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30180b55b0 con 0x7f30180cfca0 2023-10-18T10:46:54.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.740+0000 7f301dc74640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f301815c670 0x7f30180cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:54.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.740+0000 7f301d473640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f30180cd3c0 0x7f30180cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58138/0 (socket says 172.21.15.89:58138) 2023-10-18T10:46:54.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.740+0000 7f301d473640 1 -- 172.21.15.89:0/968051432 learned_addr learned my addr 172.21.15.89:0/968051432 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:54.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.740+0000 7f301dc74640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f301815c670 0x7f30180cce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57302/0 (socket says 172.21.15.89:57302) 2023-10-18T10:46:54.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.740+0000 7f301e475640 1 --2- 172.21.15.89:0/968051432 >> v2:172.21.15.154:3300/0 conn(0x7f30180cfca0 0x7f3018160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:54.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.740+0000 7f301dc74640 1 -- 172.21.15.89:0/968051432 >> v2:172.21.15.154:3300/0 conn(0x7f30180cfca0 msgr2=0x7f3018160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:54.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.740+0000 7f301dc74640 1 --2- 172.21.15.89:0/968051432 >> v2:172.21.15.154:3300/0 conn(0x7f30180cfca0 0x7f3018160280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:54.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.740+0000 7f301dc74640 1 -- 172.21.15.89:0/968051432 >> v1:172.21.15.89:6789/0 conn(0x7f30180cd3c0 legacy=0x7f30180cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:54.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.740+0000 7f301dc74640 1 -- 172.21.15.89:0/968051432 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30180b40a0 con 0x7f301815c670 2023-10-18T10:46:54.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.744+0000 7f301dc74640 1 --2- 172.21.15.89:0/968051432 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f301815c670 0x7f30180cce80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f30180b1e80 tx=0x7f301401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d02a72e7e51e0d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:54.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.744+0000 7f301cc72640 1 -- 172.21.15.89:0/968051432 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30140a59a0 con 0x7f301815c670 2023-10-18T10:46:54.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.744+0000 7f301cc72640 1 -- 172.21.15.89:0/968051432 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30140a5b00 con 0x7f301815c670 2023-10-18T10:46:54.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.744+0000 7f301cc72640 1 -- 172.21.15.89:0/968051432 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30140a5e90 con 0x7f301815c670 2023-10-18T10:46:54.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.744+0000 7f301fed8640 1 -- 172.21.15.89:0/968051432 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f301815c670 msgr2=0x7f30180cce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:54.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.744+0000 7f301fed8640 1 --2- 172.21.15.89:0/968051432 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f301815c670 0x7f30180cce80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f30180b1e80 tx=0x7f301401c8e0 comp rx=0 tx=0).stop 2023-10-18T10:46:54.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.744+0000 7f301fed8640 1 -- 172.21.15.89:0/968051432 shutdown_connections 2023-10-18T10:46:54.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.744+0000 7f301fed8640 1 --2- 172.21.15.89:0/968051432 >> v2:172.21.15.154:3300/0 conn(0x7f30180cfca0 0x7f3018160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:54.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.744+0000 7f301fed8640 1 --2- 172.21.15.89:0/968051432 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f301815c670 0x7f30180cce80 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:54.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.744+0000 7f301fed8640 1 -- 172.21.15.89:0/968051432 >> 172.21.15.89:0/968051432 conn(0x7f3018070320 msgr2=0x7f3018070730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:54.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.744+0000 7f301fed8640 1 -- 172.21.15.89:0/968051432 shutdown_connections 2023-10-18T10:46:54.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.744+0000 7f301fed8640 1 -- 172.21.15.89:0/968051432 wait complete. 2023-10-18T10:46:54.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.748+0000 7f301fed8640 1 Processor -- start 2023-10-18T10:46:54.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.748+0000 7f301fed8640 1 -- start start 2023-10-18T10:46:54.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.748+0000 7f301fed8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30180cd3c0 0x7f3018156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:54.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.748+0000 7f301fed8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f30180cfca0 0x7f30181554a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:54.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.748+0000 7f301fed8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f301815c670 0x7f3018153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:54.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.748+0000 7f301fed8640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30180ca6c0 con 0x7f30180cd3c0 2023-10-18T10:46:54.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.748+0000 7f301fed8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301815cb80 con 0x7f301815c670 2023-10-18T10:46:54.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.748+0000 7f301fed8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30180c9b40 con 0x7f30180cfca0 2023-10-18T10:46:54.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.748+0000 7f301dc74640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30180cd3c0 0x7f3018156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:54.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.748+0000 7f301dc74640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30180cd3c0 0x7f3018156c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57312/0 (socket says 172.21.15.89:57312) 2023-10-18T10:46:54.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.748+0000 7f301dc74640 1 -- 172.21.15.89:0/762043409 learned_addr learned my addr 172.21.15.89:0/762043409 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:54.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.748+0000 7f301dc74640 1 -- 172.21.15.89:0/762043409 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f30180cfca0 msgr2=0x7f30181554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:54.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.748+0000 7f301dc74640 1 --2- 172.21.15.89:0/762043409 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f30180cfca0 0x7f30181554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:54.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.748+0000 7f301dc74640 1 -- 172.21.15.89:0/762043409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f301815c670 msgr2=0x7f3018153740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:54.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.748+0000 7f301dc74640 1 --2- 172.21.15.89:0/762043409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f301815c670 0x7f3018153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:54.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.748+0000 7f301dc74640 1 -- 172.21.15.89:0/762043409 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3014009a70 con 0x7f30180cd3c0 2023-10-18T10:46:54.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.748+0000 7f301dc74640 1 --2- 172.21.15.89:0/762043409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30180cd3c0 0x7f3018156c30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f30140194f0 tx=0x7f3014019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:54.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.748+0000 7f300e7fc640 1 -- 172.21.15.89:0/762043409 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30140a59a0 con 0x7f30180cd3c0 2023-10-18T10:46:54.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.748+0000 7f300e7fc640 1 -- 172.21.15.89:0/762043409 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3014009450 con 0x7f30180cd3c0 2023-10-18T10:46:54.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.748+0000 7f301fed8640 1 -- 172.21.15.89:0/762043409 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30180c9e20 con 0x7f30180cd3c0 2023-10-18T10:46:54.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.748+0000 7f300e7fc640 1 -- 172.21.15.89:0/762043409 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30140b9690 con 0x7f30180cd3c0 2023-10-18T10:46:54.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.748+0000 7f301fed8640 1 -- 172.21.15.89:0/762043409 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30180ca3e0 con 0x7f30180cd3c0 2023-10-18T10:46:54.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.752+0000 7f301fed8640 1 -- 172.21.15.89:0/762043409 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f301815cd40 con 0x7f30180cd3c0 2023-10-18T10:46:54.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.752+0000 7f300e7fc640 1 -- 172.21.15.89:0/762043409 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f30140b97f0 con 0x7f30180cd3c0 2023-10-18T10:46:54.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.752+0000 7f300e7fc640 1 --2- 172.21.15.89:0/762043409 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2ffc0656d0 0x7f2ffc067b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:54.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.752+0000 7f300e7fc640 1 -- 172.21.15.89:0/762043409 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f301411e960 con 0x7f30180cd3c0 2023-10-18T10:46:54.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.752+0000 7f301d473640 1 --2- 172.21.15.89:0/762043409 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2ffc0656d0 0x7f2ffc067b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:54.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.752+0000 7f301d473640 1 --2- 172.21.15.89:0/762043409 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2ffc0656d0 0x7f2ffc067b90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f30181f84d0 tx=0x7f3008007a50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:54.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.756+0000 7f300e7fc640 1 -- 172.21.15.89:0/762043409 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f30140ec140 con 0x7f30180cd3c0 2023-10-18T10:46:54.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.904+0000 7f301fed8640 1 -- 172.21.15.89:0/762043409 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f301804dbe0 con 0x7f30180cd3c0 2023-10-18T10:46:54.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.904+0000 7f300e7fc640 1 -- 172.21.15.89:0/762043409 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f30140b20a0 con 0x7f30180cd3c0 2023-10-18T10:46:54.906 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-18T10:46:54.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.908+0000 7f301fed8640 1 -- 172.21.15.89:0/762043409 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2ffc0656d0 msgr2=0x7f2ffc067b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:54.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.908+0000 7f301fed8640 1 --2- 172.21.15.89:0/762043409 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2ffc0656d0 0x7f2ffc067b90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f30181f84d0 tx=0x7f3008007a50 comp rx=0 tx=0).stop 2023-10-18T10:46:54.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.908+0000 7f301fed8640 1 -- 172.21.15.89:0/762043409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30180cd3c0 msgr2=0x7f3018156c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:54.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.908+0000 7f301fed8640 1 --2- 172.21.15.89:0/762043409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30180cd3c0 0x7f3018156c30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f30140194f0 tx=0x7f3014019680 comp rx=0 tx=0).stop 2023-10-18T10:46:54.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.908+0000 7f301fed8640 1 -- 172.21.15.89:0/762043409 shutdown_connections 2023-10-18T10:46:54.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.908+0000 7f301fed8640 1 --2- 172.21.15.89:0/762043409 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2ffc0656d0 0x7f2ffc067b90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:54.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.908+0000 7f301fed8640 1 --2- 172.21.15.89:0/762043409 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f30180cfca0 0x7f30181554a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:54.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.908+0000 7f301fed8640 1 --2- 172.21.15.89:0/762043409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f301815c670 0x7f3018153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:54.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.908+0000 7f301fed8640 1 --2- 172.21.15.89:0/762043409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30180cd3c0 0x7f3018156c30 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:54.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.908+0000 7f301fed8640 1 -- 172.21.15.89:0/762043409 >> 172.21.15.89:0/762043409 conn(0x7f3018070320 msgr2=0x7f30180b6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:54.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.908+0000 7f301fed8640 1 -- 172.21.15.89:0/762043409 shutdown_connections 2023-10-18T10:46:54.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:54.908+0000 7f301fed8640 1 -- 172.21.15.89:0/762043409 wait complete. 2023-10-18T10:46:54.927 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-18T10:46:55.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.068+0000 7f47aa448640 1 Processor -- start 2023-10-18T10:46:55.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.068+0000 7f47aa448640 1 -- start start 2023-10-18T10:46:55.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.068+0000 7f47aa448640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47a4070600 0x7f47a406b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:55.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.068+0000 7f47aa448640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f47a406bd10 0x7f47a40c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:55.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.068+0000 7f47aa448640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47a4070b90 con 0x7f47a4159490 2023-10-18T10:46:55.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.068+0000 7f47aa448640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47a406c1a0 con 0x7f47a406bd10 2023-10-18T10:46:55.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.068+0000 7f47aa448640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a4070e70 con 0x7f47a4070600 2023-10-18T10:46:55.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.068+0000 7f47a3fff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f47a4159490 0x7f47a415d8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58142/0 (socket says 172.21.15.89:58142) 2023-10-18T10:46:55.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.068+0000 7f47a3fff640 1 -- 172.21.15.89:0/1805689139 learned_addr learned my addr 172.21.15.89:0/1805689139 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:55.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.068+0000 7f47a37fe640 1 --2- 172.21.15.89:0/1805689139 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47a4070600 0x7f47a406b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:55.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.068+0000 7f47a89e5640 1 --2- 172.21.15.89:0/1805689139 >> v2:172.21.15.154:3300/0 conn(0x7f47a406bd10 0x7f47a40c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:55.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.068+0000 7f47a2ffd640 1 -- 172.21.15.89:0/1805689139 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1115169853 0 0) 0x7f47a4070b90 con 0x7f47a4159490 2023-10-18T10:46:55.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.068+0000 7f47a2ffd640 1 -- 172.21.15.89:0/1805689139 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f478c003530 con 0x7f47a4159490 2023-10-18T10:46:55.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.068+0000 7f47a37fe640 1 -- 172.21.15.89:0/1805689139 >> v2:172.21.15.154:3300/0 conn(0x7f47a406bd10 msgr2=0x7f47a40c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:55.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.068+0000 7f47a37fe640 1 --2- 172.21.15.89:0/1805689139 >> v2:172.21.15.154:3300/0 conn(0x7f47a406bd10 0x7f47a40c8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:55.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.068+0000 7f47a37fe640 1 -- 172.21.15.89:0/1805689139 >> v1:172.21.15.89:6789/0 conn(0x7f47a4159490 legacy=0x7f47a415d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:55.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.068+0000 7f47a37fe640 1 -- 172.21.15.89:0/1805689139 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a4071150 con 0x7f47a4070600 2023-10-18T10:46:55.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.068+0000 7f47a89e5640 1 --2- 172.21.15.89:0/1805689139 >> v2:172.21.15.154:3300/0 conn(0x7f47a406bd10 0x7f47a40c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:46:55.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.072+0000 7f47a37fe640 1 --2- 172.21.15.89:0/1805689139 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47a4070600 0x7f47a406b7d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f47980097c0 tx=0x7f479801c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5383009c2d2f5d5a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:55.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.072+0000 7f47a2ffd640 1 -- 172.21.15.89:0/1805689139 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47980ac620 con 0x7f47a4070600 2023-10-18T10:46:55.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.072+0000 7f47a2ffd640 1 -- 172.21.15.89:0/1805689139 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47980ac780 con 0x7f47a4070600 2023-10-18T10:46:55.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.072+0000 7f47a2ffd640 1 -- 172.21.15.89:0/1805689139 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47980acb10 con 0x7f47a4070600 2023-10-18T10:46:55.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.072+0000 7f47aa448640 1 -- 172.21.15.89:0/1805689139 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47a4070600 msgr2=0x7f47a406b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:55.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.072+0000 7f47aa448640 1 --2- 172.21.15.89:0/1805689139 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47a4070600 0x7f47a406b7d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f47980097c0 tx=0x7f479801c3c0 comp rx=0 tx=0).stop 2023-10-18T10:46:55.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.072+0000 7f47aa448640 1 -- 172.21.15.89:0/1805689139 shutdown_connections 2023-10-18T10:46:55.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.072+0000 7f47aa448640 1 --2- 172.21.15.89:0/1805689139 >> v2:172.21.15.154:3300/0 conn(0x7f47a406bd10 0x7f47a40c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:55.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.072+0000 7f47aa448640 1 --2- 172.21.15.89:0/1805689139 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47a4070600 0x7f47a406b7d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:55.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.072+0000 7f47aa448640 1 -- 172.21.15.89:0/1805689139 >> 172.21.15.89:0/1805689139 conn(0x7f47a4069f00 msgr2=0x7f47a40cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:55.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.072+0000 7f47aa448640 1 -- 172.21.15.89:0/1805689139 shutdown_connections 2023-10-18T10:46:55.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.072+0000 7f47aa448640 1 -- 172.21.15.89:0/1805689139 wait complete. 2023-10-18T10:46:55.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.076+0000 7f47aa448640 1 Processor -- start 2023-10-18T10:46:55.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.076+0000 7f47aa448640 1 -- start start 2023-10-18T10:46:55.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.076+0000 7f47aa448640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47a406bd10 0x7f47a4166e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:55.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.076+0000 7f47aa448640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47a4070600 0x7f47a40c2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:55.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.076+0000 7f47aa448640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47a4159490 0x7f47a415ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:55.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.076+0000 7f47aa448640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a40c3180 con 0x7f47a4159490 2023-10-18T10:46:55.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.076+0000 7f47aa448640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a40c3460 con 0x7f47a406bd10 2023-10-18T10:46:55.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.076+0000 7f47aa448640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a40c3740 con 0x7f47a4070600 2023-10-18T10:46:55.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.076+0000 7f47a89e5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47a4159490 0x7f47a415ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:55.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.076+0000 7f47a89e5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47a4159490 0x7f47a415ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57326/0 (socket says 172.21.15.89:57326) 2023-10-18T10:46:55.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.076+0000 7f47a89e5640 1 -- 172.21.15.89:0/2891605188 learned_addr learned my addr 172.21.15.89:0/2891605188 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:55.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.076+0000 7f47a37fe640 1 --2- 172.21.15.89:0/2891605188 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47a4070600 0x7f47a40c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:55.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.076+0000 7f47a3fff640 1 --2- 172.21.15.89:0/2891605188 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47a406bd10 0x7f47a4166e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:55.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.076+0000 7f47a89e5640 1 -- 172.21.15.89:0/2891605188 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47a4070600 msgr2=0x7f47a40c2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:55.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.076+0000 7f47a89e5640 1 --2- 172.21.15.89:0/2891605188 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47a4070600 0x7f47a40c2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:55.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.076+0000 7f47a89e5640 1 -- 172.21.15.89:0/2891605188 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47a406bd10 msgr2=0x7f47a4166e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:55.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.076+0000 7f47a89e5640 1 --2- 172.21.15.89:0/2891605188 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47a406bd10 0x7f47a4166e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:55.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.076+0000 7f47a89e5640 1 -- 172.21.15.89:0/2891605188 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479c009a70 con 0x7f47a4159490 2023-10-18T10:46:55.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.076+0000 7f47a3fff640 1 --2- 172.21.15.89:0/2891605188 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47a406bd10 0x7f47a4166e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:55.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.076+0000 7f47a89e5640 1 --2- 172.21.15.89:0/2891605188 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47a4159490 0x7f47a415ba40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f479c00efc0 tx=0x7f479c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:55.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.076+0000 7f47a0ff9640 1 -- 172.21.15.89:0/2891605188 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c009cc0 con 0x7f47a4159490 2023-10-18T10:46:55.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.076+0000 7f47a0ff9640 1 -- 172.21.15.89:0/2891605188 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f479c00f040 con 0x7f47a4159490 2023-10-18T10:46:55.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.076+0000 7f47aa448640 1 -- 172.21.15.89:0/2891605188 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4798009420 con 0x7f47a4159490 2023-10-18T10:46:55.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.076+0000 7f47a0ff9640 1 -- 172.21.15.89:0/2891605188 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c0039a0 con 0x7f47a4159490 2023-10-18T10:46:55.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.076+0000 7f47aa448640 1 -- 172.21.15.89:0/2891605188 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a40c2d90 con 0x7f47a4159490 2023-10-18T10:46:55.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.080+0000 7f47aa448640 1 -- 172.21.15.89:0/2891605188 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47a41651f0 con 0x7f47a4159490 2023-10-18T10:46:55.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.080+0000 7f47a0ff9640 1 -- 172.21.15.89:0/2891605188 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f479c014480 con 0x7f47a4159490 2023-10-18T10:46:55.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.080+0000 7f47a0ff9640 1 --2- 172.21.15.89:0/2891605188 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4790065680 0x7f4790067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:55.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.080+0000 7f47a0ff9640 1 -- 172.21.15.89:0/2891605188 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f479c083cc0 con 0x7f47a4159490 2023-10-18T10:46:55.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.080+0000 7f47a3fff640 1 --2- 172.21.15.89:0/2891605188 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4790065680 0x7f4790067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:55.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.084+0000 7f47a3fff640 1 --2- 172.21.15.89:0/2891605188 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4790065680 0x7f4790067b40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f47940011f0 tx=0x7f4794009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:55.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.084+0000 7f47a0ff9640 1 -- 172.21.15.89:0/2891605188 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f47a41651f0 con 0x7f47a4159490 2023-10-18T10:46:55.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.236+0000 7f47aa448640 1 -- 172.21.15.89:0/2891605188 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f47a41651f0 con 0x7f47a4159490 2023-10-18T10:46:55.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.236+0000 7f47a0ff9640 1 -- 172.21.15.89:0/2891605188 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f479c018020 con 0x7f47a4159490 2023-10-18T10:46:55.238 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-10-18T10:46:55.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.240+0000 7f47aa448640 1 -- 172.21.15.89:0/2891605188 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4790065680 msgr2=0x7f4790067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:55.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.240+0000 7f47aa448640 1 --2- 172.21.15.89:0/2891605188 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4790065680 0x7f4790067b40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f47940011f0 tx=0x7f4794009040 comp rx=0 tx=0).stop 2023-10-18T10:46:55.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.240+0000 7f47aa448640 1 -- 172.21.15.89:0/2891605188 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47a4159490 msgr2=0x7f47a415ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:55.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.240+0000 7f47aa448640 1 --2- 172.21.15.89:0/2891605188 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47a4159490 0x7f47a415ba40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f479c00efc0 tx=0x7f479c00c490 comp rx=0 tx=0).stop 2023-10-18T10:46:55.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.240+0000 7f47aa448640 1 -- 172.21.15.89:0/2891605188 shutdown_connections 2023-10-18T10:46:55.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.240+0000 7f47aa448640 1 --2- 172.21.15.89:0/2891605188 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4790065680 0x7f4790067b40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:55.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.240+0000 7f47aa448640 1 --2- 172.21.15.89:0/2891605188 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47a4070600 0x7f47a40c2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:55.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.240+0000 7f47aa448640 1 --2- 172.21.15.89:0/2891605188 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47a4159490 0x7f47a415ba40 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:55.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.240+0000 7f47aa448640 1 --2- 172.21.15.89:0/2891605188 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47a406bd10 0x7f47a4166e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:55.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.240+0000 7f47aa448640 1 -- 172.21.15.89:0/2891605188 >> 172.21.15.89:0/2891605188 conn(0x7f47a4069f00 msgr2=0x7f47a40ae5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:55.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.240+0000 7f47aa448640 1 -- 172.21.15.89:0/2891605188 shutdown_connections 2023-10-18T10:46:55.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.240+0000 7f47aa448640 1 -- 172.21.15.89:0/2891605188 wait complete. 2023-10-18T10:46:55.259 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:46:55.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.400+0000 7f28b3cca640 1 Processor -- start 2023-10-18T10:46:55.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.400+0000 7f28b3cca640 1 -- start start 2023-10-18T10:46:55.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.400+0000 7f28b3cca640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f28ac15e600 0x7f28ac0c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:55.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.400+0000 7f28b3cca640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28ac1614e0 con 0x7f28ac06c1b0 2023-10-18T10:46:55.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.400+0000 7f28b3cca640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28ac06c640 con 0x7f28ac0afd90 2023-10-18T10:46:55.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.400+0000 7f28b3cca640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28ac160f80 con 0x7f28ac15e600 2023-10-18T10:46:55.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.400+0000 7f28b1265640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f28ac06c1b0 0x7f28ac161970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58146/0 (socket says 172.21.15.89:58146) 2023-10-18T10:46:55.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.400+0000 7f28b1265640 1 -- 172.21.15.89:0/2923984029 learned_addr learned my addr 172.21.15.89:0/2923984029 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:55.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.400+0000 7f28b1a66640 1 --2- 172.21.15.89:0/2923984029 >> v2:172.21.15.154:3300/0 conn(0x7f28ac15e600 0x7f28ac0c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:55.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.400+0000 7f28b0a64640 1 -- 172.21.15.89:0/2923984029 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3168344830 0 0) 0x7f28ac1614e0 con 0x7f28ac06c1b0 2023-10-18T10:46:55.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.400+0000 7f28b0a64640 1 -- 172.21.15.89:0/2923984029 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2898003530 con 0x7f28ac06c1b0 2023-10-18T10:46:55.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.400+0000 7f28b0a64640 1 -- 172.21.15.89:0/2923984029 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4098285799 0 0) 0x7f28ac06c640 con 0x7f28ac0afd90 2023-10-18T10:46:55.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.400+0000 7f28b0a64640 1 -- 172.21.15.89:0/2923984029 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28ac1614e0 con 0x7f28ac0afd90 2023-10-18T10:46:55.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.400+0000 7f28b1a66640 1 -- 172.21.15.89:0/2923984029 >> v1:172.21.15.154:6789/0 conn(0x7f28ac0afd90 legacy=0x7f28ac0b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:55.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.400+0000 7f28b1a66640 1 -- 172.21.15.89:0/2923984029 >> v1:172.21.15.89:6789/0 conn(0x7f28ac06c1b0 legacy=0x7f28ac161970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:55.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.400+0000 7f28b1a66640 1 -- 172.21.15.89:0/2923984029 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ac15e9e0 con 0x7f28ac15e600 2023-10-18T10:46:55.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.404+0000 7f28b1a66640 1 --2- 172.21.15.89:0/2923984029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28ac15e600 0x7f28ac0c8960 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f28a80097c0 tx=0x7f28a801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed63c579ace80742 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:55.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.404+0000 7f28b0a64640 1 -- 172.21.15.89:0/2923984029 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a80ab650 con 0x7f28ac15e600 2023-10-18T10:46:55.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.404+0000 7f28b0a64640 1 -- 172.21.15.89:0/2923984029 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28a80ab7b0 con 0x7f28ac15e600 2023-10-18T10:46:55.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.404+0000 7f28b0a64640 1 -- 172.21.15.89:0/2923984029 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a80abb20 con 0x7f28ac15e600 2023-10-18T10:46:55.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.404+0000 7f28b3cca640 1 -- 172.21.15.89:0/2923984029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28ac15e600 msgr2=0x7f28ac0c8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:55.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.404+0000 7f28b3cca640 1 --2- 172.21.15.89:0/2923984029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28ac15e600 0x7f28ac0c8960 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f28a80097c0 tx=0x7f28a801c450 comp rx=0 tx=0).stop 2023-10-18T10:46:55.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.404+0000 7f28b3cca640 1 -- 172.21.15.89:0/2923984029 shutdown_connections 2023-10-18T10:46:55.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.404+0000 7f28b3cca640 1 --2- 172.21.15.89:0/2923984029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28ac15e600 0x7f28ac0c8960 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:55.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.404+0000 7f28b3cca640 1 -- 172.21.15.89:0/2923984029 >> 172.21.15.89:0/2923984029 conn(0x7f28ac069f00 msgr2=0x7f28ac0cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:55.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.404+0000 7f28b3cca640 1 -- 172.21.15.89:0/2923984029 shutdown_connections 2023-10-18T10:46:55.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.404+0000 7f28b3cca640 1 -- 172.21.15.89:0/2923984029 wait complete. 2023-10-18T10:46:55.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.408+0000 7f28b3cca640 1 Processor -- start 2023-10-18T10:46:55.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.408+0000 7f28b3cca640 1 -- start start 2023-10-18T10:46:55.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.408+0000 7f28b3cca640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28ac06c1b0 0x7f28ac156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:55.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.408+0000 7f28b3cca640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28ac0afd90 0x7f28ac156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:55.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.408+0000 7f28b3cca640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28ac15e600 0x7f28ac154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:55.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.408+0000 7f28b3cca640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac0c7100 con 0x7f28ac0afd90 2023-10-18T10:46:55.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.408+0000 7f28b3cca640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac0c5780 con 0x7f28ac06c1b0 2023-10-18T10:46:55.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.408+0000 7f28b3cca640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac0c5a60 con 0x7f28ac15e600 2023-10-18T10:46:55.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.408+0000 7f28b1265640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28ac0afd90 0x7f28ac156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:55.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.408+0000 7f28b1265640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28ac0afd90 0x7f28ac156bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57330/0 (socket says 172.21.15.89:57330) 2023-10-18T10:46:55.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.408+0000 7f28b1265640 1 -- 172.21.15.89:0/603086007 learned_addr learned my addr 172.21.15.89:0/603086007 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:55.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.408+0000 7f28b1a66640 1 --2- 172.21.15.89:0/603086007 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28ac06c1b0 0x7f28ac156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:55.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.408+0000 7f28b2267640 1 --2- 172.21.15.89:0/603086007 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28ac15e600 0x7f28ac154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:55.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.408+0000 7f28b1265640 1 -- 172.21.15.89:0/603086007 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28ac15e600 msgr2=0x7f28ac154c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:55.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.408+0000 7f28b1265640 1 --2- 172.21.15.89:0/603086007 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28ac15e600 0x7f28ac154c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:55.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.408+0000 7f28b1265640 1 -- 172.21.15.89:0/603086007 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28ac06c1b0 msgr2=0x7f28ac156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:55.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.408+0000 7f28b1265640 1 --2- 172.21.15.89:0/603086007 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28ac06c1b0 0x7f28ac156690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:55.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.408+0000 7f28b1265640 1 -- 172.21.15.89:0/603086007 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a8009420 con 0x7f28ac0afd90 2023-10-18T10:46:55.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.408+0000 7f28b2267640 1 --2- 172.21.15.89:0/603086007 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28ac15e600 0x7f28ac154c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:55.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.408+0000 7f28b1265640 1 --2- 172.21.15.89:0/603086007 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28ac0afd90 0x7f28ac156bd0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f289c00c4f0 tx=0x7f289c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:55.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.408+0000 7f28a27fc640 1 -- 172.21.15.89:0/603086007 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289c005cf0 con 0x7f28ac0afd90 2023-10-18T10:46:55.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.408+0000 7f28a27fc640 1 -- 172.21.15.89:0/603086007 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f289c005e50 con 0x7f28ac0afd90 2023-10-18T10:46:55.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.408+0000 7f28a27fc640 1 -- 172.21.15.89:0/603086007 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289c007270 con 0x7f28ac0afd90 2023-10-18T10:46:55.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.408+0000 7f28b3cca640 1 -- 172.21.15.89:0/603086007 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28ac0c5d40 con 0x7f28ac0afd90 2023-10-18T10:46:55.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.412+0000 7f28b3cca640 1 -- 172.21.15.89:0/603086007 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28ac06f7f0 con 0x7f28ac0afd90 2023-10-18T10:46:55.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.412+0000 7f28b3cca640 1 -- 172.21.15.89:0/603086007 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28ac06fa10 con 0x7f28ac0afd90 2023-10-18T10:46:55.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.412+0000 7f28a27fc640 1 -- 172.21.15.89:0/603086007 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f289c0073d0 con 0x7f28ac0afd90 2023-10-18T10:46:55.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.412+0000 7f28a27fc640 1 --2- 172.21.15.89:0/603086007 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2890065680 0x7f2890067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:55.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.412+0000 7f28a27fc640 1 -- 172.21.15.89:0/603086007 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f289c082650 con 0x7f28ac0afd90 2023-10-18T10:46:55.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.412+0000 7f28b1a66640 1 --2- 172.21.15.89:0/603086007 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2890065680 0x7f2890067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:55.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.412+0000 7f28b1a66640 1 --2- 172.21.15.89:0/603086007 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2890065680 0x7f2890067b40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f28a80af000 tx=0x7f28a80a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:55.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.416+0000 7f28a27fc640 1 -- 172.21.15.89:0/603086007 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f289c04fe70 con 0x7f28ac0afd90 2023-10-18T10:46:55.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.564+0000 7f28b3cca640 1 -- 172.21.15.89:0/603086007 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f28ac04dbe0 con 0x7f28ac0afd90 2023-10-18T10:46:55.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.564+0000 7f28a27fc640 1 -- 172.21.15.89:0/603086007 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f289c053490 con 0x7f28ac0afd90 2023-10-18T10:46:55.566 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-18T10:46:55.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.568+0000 7f28b3cca640 1 -- 172.21.15.89:0/603086007 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2890065680 msgr2=0x7f2890067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:55.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.568+0000 7f28b3cca640 1 --2- 172.21.15.89:0/603086007 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2890065680 0x7f2890067b40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f28a80af000 tx=0x7f28a80a7000 comp rx=0 tx=0).stop 2023-10-18T10:46:55.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.568+0000 7f28b3cca640 1 -- 172.21.15.89:0/603086007 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28ac0afd90 msgr2=0x7f28ac156bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:55.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.568+0000 7f28b3cca640 1 --2- 172.21.15.89:0/603086007 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28ac0afd90 0x7f28ac156bd0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f289c00c4f0 tx=0x7f289c00c9b0 comp rx=0 tx=0).stop 2023-10-18T10:46:55.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.568+0000 7f28b3cca640 1 -- 172.21.15.89:0/603086007 shutdown_connections 2023-10-18T10:46:55.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.568+0000 7f28b3cca640 1 --2- 172.21.15.89:0/603086007 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2890065680 0x7f2890067b40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:55.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.568+0000 7f28b3cca640 1 --2- 172.21.15.89:0/603086007 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28ac15e600 0x7f28ac154c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:55.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.568+0000 7f28b3cca640 1 --2- 172.21.15.89:0/603086007 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28ac0afd90 0x7f28ac156bd0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:55.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.568+0000 7f28b3cca640 1 --2- 172.21.15.89:0/603086007 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28ac06c1b0 0x7f28ac156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:55.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.568+0000 7f28b3cca640 1 -- 172.21.15.89:0/603086007 >> 172.21.15.89:0/603086007 conn(0x7f28ac069f00 msgr2=0x7f28ac0af580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:55.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.568+0000 7f28b3cca640 1 -- 172.21.15.89:0/603086007 shutdown_connections 2023-10-18T10:46:55.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.568+0000 7f28b3cca640 1 -- 172.21.15.89:0/603086007 wait complete. 2023-10-18T10:46:55.587 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:46:55.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.728+0000 7f08c1eca640 1 Processor -- start 2023-10-18T10:46:55.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.728+0000 7f08c1eca640 1 -- start start 2023-10-18T10:46:55.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.728+0000 7f08c1eca640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08bc0cd4f0 0x7f08bc143470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:55.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.728+0000 7f08c1eca640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f08bc0ce070 0x7f08bc0ce500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:55.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.728+0000 7f08c1eca640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08bc0cd8d0 con 0x7f08bc145b70 2023-10-18T10:46:55.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.728+0000 7f08c1eca640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08bc0cea40 con 0x7f08bc0ce070 2023-10-18T10:46:55.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.728+0000 7f08c1eca640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc069900 con 0x7f08bc0cd4f0 2023-10-18T10:46:55.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.728+0000 7f08bbfff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f08bc0ce070 0x7f08bc0ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:55.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.728+0000 7f08bbfff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f08bc0ce070 0x7f08bc0ce500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57346/0 (socket says 172.21.15.89:57346) 2023-10-18T10:46:55.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.728+0000 7f08b2ffd640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f08bc145b70 0x7f08bc160cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58148/0 (socket says 172.21.15.89:58148) 2023-10-18T10:46:55.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.728+0000 7f08bbfff640 1 -- 172.21.15.89:0/2573124487 learned_addr learned my addr 172.21.15.89:0/2573124487 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:55.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.728+0000 7f08bb7fe640 1 --2- 172.21.15.89:0/2573124487 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08bc0cd4f0 0x7f08bc143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:55.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.728+0000 7f08bbfff640 1 -- 172.21.15.89:0/2573124487 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08bc0cd4f0 msgr2=0x7f08bc143470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:55.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.728+0000 7f08bbfff640 1 --2- 172.21.15.89:0/2573124487 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08bc0cd4f0 0x7f08bc143470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:55.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.728+0000 7f08bbfff640 1 -- 172.21.15.89:0/2573124487 >> v1:172.21.15.89:6789/0 conn(0x7f08bc145b70 legacy=0x7f08bc160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:55.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.728+0000 7f08bbfff640 1 -- 172.21.15.89:0/2573124487 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08bc15cbf0 con 0x7f08bc0ce070 2023-10-18T10:46:55.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.732+0000 7f08bbfff640 1 --2- 172.21.15.89:0/2573124487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08bc0ce070 0x7f08bc0ce500 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009e50 tx=0x7f08b401c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa7c1cf0eab383bc server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:55.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.732+0000 7f08baffd640 1 -- 172.21.15.89:0/2573124487 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b40a58a0 con 0x7f08bc0ce070 2023-10-18T10:46:55.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.732+0000 7f08baffd640 1 -- 172.21.15.89:0/2573124487 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b40a5a00 con 0x7f08bc0ce070 2023-10-18T10:46:55.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.732+0000 7f08baffd640 1 -- 172.21.15.89:0/2573124487 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b40a5d90 con 0x7f08bc0ce070 2023-10-18T10:46:55.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.732+0000 7f08c1eca640 1 -- 172.21.15.89:0/2573124487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08bc0ce070 msgr2=0x7f08bc0ce500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:55.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.732+0000 7f08c1eca640 1 --2- 172.21.15.89:0/2573124487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08bc0ce070 0x7f08bc0ce500 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009e50 tx=0x7f08b401c9e0 comp rx=0 tx=0).stop 2023-10-18T10:46:55.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.732+0000 7f08c1eca640 1 -- 172.21.15.89:0/2573124487 shutdown_connections 2023-10-18T10:46:55.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.732+0000 7f08c1eca640 1 --2- 172.21.15.89:0/2573124487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08bc0ce070 0x7f08bc0ce500 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:55.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.732+0000 7f08c1eca640 1 --2- 172.21.15.89:0/2573124487 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08bc0cd4f0 0x7f08bc143470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:55.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.732+0000 7f08c1eca640 1 -- 172.21.15.89:0/2573124487 >> 172.21.15.89:0/2573124487 conn(0x7f08bc06dad0 msgr2=0x7f08bc06dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:55.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.732+0000 7f08c1eca640 1 -- 172.21.15.89:0/2573124487 shutdown_connections 2023-10-18T10:46:55.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.732+0000 7f08c1eca640 1 -- 172.21.15.89:0/2573124487 wait complete. 2023-10-18T10:46:55.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.732+0000 7f08c1eca640 1 Processor -- start 2023-10-18T10:46:55.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.736+0000 7f08c1eca640 1 -- start start 2023-10-18T10:46:55.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.736+0000 7f08c1eca640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08bc0cd4f0 0x7f08bc0cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:55.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.736+0000 7f08c1eca640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f08bc0ce070 0x7f08bc0cc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:55.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.736+0000 7f08c1eca640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08bc145b70 0x7f08bc1536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:55.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.736+0000 7f08c1eca640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc0ca440 con 0x7f08bc145b70 2023-10-18T10:46:55.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.736+0000 7f08c1eca640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc0bcef0 con 0x7f08bc0ce070 2023-10-18T10:46:55.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.736+0000 7f08c1eca640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc15c020 con 0x7f08bc0cd4f0 2023-10-18T10:46:55.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.736+0000 7f08bbfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08bc145b70 0x7f08bc1536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:55.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.736+0000 7f08bbfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08bc145b70 0x7f08bc1536b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57362/0 (socket says 172.21.15.89:57362) 2023-10-18T10:46:55.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.736+0000 7f08bbfff640 1 -- 172.21.15.89:0/2105179773 learned_addr learned my addr 172.21.15.89:0/2105179773 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:55.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.736+0000 7f08b2ffd640 1 --2- 172.21.15.89:0/2105179773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f08bc0ce070 0x7f08bc0cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:55.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.736+0000 7f08bb7fe640 1 --2- 172.21.15.89:0/2105179773 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08bc0cd4f0 0x7f08bc0cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:55.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.736+0000 7f08bbfff640 1 -- 172.21.15.89:0/2105179773 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08bc0cd4f0 msgr2=0x7f08bc0cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:55.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.736+0000 7f08bbfff640 1 --2- 172.21.15.89:0/2105179773 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08bc0cd4f0 0x7f08bc0cbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:55.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.736+0000 7f08bbfff640 1 -- 172.21.15.89:0/2105179773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f08bc0ce070 msgr2=0x7f08bc0cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:55.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.736+0000 7f08bbfff640 1 --2- 172.21.15.89:0/2105179773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f08bc0ce070 0x7f08bc0cc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:55.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.736+0000 7f08bbfff640 1 -- 172.21.15.89:0/2105179773 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b4009b30 con 0x7f08bc145b70 2023-10-18T10:46:55.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.736+0000 7f08bbfff640 1 --2- 172.21.15.89:0/2105179773 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08bc145b70 0x7f08bc1536b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f08b40af040 tx=0x7f08b4019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:55.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.736+0000 7f08b8ff9640 1 -- 172.21.15.89:0/2105179773 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b40bd070 con 0x7f08bc145b70 2023-10-18T10:46:55.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.736+0000 7f08b8ff9640 1 -- 172.21.15.89:0/2105179773 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b40b8470 con 0x7f08bc145b70 2023-10-18T10:46:55.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.736+0000 7f08c1eca640 1 -- 172.21.15.89:0/2105179773 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08bc0c7f50 con 0x7f08bc145b70 2023-10-18T10:46:55.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.736+0000 7f08b8ff9640 1 -- 172.21.15.89:0/2105179773 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b40b8840 con 0x7f08bc145b70 2023-10-18T10:46:55.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.736+0000 7f08c1eca640 1 -- 172.21.15.89:0/2105179773 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08bc15b4c0 con 0x7f08bc145b70 2023-10-18T10:46:55.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.736+0000 7f08c1eca640 1 -- 172.21.15.89:0/2105179773 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08bc04dbe0 con 0x7f08bc145b70 2023-10-18T10:46:55.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.740+0000 7f08b8ff9640 1 -- 172.21.15.89:0/2105179773 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f08b40b89a0 con 0x7f08bc145b70 2023-10-18T10:46:55.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.740+0000 7f08b8ff9640 1 --2- 172.21.15.89:0/2105179773 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f089c065680 0x7f089c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:55.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.740+0000 7f08b8ff9640 1 -- 172.21.15.89:0/2105179773 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f08b411e170 con 0x7f08bc145b70 2023-10-18T10:46:55.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.740+0000 7f08bb7fe640 1 --2- 172.21.15.89:0/2105179773 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f089c065680 0x7f089c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:55.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.740+0000 7f08bb7fe640 1 --2- 172.21.15.89:0/2105179773 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f089c065680 0x7f089c067b40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f08ac00b930 tx=0x7f08ac008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:55.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.744+0000 7f08b8ff9640 1 -- 172.21.15.89:0/2105179773 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f08b40eb990 con 0x7f08bc145b70 2023-10-18T10:46:55.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.896+0000 7f08c1eca640 1 -- 172.21.15.89:0/2105179773 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f08bc144990 con 0x7f08bc145b70 2023-10-18T10:46:55.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.900+0000 7f08b8ff9640 1 -- 172.21.15.89:0/2105179773 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f08b40eefb0 con 0x7f08bc145b70 2023-10-18T10:46:55.900 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-18T10:46:55.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.900+0000 7f08c1eca640 1 -- 172.21.15.89:0/2105179773 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f089c065680 msgr2=0x7f089c067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:55.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.900+0000 7f08c1eca640 1 --2- 172.21.15.89:0/2105179773 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f089c065680 0x7f089c067b40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f08ac00b930 tx=0x7f08ac008040 comp rx=0 tx=0).stop 2023-10-18T10:46:55.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.900+0000 7f08c1eca640 1 -- 172.21.15.89:0/2105179773 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08bc145b70 msgr2=0x7f08bc1536b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:55.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.900+0000 7f08c1eca640 1 --2- 172.21.15.89:0/2105179773 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08bc145b70 0x7f08bc1536b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f08b40af040 tx=0x7f08b4019780 comp rx=0 tx=0).stop 2023-10-18T10:46:55.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.900+0000 7f08c1eca640 1 -- 172.21.15.89:0/2105179773 shutdown_connections 2023-10-18T10:46:55.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.900+0000 7f08c1eca640 1 --2- 172.21.15.89:0/2105179773 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f089c065680 0x7f089c067b40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:55.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.900+0000 7f08c1eca640 1 --2- 172.21.15.89:0/2105179773 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08bc145b70 0x7f08bc1536b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:55.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.900+0000 7f08c1eca640 1 --2- 172.21.15.89:0/2105179773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f08bc0ce070 0x7f08bc0cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:55.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.900+0000 7f08c1eca640 1 --2- 172.21.15.89:0/2105179773 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08bc0cd4f0 0x7f08bc0cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:55.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.900+0000 7f08c1eca640 1 -- 172.21.15.89:0/2105179773 >> 172.21.15.89:0/2105179773 conn(0x7f08bc06dad0 msgr2=0x7f08bc15d830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:55.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.900+0000 7f08c1eca640 1 -- 172.21.15.89:0/2105179773 shutdown_connections 2023-10-18T10:46:55.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:55.900+0000 7f08c1eca640 1 -- 172.21.15.89:0/2105179773 wait complete. 2023-10-18T10:46:55.920 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T10:46:56.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5efb5ba640 1 Processor -- start 2023-10-18T10:46:56.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5efb5ba640 1 -- start start 2023-10-18T10:46:56.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5efb5ba640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5ef40cd4f0 0x7f5ef4143470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:56.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5efb5ba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ef40ce070 0x7f5ef40ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:56.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5efb5ba640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ef40cd8d0 con 0x7f5ef4145b70 2023-10-18T10:46:56.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5efb5ba640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ef4069900 con 0x7f5ef40cd4f0 2023-10-18T10:46:56.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5efb5ba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef40cea20 con 0x7f5ef40ce070 2023-10-18T10:46:56.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5ef8b55640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5ef4145b70 0x7f5ef4160cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58152/0 (socket says 172.21.15.89:58152) 2023-10-18T10:46:56.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5ef8b55640 1 -- 172.21.15.89:0/3155722655 learned_addr learned my addr 172.21.15.89:0/3155722655 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:56.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5ef9b57640 1 --2- 172.21.15.89:0/3155722655 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ef40ce070 0x7f5ef40ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:56.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5ef9356640 1 --2- 172.21.15.89:0/3155722655 >> v2:172.21.15.154:3300/0 conn(0x7f5ef40cd4f0 0x7f5ef4143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:56.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5eebfff640 1 -- 172.21.15.89:0/3155722655 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3221697283 0 0) 0x7f5ef40cd8d0 con 0x7f5ef4145b70 2023-10-18T10:46:56.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5eebfff640 1 -- 172.21.15.89:0/3155722655 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ee0003530 con 0x7f5ef4145b70 2023-10-18T10:46:56.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5eebfff640 1 -- 172.21.15.89:0/3155722655 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4064391572 0 0) 0x7f5ee0003530 con 0x7f5ef4145b70 2023-10-18T10:46:56.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5eebfff640 1 -- 172.21.15.89:0/3155722655 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ef40ce070 msgr2=0x7f5ef40ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:56.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5eebfff640 1 --2- 172.21.15.89:0/3155722655 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ef40ce070 0x7f5ef40ce4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:56.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5eebfff640 1 -- 172.21.15.89:0/3155722655 >> v2:172.21.15.154:3300/0 conn(0x7f5ef40cd4f0 msgr2=0x7f5ef4143470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:56.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5eebfff640 1 --2- 172.21.15.89:0/3155722655 >> v2:172.21.15.154:3300/0 conn(0x7f5ef40cd4f0 0x7f5ef4143470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:56.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5eebfff640 1 -- 172.21.15.89:0/3155722655 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef415cbf0 con 0x7f5ef4145b70 2023-10-18T10:46:56.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5eebfff640 1 -- 172.21.15.89:0/3155722655 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7f5ee4001680 con 0x7f5ef4145b70 2023-10-18T10:46:56.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5eebfff640 1 -- 172.21.15.89:0/3155722655 >> v1:172.21.15.89:6789/0 conn(0x7f5ef4145b70 legacy=0x7f5ef4160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:56.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5eebfff640 1 --2- 172.21.15.89:0/3155722655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ee0003980 0x7f5ee0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:56.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5eebfff640 1 --2- 172.21.15.89:0/3155722655 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ef40ce070 0x7f5ee00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:56.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5eebfff640 1 --2- 172.21.15.89:0/3155722655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ee0004d10 0x7f5ee0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:56.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5eebfff640 1 -- 172.21.15.89:0/3155722655 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef4069900 con 0x7f5ee0003980 2023-10-18T10:46:56.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5eebfff640 1 -- 172.21.15.89:0/3155722655 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef40cea20 con 0x7f5ee0004d10 2023-10-18T10:46:56.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5eebfff640 1 -- 172.21.15.89:0/3155722655 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee000b990 con 0x7f5ef40ce070 2023-10-18T10:46:56.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5ef9356640 1 --2- 172.21.15.89:0/3155722655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ee0003980 0x7f5ee0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:56.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5ef8b55640 1 --2- 172.21.15.89:0/3155722655 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ef40ce070 0x7f5ee00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:56.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5ef9b57640 1 --2- 172.21.15.89:0/3155722655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ee0004d10 0x7f5ee0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:56.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5ef9356640 1 -- 172.21.15.89:0/3155722655 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ef40ce070 msgr2=0x7f5ee00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:56.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5ef9356640 1 --2- 172.21.15.89:0/3155722655 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ef40ce070 0x7f5ee00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:56.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5ef9356640 1 -- 172.21.15.89:0/3155722655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ee0004d10 msgr2=0x7f5ee0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:56.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5ef9356640 1 --2- 172.21.15.89:0/3155722655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ee0004d10 0x7f5ee0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:56.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5ef9356640 1 -- 172.21.15.89:0/3155722655 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee0003530 con 0x7f5ee0003980 2023-10-18T10:46:56.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.059+0000 7f5ef9b57640 1 --2- 172.21.15.89:0/3155722655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ee0004d10 0x7f5ee0005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:56.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.063+0000 7f5ef9356640 1 --2- 172.21.15.89:0/3155722655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ee0003980 0x7f5ee0003dc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0009ae0 tx=0x7f5ef001ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d46e43fb54acc397 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:56.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.063+0000 7f5eebfff640 1 -- 172.21.15.89:0/3155722655 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef00a5540 con 0x7f5ee0003980 2023-10-18T10:46:56.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.063+0000 7f5eebfff640 1 -- 172.21.15.89:0/3155722655 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef00a56a0 con 0x7f5ee0003980 2023-10-18T10:46:56.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.063+0000 7f5eebfff640 1 -- 172.21.15.89:0/3155722655 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef00a5a50 con 0x7f5ee0003980 2023-10-18T10:46:56.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.063+0000 7f5efb5ba640 1 -- 172.21.15.89:0/3155722655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ee0003980 msgr2=0x7f5ee0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:56.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.063+0000 7f5efb5ba640 1 --2- 172.21.15.89:0/3155722655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ee0003980 0x7f5ee0003dc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0009ae0 tx=0x7f5ef001ca60 comp rx=0 tx=0).stop 2023-10-18T10:46:56.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.063+0000 7f5ef9b57640 1 -- 172.21.15.89:0/3155722655 reap_dead start 2023-10-18T10:46:56.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.063+0000 7f5efb5ba640 1 -- 172.21.15.89:0/3155722655 shutdown_connections 2023-10-18T10:46:56.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.063+0000 7f5efb5ba640 1 -- 172.21.15.89:0/3155722655 >> 172.21.15.89:0/3155722655 conn(0x7f5ef406dad0 msgr2=0x7f5ef406dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:56.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.063+0000 7f5efb5ba640 1 -- 172.21.15.89:0/3155722655 shutdown_connections 2023-10-18T10:46:56.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.063+0000 7f5efb5ba640 1 -- 172.21.15.89:0/3155722655 wait complete. 2023-10-18T10:46:56.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.067+0000 7f5efb5ba640 1 Processor -- start 2023-10-18T10:46:56.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.067+0000 7f5efb5ba640 1 -- start start 2023-10-18T10:46:56.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.067+0000 7f5efb5ba640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ee0003980 0x7f5ef40c3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:56.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.067+0000 7f5efb5ba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ef40c83d0 0x7f5ef40c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:56.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.067+0000 7f5efb5ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ef40c4110 0x7f5ef4156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:56.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.067+0000 7f5efb5ba640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef40ca450 con 0x7f5ee0003980 2023-10-18T10:46:56.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.067+0000 7f5efb5ba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef4156970 con 0x7f5ef40c4110 2023-10-18T10:46:56.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.067+0000 7f5efb5ba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef40c9e90 con 0x7f5ef40c83d0 2023-10-18T10:46:56.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.067+0000 7f5ef9356640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ee0003980 0x7f5ef40c3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:56.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.067+0000 7f5ef9356640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ee0003980 0x7f5ef40c3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57382/0 (socket says 172.21.15.89:57382) 2023-10-18T10:46:56.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.067+0000 7f5ef9356640 1 -- 172.21.15.89:0/2004434313 learned_addr learned my addr 172.21.15.89:0/2004434313 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:56.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.067+0000 7f5ef9b57640 1 --2- 172.21.15.89:0/2004434313 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ef40c4110 0x7f5ef4156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:56.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.067+0000 7f5ef8b55640 1 --2- 172.21.15.89:0/2004434313 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ef40c83d0 0x7f5ef40c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:56.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.067+0000 7f5ef9356640 1 -- 172.21.15.89:0/2004434313 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ef40c83d0 msgr2=0x7f5ef40c3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:56.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.067+0000 7f5ef9356640 1 --2- 172.21.15.89:0/2004434313 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ef40c83d0 0x7f5ef40c3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:56.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.067+0000 7f5ef9356640 1 -- 172.21.15.89:0/2004434313 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ef40c4110 msgr2=0x7f5ef4156e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:56.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.067+0000 7f5ef9356640 1 --2- 172.21.15.89:0/2004434313 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ef40c4110 0x7f5ef4156e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:56.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.067+0000 7f5ef9356640 1 -- 172.21.15.89:0/2004434313 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef40ca170 con 0x7f5ee0003980 2023-10-18T10:46:56.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.067+0000 7f5ef9356640 1 --2- 172.21.15.89:0/2004434313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ee0003980 0x7f5ef40c3390 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0009ae0 tx=0x7f5ee0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:56.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.067+0000 7f5ee9ffb640 1 -- 172.21.15.89:0/2004434313 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef00a5e90 con 0x7f5ee0003980 2023-10-18T10:46:56.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.067+0000 7f5ee9ffb640 1 -- 172.21.15.89:0/2004434313 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef0004020 con 0x7f5ee0003980 2023-10-18T10:46:56.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.067+0000 7f5ee9ffb640 1 -- 172.21.15.89:0/2004434313 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0004390 con 0x7f5ee0003980 2023-10-18T10:46:56.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.067+0000 7f5efb5ba640 1 -- 172.21.15.89:0/2004434313 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef4156600 con 0x7f5ee0003980 2023-10-18T10:46:56.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.071+0000 7f5efb5ba640 1 -- 172.21.15.89:0/2004434313 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef40c9b40 con 0x7f5ee0003980 2023-10-18T10:46:56.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.071+0000 7f5efb5ba640 1 -- 172.21.15.89:0/2004434313 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef4156790 con 0x7f5ee0003980 2023-10-18T10:46:56.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.071+0000 7f5ee9ffb640 1 -- 172.21.15.89:0/2004434313 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f5ef00b4040 con 0x7f5ee0003980 2023-10-18T10:46:56.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.071+0000 7f5ee9ffb640 1 --2- 172.21.15.89:0/2004434313 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5ed8065680 0x7f5ed8067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:56.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.071+0000 7f5ee9ffb640 1 -- 172.21.15.89:0/2004434313 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5ef011d2e0 con 0x7f5ee0003980 2023-10-18T10:46:56.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.071+0000 7f5ef8b55640 1 --2- 172.21.15.89:0/2004434313 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5ed8065680 0x7f5ed8067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:56.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.075+0000 7f5ef8b55640 1 --2- 172.21.15.89:0/2004434313 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5ed8065680 0x7f5ed8067b40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4001ec0 tx=0x7f5ee4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:56.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.075+0000 7f5ee9ffb640 1 -- 172.21.15.89:0/2004434313 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f5ef00eac40 con 0x7f5ee0003980 2023-10-18T10:46:56.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.223+0000 7f5efb5ba640 1 -- 172.21.15.89:0/2004434313 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5ef404dbe0 con 0x7f5ee0003980 2023-10-18T10:46:56.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.223+0000 7f5ee9ffb640 1 -- 172.21.15.89:0/2004434313 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5ef00ee260 con 0x7f5ee0003980 2023-10-18T10:46:56.229 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-18T10:46:56.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.227+0000 7f5efb5ba640 1 -- 172.21.15.89:0/2004434313 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5ed8065680 msgr2=0x7f5ed8067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:56.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.227+0000 7f5efb5ba640 1 --2- 172.21.15.89:0/2004434313 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5ed8065680 0x7f5ed8067b40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4001ec0 tx=0x7f5ee4009040 comp rx=0 tx=0).stop 2023-10-18T10:46:56.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.227+0000 7f5efb5ba640 1 -- 172.21.15.89:0/2004434313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ee0003980 msgr2=0x7f5ef40c3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:56.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.227+0000 7f5efb5ba640 1 --2- 172.21.15.89:0/2004434313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ee0003980 0x7f5ef40c3390 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0009ae0 tx=0x7f5ee0005800 comp rx=0 tx=0).stop 2023-10-18T10:46:56.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.227+0000 7f5efb5ba640 1 -- 172.21.15.89:0/2004434313 shutdown_connections 2023-10-18T10:46:56.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.227+0000 7f5efb5ba640 1 --2- 172.21.15.89:0/2004434313 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5ed8065680 0x7f5ed8067b40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:56.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.227+0000 7f5efb5ba640 1 --2- 172.21.15.89:0/2004434313 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ef40c83d0 0x7f5ef40c3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:56.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.227+0000 7f5efb5ba640 1 --2- 172.21.15.89:0/2004434313 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ef40c4110 0x7f5ef4156e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:56.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.227+0000 7f5efb5ba640 1 --2- 172.21.15.89:0/2004434313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ee0003980 0x7f5ef40c3390 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:56.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.227+0000 7f5efb5ba640 1 -- 172.21.15.89:0/2004434313 >> 172.21.15.89:0/2004434313 conn(0x7f5ef406dad0 msgr2=0x7f5ef415ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:56.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.227+0000 7f5efb5ba640 1 -- 172.21.15.89:0/2004434313 shutdown_connections 2023-10-18T10:46:56.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.227+0000 7f5efb5ba640 1 -- 172.21.15.89:0/2004434313 wait complete. 2023-10-18T10:46:56.248 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T10:46:56.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.387+0000 7f928932c640 1 Processor -- start 2023-10-18T10:46:56.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.387+0000 7f928932c640 1 -- start start 2023-10-18T10:46:56.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.387+0000 7f928932c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f928415c570 0x7f92840d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:56.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.387+0000 7f928932c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92840d1a60 0x7f92840cee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:56.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.387+0000 7f928932c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92840cf800 con 0x7f92840cf390 2023-10-18T10:46:56.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.387+0000 7f928932c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92840d0e60 con 0x7f928415c570 2023-10-18T10:46:56.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.387+0000 7f928932c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92840d1140 con 0x7f92840d1a60 2023-10-18T10:46:56.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.387+0000 7f927bfff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92840d1a60 0x7f92840cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:56.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.387+0000 7f9282ffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f928415c570 0x7f92840d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:56.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.387+0000 7f927bfff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92840d1a60 0x7f92840cee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:58466/0 (socket says 172.21.15.89:58466) 2023-10-18T10:46:56.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.387+0000 7f92837fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f92840cf390 0x7f9284069900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.89:41236/0 (socket says 172.21.15.89:41236) 2023-10-18T10:46:56.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.387+0000 7f927bfff640 1 -- 172.21.15.89:0/2448337014 learned_addr learned my addr 172.21.15.89:0/2448337014 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:56.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.387+0000 7f9282ffd640 1 -- 172.21.15.89:0/2448337014 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92840d1a60 msgr2=0x7f92840cee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:56.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.387+0000 7f9282ffd640 1 --2- 172.21.15.89:0/2448337014 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92840d1a60 0x7f92840cee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:56.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.387+0000 7f9282ffd640 1 -- 172.21.15.89:0/2448337014 >> v1:172.21.15.154:6789/0 conn(0x7f92840cf390 legacy=0x7f9284069900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:56.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.387+0000 7f9282ffd640 1 -- 172.21.15.89:0/2448337014 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92840d1420 con 0x7f928415c570 2023-10-18T10:46:56.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.387+0000 7f927bfff640 1 --2- 172.21.15.89:0/2448337014 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92840d1a60 0x7f92840cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:46:56.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.387+0000 7f92827fc640 1 -- 172.21.15.89:0/2448337014 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 27037123 0 0) 0x7f92840cf800 con 0x7f92840cf390 2023-10-18T10:46:56.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.391+0000 7f9282ffd640 1 --2- 172.21.15.89:0/2448337014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f928415c570 0x7f92840d0560 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f92740097c0 tx=0x7f927401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=783c1eab05142201 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:56.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.391+0000 7f92827fc640 1 -- 172.21.15.89:0/2448337014 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92740ab670 con 0x7f928415c570 2023-10-18T10:46:56.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.391+0000 7f92827fc640 1 -- 172.21.15.89:0/2448337014 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92740ab7d0 con 0x7f928415c570 2023-10-18T10:46:56.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.391+0000 7f92827fc640 1 -- 172.21.15.89:0/2448337014 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92740abb60 con 0x7f928415c570 2023-10-18T10:46:56.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.391+0000 7f928932c640 1 -- 172.21.15.89:0/2448337014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f928415c570 msgr2=0x7f92840d0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:56.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.391+0000 7f928932c640 1 --2- 172.21.15.89:0/2448337014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f928415c570 0x7f92840d0560 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f92740097c0 tx=0x7f927401c450 comp rx=0 tx=0).stop 2023-10-18T10:46:56.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.391+0000 7f928932c640 1 -- 172.21.15.89:0/2448337014 shutdown_connections 2023-10-18T10:46:56.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.391+0000 7f928932c640 1 --2- 172.21.15.89:0/2448337014 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92840d1a60 0x7f92840cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:56.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.391+0000 7f928932c640 1 --2- 172.21.15.89:0/2448337014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f928415c570 0x7f92840d0560 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:56.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.391+0000 7f928932c640 1 -- 172.21.15.89:0/2448337014 >> 172.21.15.89:0/2448337014 conn(0x7f9284070320 msgr2=0x7f9284070730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:56.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.391+0000 7f928932c640 1 -- 172.21.15.89:0/2448337014 shutdown_connections 2023-10-18T10:46:56.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.391+0000 7f928932c640 1 -- 172.21.15.89:0/2448337014 wait complete. 2023-10-18T10:46:56.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.395+0000 7f928932c640 1 Processor -- start 2023-10-18T10:46:56.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.395+0000 7f928932c640 1 -- start start 2023-10-18T10:46:56.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.395+0000 7f928932c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92840cf390 0x7f92840cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:56.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.395+0000 7f928932c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92840d1a60 0x7f92840c8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:56.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.395+0000 7f928932c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f928415c570 0x7f928415b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:56.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.395+0000 7f928932c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92840cc6e0 con 0x7f928415c570 2023-10-18T10:46:56.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.395+0000 7f928932c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92840c8e50 con 0x7f92840d1a60 2023-10-18T10:46:56.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.395+0000 7f928932c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f928415d130 con 0x7f92840cf390 2023-10-18T10:46:56.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.395+0000 7f92837fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f928415c570 0x7f928415b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:56.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.395+0000 7f92837fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f928415c570 0x7f928415b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57390/0 (socket says 172.21.15.89:57390) 2023-10-18T10:46:56.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.395+0000 7f92837fe640 1 -- 172.21.15.89:0/2182398133 learned_addr learned my addr 172.21.15.89:0/2182398133 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:56.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.395+0000 7f9282ffd640 1 --2- 172.21.15.89:0/2182398133 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92840cf390 0x7f92840cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:56.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.395+0000 7f927bfff640 1 --2- 172.21.15.89:0/2182398133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92840d1a60 0x7f92840c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:56.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.395+0000 7f92837fe640 1 -- 172.21.15.89:0/2182398133 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92840cf390 msgr2=0x7f92840cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:56.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.395+0000 7f92837fe640 1 --2- 172.21.15.89:0/2182398133 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92840cf390 0x7f92840cc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:56.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.395+0000 7f92837fe640 1 -- 172.21.15.89:0/2182398133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92840d1a60 msgr2=0x7f92840c8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:56.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.395+0000 7f92837fe640 1 --2- 172.21.15.89:0/2182398133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92840d1a60 0x7f92840c8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:56.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.395+0000 7f92837fe640 1 -- 172.21.15.89:0/2182398133 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f926c0094d0 con 0x7f928415c570 2023-10-18T10:46:56.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.395+0000 7f9282ffd640 1 --2- 172.21.15.89:0/2182398133 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92840cf390 0x7f92840cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:56.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.395+0000 7f92837fe640 1 --2- 172.21.15.89:0/2182398133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f928415c570 0x7f928415b1a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f927c00c7b0 tx=0x7f927c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:56.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.395+0000 7f927b7fe640 1 -- 172.21.15.89:0/2182398133 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c00a640 con 0x7f928415c570 2023-10-18T10:46:56.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.395+0000 7f927b7fe640 1 -- 172.21.15.89:0/2182398133 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927c00a7c0 con 0x7f928415c570 2023-10-18T10:46:56.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.395+0000 7f927b7fe640 1 -- 172.21.15.89:0/2182398133 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c00ab70 con 0x7f928415c570 2023-10-18T10:46:56.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.395+0000 7f928932c640 1 -- 172.21.15.89:0/2182398133 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9274009420 con 0x7f928415c570 2023-10-18T10:46:56.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.395+0000 7f928932c640 1 -- 172.21.15.89:0/2182398133 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f928415cad0 con 0x7f928415c570 2023-10-18T10:46:56.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.399+0000 7f928932c640 1 -- 172.21.15.89:0/2182398133 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92840c8600 con 0x7f928415c570 2023-10-18T10:46:56.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.399+0000 7f927b7fe640 1 -- 172.21.15.89:0/2182398133 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f927c00acd0 con 0x7f928415c570 2023-10-18T10:46:56.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.399+0000 7f927b7fe640 1 --2- 172.21.15.89:0/2182398133 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f92640656b0 0x7f9264067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:56.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.399+0000 7f927b7fe640 1 -- 172.21.15.89:0/2182398133 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f927c0824f0 con 0x7f928415c570 2023-10-18T10:46:56.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.399+0000 7f9282ffd640 1 --2- 172.21.15.89:0/2182398133 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f92640656b0 0x7f9264067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:56.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.403+0000 7f9282ffd640 1 --2- 172.21.15.89:0/2182398133 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f92640656b0 0x7f9264067b70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f92740af000 tx=0x7f92740a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:56.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.403+0000 7f927b7fe640 1 -- 172.21.15.89:0/2182398133 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f927c04fc70 con 0x7f928415c570 2023-10-18T10:46:56.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.563+0000 7f928932c640 1 -- 172.21.15.89:0/2182398133 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9284073b50 con 0x7f928415c570 2023-10-18T10:46:56.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.563+0000 7f927b7fe640 1 -- 172.21.15.89:0/2182398133 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7f927c053290 con 0x7f928415c570 2023-10-18T10:46:56.565 INFO:teuthology.orchestra.run.smithi089.stderr:listed 0 entries 2023-10-18T10:46:56.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.563+0000 7f928932c640 1 -- 172.21.15.89:0/2182398133 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f92640656b0 msgr2=0x7f9264067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:56.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.563+0000 7f928932c640 1 --2- 172.21.15.89:0/2182398133 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f92640656b0 0x7f9264067b70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f92740af000 tx=0x7f92740a7000 comp rx=0 tx=0).stop 2023-10-18T10:46:56.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.563+0000 7f928932c640 1 -- 172.21.15.89:0/2182398133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f928415c570 msgr2=0x7f928415b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:56.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.563+0000 7f928932c640 1 --2- 172.21.15.89:0/2182398133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f928415c570 0x7f928415b1a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f927c00c7b0 tx=0x7f927c00cc70 comp rx=0 tx=0).stop 2023-10-18T10:46:56.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.563+0000 7f928932c640 1 -- 172.21.15.89:0/2182398133 shutdown_connections 2023-10-18T10:46:56.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.563+0000 7f928932c640 1 --2- 172.21.15.89:0/2182398133 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f92640656b0 0x7f9264067b70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:56.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.563+0000 7f928932c640 1 --2- 172.21.15.89:0/2182398133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f928415c570 0x7f928415b1a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:56.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.563+0000 7f928932c640 1 --2- 172.21.15.89:0/2182398133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92840d1a60 0x7f92840c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:56.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.563+0000 7f928932c640 1 --2- 172.21.15.89:0/2182398133 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92840cf390 0x7f92840cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:56.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.563+0000 7f928932c640 1 -- 172.21.15.89:0/2182398133 >> 172.21.15.89:0/2182398133 conn(0x7f9284070320 msgr2=0x7f9284070700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:56.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.563+0000 7f928932c640 1 -- 172.21.15.89:0/2182398133 shutdown_connections 2023-10-18T10:46:56.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.563+0000 7f928932c640 1 -- 172.21.15.89:0/2182398133 wait complete. 2023-10-18T10:46:56.586 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-10-18T10:46:56.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:56.587 DEBUG:teuthology.orchestra.run.smithi089:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-10-18T10:46:56.612 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi089.front.sepia.ceph.com 2023-10-18T10:46:56.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:56.613 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-18T10:46:56.668 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T10:46:56.669 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:46:56.680 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T10:46:56.680 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:46:56.730 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T10:46:56.731 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:46:56.782 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:46:56.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.915+0000 7fd0a0c34640 1 Processor -- start 2023-10-18T10:46:56.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.915+0000 7fd0a0c34640 1 -- start start 2023-10-18T10:46:56.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.919+0000 7fd0a0c34640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd09c066850 0x7fd09c15cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:56.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.919+0000 7fd0a0c34640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd09c0cc950 0x7fd09c163570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:56.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.919+0000 7fd0a0c34640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd09c0cb620 con 0x7fd09c158540 2023-10-18T10:46:56.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.919+0000 7fd0a0c34640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd09c0cbab0 con 0x7fd09c066850 2023-10-18T10:46:56.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.919+0000 7fd0a0c34640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd09c15d150 con 0x7fd09c0cc950 2023-10-18T10:46:56.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.919+0000 7fd09a59c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd09c066850 0x7fd09c15cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:56.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.919+0000 7fd09a59c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd09c066850 0x7fd09c15cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57398/0 (socket says 172.21.15.89:57398) 2023-10-18T10:46:56.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.919+0000 7fd09a59c640 1 -- 172.21.15.89:0/1846318084 learned_addr learned my addr 172.21.15.89:0/1846318084 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:56.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.919+0000 7fd09ad9d640 1 --2- 172.21.15.89:0/1846318084 >> v2:172.21.15.154:3300/0 conn(0x7fd09c0cc950 0x7fd09c163570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:56.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.919+0000 7fd09a59c640 1 -- 172.21.15.89:0/1846318084 >> v2:172.21.15.154:3300/0 conn(0x7fd09c0cc950 msgr2=0x7fd09c163570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:56.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.919+0000 7fd09a59c640 1 --2- 172.21.15.89:0/1846318084 >> v2:172.21.15.154:3300/0 conn(0x7fd09c0cc950 0x7fd09c163570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:56.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.919+0000 7fd09a59c640 1 -- 172.21.15.89:0/1846318084 >> v1:172.21.15.89:6789/0 conn(0x7fd09c158540 legacy=0x7fd09c0cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:56.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.919+0000 7fd09a59c640 1 -- 172.21.15.89:0/1846318084 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd09c0caa10 con 0x7fd09c066850 2023-10-18T10:46:56.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.923+0000 7fd09a59c640 1 --2- 172.21.15.89:0/1846318084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd09c066850 0x7fd09c15cc10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0cb590 tx=0x7fd08c01eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc8cc0db279a96b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:56.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.923+0000 7fd09959a640 1 -- 172.21.15.89:0/1846318084 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c00b2d0 con 0x7fd09c066850 2023-10-18T10:46:56.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.923+0000 7fd09959a640 1 -- 172.21.15.89:0/1846318084 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08c00b430 con 0x7fd09c066850 2023-10-18T10:46:56.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.923+0000 7fd09959a640 1 -- 172.21.15.89:0/1846318084 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c0a7c90 con 0x7fd09c066850 2023-10-18T10:46:56.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.923+0000 7fd0a0c34640 1 -- 172.21.15.89:0/1846318084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd09c066850 msgr2=0x7fd09c15cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:56.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.923+0000 7fd0a0c34640 1 --2- 172.21.15.89:0/1846318084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd09c066850 0x7fd09c15cc10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0cb590 tx=0x7fd08c01eba0 comp rx=0 tx=0).stop 2023-10-18T10:46:56.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.923+0000 7fd0a0c34640 1 -- 172.21.15.89:0/1846318084 shutdown_connections 2023-10-18T10:46:56.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.923+0000 7fd0a0c34640 1 --2- 172.21.15.89:0/1846318084 >> v2:172.21.15.154:3300/0 conn(0x7fd09c0cc950 0x7fd09c163570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:56.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.923+0000 7fd0a0c34640 1 --2- 172.21.15.89:0/1846318084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd09c066850 0x7fd09c15cc10 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:56.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.923+0000 7fd0a0c34640 1 -- 172.21.15.89:0/1846318084 >> 172.21.15.89:0/1846318084 conn(0x7fd09c0b0470 msgr2=0x7fd09c0b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:56.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.923+0000 7fd0a0c34640 1 -- 172.21.15.89:0/1846318084 shutdown_connections 2023-10-18T10:46:56.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.923+0000 7fd0a0c34640 1 -- 172.21.15.89:0/1846318084 wait complete. 2023-10-18T10:46:56.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.923+0000 7fd0a0c34640 1 Processor -- start 2023-10-18T10:46:56.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.923+0000 7fd0a0c34640 1 -- start start 2023-10-18T10:46:56.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.927+0000 7fd0a0c34640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd09c066850 0x7fd09c0c5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:56.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.927+0000 7fd0a0c34640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd09c0cc950 0x7fd09c162d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:56.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.927+0000 7fd0a0c34640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd09c158540 0x7fd09c0b1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:56.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.927+0000 7fd0a0c34640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09c0c6df0 con 0x7fd09c066850 2023-10-18T10:46:56.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.927+0000 7fd0a0c34640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09c0c6b10 con 0x7fd09c0cc950 2023-10-18T10:46:56.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.927+0000 7fd0a0c34640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd09c0c5f70 con 0x7fd09c158540 2023-10-18T10:46:56.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.927+0000 7fd09a59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd09c066850 0x7fd09c0c5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:56.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.927+0000 7fd09a59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd09c066850 0x7fd09c0c5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57408/0 (socket says 172.21.15.89:57408) 2023-10-18T10:46:56.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.927+0000 7fd09a59c640 1 -- 172.21.15.89:0/3003272379 learned_addr learned my addr 172.21.15.89:0/3003272379 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:56.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.927+0000 7fd099d9b640 1 --2- 172.21.15.89:0/3003272379 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd09c0cc950 0x7fd09c162d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:56.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.927+0000 7fd09ad9d640 1 --2- 172.21.15.89:0/3003272379 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd09c158540 0x7fd09c0b1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:56.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.927+0000 7fd09a59c640 1 -- 172.21.15.89:0/3003272379 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd09c158540 msgr2=0x7fd09c0b1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:56.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.927+0000 7fd09a59c640 1 --2- 172.21.15.89:0/3003272379 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd09c158540 0x7fd09c0b1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:56.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.927+0000 7fd09a59c640 1 -- 172.21.15.89:0/3003272379 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd09c0cc950 msgr2=0x7fd09c162d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:56.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.927+0000 7fd09a59c640 1 --2- 172.21.15.89:0/3003272379 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd09c0cc950 0x7fd09c162d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:56.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.927+0000 7fd09a59c640 1 -- 172.21.15.89:0/3003272379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08c00bd40 con 0x7fd09c066850 2023-10-18T10:46:56.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.927+0000 7fd099d9b640 1 --2- 172.21.15.89:0/3003272379 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd09c0cc950 0x7fd09c162d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:56.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.927+0000 7fd09a59c640 1 --2- 172.21.15.89:0/3003272379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd09c066850 0x7fd09c0c5740 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd08c01b7b0 tx=0x7fd08c01b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:56.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.927+0000 7fd08affd640 1 -- 172.21.15.89:0/3003272379 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c0bd070 con 0x7fd09c066850 2023-10-18T10:46:56.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.927+0000 7fd08affd640 1 -- 172.21.15.89:0/3003272379 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08c0a7c80 con 0x7fd09c066850 2023-10-18T10:46:56.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.927+0000 7fd08affd640 1 -- 172.21.15.89:0/3003272379 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c009820 con 0x7fd09c066850 2023-10-18T10:46:56.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.927+0000 7fd0a0c34640 1 -- 172.21.15.89:0/3003272379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd09c0c6250 con 0x7fd09c066850 2023-10-18T10:46:56.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.927+0000 7fd0a0c34640 1 -- 172.21.15.89:0/3003272379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd09c0c5c80 con 0x7fd09c066850 2023-10-18T10:46:56.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.927+0000 7fd0a0c34640 1 -- 172.21.15.89:0/3003272379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd09c0c6770 con 0x7fd09c066850 2023-10-18T10:46:56.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.927+0000 7fd08affd640 1 -- 172.21.15.89:0/3003272379 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fd08c0039a0 con 0x7fd09c066850 2023-10-18T10:46:56.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.931+0000 7fd08affd640 1 --2- 172.21.15.89:0/3003272379 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd07c0656b0 0x7fd07c067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:56.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.931+0000 7fd08affd640 1 -- 172.21.15.89:0/3003272379 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd08c11df90 con 0x7fd09c066850 2023-10-18T10:46:56.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.931+0000 7fd099d9b640 1 --2- 172.21.15.89:0/3003272379 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd07c0656b0 0x7fd07c067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:56.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.931+0000 7fd099d9b640 1 --2- 172.21.15.89:0/3003272379 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd07c0656b0 0x7fd07c067b70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd09c14cd90 tx=0x7fd090009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:56.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:56.935+0000 7fd08affd640 1 -- 172.21.15.89:0/3003272379 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fd08c0eb7a0 con 0x7fd09c066850 2023-10-18T10:46:57.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.087+0000 7fd0a0c34640 1 -- 172.21.15.89:0/3003272379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd09c0c6c50 con 0x7fd09c066850 2023-10-18T10:46:57.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.087+0000 7fd08affd640 1 -- 172.21.15.89:0/3003272379 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20192 (secure 0 0 0) 0x7fd09c0c6c50 con 0x7fd09c066850 2023-10-18T10:46:57.089 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:57.089 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":29,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:46:36.958604+0000","last_up_change":"2023-10-18T10:45:42.164688+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:45:53.309122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-18T10:45:54.659763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:45:57.796514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6824","nonce":773078364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6827","nonce":773078364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6831","nonce":773078364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6829","nonce":773078364}]},"public_addr":"172.21.15.154:6824/773078364","cluster_addr":"172.21.15.154:6827/773078364","heartbeat_back_addr":"172.21.15.154:6831/773078364","heartbeat_front_addr":"172.21.15.154:6829/773078364","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":1,"to":5}]},{"pgid":"4.1a","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:46:57.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.087+0000 7fd0a0c34640 1 -- 172.21.15.89:0/3003272379 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd07c0656b0 msgr2=0x7fd07c067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:57.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.087+0000 7fd0a0c34640 1 --2- 172.21.15.89:0/3003272379 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd07c0656b0 0x7fd07c067b70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd09c14cd90 tx=0x7fd090009040 comp rx=0 tx=0).stop 2023-10-18T10:46:57.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.087+0000 7fd0a0c34640 1 -- 172.21.15.89:0/3003272379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd09c066850 msgr2=0x7fd09c0c5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:57.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.087+0000 7fd0a0c34640 1 --2- 172.21.15.89:0/3003272379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd09c066850 0x7fd09c0c5740 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd08c01b7b0 tx=0x7fd08c01b940 comp rx=0 tx=0).stop 2023-10-18T10:46:57.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.087+0000 7fd0a0c34640 1 -- 172.21.15.89:0/3003272379 shutdown_connections 2023-10-18T10:46:57.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.087+0000 7fd0a0c34640 1 --2- 172.21.15.89:0/3003272379 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd07c0656b0 0x7fd07c067b70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:57.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.087+0000 7fd0a0c34640 1 --2- 172.21.15.89:0/3003272379 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd09c158540 0x7fd09c0b1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:57.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.087+0000 7fd0a0c34640 1 --2- 172.21.15.89:0/3003272379 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd09c0cc950 0x7fd09c162d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:57.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.087+0000 7fd0a0c34640 1 --2- 172.21.15.89:0/3003272379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd09c066850 0x7fd09c0c5740 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:57.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.087+0000 7fd0a0c34640 1 -- 172.21.15.89:0/3003272379 >> 172.21.15.89:0/3003272379 conn(0x7fd09c0b0470 msgr2=0x7fd09c0c3810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:57.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.087+0000 7fd0a0c34640 1 -- 172.21.15.89:0/3003272379 shutdown_connections 2023-10-18T10:46:57.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.087+0000 7fd0a0c34640 1 -- 172.21.15.89:0/3003272379 wait complete. 2023-10-18T10:46:57.110 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:45:43.114184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-18T10:45:48.730968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:45:53.309122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:45:54.659763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T10:45:57.796514+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T10:46:57.112 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:46:57.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.251+0000 7f486a8bd640 1 Processor -- start 2023-10-18T10:46:57.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.251+0000 7f486a8bd640 1 -- start start 2023-10-18T10:46:57.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.251+0000 7f486a8bd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f486415c7f0 0x7f486406b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:57.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.251+0000 7f486a8bd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f48640ad5b0 0x7f4864160a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:57.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.251+0000 7f486a8bd640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48641605c0 con 0x7f48640afd90 2023-10-18T10:46:57.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.251+0000 7f486a8bd640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f486406c000 con 0x7f486415c7f0 2023-10-18T10:46:57.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.251+0000 7f486a8bd640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f486406c2e0 con 0x7f48640ad5b0 2023-10-18T10:46:57.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.251+0000 7f48637fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f486415c7f0 0x7f486406b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:57.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.251+0000 7f48637fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f486415c7f0 0x7f486406b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57416/0 (socket says 172.21.15.89:57416) 2023-10-18T10:46:57.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.251+0000 7f48637fe640 1 -- 172.21.15.89:0/3471461101 learned_addr learned my addr 172.21.15.89:0/3471461101 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:57.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.251+0000 7f48637fe640 1 -- 172.21.15.89:0/3471461101 >> v2:172.21.15.154:3300/0 conn(0x7f48640ad5b0 msgr2=0x7f4864160a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:57.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.251+0000 7f4868e5a640 1 --2- 172.21.15.89:0/3471461101 >> v2:172.21.15.154:3300/0 conn(0x7f48640ad5b0 0x7f4864160a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:57.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.251+0000 7f48637fe640 1 --2- 172.21.15.89:0/3471461101 >> v2:172.21.15.154:3300/0 conn(0x7f48640ad5b0 0x7f4864160a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:57.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.251+0000 7f48637fe640 1 -- 172.21.15.89:0/3471461101 >> v1:172.21.15.89:6789/0 conn(0x7f48640afd90 legacy=0x7f48640b0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:57.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.251+0000 7f48637fe640 1 -- 172.21.15.89:0/3471461101 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f486406c5c0 con 0x7f486415c7f0 2023-10-18T10:46:57.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.251+0000 7f4868e5a640 1 --2- 172.21.15.89:0/3471461101 >> v2:172.21.15.154:3300/0 conn(0x7f48640ad5b0 0x7f4864160a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:57.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.251+0000 7f4862ffd640 1 -- 172.21.15.89:0/3471461101 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 19232023 0 0) 0x7f48641605c0 con 0x7f48640afd90 2023-10-18T10:46:57.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.255+0000 7f48637fe640 1 --2- 172.21.15.89:0/3471461101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f486415c7f0 0x7f486406b7d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f48640cb750 tx=0x7f485801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d75a69c60446529f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:57.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.255+0000 7f4862ffd640 1 -- 172.21.15.89:0/3471461101 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48580ab600 con 0x7f486415c7f0 2023-10-18T10:46:57.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.255+0000 7f4862ffd640 1 -- 172.21.15.89:0/3471461101 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48580a7050 con 0x7f486415c7f0 2023-10-18T10:46:57.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.255+0000 7f4862ffd640 1 -- 172.21.15.89:0/3471461101 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48580ab980 con 0x7f486415c7f0 2023-10-18T10:46:57.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.255+0000 7f486a8bd640 1 -- 172.21.15.89:0/3471461101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f486415c7f0 msgr2=0x7f486406b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:57.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.255+0000 7f486a8bd640 1 --2- 172.21.15.89:0/3471461101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f486415c7f0 0x7f486406b7d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f48640cb750 tx=0x7f485801c470 comp rx=0 tx=0).stop 2023-10-18T10:46:57.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.255+0000 7f486a8bd640 1 -- 172.21.15.89:0/3471461101 shutdown_connections 2023-10-18T10:46:57.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.255+0000 7f486a8bd640 1 --2- 172.21.15.89:0/3471461101 >> v2:172.21.15.154:3300/0 conn(0x7f48640ad5b0 0x7f4864160a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:57.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.255+0000 7f486a8bd640 1 --2- 172.21.15.89:0/3471461101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f486415c7f0 0x7f486406b7d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:57.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.255+0000 7f486a8bd640 1 -- 172.21.15.89:0/3471461101 >> 172.21.15.89:0/3471461101 conn(0x7f4864069f00 msgr2=0x7f48640cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:57.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.255+0000 7f486a8bd640 1 -- 172.21.15.89:0/3471461101 shutdown_connections 2023-10-18T10:46:57.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.255+0000 7f486a8bd640 1 -- 172.21.15.89:0/3471461101 wait complete. 2023-10-18T10:46:57.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.259+0000 7f486a8bd640 1 Processor -- start 2023-10-18T10:46:57.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.259+0000 7f486a8bd640 1 -- start start 2023-10-18T10:46:57.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.259+0000 7f486a8bd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48640ad5b0 0x7f486414e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:57.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.259+0000 7f486a8bd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48640afd90 0x7f486406ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:57.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.259+0000 7f486a8bd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f486415c7f0 0x7f486415eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:57.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.259+0000 7f486a8bd640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486414c6b0 con 0x7f48640ad5b0 2023-10-18T10:46:57.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.259+0000 7f486a8bd640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4864153690 con 0x7f48640afd90 2023-10-18T10:46:57.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.259+0000 7f486a8bd640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4864153970 con 0x7f486415c7f0 2023-10-18T10:46:57.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.259+0000 7f4863fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48640ad5b0 0x7f486414e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:57.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.259+0000 7f4863fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48640ad5b0 0x7f486414e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57428/0 (socket says 172.21.15.89:57428) 2023-10-18T10:46:57.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.259+0000 7f4863fff640 1 -- 172.21.15.89:0/948787949 learned_addr learned my addr 172.21.15.89:0/948787949 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:57.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.259+0000 7f48637fe640 1 --2- 172.21.15.89:0/948787949 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48640afd90 0x7f486406ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:57.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.259+0000 7f4868e5a640 1 --2- 172.21.15.89:0/948787949 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f486415c7f0 0x7f486415eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:57.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.259+0000 7f4863fff640 1 -- 172.21.15.89:0/948787949 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f486415c7f0 msgr2=0x7f486415eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:57.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.259+0000 7f4863fff640 1 --2- 172.21.15.89:0/948787949 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f486415c7f0 0x7f486415eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:57.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.259+0000 7f4863fff640 1 -- 172.21.15.89:0/948787949 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48640afd90 msgr2=0x7f486406ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:57.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.259+0000 7f4863fff640 1 --2- 172.21.15.89:0/948787949 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48640afd90 0x7f486406ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:57.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.259+0000 7f4863fff640 1 -- 172.21.15.89:0/948787949 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48580094d0 con 0x7f48640ad5b0 2023-10-18T10:46:57.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.259+0000 7f4868e5a640 1 --2- 172.21.15.89:0/948787949 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f486415c7f0 0x7f486415eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:57.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.259+0000 7f4863fff640 1 --2- 172.21.15.89:0/948787949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48640ad5b0 0x7f486414e5c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f485400c4d0 tx=0x7f485400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:57.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.259+0000 7f4860ff9640 1 -- 172.21.15.89:0/948787949 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4854005480 con 0x7f48640ad5b0 2023-10-18T10:46:57.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.259+0000 7f4860ff9640 1 -- 172.21.15.89:0/948787949 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4854003600 con 0x7f48640ad5b0 2023-10-18T10:46:57.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.259+0000 7f4860ff9640 1 -- 172.21.15.89:0/948787949 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48540039b0 con 0x7f48640ad5b0 2023-10-18T10:46:57.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.259+0000 7f486a8bd640 1 -- 172.21.15.89:0/948787949 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4864153c50 con 0x7f48640ad5b0 2023-10-18T10:46:57.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.259+0000 7f486a8bd640 1 -- 172.21.15.89:0/948787949 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4864154210 con 0x7f48640ad5b0 2023-10-18T10:46:57.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.263+0000 7f486a8bd640 1 -- 172.21.15.89:0/948787949 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f486404dbe0 con 0x7f48640ad5b0 2023-10-18T10:46:57.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.263+0000 7f4860ff9640 1 -- 172.21.15.89:0/948787949 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f4854003b10 con 0x7f48640ad5b0 2023-10-18T10:46:57.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.263+0000 7f4860ff9640 1 --2- 172.21.15.89:0/948787949 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4850065680 0x7f4850067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:57.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.263+0000 7f4860ff9640 1 -- 172.21.15.89:0/948787949 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4854082730 con 0x7f48640ad5b0 2023-10-18T10:46:57.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.263+0000 7f48637fe640 1 --2- 172.21.15.89:0/948787949 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4850065680 0x7f4850067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:57.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.263+0000 7f48637fe640 1 --2- 172.21.15.89:0/948787949 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4850065680 0x7f4850067b40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f48640cb5c0 tx=0x7f4858008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:57.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.267+0000 7f4860ff9640 1 -- 172.21.15.89:0/948787949 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f4854053a20 con 0x7f48640ad5b0 2023-10-18T10:46:57.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.419+0000 7f486a8bd640 1 -- 172.21.15.89:0/948787949 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f48640cba40 con 0x7f48640ad5b0 2023-10-18T10:46:57.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.419+0000 7f4860ff9640 1 -- 172.21.15.89:0/948787949 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f48540535b0 con 0x7f48640ad5b0 2023-10-18T10:46:57.421 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-18T10:46:57.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.419+0000 7f486a8bd640 1 -- 172.21.15.89:0/948787949 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4850065680 msgr2=0x7f4850067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:57.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.419+0000 7f486a8bd640 1 --2- 172.21.15.89:0/948787949 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4850065680 0x7f4850067b40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f48640cb5c0 tx=0x7f4858008040 comp rx=0 tx=0).stop 2023-10-18T10:46:57.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.419+0000 7f486a8bd640 1 -- 172.21.15.89:0/948787949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48640ad5b0 msgr2=0x7f486414e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:57.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.419+0000 7f486a8bd640 1 --2- 172.21.15.89:0/948787949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48640ad5b0 0x7f486414e5c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f485400c4d0 tx=0x7f485400c990 comp rx=0 tx=0).stop 2023-10-18T10:46:57.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.419+0000 7f486a8bd640 1 -- 172.21.15.89:0/948787949 shutdown_connections 2023-10-18T10:46:57.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.419+0000 7f486a8bd640 1 --2- 172.21.15.89:0/948787949 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4850065680 0x7f4850067b40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:57.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.419+0000 7f486a8bd640 1 --2- 172.21.15.89:0/948787949 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f486415c7f0 0x7f486415eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:57.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.419+0000 7f486a8bd640 1 --2- 172.21.15.89:0/948787949 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48640afd90 0x7f486406ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:57.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.419+0000 7f486a8bd640 1 --2- 172.21.15.89:0/948787949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48640ad5b0 0x7f486414e5c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:57.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.419+0000 7f486a8bd640 1 -- 172.21.15.89:0/948787949 >> 172.21.15.89:0/948787949 conn(0x7f4864069f00 msgr2=0x7f486415ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:57.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.419+0000 7f486a8bd640 1 -- 172.21.15.89:0/948787949 shutdown_connections 2023-10-18T10:46:57.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.419+0000 7f486a8bd640 1 -- 172.21.15.89:0/948787949 wait complete. 2023-10-18T10:46:57.442 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-18T10:46:57.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.579+0000 7fddc946a640 1 Processor -- start 2023-10-18T10:46:57.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.579+0000 7fddc946a640 1 -- start start 2023-10-18T10:46:57.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.583+0000 7fddc946a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fddc4159490 0x7fddc415d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:57.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.583+0000 7fddc946a640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fddc406bee0 0x7fddc40c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:57.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.583+0000 7fddc946a640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddc415ddf0 con 0x7fddc4070600 2023-10-18T10:46:57.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.583+0000 7fddc946a640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddc4070b90 con 0x7fddc406bee0 2023-10-18T10:46:57.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.583+0000 7fddc946a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc4070e70 con 0x7fddc4159490 2023-10-18T10:46:57.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.583+0000 7fddc37fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fddc406bee0 0x7fddc40c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:57.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.583+0000 7fddc37fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fddc406bee0 0x7fddc40c8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57442/0 (socket says 172.21.15.89:57442) 2023-10-18T10:46:57.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.583+0000 7fddc37fe640 1 -- 172.21.15.89:0/2260688779 learned_addr learned my addr 172.21.15.89:0/2260688779 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:57.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.583+0000 7fddc2ffd640 1 --2- 172.21.15.89:0/2260688779 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fddc4159490 0x7fddc415d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:57.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.583+0000 7fddc37fe640 1 -- 172.21.15.89:0/2260688779 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fddc4159490 msgr2=0x7fddc415d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:57.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.583+0000 7fddc37fe640 1 --2- 172.21.15.89:0/2260688779 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fddc4159490 0x7fddc415d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:57.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.583+0000 7fddc37fe640 1 -- 172.21.15.89:0/2260688779 >> v1:172.21.15.89:6789/0 conn(0x7fddc4070600 legacy=0x7fddc406b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:57.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.583+0000 7fddc37fe640 1 -- 172.21.15.89:0/2260688779 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc4071150 con 0x7fddc406bee0 2023-10-18T10:46:57.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.587+0000 7fddc37fe640 1 --2- 172.21.15.89:0/2260688779 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fddc406bee0 0x7fddc40c8960 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fddbc00d150 tx=0x7fddbc01eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b96c767a1a03a04 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:57.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.587+0000 7fddc27fc640 1 -- 172.21.15.89:0/2260688779 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddbc00b2d0 con 0x7fddc406bee0 2023-10-18T10:46:57.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.587+0000 7fddc27fc640 1 -- 172.21.15.89:0/2260688779 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddbc00b430 con 0x7fddc406bee0 2023-10-18T10:46:57.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.587+0000 7fddc27fc640 1 -- 172.21.15.89:0/2260688779 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddbc0a7c40 con 0x7fddc406bee0 2023-10-18T10:46:57.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.587+0000 7fddc946a640 1 -- 172.21.15.89:0/2260688779 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fddc406bee0 msgr2=0x7fddc40c8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:57.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.587+0000 7fddc946a640 1 --2- 172.21.15.89:0/2260688779 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fddc406bee0 0x7fddc40c8960 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fddbc00d150 tx=0x7fddbc01eb50 comp rx=0 tx=0).stop 2023-10-18T10:46:57.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.587+0000 7fddc946a640 1 -- 172.21.15.89:0/2260688779 shutdown_connections 2023-10-18T10:46:57.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.587+0000 7fddc946a640 1 --2- 172.21.15.89:0/2260688779 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fddc406bee0 0x7fddc40c8960 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:57.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.587+0000 7fddc946a640 1 --2- 172.21.15.89:0/2260688779 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fddc4159490 0x7fddc415d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:57.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.587+0000 7fddc946a640 1 -- 172.21.15.89:0/2260688779 >> 172.21.15.89:0/2260688779 conn(0x7fddc4069f00 msgr2=0x7fddc40cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:57.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.587+0000 7fddc946a640 1 -- 172.21.15.89:0/2260688779 shutdown_connections 2023-10-18T10:46:57.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.587+0000 7fddc946a640 1 -- 172.21.15.89:0/2260688779 wait complete. 2023-10-18T10:46:57.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.587+0000 7fddc946a640 1 Processor -- start 2023-10-18T10:46:57.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.587+0000 7fddc946a640 1 -- start start 2023-10-18T10:46:57.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.591+0000 7fddc946a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fddc406bee0 0x7fddc4157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:57.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.591+0000 7fddc946a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fddc4070600 0x7fddc4144670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:57.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.591+0000 7fddc946a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fddc4159490 0x7fddc4156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:57.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.591+0000 7fddc946a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc4163d40 con 0x7fddc4159490 2023-10-18T10:46:57.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.591+0000 7fddc946a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc4159bc0 con 0x7fddc4070600 2023-10-18T10:46:57.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.591+0000 7fddc946a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc4163780 con 0x7fddc406bee0 2023-10-18T10:46:57.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.591+0000 7fddc37fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fddc4159490 0x7fddc4156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.591+0000 7fddc37fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fddc4159490 0x7fddc4156010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57450/0 (socket says 172.21.15.89:57450) 2023-10-18T10:46:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.591+0000 7fddc37fe640 1 -- 172.21.15.89:0/3662231295 learned_addr learned my addr 172.21.15.89:0/3662231295 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.591+0000 7fddba7fc640 1 --2- 172.21.15.89:0/3662231295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fddc4070600 0x7fddc4144670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.591+0000 7fddc2ffd640 1 --2- 172.21.15.89:0/3662231295 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fddc406bee0 0x7fddc4157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.591+0000 7fddc37fe640 1 -- 172.21.15.89:0/3662231295 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fddc406bee0 msgr2=0x7fddc4157c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.591+0000 7fddc37fe640 1 --2- 172.21.15.89:0/3662231295 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fddc406bee0 0x7fddc4157c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.591+0000 7fddc37fe640 1 -- 172.21.15.89:0/3662231295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fddc4070600 msgr2=0x7fddc4144670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.591+0000 7fddc37fe640 1 --2- 172.21.15.89:0/3662231295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fddc4070600 0x7fddc4144670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.591+0000 7fddc37fe640 1 -- 172.21.15.89:0/3662231295 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddbc00bd40 con 0x7fddc4159490 2023-10-18T10:46:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.591+0000 7fddc37fe640 1 --2- 172.21.15.89:0/3662231295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fddc4159490 0x7fddc4156010 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fddbc01c780 tx=0x7fddbc01b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.591+0000 7fddbbfff640 1 -- 172.21.15.89:0/3662231295 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddbc009800 con 0x7fddc4159490 2023-10-18T10:46:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.591+0000 7fddbbfff640 1 -- 172.21.15.89:0/3662231295 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddbc009980 con 0x7fddc4159490 2023-10-18T10:46:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.591+0000 7fddc946a640 1 -- 172.21.15.89:0/3662231295 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc4163a60 con 0x7fddc4159490 2023-10-18T10:46:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.591+0000 7fddbbfff640 1 -- 172.21.15.89:0/3662231295 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddbc009d30 con 0x7fddc4159490 2023-10-18T10:46:57.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.591+0000 7fddc946a640 1 -- 172.21.15.89:0/3662231295 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc4156fc0 con 0x7fddc4159490 2023-10-18T10:46:57.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.591+0000 7fddc946a640 1 -- 172.21.15.89:0/3662231295 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc404dbe0 con 0x7fddc4159490 2023-10-18T10:46:57.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.591+0000 7fddbbfff640 1 -- 172.21.15.89:0/3662231295 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fddbc0c3460 con 0x7fddc4159490 2023-10-18T10:46:57.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.595+0000 7fddbbfff640 1 --2- 172.21.15.89:0/3662231295 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdda4065680 0x7fdda4067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:57.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.595+0000 7fddbbfff640 1 -- 172.21.15.89:0/3662231295 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fddbc11e800 con 0x7fddc4159490 2023-10-18T10:46:57.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.595+0000 7fddc2ffd640 1 --2- 172.21.15.89:0/3662231295 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdda4065680 0x7fdda4067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:57.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.595+0000 7fddc2ffd640 1 --2- 172.21.15.89:0/3662231295 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdda4065680 0x7fdda4067b40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fddb4009a50 tx=0x7fddb4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:57.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.599+0000 7fddbbfff640 1 -- 172.21.15.89:0/3662231295 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fddbc0ebf90 con 0x7fddc4159490 2023-10-18T10:46:57.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.747+0000 7fddc946a640 1 -- 172.21.15.89:0/3662231295 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fddc41571b0 con 0x7fddc4159490 2023-10-18T10:46:57.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.747+0000 7fddbbfff640 1 -- 172.21.15.89:0/3662231295 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7fddbc0ef5b0 con 0x7fddc4159490 2023-10-18T10:46:57.748 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-10-18T10:46:57.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.747+0000 7fddc946a640 1 -- 172.21.15.89:0/3662231295 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdda4065680 msgr2=0x7fdda4067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:57.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.747+0000 7fddc946a640 1 --2- 172.21.15.89:0/3662231295 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdda4065680 0x7fdda4067b40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fddb4009a50 tx=0x7fddb4008040 comp rx=0 tx=0).stop 2023-10-18T10:46:57.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.747+0000 7fddc946a640 1 -- 172.21.15.89:0/3662231295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fddc4159490 msgr2=0x7fddc4156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:57.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.747+0000 7fddc946a640 1 --2- 172.21.15.89:0/3662231295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fddc4159490 0x7fddc4156010 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fddbc01c780 tx=0x7fddbc01b8f0 comp rx=0 tx=0).stop 2023-10-18T10:46:57.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.747+0000 7fddc946a640 1 -- 172.21.15.89:0/3662231295 shutdown_connections 2023-10-18T10:46:57.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.747+0000 7fddc946a640 1 --2- 172.21.15.89:0/3662231295 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdda4065680 0x7fdda4067b40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:57.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.747+0000 7fddc946a640 1 --2- 172.21.15.89:0/3662231295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fddc4159490 0x7fddc4156010 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:57.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.747+0000 7fddc946a640 1 --2- 172.21.15.89:0/3662231295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fddc4070600 0x7fddc4144670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:57.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.747+0000 7fddc946a640 1 --2- 172.21.15.89:0/3662231295 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fddc406bee0 0x7fddc4157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:57.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.747+0000 7fddc946a640 1 -- 172.21.15.89:0/3662231295 >> 172.21.15.89:0/3662231295 conn(0x7fddc4069f00 msgr2=0x7fddc40c1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:57.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.747+0000 7fddc946a640 1 -- 172.21.15.89:0/3662231295 shutdown_connections 2023-10-18T10:46:57.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.747+0000 7fddc946a640 1 -- 172.21.15.89:0/3662231295 wait complete. 2023-10-18T10:46:57.769 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:46:57.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.907+0000 7f6eec940640 1 Processor -- start 2023-10-18T10:46:57.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.907+0000 7f6eec940640 1 -- start start 2023-10-18T10:46:57.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.907+0000 7f6eec940640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6ee815c550 0x7f6ee80d2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:57.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.911+0000 7f6eec940640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ee80d1e50 con 0x7f6ee80cfd20 2023-10-18T10:46:57.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.911+0000 7f6eec940640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ee80d2130 con 0x7f6ee80b1860 2023-10-18T10:46:57.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.911+0000 7f6eec940640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ee80d2410 con 0x7f6ee815c550 2023-10-18T10:46:57.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.911+0000 7f6ee6d9d640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6ee80cfd20 0x7f6ee80b55f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58178/0 (socket says 172.21.15.89:58178) 2023-10-18T10:46:57.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.911+0000 7f6ee6d9d640 1 -- 172.21.15.89:0/359367878 learned_addr learned my addr 172.21.15.89:0/359367878 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:57.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.911+0000 7f6ee659c640 1 --2- 172.21.15.89:0/359367878 >> v2:172.21.15.154:3300/0 conn(0x7f6ee815c550 0x7f6ee80d2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:57.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.911+0000 7f6ee5d9b640 1 -- 172.21.15.89:0/359367878 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1233697444 0 0) 0x7f6ee80d1e50 con 0x7f6ee80cfd20 2023-10-18T10:46:57.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.911+0000 7f6ee5d9b640 1 -- 172.21.15.89:0/359367878 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ed4003530 con 0x7f6ee80cfd20 2023-10-18T10:46:57.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.911+0000 7f6ee5d9b640 1 -- 172.21.15.89:0/359367878 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 515215357 0 0) 0x7f6ee80d2130 con 0x7f6ee80b1860 2023-10-18T10:46:57.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.911+0000 7f6ee5d9b640 1 -- 172.21.15.89:0/359367878 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ee80d1e50 con 0x7f6ee80b1860 2023-10-18T10:46:57.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.911+0000 7f6ee5d9b640 1 -- 172.21.15.89:0/359367878 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3032228359 0 0) 0x7f6ed4003530 con 0x7f6ee80cfd20 2023-10-18T10:46:57.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.911+0000 7f6ee659c640 1 -- 172.21.15.89:0/359367878 >> v1:172.21.15.154:6789/0 conn(0x7f6ee80b1860 legacy=0x7f6ee80cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:57.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.911+0000 7f6ee659c640 1 -- 172.21.15.89:0/359367878 >> v1:172.21.15.89:6789/0 conn(0x7f6ee80cfd20 legacy=0x7f6ee80b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:57.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.911+0000 7f6ee659c640 1 -- 172.21.15.89:0/359367878 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee80d26f0 con 0x7f6ee815c550 2023-10-18T10:46:57.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.915+0000 7f6ee659c640 1 --2- 172.21.15.89:0/359367878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ee815c550 0x7f6ee80d2ef0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80097c0 tx=0x7f6ed801cdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5bcc1c2dfba3719 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:57.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.915+0000 7f6ee5d9b640 1 -- 172.21.15.89:0/359367878 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0001a00 con 0x7f6ee815c550 2023-10-18T10:46:57.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.915+0000 7f6ee5d9b640 1 -- 172.21.15.89:0/359367878 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed80ac440 con 0x7f6ee815c550 2023-10-18T10:46:57.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.915+0000 7f6ee5d9b640 1 -- 172.21.15.89:0/359367878 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed80ac7d0 con 0x7f6ee815c550 2023-10-18T10:46:57.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.915+0000 7f6eec940640 1 -- 172.21.15.89:0/359367878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ee815c550 msgr2=0x7f6ee80d2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:57.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.915+0000 7f6eec940640 1 --2- 172.21.15.89:0/359367878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ee815c550 0x7f6ee80d2ef0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80097c0 tx=0x7f6ed801cdd0 comp rx=0 tx=0).stop 2023-10-18T10:46:57.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.915+0000 7f6eec940640 1 -- 172.21.15.89:0/359367878 shutdown_connections 2023-10-18T10:46:57.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.915+0000 7f6eec940640 1 --2- 172.21.15.89:0/359367878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ee815c550 0x7f6ee80d2ef0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:57.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.915+0000 7f6eec940640 1 -- 172.21.15.89:0/359367878 >> 172.21.15.89:0/359367878 conn(0x7f6ee8069d00 msgr2=0x7f6ee806a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:57.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.915+0000 7f6eec940640 1 -- 172.21.15.89:0/359367878 shutdown_connections 2023-10-18T10:46:57.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.915+0000 7f6eec940640 1 -- 172.21.15.89:0/359367878 wait complete. 2023-10-18T10:46:57.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.915+0000 7f6eec940640 1 Processor -- start 2023-10-18T10:46:57.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.915+0000 7f6eec940640 1 -- start start 2023-10-18T10:46:57.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.915+0000 7f6eec940640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ee80b1860 0x7f6ee814c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:57.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.919+0000 7f6eec940640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ee80cfd20 0x7f6ee80c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:57.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.919+0000 7f6eec940640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ee815c550 0x7f6ee8150990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:57.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.919+0000 7f6eec940640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee8157370 con 0x7f6ee80b1860 2023-10-18T10:46:57.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.919+0000 7f6eec940640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee815ed20 con 0x7f6ee80cfd20 2023-10-18T10:46:57.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.919+0000 7f6eec940640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee8154f60 con 0x7f6ee815c550 2023-10-18T10:46:57.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.919+0000 7f6ee659c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ee80b1860 0x7f6ee814c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:57.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.919+0000 7f6ee659c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ee80b1860 0x7f6ee814c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57464/0 (socket says 172.21.15.89:57464) 2023-10-18T10:46:57.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.919+0000 7f6ee659c640 1 -- 172.21.15.89:0/1046103862 learned_addr learned my addr 172.21.15.89:0/1046103862 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:57.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.919+0000 7f6ee659c640 1 -- 172.21.15.89:0/1046103862 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ee815c550 msgr2=0x7f6ee8150990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:57.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.919+0000 7f6ee659c640 1 --2- 172.21.15.89:0/1046103862 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ee815c550 0x7f6ee8150990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:57.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.919+0000 7f6ee659c640 1 -- 172.21.15.89:0/1046103862 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ee80cfd20 msgr2=0x7f6ee80c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:57.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.919+0000 7f6ee659c640 1 --2- 172.21.15.89:0/1046103862 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ee80cfd20 0x7f6ee80c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:57.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.919+0000 7f6ee659c640 1 -- 172.21.15.89:0/1046103862 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed8009420 con 0x7f6ee80b1860 2023-10-18T10:46:57.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.919+0000 7f6ee659c640 1 --2- 172.21.15.89:0/1046103862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ee80b1860 0x7f6ee814c680 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80af040 tx=0x7f6ed801abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:57.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.919+0000 7f6edf7fe640 1 -- 172.21.15.89:0/1046103862 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed80acea0 con 0x7f6ee80b1860 2023-10-18T10:46:57.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.919+0000 7f6edf7fe640 1 -- 172.21.15.89:0/1046103862 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed8004220 con 0x7f6ee80b1860 2023-10-18T10:46:57.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.919+0000 7f6eec940640 1 -- 172.21.15.89:0/1046103862 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ee8155240 con 0x7f6ee80b1860 2023-10-18T10:46:57.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.919+0000 7f6edf7fe640 1 -- 172.21.15.89:0/1046103862 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8003620 con 0x7f6ee80b1860 2023-10-18T10:46:57.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.919+0000 7f6eec940640 1 -- 172.21.15.89:0/1046103862 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee8155800 con 0x7f6ee80b1860 2023-10-18T10:46:57.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.919+0000 7f6eec940640 1 -- 172.21.15.89:0/1046103862 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ee804dbe0 con 0x7f6ee80b1860 2023-10-18T10:46:57.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.919+0000 7f6edf7fe640 1 -- 172.21.15.89:0/1046103862 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f6ed8003780 con 0x7f6ee80b1860 2023-10-18T10:46:57.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.923+0000 7f6edf7fe640 1 --2- 172.21.15.89:0/1046103862 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6ec8065680 0x7f6ec8067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:57.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.923+0000 7f6edf7fe640 1 -- 172.21.15.89:0/1046103862 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6ed811cc50 con 0x7f6ee80b1860 2023-10-18T10:46:57.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.923+0000 7f6eddd9b640 1 --2- 172.21.15.89:0/1046103862 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6ec8065680 0x7f6ec8067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:57.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.923+0000 7f6eddd9b640 1 --2- 172.21.15.89:0/1046103862 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6ec8065680 0x7f6ec8067b40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0007fd0 tx=0x7f6ed0007960 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:57.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:57.923+0000 7f6edf7fe640 1 -- 172.21.15.89:0/1046103862 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f6ed80ea340 con 0x7f6ee80b1860 2023-10-18T10:46:58.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.071+0000 7f6eec940640 1 -- 172.21.15.89:0/1046103862 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6ee80cd300 con 0x7f6ee80b1860 2023-10-18T10:46:58.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.075+0000 7f6edf7fe640 1 -- 172.21.15.89:0/1046103862 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6ed80ed960 con 0x7f6ee80b1860 2023-10-18T10:46:58.076 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-18T10:46:58.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.075+0000 7f6eec940640 1 -- 172.21.15.89:0/1046103862 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6ec8065680 msgr2=0x7f6ec8067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:58.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.075+0000 7f6eec940640 1 --2- 172.21.15.89:0/1046103862 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6ec8065680 0x7f6ec8067b40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0007fd0 tx=0x7f6ed0007960 comp rx=0 tx=0).stop 2023-10-18T10:46:58.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.075+0000 7f6eec940640 1 -- 172.21.15.89:0/1046103862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ee80b1860 msgr2=0x7f6ee814c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:58.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.075+0000 7f6eec940640 1 --2- 172.21.15.89:0/1046103862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ee80b1860 0x7f6ee814c680 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80af040 tx=0x7f6ed801abb0 comp rx=0 tx=0).stop 2023-10-18T10:46:58.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.075+0000 7f6eec940640 1 -- 172.21.15.89:0/1046103862 shutdown_connections 2023-10-18T10:46:58.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.075+0000 7f6eec940640 1 --2- 172.21.15.89:0/1046103862 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6ec8065680 0x7f6ec8067b40 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:58.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.075+0000 7f6eec940640 1 --2- 172.21.15.89:0/1046103862 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ee815c550 0x7f6ee8150990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:58.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.075+0000 7f6eec940640 1 --2- 172.21.15.89:0/1046103862 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ee80cfd20 0x7f6ee80c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:58.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.075+0000 7f6eec940640 1 --2- 172.21.15.89:0/1046103862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ee80b1860 0x7f6ee814c680 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:58.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.075+0000 7f6eec940640 1 -- 172.21.15.89:0/1046103862 >> 172.21.15.89:0/1046103862 conn(0x7f6ee8069d00 msgr2=0x7f6ee80b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:58.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.075+0000 7f6eec940640 1 -- 172.21.15.89:0/1046103862 shutdown_connections 2023-10-18T10:46:58.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.075+0000 7f6eec940640 1 -- 172.21.15.89:0/1046103862 wait complete. 2023-10-18T10:46:58.096 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:46:58.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.235+0000 7ff7fbd8f640 1 Processor -- start 2023-10-18T10:46:58.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.235+0000 7ff7fbd8f640 1 -- start start 2023-10-18T10:46:58.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.235+0000 7ff7fbd8f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff7f415c940 0x7ff7f40d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:58.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.235+0000 7ff7fbd8f640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7f415da50 con 0x7ff7f40d0e70 2023-10-18T10:46:58.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.235+0000 7ff7fbd8f640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7f415ce40 con 0x7ff7f40d2f40 2023-10-18T10:46:58.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.235+0000 7ff7fbd8f640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7f415d120 con 0x7ff7f415c940 2023-10-18T10:46:58.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.235+0000 7ff7f9b2b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff7f415c940 0x7ff7f40d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:58.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.235+0000 7ff7f9b2b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff7f415c940 0x7ff7f40d2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57472/0 (socket says 172.21.15.89:57472) 2023-10-18T10:46:58.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.235+0000 7ff7f9b2b640 1 -- 172.21.15.89:0/3233887592 learned_addr learned my addr 172.21.15.89:0/3233887592 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:58.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.235+0000 7ff7f9b2b640 1 -- 172.21.15.89:0/3233887592 >> v1:172.21.15.154:6789/0 conn(0x7ff7f40d2f40 legacy=0x7ff7f40d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:58.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.235+0000 7ff7f9b2b640 1 -- 172.21.15.89:0/3233887592 >> v1:172.21.15.89:6789/0 conn(0x7ff7f40d0e70 legacy=0x7ff7f40cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:58.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.235+0000 7ff7f9b2b640 1 -- 172.21.15.89:0/3233887592 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f415d400 con 0x7ff7f415c940 2023-10-18T10:46:58.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.239+0000 7ff7f9b2b640 1 --2- 172.21.15.89:0/3233887592 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7f415c940 0x7ff7f40d2a00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0009870 tx=0x7ff7f001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=563c9651ee70c8c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:58.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.239+0000 7ff7f8b29640 1 -- 172.21.15.89:0/3233887592 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f00a95f0 con 0x7ff7f415c940 2023-10-18T10:46:58.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.239+0000 7ff7f8b29640 1 -- 172.21.15.89:0/3233887592 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f00a9750 con 0x7ff7f415c940 2023-10-18T10:46:58.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.239+0000 7ff7f8b29640 1 -- 172.21.15.89:0/3233887592 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f00a9ae0 con 0x7ff7f415c940 2023-10-18T10:46:58.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.239+0000 7ff7fbd8f640 1 -- 172.21.15.89:0/3233887592 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7f415c940 msgr2=0x7ff7f40d2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:58.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.239+0000 7ff7fbd8f640 1 --2- 172.21.15.89:0/3233887592 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7f415c940 0x7ff7f40d2a00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0009870 tx=0x7ff7f001c510 comp rx=0 tx=0).stop 2023-10-18T10:46:58.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.239+0000 7ff7fbd8f640 1 -- 172.21.15.89:0/3233887592 shutdown_connections 2023-10-18T10:46:58.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.239+0000 7ff7fbd8f640 1 --2- 172.21.15.89:0/3233887592 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7f415c940 0x7ff7f40d2a00 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:58.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.239+0000 7ff7fbd8f640 1 -- 172.21.15.89:0/3233887592 >> 172.21.15.89:0/3233887592 conn(0x7ff7f4089fd0 msgr2=0x7ff7f409b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:58.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.239+0000 7ff7fbd8f640 1 -- 172.21.15.89:0/3233887592 shutdown_connections 2023-10-18T10:46:58.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.239+0000 7ff7fbd8f640 1 -- 172.21.15.89:0/3233887592 wait complete. 2023-10-18T10:46:58.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.243+0000 7ff7fbd8f640 1 Processor -- start 2023-10-18T10:46:58.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.243+0000 7ff7fbd8f640 1 -- start start 2023-10-18T10:46:58.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.243+0000 7ff7fbd8f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7f40d0e70 0x7ff7f40b6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:58.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.243+0000 7ff7fbd8f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7f40b7c20 0x7ff7f4153690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:58.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.243+0000 7ff7fbd8f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7f40cbfe0 0x7ff7f4154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:58.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.243+0000 7ff7fbd8f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f4153bd0 con 0x7ff7f40cbfe0 2023-10-18T10:46:58.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.243+0000 7ff7fbd8f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f40b5310 con 0x7ff7f40b7c20 2023-10-18T10:46:58.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.243+0000 7ff7fbd8f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f40b5580 con 0x7ff7f40d0e70 2023-10-18T10:46:58.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.243+0000 7ff7fa32c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7f40cbfe0 0x7ff7f4154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:58.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.243+0000 7ff7fa32c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7f40cbfe0 0x7ff7f4154b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57488/0 (socket says 172.21.15.89:57488) 2023-10-18T10:46:58.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.243+0000 7ff7fa32c640 1 -- 172.21.15.89:0/1350475569 learned_addr learned my addr 172.21.15.89:0/1350475569 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:58.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.243+0000 7ff7f9b2b640 1 --2- 172.21.15.89:0/1350475569 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7f40d0e70 0x7ff7f40b6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:58.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.243+0000 7ff7f932a640 1 --2- 172.21.15.89:0/1350475569 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7f40b7c20 0x7ff7f4153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:58.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.243+0000 7ff7fa32c640 1 -- 172.21.15.89:0/1350475569 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7f40d0e70 msgr2=0x7ff7f40b6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:58.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.243+0000 7ff7fa32c640 1 --2- 172.21.15.89:0/1350475569 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7f40d0e70 0x7ff7f40b6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:58.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.243+0000 7ff7fa32c640 1 -- 172.21.15.89:0/1350475569 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7f40b7c20 msgr2=0x7ff7f4153690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:58.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.243+0000 7ff7fa32c640 1 --2- 172.21.15.89:0/1350475569 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7f40b7c20 0x7ff7f4153690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:58.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.243+0000 7ff7fa32c640 1 -- 172.21.15.89:0/1350475569 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f00094d0 con 0x7ff7f40cbfe0 2023-10-18T10:46:58.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.243+0000 7ff7fa32c640 1 --2- 172.21.15.89:0/1350475569 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7f40cbfe0 0x7ff7f4154b60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00c7a0 tx=0x7ff7ec00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:58.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.243+0000 7ff7ea7fc640 1 -- 172.21.15.89:0/1350475569 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec003b80 con 0x7ff7f40cbfe0 2023-10-18T10:46:58.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.243+0000 7ff7ea7fc640 1 -- 172.21.15.89:0/1350475569 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ec003ce0 con 0x7ff7f40cbfe0 2023-10-18T10:46:58.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.243+0000 7ff7ea7fc640 1 -- 172.21.15.89:0/1350475569 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec00aa30 con 0x7ff7f40cbfe0 2023-10-18T10:46:58.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.243+0000 7ff7fbd8f640 1 -- 172.21.15.89:0/1350475569 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f40b5860 con 0x7ff7f40cbfe0 2023-10-18T10:46:58.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.243+0000 7ff7fbd8f640 1 -- 172.21.15.89:0/1350475569 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f40b5e20 con 0x7ff7f40cbfe0 2023-10-18T10:46:58.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.247+0000 7ff7fbd8f640 1 -- 172.21.15.89:0/1350475569 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7f40b8000 con 0x7ff7f40cbfe0 2023-10-18T10:46:58.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.247+0000 7ff7ea7fc640 1 -- 172.21.15.89:0/1350475569 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7ff7ec00ab90 con 0x7ff7f40cbfe0 2023-10-18T10:46:58.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.247+0000 7ff7ea7fc640 1 --2- 172.21.15.89:0/1350475569 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff7d8065680 0x7ff7d8067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:58.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.247+0000 7ff7ea7fc640 1 -- 172.21.15.89:0/1350475569 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff7ec082f80 con 0x7ff7f40cbfe0 2023-10-18T10:46:58.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.247+0000 7ff7f9b2b640 1 --2- 172.21.15.89:0/1350475569 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff7d8065680 0x7ff7d8067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:58.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.247+0000 7ff7f9b2b640 1 --2- 172.21.15.89:0/1350475569 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff7d8065680 0x7ff7d8067b40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff7f00ae000 tx=0x7ff7f00b3040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:58.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.251+0000 7ff7ea7fc640 1 -- 172.21.15.89:0/1350475569 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7ff7ec050700 con 0x7ff7f40cbfe0 2023-10-18T10:46:58.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.399+0000 7ff7fbd8f640 1 -- 172.21.15.89:0/1350475569 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff7f40b56c0 con 0x7ff7f40cbfe0 2023-10-18T10:46:58.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.399+0000 7ff7ea7fc640 1 -- 172.21.15.89:0/1350475569 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff7f40b56c0 con 0x7ff7f40cbfe0 2023-10-18T10:46:58.403 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-18T10:46:58.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.403+0000 7ff7fbd8f640 1 -- 172.21.15.89:0/1350475569 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff7d8065680 msgr2=0x7ff7d8067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:58.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.403+0000 7ff7fbd8f640 1 --2- 172.21.15.89:0/1350475569 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff7d8065680 0x7ff7d8067b40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff7f00ae000 tx=0x7ff7f00b3040 comp rx=0 tx=0).stop 2023-10-18T10:46:58.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.403+0000 7ff7fbd8f640 1 -- 172.21.15.89:0/1350475569 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7f40cbfe0 msgr2=0x7ff7f4154b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:58.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.403+0000 7ff7fbd8f640 1 --2- 172.21.15.89:0/1350475569 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7f40cbfe0 0x7ff7f4154b60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00c7a0 tx=0x7ff7ec00cc60 comp rx=0 tx=0).stop 2023-10-18T10:46:58.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.403+0000 7ff7fbd8f640 1 -- 172.21.15.89:0/1350475569 shutdown_connections 2023-10-18T10:46:58.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.403+0000 7ff7fbd8f640 1 --2- 172.21.15.89:0/1350475569 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff7d8065680 0x7ff7d8067b40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:58.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.403+0000 7ff7fbd8f640 1 --2- 172.21.15.89:0/1350475569 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7f40cbfe0 0x7ff7f4154b60 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:58.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.403+0000 7ff7fbd8f640 1 --2- 172.21.15.89:0/1350475569 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7f40b7c20 0x7ff7f4153690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:58.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.403+0000 7ff7fbd8f640 1 --2- 172.21.15.89:0/1350475569 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7f40d0e70 0x7ff7f40b6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:58.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.403+0000 7ff7fbd8f640 1 -- 172.21.15.89:0/1350475569 >> 172.21.15.89:0/1350475569 conn(0x7ff7f4089fd0 msgr2=0x7ff7f415dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:58.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.403+0000 7ff7fbd8f640 1 -- 172.21.15.89:0/1350475569 shutdown_connections 2023-10-18T10:46:58.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.403+0000 7ff7fbd8f640 1 -- 172.21.15.89:0/1350475569 wait complete. 2023-10-18T10:46:58.424 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T10:46:58.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.563+0000 7f70f3ae6640 1 Processor -- start 2023-10-18T10:46:58.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.563+0000 7f70f3ae6640 1 -- start start 2023-10-18T10:46:58.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.563+0000 7f70f3ae6640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f70ec15c940 0x7f70ec0d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:58.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.563+0000 7f70f3ae6640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70ec0d0e70 0x7f70ec0cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:58.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.563+0000 7f70f3ae6640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70ec15da50 con 0x7f70ec0d2f40 2023-10-18T10:46:58.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.563+0000 7f70f3ae6640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70ec15ce40 con 0x7f70ec15c940 2023-10-18T10:46:58.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.563+0000 7f70f3ae6640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec15d120 con 0x7f70ec0d0e70 2023-10-18T10:46:58.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.563+0000 7f70f1882640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f70ec15c940 0x7f70ec0d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:58.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.563+0000 7f70f1882640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f70ec15c940 0x7f70ec0d2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57502/0 (socket says 172.21.15.89:57502) 2023-10-18T10:46:58.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.563+0000 7f70f1882640 1 -- 172.21.15.89:0/3686105372 learned_addr learned my addr 172.21.15.89:0/3686105372 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:58.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.563+0000 7f70f1081640 1 --1- 172.21.15.89:0/3686105372 >> v1:172.21.15.89:6789/0 conn(0x7f70ec0d2f40 0x7f70ec0d0760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58188/0 (socket says 172.21.15.89:58188) 2023-10-18T10:46:58.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.563+0000 7f70f2083640 1 --2- 172.21.15.89:0/3686105372 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70ec0d0e70 0x7f70ec0cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:58.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.563+0000 7f70f1882640 1 -- 172.21.15.89:0/3686105372 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70ec0d0e70 msgr2=0x7f70ec0cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:58.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.563+0000 7f70f1882640 1 --2- 172.21.15.89:0/3686105372 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70ec0d0e70 0x7f70ec0cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:58.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.563+0000 7f70f1882640 1 -- 172.21.15.89:0/3686105372 >> v1:172.21.15.89:6789/0 conn(0x7f70ec0d2f40 legacy=0x7f70ec0d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:58.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.563+0000 7f70f1882640 1 -- 172.21.15.89:0/3686105372 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ec15d400 con 0x7f70ec15c940 2023-10-18T10:46:58.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.563+0000 7f70f2083640 1 --2- 172.21.15.89:0/3686105372 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70ec0d0e70 0x7f70ec0cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:58.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.567+0000 7f70f1882640 1 --2- 172.21.15.89:0/3686105372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70ec15c940 0x7f70ec0d2a00 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f70e8009870 tx=0x7f70e801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d1b19b508053f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:58.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.567+0000 7f70f0880640 1 -- 172.21.15.89:0/3686105372 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e80aa610 con 0x7f70ec15c940 2023-10-18T10:46:58.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.567+0000 7f70f0880640 1 -- 172.21.15.89:0/3686105372 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e80a6040 con 0x7f70ec15c940 2023-10-18T10:46:58.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.567+0000 7f70f0880640 1 -- 172.21.15.89:0/3686105372 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e80aa990 con 0x7f70ec15c940 2023-10-18T10:46:58.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.567+0000 7f70f3ae6640 1 -- 172.21.15.89:0/3686105372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70ec15c940 msgr2=0x7f70ec0d2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:58.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.567+0000 7f70f3ae6640 1 --2- 172.21.15.89:0/3686105372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70ec15c940 0x7f70ec0d2a00 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f70e8009870 tx=0x7f70e801c510 comp rx=0 tx=0).stop 2023-10-18T10:46:58.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.567+0000 7f70f3ae6640 1 -- 172.21.15.89:0/3686105372 shutdown_connections 2023-10-18T10:46:58.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.567+0000 7f70f3ae6640 1 --2- 172.21.15.89:0/3686105372 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70ec0d0e70 0x7f70ec0cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:58.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.567+0000 7f70f3ae6640 1 --2- 172.21.15.89:0/3686105372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70ec15c940 0x7f70ec0d2a00 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:58.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.567+0000 7f70f3ae6640 1 -- 172.21.15.89:0/3686105372 >> 172.21.15.89:0/3686105372 conn(0x7f70ec089fd0 msgr2=0x7f70ec09b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:58.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.567+0000 7f70f3ae6640 1 -- 172.21.15.89:0/3686105372 shutdown_connections 2023-10-18T10:46:58.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.567+0000 7f70f3ae6640 1 -- 172.21.15.89:0/3686105372 wait complete. 2023-10-18T10:46:58.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.571+0000 7f70f3ae6640 1 Processor -- start 2023-10-18T10:46:58.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.571+0000 7f70f3ae6640 1 -- start start 2023-10-18T10:46:58.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.571+0000 7f70f3ae6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70ec0d0e70 0x7f70ec0b6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:58.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.571+0000 7f70f3ae6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70ec152460 0x7f70ec0b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:58.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.571+0000 7f70f3ae6640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70ec150dd0 0x7f70ec153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:58.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.571+0000 7f70f3ae6640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec0b8140 con 0x7f70ec0d0e70 2023-10-18T10:46:58.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.571+0000 7f70f3ae6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec1528e0 con 0x7f70ec152460 2023-10-18T10:46:58.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.571+0000 7f70f3ae6640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec151fd0 con 0x7f70ec150dd0 2023-10-18T10:46:58.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.571+0000 7f70f1882640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70ec0d0e70 0x7f70ec0b6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:58.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.571+0000 7f70f1882640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70ec0d0e70 0x7f70ec0b6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57512/0 (socket says 172.21.15.89:57512) 2023-10-18T10:46:58.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.571+0000 7f70f1882640 1 -- 172.21.15.89:0/2025601651 learned_addr learned my addr 172.21.15.89:0/2025601651 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:58.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.571+0000 7f70f1081640 1 --2- 172.21.15.89:0/2025601651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70ec152460 0x7f70ec0b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:58.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.571+0000 7f70f2083640 1 --2- 172.21.15.89:0/2025601651 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70ec150dd0 0x7f70ec153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:58.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.571+0000 7f70f1882640 1 -- 172.21.15.89:0/2025601651 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70ec150dd0 msgr2=0x7f70ec153570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:58.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.571+0000 7f70f1882640 1 --2- 172.21.15.89:0/2025601651 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70ec150dd0 0x7f70ec153570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:58.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.571+0000 7f70f1882640 1 -- 172.21.15.89:0/2025601651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70ec152460 msgr2=0x7f70ec0b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:58.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.571+0000 7f70f1882640 1 --2- 172.21.15.89:0/2025601651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70ec152460 0x7f70ec0b7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:58.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.571+0000 7f70f1882640 1 -- 172.21.15.89:0/2025601651 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e80094d0 con 0x7f70ec0d0e70 2023-10-18T10:46:58.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.571+0000 7f70f2083640 1 --2- 172.21.15.89:0/2025601651 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70ec150dd0 0x7f70ec153570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:58.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.571+0000 7f70f1882640 1 --2- 172.21.15.89:0/2025601651 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70ec0d0e70 0x7f70ec0b6140 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f70e801abc0 tx=0x7f70e801ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:58.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.571+0000 7f70e27fc640 1 -- 172.21.15.89:0/2025601651 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e80bd5f0 con 0x7f70ec0d0e70 2023-10-18T10:46:58.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.571+0000 7f70e27fc640 1 -- 172.21.15.89:0/2025601651 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e80bd770 con 0x7f70ec0d0e70 2023-10-18T10:46:58.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.571+0000 7f70f3ae6640 1 -- 172.21.15.89:0/2025601651 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70ec1516a0 con 0x7f70ec0d0e70 2023-10-18T10:46:58.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.571+0000 7f70e27fc640 1 -- 172.21.15.89:0/2025601651 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e80bdb20 con 0x7f70ec0d0e70 2023-10-18T10:46:58.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.571+0000 7f70f3ae6640 1 -- 172.21.15.89:0/2025601651 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70ec151c60 con 0x7f70ec0d0e70 2023-10-18T10:46:58.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.575+0000 7f70f3ae6640 1 -- 172.21.15.89:0/2025601651 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70ec04dbe0 con 0x7f70ec0d0e70 2023-10-18T10:46:58.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.575+0000 7f70e27fc640 1 -- 172.21.15.89:0/2025601651 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f70e80b33f0 con 0x7f70ec0d0e70 2023-10-18T10:46:58.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.575+0000 7f70e27fc640 1 --2- 172.21.15.89:0/2025601651 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f70d00656b0 0x7f70d0067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:58.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.575+0000 7f70e27fc640 1 -- 172.21.15.89:0/2025601651 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f70e811da40 con 0x7f70ec0d0e70 2023-10-18T10:46:58.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.575+0000 7f70f1081640 1 --2- 172.21.15.89:0/2025601651 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f70d00656b0 0x7f70d0067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:58.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.575+0000 7f70f1081640 1 --2- 172.21.15.89:0/2025601651 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f70d00656b0 0x7f70d0067b70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f70dc000f70 tx=0x7f70dc007e50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:58.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.579+0000 7f70e27fc640 1 -- 172.21.15.89:0/2025601651 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f70e80eb270 con 0x7f70ec0d0e70 2023-10-18T10:46:58.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.727+0000 7f70f3ae6640 1 -- 172.21.15.89:0/2025601651 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f70ec0b85d0 con 0x7f70ec0d0e70 2023-10-18T10:46:58.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.727+0000 7f70e27fc640 1 -- 172.21.15.89:0/2025601651 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f70e811be60 con 0x7f70ec0d0e70 2023-10-18T10:46:58.731 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-18T10:46:58.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.731+0000 7f70f3ae6640 1 -- 172.21.15.89:0/2025601651 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f70d00656b0 msgr2=0x7f70d0067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:58.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.731+0000 7f70f3ae6640 1 --2- 172.21.15.89:0/2025601651 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f70d00656b0 0x7f70d0067b70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f70dc000f70 tx=0x7f70dc007e50 comp rx=0 tx=0).stop 2023-10-18T10:46:58.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.731+0000 7f70f3ae6640 1 -- 172.21.15.89:0/2025601651 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70ec0d0e70 msgr2=0x7f70ec0b6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:58.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.731+0000 7f70f3ae6640 1 --2- 172.21.15.89:0/2025601651 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70ec0d0e70 0x7f70ec0b6140 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f70e801abc0 tx=0x7f70e801ad50 comp rx=0 tx=0).stop 2023-10-18T10:46:58.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.731+0000 7f70f3ae6640 1 -- 172.21.15.89:0/2025601651 shutdown_connections 2023-10-18T10:46:58.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.731+0000 7f70f3ae6640 1 --2- 172.21.15.89:0/2025601651 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f70d00656b0 0x7f70d0067b70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:58.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.731+0000 7f70f3ae6640 1 --2- 172.21.15.89:0/2025601651 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70ec150dd0 0x7f70ec153570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:58.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.731+0000 7f70f3ae6640 1 --2- 172.21.15.89:0/2025601651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70ec152460 0x7f70ec0b7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:58.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.731+0000 7f70f3ae6640 1 --2- 172.21.15.89:0/2025601651 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70ec0d0e70 0x7f70ec0b6140 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:58.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.731+0000 7f70f3ae6640 1 -- 172.21.15.89:0/2025601651 >> 172.21.15.89:0/2025601651 conn(0x7f70ec089fd0 msgr2=0x7f70ec15dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:58.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.731+0000 7f70f3ae6640 1 -- 172.21.15.89:0/2025601651 shutdown_connections 2023-10-18T10:46:58.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.731+0000 7f70f3ae6640 1 -- 172.21.15.89:0/2025601651 wait complete. 2023-10-18T10:46:58.753 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T10:46:58.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.887+0000 7f6777c6a640 1 Processor -- start 2023-10-18T10:46:58.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.891+0000 7f6777c6a640 1 -- start start 2023-10-18T10:46:58.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.891+0000 7f6777c6a640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f67700cd4f0 0x7f6770143470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:58.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.891+0000 7f6777c6a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f67700ce070 0x7f67700ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:58.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.891+0000 7f6777c6a640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67700cd8d0 con 0x7f6770145b70 2023-10-18T10:46:58.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.891+0000 7f6777c6a640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6770069900 con 0x7f67700cd4f0 2023-10-18T10:46:58.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.891+0000 7f6777c6a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67700ce9e0 con 0x7f67700ce070 2023-10-18T10:46:58.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.891+0000 7f6775a06640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f67700cd4f0 0x7f6770143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:58.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.891+0000 7f6775a06640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f67700cd4f0 0x7f6770143470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57514/0 (socket says 172.21.15.89:57514) 2023-10-18T10:46:58.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.891+0000 7f6775a06640 1 -- 172.21.15.89:0/3092023130 learned_addr learned my addr 172.21.15.89:0/3092023130 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:58.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.891+0000 7f6776207640 1 --2- 172.21.15.89:0/3092023130 >> v2:172.21.15.154:3300/0 conn(0x7f67700ce070 0x7f67700ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:58.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.891+0000 7f6775a06640 1 -- 172.21.15.89:0/3092023130 >> v2:172.21.15.154:3300/0 conn(0x7f67700ce070 msgr2=0x7f67700ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:58.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.891+0000 7f6775a06640 1 --2- 172.21.15.89:0/3092023130 >> v2:172.21.15.154:3300/0 conn(0x7f67700ce070 0x7f67700ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:58.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.891+0000 7f6775a06640 1 -- 172.21.15.89:0/3092023130 >> v1:172.21.15.89:6789/0 conn(0x7f6770145b70 legacy=0x7f6770160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:58.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.891+0000 7f6775a06640 1 -- 172.21.15.89:0/3092023130 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f677015cbf0 con 0x7f67700cd4f0 2023-10-18T10:46:58.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.891+0000 7f6776207640 1 --2- 172.21.15.89:0/3092023130 >> v2:172.21.15.154:3300/0 conn(0x7f67700ce070 0x7f67700ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:58.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.895+0000 7f6775a06640 1 --2- 172.21.15.89:0/3092023130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67700cd4f0 0x7f6770143470 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f67700b63d0 tx=0x7f676c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f745f2ccc57ae609 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:58.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.895+0000 7f6774a04640 1 -- 172.21.15.89:0/3092023130 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676c0aa430 con 0x7f67700cd4f0 2023-10-18T10:46:58.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.895+0000 7f6774a04640 1 -- 172.21.15.89:0/3092023130 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676c0aa590 con 0x7f67700cd4f0 2023-10-18T10:46:58.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.895+0000 7f6774a04640 1 -- 172.21.15.89:0/3092023130 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676c0aa920 con 0x7f67700cd4f0 2023-10-18T10:46:58.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.895+0000 7f6777c6a640 1 -- 172.21.15.89:0/3092023130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67700cd4f0 msgr2=0x7f6770143470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:58.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.895+0000 7f6777c6a640 1 --2- 172.21.15.89:0/3092023130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67700cd4f0 0x7f6770143470 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f67700b63d0 tx=0x7f676c01c4d0 comp rx=0 tx=0).stop 2023-10-18T10:46:58.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.895+0000 7f6777c6a640 1 -- 172.21.15.89:0/3092023130 shutdown_connections 2023-10-18T10:46:58.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.895+0000 7f6777c6a640 1 --2- 172.21.15.89:0/3092023130 >> v2:172.21.15.154:3300/0 conn(0x7f67700ce070 0x7f67700ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:58.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.895+0000 7f6777c6a640 1 --2- 172.21.15.89:0/3092023130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67700cd4f0 0x7f6770143470 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:58.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.895+0000 7f6777c6a640 1 -- 172.21.15.89:0/3092023130 >> 172.21.15.89:0/3092023130 conn(0x7f677006dad0 msgr2=0x7f677006dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:58.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.895+0000 7f6777c6a640 1 -- 172.21.15.89:0/3092023130 shutdown_connections 2023-10-18T10:46:58.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.895+0000 7f6777c6a640 1 -- 172.21.15.89:0/3092023130 wait complete. 2023-10-18T10:46:58.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.895+0000 7f6777c6a640 1 Processor -- start 2023-10-18T10:46:58.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.895+0000 7f6777c6a640 1 -- start start 2023-10-18T10:46:58.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.899+0000 7f6777c6a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67700cd4f0 0x7f6770155080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:58.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.899+0000 7f6777c6a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67700ce070 0x7f6770159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:58.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.899+0000 7f6777c6a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6770145b70 0x7f677015a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:58.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.899+0000 7f6777c6a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677015b2f0 con 0x7f6770145b70 2023-10-18T10:46:58.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.899+0000 7f6777c6a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67700b20d0 con 0x7f67700cd4f0 2023-10-18T10:46:58.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.899+0000 7f6777c6a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f677015c490 con 0x7f67700ce070 2023-10-18T10:46:58.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.899+0000 7f6776207640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6770145b70 0x7f677015a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:58.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.899+0000 7f6776207640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6770145b70 0x7f677015a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57522/0 (socket says 172.21.15.89:57522) 2023-10-18T10:46:58.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.899+0000 7f6776207640 1 -- 172.21.15.89:0/1551536922 learned_addr learned my addr 172.21.15.89:0/1551536922 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:58.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.899+0000 7f6775a06640 1 --2- 172.21.15.89:0/1551536922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67700cd4f0 0x7f6770155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:58.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.899+0000 7f6775205640 1 --2- 172.21.15.89:0/1551536922 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67700ce070 0x7f6770159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:58.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.899+0000 7f6776207640 1 -- 172.21.15.89:0/1551536922 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67700ce070 msgr2=0x7f6770159f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:58.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.899+0000 7f6776207640 1 --2- 172.21.15.89:0/1551536922 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67700ce070 0x7f6770159f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:58.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.899+0000 7f6776207640 1 -- 172.21.15.89:0/1551536922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67700cd4f0 msgr2=0x7f6770155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:58.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.899+0000 7f6776207640 1 --2- 172.21.15.89:0/1551536922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67700cd4f0 0x7f6770155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:58.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.899+0000 7f6776207640 1 -- 172.21.15.89:0/1551536922 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f676c0094d0 con 0x7f6770145b70 2023-10-18T10:46:58.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.899+0000 7f6776207640 1 --2- 172.21.15.89:0/1551536922 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6770145b70 0x7f677015a4d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f676800cef0 tx=0x7f676800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:58.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.899+0000 7f67667fc640 1 -- 172.21.15.89:0/1551536922 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676800d840 con 0x7f6770145b70 2023-10-18T10:46:58.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.899+0000 7f67667fc640 1 -- 172.21.15.89:0/1551536922 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676800d9c0 con 0x7f6770145b70 2023-10-18T10:46:58.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.899+0000 7f67667fc640 1 -- 172.21.15.89:0/1551536922 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676800dd70 con 0x7f6770145b70 2023-10-18T10:46:58.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.899+0000 7f6777c6a640 1 -- 172.21.15.89:0/1551536922 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f677015c770 con 0x7f6770145b70 2023-10-18T10:46:58.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.899+0000 7f6777c6a640 1 -- 172.21.15.89:0/1551536922 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67700b2600 con 0x7f6770145b70 2023-10-18T10:46:58.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.899+0000 7f6777c6a640 1 -- 172.21.15.89:0/1551536922 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f677004dbe0 con 0x7f6770145b70 2023-10-18T10:46:58.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.903+0000 7f67667fc640 1 -- 172.21.15.89:0/1551536922 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f676801e3b0 con 0x7f6770145b70 2023-10-18T10:46:58.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.903+0000 7f67667fc640 1 --2- 172.21.15.89:0/1551536922 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6754065680 0x7f6754067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:58.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.903+0000 7f67667fc640 1 -- 172.21.15.89:0/1551536922 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6768083b90 con 0x7f6770145b70 2023-10-18T10:46:58.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.903+0000 7f6775a06640 1 --2- 172.21.15.89:0/1551536922 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6754065680 0x7f6754067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:58.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.903+0000 7f6775a06640 1 --2- 172.21.15.89:0/1551536922 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6754065680 0x7f6754067b40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f676c0ae000 tx=0x7f676c0b5040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:58.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:58.907+0000 7f67667fc640 1 -- 172.21.15.89:0/1551536922 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f67680513f0 con 0x7f6770145b70 2023-10-18T10:46:59.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.067+0000 7f6777c6a640 1 -- 172.21.15.89:0/1551536922 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f67700b6420 con 0x7f6770145b70 2023-10-18T10:46:59.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.067+0000 7f67667fc640 1 -- 172.21.15.89:0/1551536922 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7f676801a080 con 0x7f6770145b70 2023-10-18T10:46:59.070 INFO:teuthology.orchestra.run.smithi089.stderr:listed 0 entries 2023-10-18T10:46:59.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.067+0000 7f6777c6a640 1 -- 172.21.15.89:0/1551536922 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6754065680 msgr2=0x7f6754067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:59.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.067+0000 7f6777c6a640 1 --2- 172.21.15.89:0/1551536922 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6754065680 0x7f6754067b40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f676c0ae000 tx=0x7f676c0b5040 comp rx=0 tx=0).stop 2023-10-18T10:46:59.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.067+0000 7f6777c6a640 1 -- 172.21.15.89:0/1551536922 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6770145b70 msgr2=0x7f677015a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:59.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.067+0000 7f6777c6a640 1 --2- 172.21.15.89:0/1551536922 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6770145b70 0x7f677015a4d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f676800cef0 tx=0x7f676800a710 comp rx=0 tx=0).stop 2023-10-18T10:46:59.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.067+0000 7f6777c6a640 1 -- 172.21.15.89:0/1551536922 shutdown_connections 2023-10-18T10:46:59.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.067+0000 7f6777c6a640 1 --2- 172.21.15.89:0/1551536922 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6754065680 0x7f6754067b40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:59.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.067+0000 7f6777c6a640 1 --2- 172.21.15.89:0/1551536922 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f67700ce070 0x7f6770159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:59.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.067+0000 7f6777c6a640 1 --2- 172.21.15.89:0/1551536922 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6770145b70 0x7f677015a4d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:59.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.067+0000 7f6777c6a640 1 --2- 172.21.15.89:0/1551536922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67700cd4f0 0x7f6770155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:59.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.067+0000 7f6777c6a640 1 -- 172.21.15.89:0/1551536922 >> 172.21.15.89:0/1551536922 conn(0x7f677006dad0 msgr2=0x7f67700c7570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:59.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.067+0000 7f6777c6a640 1 -- 172.21.15.89:0/1551536922 shutdown_connections 2023-10-18T10:46:59.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.071+0000 7f6777c6a640 1 -- 172.21.15.89:0/1551536922 wait complete. 2023-10-18T10:46:59.090 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-10-18T10:46:59.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:59.091 DEBUG:teuthology.orchestra.run.smithi154:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-10-18T10:46:59.119 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2023-10-18T10:46:59.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:46:59.120 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-18T10:46:59.174 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-18T10:46:59.175 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:46:59.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.311+0000 7f5c3ea93640 1 Processor -- start 2023-10-18T10:46:59.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.311+0000 7f5c3ea93640 1 -- start start 2023-10-18T10:46:59.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.311+0000 7f5c3ea93640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5c3815c550 0x7f5c380d2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:59.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.311+0000 7f5c3ea93640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c380cfd20 0x7f5c380b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:59.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.311+0000 7f5c3ea93640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c380d1e50 con 0x7f5c380b1860 2023-10-18T10:46:59.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.311+0000 7f5c3ea93640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c380d2130 con 0x7f5c3815c550 2023-10-18T10:46:59.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.311+0000 7f5c3ea93640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c380d2410 con 0x7f5c380cfd20 2023-10-18T10:46:59.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.311+0000 7f5c37fff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5c380b1860 0x7f5c380cf610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58198/0 (socket says 172.21.15.89:58198) 2023-10-18T10:46:59.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.311+0000 7f5c37fff640 1 -- 172.21.15.89:0/3552341216 learned_addr learned my addr 172.21.15.89:0/3552341216 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:46:59.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.311+0000 7f5c3c82f640 1 --2- 172.21.15.89:0/3552341216 >> v2:172.21.15.154:3300/0 conn(0x7f5c3815c550 0x7f5c380d2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:59.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.311+0000 7f5c3d030640 1 --2- 172.21.15.89:0/3552341216 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c380cfd20 0x7f5c380b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:59.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.315+0000 7f5c377fe640 1 -- 172.21.15.89:0/3552341216 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 331783554 0 0) 0x7f5c380d1e50 con 0x7f5c380b1860 2023-10-18T10:46:59.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.315+0000 7f5c377fe640 1 -- 172.21.15.89:0/3552341216 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c28003530 con 0x7f5c380b1860 2023-10-18T10:46:59.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.315+0000 7f5c377fe640 1 -- 172.21.15.89:0/3552341216 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3455285830 0 0) 0x7f5c28003530 con 0x7f5c380b1860 2023-10-18T10:46:59.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.315+0000 7f5c377fe640 1 -- 172.21.15.89:0/3552341216 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c380cfd20 msgr2=0x7f5c380b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:59.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.315+0000 7f5c377fe640 1 --2- 172.21.15.89:0/3552341216 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c380cfd20 0x7f5c380b55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:59.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.315+0000 7f5c377fe640 1 -- 172.21.15.89:0/3552341216 >> v2:172.21.15.154:3300/0 conn(0x7f5c3815c550 msgr2=0x7f5c380d2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:59.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.315+0000 7f5c377fe640 1 --2- 172.21.15.89:0/3552341216 >> v2:172.21.15.154:3300/0 conn(0x7f5c3815c550 0x7f5c380d2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:59.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.315+0000 7f5c377fe640 1 -- 172.21.15.89:0/3552341216 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c380d26f0 con 0x7f5c380b1860 2023-10-18T10:46:59.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.315+0000 7f5c377fe640 1 -- 172.21.15.89:0/3552341216 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7f5c24001680 con 0x7f5c380b1860 2023-10-18T10:46:59.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.315+0000 7f5c3c82f640 1 --2- 172.21.15.89:0/3552341216 >> v2:172.21.15.154:3300/0 conn(0x7f5c3815c550 0x7f5c380d2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:46:59.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.315+0000 7f5c377fe640 1 -- 172.21.15.89:0/3552341216 >> v1:172.21.15.89:6789/0 conn(0x7f5c380b1860 legacy=0x7f5c380cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:59.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.315+0000 7f5c377fe640 1 --2- 172.21.15.89:0/3552341216 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c28003980 0x7f5c28003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:59.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.315+0000 7f5c377fe640 1 --2- 172.21.15.89:0/3552341216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c280047d0 0x7f5c28004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:59.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.315+0000 7f5c377fe640 1 --2- 172.21.15.89:0/3552341216 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c380cfd20 0x7f5c28005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:59.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.315+0000 7f5c377fe640 1 -- 172.21.15.89:0/3552341216 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c380d2130 con 0x7f5c28003980 2023-10-18T10:46:59.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.315+0000 7f5c377fe640 1 -- 172.21.15.89:0/3552341216 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c380d2410 con 0x7f5c280047d0 2023-10-18T10:46:59.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.315+0000 7f5c377fe640 1 -- 172.21.15.89:0/3552341216 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2800b990 con 0x7f5c380cfd20 2023-10-18T10:46:59.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.315+0000 7f5c3c82f640 1 --2- 172.21.15.89:0/3552341216 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c28003980 0x7f5c28003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:59.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.315+0000 7f5c37fff640 1 --2- 172.21.15.89:0/3552341216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c280047d0 0x7f5c28004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:59.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.315+0000 7f5c3d030640 1 --2- 172.21.15.89:0/3552341216 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c380cfd20 0x7f5c28005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:59.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.315+0000 7f5c3c82f640 1 -- 172.21.15.89:0/3552341216 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c380cfd20 msgr2=0x7f5c28005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:59.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.315+0000 7f5c3c82f640 1 --2- 172.21.15.89:0/3552341216 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c380cfd20 0x7f5c28005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:59.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.315+0000 7f5c3c82f640 1 -- 172.21.15.89:0/3552341216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c280047d0 msgr2=0x7f5c28004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:59.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.315+0000 7f5c3c82f640 1 --2- 172.21.15.89:0/3552341216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c280047d0 0x7f5c28004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:59.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.315+0000 7f5c3c82f640 1 -- 172.21.15.89:0/3552341216 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c28003530 con 0x7f5c28003980 2023-10-18T10:46:59.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.319+0000 7f5c3c82f640 1 --2- 172.21.15.89:0/3552341216 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c28003980 0x7f5c28003da0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c009ce0 tx=0x7f5c2c01d680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=355f1ac2982ffdc4 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:59.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.319+0000 7f5c377fe640 1 -- 172.21.15.89:0/3552341216 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c0a58e0 con 0x7f5c28003980 2023-10-18T10:46:59.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.319+0000 7f5c377fe640 1 -- 172.21.15.89:0/3552341216 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c2c0a5a40 con 0x7f5c28003980 2023-10-18T10:46:59.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.319+0000 7f5c377fe640 1 -- 172.21.15.89:0/3552341216 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c0a5db0 con 0x7f5c28003980 2023-10-18T10:46:59.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.319+0000 7f5c3ea93640 1 -- 172.21.15.89:0/3552341216 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c28003980 msgr2=0x7f5c28003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:59.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.319+0000 7f5c3ea93640 1 --2- 172.21.15.89:0/3552341216 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c28003980 0x7f5c28003da0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c009ce0 tx=0x7f5c2c01d680 comp rx=0 tx=0).stop 2023-10-18T10:46:59.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.319+0000 7f5c3d030640 1 -- 172.21.15.89:0/3552341216 reap_dead start 2023-10-18T10:46:59.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.319+0000 7f5c3ea93640 1 -- 172.21.15.89:0/3552341216 shutdown_connections 2023-10-18T10:46:59.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.319+0000 7f5c3ea93640 1 -- 172.21.15.89:0/3552341216 >> 172.21.15.89:0/3552341216 conn(0x7f5c38069d00 msgr2=0x7f5c3806a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:59.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.319+0000 7f5c3ea93640 1 -- 172.21.15.89:0/3552341216 shutdown_connections 2023-10-18T10:46:59.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.319+0000 7f5c3ea93640 1 -- 172.21.15.89:0/3552341216 wait complete. 2023-10-18T10:46:59.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.319+0000 7f5c3ea93640 1 Processor -- start 2023-10-18T10:46:59.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.319+0000 7f5c3ea93640 1 -- start start 2023-10-18T10:46:59.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.323+0000 7f5c3ea93640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c28003980 0x7f5c380cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:59.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.323+0000 7f5c3ea93640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c3814d600 0x7f5c3814cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:59.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.323+0000 7f5c3ea93640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c3815f380 0x7f5c38155e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:59.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.323+0000 7f5c3ea93640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c380cb040 con 0x7f5c3815f380 2023-10-18T10:46:59.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.323+0000 7f5c3ea93640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3815e2e0 con 0x7f5c3814d600 2023-10-18T10:46:59.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.323+0000 7f5c3ea93640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3815e5c0 con 0x7f5c28003980 2023-10-18T10:46:59.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.323+0000 7f5c3d030640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c3815f380 0x7f5c38155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:59.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.323+0000 7f5c3d030640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c3815f380 0x7f5c38155e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57538/0 (socket says 172.21.15.89:57538) 2023-10-18T10:46:59.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.323+0000 7f5c3d030640 1 -- 172.21.15.89:0/1087320162 learned_addr learned my addr 172.21.15.89:0/1087320162 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:59.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.323+0000 7f5c37fff640 1 --2- 172.21.15.89:0/1087320162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c3814d600 0x7f5c3814cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:59.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.323+0000 7f5c3c82f640 1 --2- 172.21.15.89:0/1087320162 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c28003980 0x7f5c380cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:59.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.323+0000 7f5c3d030640 1 -- 172.21.15.89:0/1087320162 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c28003980 msgr2=0x7f5c380cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:59.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.323+0000 7f5c3d030640 1 --2- 172.21.15.89:0/1087320162 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c28003980 0x7f5c380cab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:59.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.323+0000 7f5c3d030640 1 -- 172.21.15.89:0/1087320162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c3814d600 msgr2=0x7f5c3814cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:59.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.323+0000 7f5c3d030640 1 --2- 172.21.15.89:0/1087320162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c3814d600 0x7f5c3814cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:59.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.323+0000 7f5c3d030640 1 -- 172.21.15.89:0/1087320162 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c3815e8a0 con 0x7f5c3815f380 2023-10-18T10:46:59.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.323+0000 7f5c3d030640 1 --2- 172.21.15.89:0/1087320162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c3815f380 0x7f5c38155e80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5c380b1610 tx=0x7f5c3000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:59.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.323+0000 7f5c357fa640 1 -- 172.21.15.89:0/1087320162 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3000e040 con 0x7f5c3815f380 2023-10-18T10:46:59.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.323+0000 7f5c357fa640 1 -- 172.21.15.89:0/1087320162 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3000f040 con 0x7f5c3815f380 2023-10-18T10:46:59.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.323+0000 7f5c3ea93640 1 -- 172.21.15.89:0/1087320162 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c3815eb80 con 0x7f5c3815f380 2023-10-18T10:46:59.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.323+0000 7f5c357fa640 1 -- 172.21.15.89:0/1087320162 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c30015430 con 0x7f5c3815f380 2023-10-18T10:46:59.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.323+0000 7f5c3ea93640 1 -- 172.21.15.89:0/1087320162 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c3815dfb0 con 0x7f5c3815f380 2023-10-18T10:46:59.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.323+0000 7f5c3ea93640 1 -- 172.21.15.89:0/1087320162 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c3815f0a0 con 0x7f5c3815f380 2023-10-18T10:46:59.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.323+0000 7f5c357fa640 1 -- 172.21.15.89:0/1087320162 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f5c30015590 con 0x7f5c3815f380 2023-10-18T10:46:59.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.327+0000 7f5c357fa640 1 --2- 172.21.15.89:0/1087320162 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5c1c0656b0 0x7f5c1c067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:59.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.327+0000 7f5c357fa640 1 -- 172.21.15.89:0/1087320162 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5c30083900 con 0x7f5c3815f380 2023-10-18T10:46:59.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.327+0000 7f5c3c82f640 1 --2- 172.21.15.89:0/1087320162 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5c1c0656b0 0x7f5c1c067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:59.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.327+0000 7f5c3c82f640 1 --2- 172.21.15.89:0/1087320162 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5c1c0656b0 0x7f5c1c067b70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c0af000 tx=0x7f5c2c0a7040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:59.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.331+0000 7f5c357fa640 1 -- 172.21.15.89:0/1087320162 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f5c3815f0a0 con 0x7f5c3815f380 2023-10-18T10:46:59.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.511+0000 7f5c3ea93640 1 -- 172.21.15.89:0/1087320162 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c3804dbe0 con 0x7f5c3815f380 2023-10-18T10:46:59.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.511+0000 7f5c357fa640 1 -- 172.21.15.89:0/1087320162 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4874 (secure 0 0 0) 0x7f5c30050ff0 con 0x7f5c3815f380 2023-10-18T10:46:59.514 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:59.515 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/1247973498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1247973498},{"type":"v1","addr":"172.21.15.154:6836","nonce":1247973498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4284,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/516057316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":516057316},{"type":"v1","addr":"172.21.15.154:6837","nonce":516057316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4299,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6834/3957051407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3957051407},{"type":"v1","addr":"172.21.15.89:6834","nonce":3957051407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:45:55.655016+0000","modified":"2023-10-18T10:46:13.833215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4312":{"gid":4312,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6835/2906113039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6833","nonce":2906113039},{"type":"v1","addr":"172.21.15.89:6835","nonce":2906113039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T10:46:59.516 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 9 2023-10-18T10:46:59.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.515+0000 7f5c3ea93640 1 -- 172.21.15.89:0/1087320162 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5c1c0656b0 msgr2=0x7f5c1c067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:59.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.515+0000 7f5c3ea93640 1 --2- 172.21.15.89:0/1087320162 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5c1c0656b0 0x7f5c1c067b70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c0af000 tx=0x7f5c2c0a7040 comp rx=0 tx=0).stop 2023-10-18T10:46:59.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.515+0000 7f5c3ea93640 1 -- 172.21.15.89:0/1087320162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c3815f380 msgr2=0x7f5c38155e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:59.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.515+0000 7f5c3ea93640 1 --2- 172.21.15.89:0/1087320162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c3815f380 0x7f5c38155e80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5c380b1610 tx=0x7f5c3000f580 comp rx=0 tx=0).stop 2023-10-18T10:46:59.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.515+0000 7f5c3ea93640 1 -- 172.21.15.89:0/1087320162 shutdown_connections 2023-10-18T10:46:59.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.515+0000 7f5c3ea93640 1 --2- 172.21.15.89:0/1087320162 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5c1c0656b0 0x7f5c1c067b70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:59.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.515+0000 7f5c3ea93640 1 --2- 172.21.15.89:0/1087320162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c3815f380 0x7f5c38155e80 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:59.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.515+0000 7f5c3ea93640 1 --2- 172.21.15.89:0/1087320162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c3814d600 0x7f5c3814cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:59.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.515+0000 7f5c3ea93640 1 --2- 172.21.15.89:0/1087320162 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c28003980 0x7f5c380cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:59.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.515+0000 7f5c3ea93640 1 -- 172.21.15.89:0/1087320162 >> 172.21.15.89:0/1087320162 conn(0x7f5c38069d00 msgr2=0x7f5c38157280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:59.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.515+0000 7f5c3ea93640 1 -- 172.21.15.89:0/1087320162 shutdown_connections 2023-10-18T10:46:59.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.515+0000 7f5c3ea93640 1 -- 172.21.15.89:0/1087320162 wait complete. 2023-10-18T10:46:59.535 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:46:59.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.671+0000 7efcb1140640 1 Processor -- start 2023-10-18T10:46:59.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.671+0000 7efcb1140640 1 -- start start 2023-10-18T10:46:59.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.671+0000 7efcb1140640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcac15c940 0x7efcac0d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:59.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.671+0000 7efcb1140640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efcac0d0e70 0x7efcac0cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:59.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.671+0000 7efcb1140640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcac15da50 con 0x7efcac0d2f40 2023-10-18T10:46:59.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.671+0000 7efcb1140640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcac15ce40 con 0x7efcac0d0e70 2023-10-18T10:46:59.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.671+0000 7efcb1140640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcac15d120 con 0x7efcac15c940 2023-10-18T10:46:59.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.671+0000 7efcab59e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efcac0d0e70 0x7efcac0cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:59.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.671+0000 7efcab59e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efcac0d0e70 0x7efcac0cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57548/0 (socket says 172.21.15.89:57548) 2023-10-18T10:46:59.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.671+0000 7efcab59e640 1 -- 172.21.15.89:0/2294884075 learned_addr learned my addr 172.21.15.89:0/2294884075 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:59.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.671+0000 7efcaad9d640 1 --2- 172.21.15.89:0/2294884075 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcac15c940 0x7efcac0d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:59.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.671+0000 7efcab59e640 1 -- 172.21.15.89:0/2294884075 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcac15c940 msgr2=0x7efcac0d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:46:59.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.671+0000 7efcab59e640 1 --2- 172.21.15.89:0/2294884075 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcac15c940 0x7efcac0d2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:59.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.671+0000 7efcab59e640 1 -- 172.21.15.89:0/2294884075 >> v1:172.21.15.154:6789/0 conn(0x7efcac0d2f40 legacy=0x7efcac0d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:59.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.671+0000 7efcab59e640 1 -- 172.21.15.89:0/2294884075 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcac15d400 con 0x7efcac0d0e70 2023-10-18T10:46:59.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.671+0000 7efcaad9d640 1 --2- 172.21.15.89:0/2294884075 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcac15c940 0x7efcac0d2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:46:59.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.675+0000 7efcab59e640 1 --2- 172.21.15.89:0/2294884075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcac0d0e70 0x7efcac0cd160 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7efca40098a0 tx=0x7efca401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d86904864bc34fd server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:59.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.675+0000 7efca9d9b640 1 -- 172.21.15.89:0/2294884075 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca40aa5f0 con 0x7efcac0d0e70 2023-10-18T10:46:59.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.675+0000 7efca9d9b640 1 -- 172.21.15.89:0/2294884075 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca40aa750 con 0x7efcac0d0e70 2023-10-18T10:46:59.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.675+0000 7efca9d9b640 1 -- 172.21.15.89:0/2294884075 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca40aaae0 con 0x7efcac0d0e70 2023-10-18T10:46:59.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.675+0000 7efcb1140640 1 -- 172.21.15.89:0/2294884075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcac0d0e70 msgr2=0x7efcac0cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:59.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.675+0000 7efcb1140640 1 --2- 172.21.15.89:0/2294884075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcac0d0e70 0x7efcac0cd160 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7efca40098a0 tx=0x7efca401c4f0 comp rx=0 tx=0).stop 2023-10-18T10:46:59.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.675+0000 7efcb1140640 1 -- 172.21.15.89:0/2294884075 shutdown_connections 2023-10-18T10:46:59.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.675+0000 7efcb1140640 1 --2- 172.21.15.89:0/2294884075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcac0d0e70 0x7efcac0cd160 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:59.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.675+0000 7efcb1140640 1 --2- 172.21.15.89:0/2294884075 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcac15c940 0x7efcac0d2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:59.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.675+0000 7efcb1140640 1 -- 172.21.15.89:0/2294884075 >> 172.21.15.89:0/2294884075 conn(0x7efcac089fd0 msgr2=0x7efcac09b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:59.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.675+0000 7efcb1140640 1 -- 172.21.15.89:0/2294884075 shutdown_connections 2023-10-18T10:46:59.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.675+0000 7efcb1140640 1 -- 172.21.15.89:0/2294884075 wait complete. 2023-10-18T10:46:59.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.679+0000 7efcb1140640 1 Processor -- start 2023-10-18T10:46:59.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.679+0000 7efcb1140640 1 -- start start 2023-10-18T10:46:59.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.679+0000 7efcb1140640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcac0d0e70 0x7efcac154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:59.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.679+0000 7efcb1140640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcac154c10 0x7efcac0c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:59.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.679+0000 7efcb1140640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcac15dde0 0x7efcac15e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:59.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.679+0000 7efcb1140640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcac0c9950 con 0x7efcac15dde0 2023-10-18T10:46:59.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.679+0000 7efcb1140640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcac15e7a0 con 0x7efcac154c10 2023-10-18T10:46:59.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.679+0000 7efcb1140640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcac0c9150 con 0x7efcac0d0e70 2023-10-18T10:46:59.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.679+0000 7efcab59e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcac15dde0 0x7efcac15e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:59.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.679+0000 7efcab59e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcac15dde0 0x7efcac15e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57552/0 (socket says 172.21.15.89:57552) 2023-10-18T10:46:59.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.679+0000 7efcab59e640 1 -- 172.21.15.89:0/814787842 learned_addr learned my addr 172.21.15.89:0/814787842 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:46:59.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.679+0000 7efcab59e640 1 -- 172.21.15.89:0/814787842 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcac0d0e70 msgr2=0x7efcac154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:59.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.679+0000 7efcab59e640 1 --2- 172.21.15.89:0/814787842 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcac0d0e70 0x7efcac154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:59.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.679+0000 7efcaa59c640 1 --2- 172.21.15.89:0/814787842 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcac154c10 0x7efcac0c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:59.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.679+0000 7efcab59e640 1 -- 172.21.15.89:0/814787842 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcac154c10 msgr2=0x7efcac0c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:59.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.679+0000 7efcab59e640 1 --2- 172.21.15.89:0/814787842 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcac154c10 0x7efcac0c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:59.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.679+0000 7efcab59e640 1 -- 172.21.15.89:0/814787842 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca40094d0 con 0x7efcac15dde0 2023-10-18T10:46:59.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.679+0000 7efcaa59c640 1 --2- 172.21.15.89:0/814787842 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcac154c10 0x7efcac0c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:46:59.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.679+0000 7efcab59e640 1 --2- 172.21.15.89:0/814787842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcac15dde0 0x7efcac15e260 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7efca40ac040 tx=0x7efca401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:59.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.679+0000 7efc937fe640 1 -- 172.21.15.89:0/814787842 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca40b3800 con 0x7efcac15dde0 2023-10-18T10:46:59.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.679+0000 7efc937fe640 1 -- 172.21.15.89:0/814787842 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca40b3960 con 0x7efcac15dde0 2023-10-18T10:46:59.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.679+0000 7efcb1140640 1 -- 172.21.15.89:0/814787842 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcac0c88b0 con 0x7efcac15dde0 2023-10-18T10:46:59.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.679+0000 7efc937fe640 1 -- 172.21.15.89:0/814787842 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca40b3cf0 con 0x7efcac15dde0 2023-10-18T10:46:59.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.679+0000 7efcb1140640 1 -- 172.21.15.89:0/814787842 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcac0c8e70 con 0x7efcac15dde0 2023-10-18T10:46:59.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.683+0000 7efcb1140640 1 -- 172.21.15.89:0/814787842 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcac0c9720 con 0x7efcac15dde0 2023-10-18T10:46:59.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.683+0000 7efc937fe640 1 -- 172.21.15.89:0/814787842 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7efca40c6390 con 0x7efcac15dde0 2023-10-18T10:46:59.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.683+0000 7efc937fe640 1 --2- 172.21.15.89:0/814787842 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efc98065680 0x7efc98067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:46:59.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.683+0000 7efc937fe640 1 -- 172.21.15.89:0/814787842 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7efca40a7070 con 0x7efcac15dde0 2023-10-18T10:46:59.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.683+0000 7efcaad9d640 1 --2- 172.21.15.89:0/814787842 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efc98065680 0x7efc98067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:46:59.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.683+0000 7efcaad9d640 1 --2- 172.21.15.89:0/814787842 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efc98065680 0x7efc98067b40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7efc9c0094c0 tx=0x7efc9c009570 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:46:59.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.687+0000 7efc937fe640 1 -- 172.21.15.89:0/814787842 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7efcac0c9720 con 0x7efcac15dde0 2023-10-18T10:46:59.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.843+0000 7efcb1140640 1 -- 172.21.15.89:0/814787842 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efcac0c9720 con 0x7efcac15dde0 2023-10-18T10:46:59.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.843+0000 7efc937fe640 1 -- 172.21.15.89:0/814787842 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20192 (secure 0 0 0) 0x7efcac0c9720 con 0x7efcac15dde0 2023-10-18T10:46:59.845 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:46:59.846 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":29,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:46:36.958604+0000","last_up_change":"2023-10-18T10:45:42.164688+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:45:53.309122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-18T10:45:54.659763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:45:57.796514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6824","nonce":773078364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6827","nonce":773078364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6831","nonce":773078364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6829","nonce":773078364}]},"public_addr":"172.21.15.154:6824/773078364","cluster_addr":"172.21.15.154:6827/773078364","heartbeat_back_addr":"172.21.15.154:6831/773078364","heartbeat_front_addr":"172.21.15.154:6829/773078364","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":1,"to":5}]},{"pgid":"4.1a","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:46:59.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.843+0000 7efcb1140640 1 -- 172.21.15.89:0/814787842 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efc98065680 msgr2=0x7efc98067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:59.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.843+0000 7efcb1140640 1 --2- 172.21.15.89:0/814787842 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efc98065680 0x7efc98067b40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7efc9c0094c0 tx=0x7efc9c009570 comp rx=0 tx=0).stop 2023-10-18T10:46:59.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.843+0000 7efcb1140640 1 -- 172.21.15.89:0/814787842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcac15dde0 msgr2=0x7efcac15e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:46:59.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.843+0000 7efcb1140640 1 --2- 172.21.15.89:0/814787842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcac15dde0 0x7efcac15e260 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7efca40ac040 tx=0x7efca401ad30 comp rx=0 tx=0).stop 2023-10-18T10:46:59.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.843+0000 7efcb1140640 1 -- 172.21.15.89:0/814787842 shutdown_connections 2023-10-18T10:46:59.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.843+0000 7efcb1140640 1 --2- 172.21.15.89:0/814787842 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efc98065680 0x7efc98067b40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:59.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.843+0000 7efcb1140640 1 --2- 172.21.15.89:0/814787842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcac15dde0 0x7efcac15e260 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:59.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.843+0000 7efcb1140640 1 --2- 172.21.15.89:0/814787842 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcac154c10 0x7efcac0c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:59.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.843+0000 7efcb1140640 1 --2- 172.21.15.89:0/814787842 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcac0d0e70 0x7efcac154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:46:59.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.843+0000 7efcb1140640 1 -- 172.21.15.89:0/814787842 >> 172.21.15.89:0/814787842 conn(0x7efcac089fd0 msgr2=0x7efcac0d0760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:46:59.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.843+0000 7efcb1140640 1 -- 172.21.15.89:0/814787842 shutdown_connections 2023-10-18T10:46:59.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:46:59.843+0000 7efcb1140640 1 -- 172.21.15.89:0/814787842 wait complete. 2023-10-18T10:46:59.867 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:45:43.114184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-18T10:45:48.730968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:45:53.309122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:45:54.659763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T10:45:57.796514+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T10:46:59.868 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:47:00.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.007+0000 7f1b90a19640 1 Processor -- start 2023-10-18T10:47:00.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.007+0000 7f1b90a19640 1 -- start start 2023-10-18T10:47:00.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.007+0000 7f1b90a19640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b8c159490 0x7f1b8c15d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:00.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.007+0000 7f1b90a19640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1b8c070600 0x7f1b8c06b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:00.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.007+0000 7f1b90a19640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b8c15ddf0 con 0x7f1b8c06bd10 2023-10-18T10:47:00.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.007+0000 7f1b90a19640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b8c06c1a0 con 0x7f1b8c070600 2023-10-18T10:47:00.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.007+0000 7f1b90a19640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c070b90 con 0x7f1b8c159490 2023-10-18T10:47:00.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.007+0000 7f1b8ad9d640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1b8c06bd10 0x7f1b8c0cbee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58202/0 (socket says 172.21.15.89:58202) 2023-10-18T10:47:00.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.007+0000 7f1b8ad9d640 1 -- 172.21.15.89:0/3950994920 learned_addr learned my addr 172.21.15.89:0/3950994920 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:47:00.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.007+0000 7f1b8a59c640 1 --2- 172.21.15.89:0/3950994920 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b8c159490 0x7f1b8c15d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:00.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.007+0000 7f1b89d9b640 1 --2- 172.21.15.89:0/3950994920 >> v2:172.21.15.154:3300/0 conn(0x7f1b8c070600 0x7f1b8c06b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:00.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.007+0000 7f1b8959a640 1 -- 172.21.15.89:0/3950994920 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3828146151 0 0) 0x7f1b8c15ddf0 con 0x7f1b8c06bd10 2023-10-18T10:47:00.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.007+0000 7f1b8959a640 1 -- 172.21.15.89:0/3950994920 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b74003530 con 0x7f1b8c06bd10 2023-10-18T10:47:00.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.007+0000 7f1b8a59c640 1 -- 172.21.15.89:0/3950994920 >> v2:172.21.15.154:3300/0 conn(0x7f1b8c070600 msgr2=0x7f1b8c06b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:00.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.007+0000 7f1b8a59c640 1 --2- 172.21.15.89:0/3950994920 >> v2:172.21.15.154:3300/0 conn(0x7f1b8c070600 0x7f1b8c06b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:00.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.007+0000 7f1b8a59c640 1 -- 172.21.15.89:0/3950994920 >> v1:172.21.15.89:6789/0 conn(0x7f1b8c06bd10 legacy=0x7f1b8c0cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:00.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.007+0000 7f1b8a59c640 1 -- 172.21.15.89:0/3950994920 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b8c070e70 con 0x7f1b8c159490 2023-10-18T10:47:00.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.011+0000 7f1b8a59c640 1 --2- 172.21.15.89:0/3950994920 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b8c159490 0x7f1b8c15d8b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c0097c0 tx=0x7f1b7c01c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=826ee23271cf49c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:00.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.011+0000 7f1b8959a640 1 -- 172.21.15.89:0/3950994920 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c0ac620 con 0x7f1b8c159490 2023-10-18T10:47:00.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.011+0000 7f1b8959a640 1 -- 172.21.15.89:0/3950994920 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b7c0ac780 con 0x7f1b8c159490 2023-10-18T10:47:00.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.011+0000 7f1b8959a640 1 -- 172.21.15.89:0/3950994920 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c0acb10 con 0x7f1b8c159490 2023-10-18T10:47:00.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.011+0000 7f1b90a19640 1 -- 172.21.15.89:0/3950994920 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b8c159490 msgr2=0x7f1b8c15d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:00.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.011+0000 7f1b90a19640 1 --2- 172.21.15.89:0/3950994920 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b8c159490 0x7f1b8c15d8b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c0097c0 tx=0x7f1b7c01c3c0 comp rx=0 tx=0).stop 2023-10-18T10:47:00.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.011+0000 7f1b90a19640 1 -- 172.21.15.89:0/3950994920 shutdown_connections 2023-10-18T10:47:00.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.011+0000 7f1b90a19640 1 --2- 172.21.15.89:0/3950994920 >> v2:172.21.15.154:3300/0 conn(0x7f1b8c070600 0x7f1b8c06b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:00.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.011+0000 7f1b90a19640 1 --2- 172.21.15.89:0/3950994920 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b8c159490 0x7f1b8c15d8b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:00.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.011+0000 7f1b90a19640 1 -- 172.21.15.89:0/3950994920 >> 172.21.15.89:0/3950994920 conn(0x7f1b8c069f00 msgr2=0x7f1b8c0cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:00.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.011+0000 7f1b90a19640 1 -- 172.21.15.89:0/3950994920 shutdown_connections 2023-10-18T10:47:00.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.011+0000 7f1b90a19640 1 -- 172.21.15.89:0/3950994920 wait complete. 2023-10-18T10:47:00.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.015+0000 7f1b90a19640 1 Processor -- start 2023-10-18T10:47:00.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.015+0000 7f1b90a19640 1 -- start start 2023-10-18T10:47:00.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.015+0000 7f1b90a19640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b8c06bd10 0x7f1b8c155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:00.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.015+0000 7f1b90a19640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b8c070600 0x7f1b8c159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:00.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.015+0000 7f1b90a19640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b8c159490 0x7f1b8c151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:00.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.015+0000 7f1b90a19640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c151d60 con 0x7f1b8c159490 2023-10-18T10:47:00.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.015+0000 7f1b90a19640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c156570 con 0x7f1b8c06bd10 2023-10-18T10:47:00.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.015+0000 7f1b90a19640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c156850 con 0x7f1b8c070600 2023-10-18T10:47:00.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.015+0000 7f1b8ad9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b8c159490 0x7f1b8c151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:00.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.015+0000 7f1b8ad9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b8c159490 0x7f1b8c151820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57562/0 (socket says 172.21.15.89:57562) 2023-10-18T10:47:00.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.015+0000 7f1b8ad9d640 1 -- 172.21.15.89:0/2589609705 learned_addr learned my addr 172.21.15.89:0/2589609705 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:00.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.015+0000 7f1b8a59c640 1 --2- 172.21.15.89:0/2589609705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b8c06bd10 0x7f1b8c155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:00.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.015+0000 7f1b89d9b640 1 --2- 172.21.15.89:0/2589609705 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b8c070600 0x7f1b8c159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:00.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.015+0000 7f1b8ad9d640 1 -- 172.21.15.89:0/2589609705 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b8c070600 msgr2=0x7f1b8c159870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:00.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.015+0000 7f1b8ad9d640 1 --2- 172.21.15.89:0/2589609705 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b8c070600 0x7f1b8c159870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:00.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.015+0000 7f1b8ad9d640 1 -- 172.21.15.89:0/2589609705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b8c06bd10 msgr2=0x7f1b8c155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:00.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.015+0000 7f1b8ad9d640 1 --2- 172.21.15.89:0/2589609705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b8c06bd10 0x7f1b8c155ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:00.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.015+0000 7f1b8ad9d640 1 -- 172.21.15.89:0/2589609705 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b7c009420 con 0x7f1b8c159490 2023-10-18T10:47:00.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.015+0000 7f1b8ad9d640 1 --2- 172.21.15.89:0/2589609705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b8c159490 0x7f1b8c151820 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1b8400caa0 tx=0x7f1b8400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:00.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.015+0000 7f1b7affd640 1 -- 172.21.15.89:0/2589609705 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8400ac00 con 0x7f1b8c159490 2023-10-18T10:47:00.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.015+0000 7f1b7affd640 1 -- 172.21.15.89:0/2589609705 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8400ad60 con 0x7f1b8c159490 2023-10-18T10:47:00.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.015+0000 7f1b7affd640 1 -- 172.21.15.89:0/2589609705 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b84012620 con 0x7f1b8c159490 2023-10-18T10:47:00.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.015+0000 7f1b90a19640 1 -- 172.21.15.89:0/2589609705 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b8c156b30 con 0x7f1b8c159490 2023-10-18T10:47:00.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.015+0000 7f1b90a19640 1 -- 172.21.15.89:0/2589609705 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b8c1570f0 con 0x7f1b8c159490 2023-10-18T10:47:00.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.019+0000 7f1b90a19640 1 -- 172.21.15.89:0/2589609705 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b8c157310 con 0x7f1b8c159490 2023-10-18T10:47:00.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.019+0000 7f1b7affd640 1 -- 172.21.15.89:0/2589609705 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f1b84007070 con 0x7f1b8c159490 2023-10-18T10:47:00.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.019+0000 7f1b7affd640 1 --2- 172.21.15.89:0/2589609705 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1b6c065680 0x7f1b6c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:00.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.019+0000 7f1b7affd640 1 -- 172.21.15.89:0/2589609705 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1b84083110 con 0x7f1b8c159490 2023-10-18T10:47:00.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.019+0000 7f1b8a59c640 1 --2- 172.21.15.89:0/2589609705 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1b6c065680 0x7f1b6c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:00.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.019+0000 7f1b8a59c640 1 --2- 172.21.15.89:0/2589609705 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1b6c065680 0x7f1b6c067b40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c0ae000 tx=0x7f1b7c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:00.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.023+0000 7f1b7affd640 1 -- 172.21.15.89:0/2589609705 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f1b84050820 con 0x7f1b8c159490 2023-10-18T10:47:00.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.175+0000 7f1b90a19640 1 -- 172.21.15.89:0/2589609705 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1b8c04dbe0 con 0x7f1b8c159490 2023-10-18T10:47:00.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.175+0000 7f1b7affd640 1 -- 172.21.15.89:0/2589609705 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1b84017020 con 0x7f1b8c159490 2023-10-18T10:47:00.177 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-18T10:47:00.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.175+0000 7f1b90a19640 1 -- 172.21.15.89:0/2589609705 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1b6c065680 msgr2=0x7f1b6c067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:00.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.175+0000 7f1b90a19640 1 --2- 172.21.15.89:0/2589609705 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1b6c065680 0x7f1b6c067b40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c0ae000 tx=0x7f1b7c008040 comp rx=0 tx=0).stop 2023-10-18T10:47:00.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.175+0000 7f1b90a19640 1 -- 172.21.15.89:0/2589609705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b8c159490 msgr2=0x7f1b8c151820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:00.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.175+0000 7f1b90a19640 1 --2- 172.21.15.89:0/2589609705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b8c159490 0x7f1b8c151820 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1b8400caa0 tx=0x7f1b8400cf60 comp rx=0 tx=0).stop 2023-10-18T10:47:00.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.175+0000 7f1b90a19640 1 -- 172.21.15.89:0/2589609705 shutdown_connections 2023-10-18T10:47:00.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.175+0000 7f1b90a19640 1 --2- 172.21.15.89:0/2589609705 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1b6c065680 0x7f1b6c067b40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:00.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.175+0000 7f1b90a19640 1 --2- 172.21.15.89:0/2589609705 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b8c070600 0x7f1b8c159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:00.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.175+0000 7f1b90a19640 1 --2- 172.21.15.89:0/2589609705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b8c159490 0x7f1b8c151820 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:00.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.175+0000 7f1b90a19640 1 --2- 172.21.15.89:0/2589609705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b8c06bd10 0x7f1b8c155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:00.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.175+0000 7f1b90a19640 1 -- 172.21.15.89:0/2589609705 >> 172.21.15.89:0/2589609705 conn(0x7f1b8c069f00 msgr2=0x7f1b8c0c53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:00.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.175+0000 7f1b90a19640 1 -- 172.21.15.89:0/2589609705 shutdown_connections 2023-10-18T10:47:00.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.175+0000 7f1b90a19640 1 -- 172.21.15.89:0/2589609705 wait complete. 2023-10-18T10:47:00.200 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-18T10:47:00.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.339+0000 7f2fb297f640 1 Processor -- start 2023-10-18T10:47:00.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.339+0000 7f2fb297f640 1 -- start start 2023-10-18T10:47:00.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.343+0000 7f2fb297f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2fac15c330 0x7f2fac0d1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:00.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.343+0000 7f2fb297f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fac15c710 0x7f2fac0b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:00.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.343+0000 7f2fb297f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2fac0b44f0 0x7f2fac0d19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:00.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.343+0000 7f2fb297f640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fac0d1650 con 0x7f2fac0b44f0 2023-10-18T10:47:00.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.343+0000 7f2fb297f640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fac0d0070 con 0x7f2fac15c330 2023-10-18T10:47:00.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.343+0000 7f2fb297f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac0d0350 con 0x7f2fac15c710 2023-10-18T10:47:00.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.343+0000 7f2fb217e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2fac0b44f0 0x7f2fac0d19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:00.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.343+0000 7f2fb217e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2fac0b44f0 0x7f2fac0d19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57576/0 (socket says 172.21.15.89:57576) 2023-10-18T10:47:00.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.343+0000 7f2fb197d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2fac15c330 0x7f2fac0d1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:00.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.343+0000 7f2fb217e640 1 -- 172.21.15.89:0/2891816685 learned_addr learned my addr 172.21.15.89:0/2891816685 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:00.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.343+0000 7f2fb117c640 1 --2- 172.21.15.89:0/2891816685 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fac15c710 0x7f2fac0b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:00.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.343+0000 7f2fb217e640 1 -- 172.21.15.89:0/2891816685 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fac15c710 msgr2=0x7f2fac0b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:00.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.343+0000 7f2fb217e640 1 --2- 172.21.15.89:0/2891816685 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fac15c710 0x7f2fac0b3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:00.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.343+0000 7f2fb217e640 1 -- 172.21.15.89:0/2891816685 >> v2:172.21.15.154:3300/0 conn(0x7f2fac15c330 msgr2=0x7f2fac0d1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:00.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.343+0000 7f2fb217e640 1 --2- 172.21.15.89:0/2891816685 >> v2:172.21.15.154:3300/0 conn(0x7f2fac15c330 0x7f2fac0d1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:00.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.343+0000 7f2fb217e640 1 -- 172.21.15.89:0/2891816685 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fac0d0630 con 0x7f2fac0b44f0 2023-10-18T10:47:00.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.343+0000 7f2fb217e640 1 --2- 172.21.15.89:0/2891816685 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fac0b44f0 0x7f2fac0d19e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2fac0cfb00 tx=0x7f2fa401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4879d5eecbb23361 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:00.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.347+0000 7f2fb097b640 1 -- 172.21.15.89:0/2891816685 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa40a55c0 con 0x7f2fac0b44f0 2023-10-18T10:47:00.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.347+0000 7f2fb097b640 1 -- 172.21.15.89:0/2891816685 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa40a5720 con 0x7f2fac0b44f0 2023-10-18T10:47:00.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.347+0000 7f2fb097b640 1 -- 172.21.15.89:0/2891816685 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa40a5ab0 con 0x7f2fac0b44f0 2023-10-18T10:47:00.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.347+0000 7f2fb297f640 1 -- 172.21.15.89:0/2891816685 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fac0b44f0 msgr2=0x7f2fac0d19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:00.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.347+0000 7f2fb297f640 1 --2- 172.21.15.89:0/2891816685 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fac0b44f0 0x7f2fac0d19e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2fac0cfb00 tx=0x7f2fa401c940 comp rx=0 tx=0).stop 2023-10-18T10:47:00.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.347+0000 7f2fb297f640 1 -- 172.21.15.89:0/2891816685 shutdown_connections 2023-10-18T10:47:00.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.347+0000 7f2fb297f640 1 --2- 172.21.15.89:0/2891816685 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fac0b44f0 0x7f2fac0d19e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:00.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.347+0000 7f2fb297f640 1 --2- 172.21.15.89:0/2891816685 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fac15c710 0x7f2fac0b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:00.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.347+0000 7f2fb297f640 1 --2- 172.21.15.89:0/2891816685 >> v2:172.21.15.154:3300/0 conn(0x7f2fac15c330 0x7f2fac0d1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:00.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.347+0000 7f2fb297f640 1 -- 172.21.15.89:0/2891816685 >> 172.21.15.89:0/2891816685 conn(0x7f2fac070320 msgr2=0x7f2fac070730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:00.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.347+0000 7f2fb297f640 1 -- 172.21.15.89:0/2891816685 shutdown_connections 2023-10-18T10:47:00.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.347+0000 7f2fb297f640 1 -- 172.21.15.89:0/2891816685 wait complete. 2023-10-18T10:47:00.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.347+0000 7f2fb297f640 1 Processor -- start 2023-10-18T10:47:00.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.347+0000 7f2fb297f640 1 -- start start 2023-10-18T10:47:00.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.347+0000 7f2fb297f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fac0b44f0 0x7f2fac1592e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:00.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.347+0000 7f2fb297f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fac15c330 0x7f2fac159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:00.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.347+0000 7f2fb297f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fac15c710 0x7f2fac157890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:00.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.347+0000 7f2fb297f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac150210 con 0x7f2fac15c710 2023-10-18T10:47:00.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.347+0000 7f2fb297f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac154a30 con 0x7f2fac15c330 2023-10-18T10:47:00.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.347+0000 7f2fb297f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac154750 con 0x7f2fac0b44f0 2023-10-18T10:47:00.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.347+0000 7f2fb217e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fac15c710 0x7f2fac157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:00.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.351+0000 7f2fb217e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fac15c710 0x7f2fac157890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57586/0 (socket says 172.21.15.89:57586) 2023-10-18T10:47:00.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.351+0000 7f2fb217e640 1 -- 172.21.15.89:0/1571060263 learned_addr learned my addr 172.21.15.89:0/1571060263 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:00.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.351+0000 7f2fb117c640 1 --2- 172.21.15.89:0/1571060263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fac15c330 0x7f2fac159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:00.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.351+0000 7f2fb197d640 1 --2- 172.21.15.89:0/1571060263 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fac0b44f0 0x7f2fac1592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:00.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.351+0000 7f2fb217e640 1 -- 172.21.15.89:0/1571060263 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fac0b44f0 msgr2=0x7f2fac1592e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:00.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.351+0000 7f2fb217e640 1 --2- 172.21.15.89:0/1571060263 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fac0b44f0 0x7f2fac1592e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:00.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.351+0000 7f2fb217e640 1 -- 172.21.15.89:0/1571060263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fac15c330 msgr2=0x7f2fac159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:00.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.351+0000 7f2fb217e640 1 --2- 172.21.15.89:0/1571060263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fac15c330 0x7f2fac159820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:00.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.351+0000 7f2fb217e640 1 -- 172.21.15.89:0/1571060263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa4009b30 con 0x7f2fac15c710 2023-10-18T10:47:00.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.351+0000 7f2fb217e640 1 --2- 172.21.15.89:0/1571060263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fac15c710 0x7f2fac157890 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40af040 tx=0x7f2fa40196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:00.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.351+0000 7f2fa27fc640 1 -- 172.21.15.89:0/1571060263 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa40a55c0 con 0x7f2fac15c710 2023-10-18T10:47:00.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.351+0000 7f2fa27fc640 1 -- 172.21.15.89:0/1571060263 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa40b8690 con 0x7f2fac15c710 2023-10-18T10:47:00.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.351+0000 7f2fa27fc640 1 -- 172.21.15.89:0/1571060263 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa40b8a40 con 0x7f2fac15c710 2023-10-18T10:47:00.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.351+0000 7f2fb297f640 1 -- 172.21.15.89:0/1571060263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fac0cb100 con 0x7f2fac15c710 2023-10-18T10:47:00.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.351+0000 7f2fb297f640 1 -- 172.21.15.89:0/1571060263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fac0cb6c0 con 0x7f2fac15c710 2023-10-18T10:47:00.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.351+0000 7f2fb297f640 1 -- 172.21.15.89:0/1571060263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fac0cb8b0 con 0x7f2fac15c710 2023-10-18T10:47:00.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.351+0000 7f2fa27fc640 1 -- 172.21.15.89:0/1571060263 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f2fa40b8ba0 con 0x7f2fac15c710 2023-10-18T10:47:00.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.351+0000 7f2fa27fc640 1 --2- 172.21.15.89:0/1571060263 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2f900656b0 0x7f2f90067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:00.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.351+0000 7f2fa27fc640 1 -- 172.21.15.89:0/1571060263 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2fa411e570 con 0x7f2fac15c710 2023-10-18T10:47:00.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.355+0000 7f2fb197d640 1 --2- 172.21.15.89:0/1571060263 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2f900656b0 0x7f2f90067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:00.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.355+0000 7f2fb197d640 1 --2- 172.21.15.89:0/1571060263 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2f900656b0 0x7f2f90067b70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8009b60 tx=0x7f2fa8009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:00.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.355+0000 7f2fa27fc640 1 -- 172.21.15.89:0/1571060263 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f2fa40ec020 con 0x7f2fac15c710 2023-10-18T10:47:00.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.511+0000 7f2fb297f640 1 -- 172.21.15.89:0/1571060263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2fac073b50 con 0x7f2fac15c710 2023-10-18T10:47:00.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.511+0000 7f2fa27fc640 1 -- 172.21.15.89:0/1571060263 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2fa4009510 con 0x7f2fac15c710 2023-10-18T10:47:00.513 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-10-18T10:47:00.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.511+0000 7f2fb297f640 1 -- 172.21.15.89:0/1571060263 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2f900656b0 msgr2=0x7f2f90067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:00.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.511+0000 7f2fb297f640 1 --2- 172.21.15.89:0/1571060263 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2f900656b0 0x7f2f90067b70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8009b60 tx=0x7f2fa8009190 comp rx=0 tx=0).stop 2023-10-18T10:47:00.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.511+0000 7f2fb297f640 1 -- 172.21.15.89:0/1571060263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fac15c710 msgr2=0x7f2fac157890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:00.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.511+0000 7f2fb297f640 1 --2- 172.21.15.89:0/1571060263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fac15c710 0x7f2fac157890 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40af040 tx=0x7f2fa40196e0 comp rx=0 tx=0).stop 2023-10-18T10:47:00.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.511+0000 7f2fb297f640 1 -- 172.21.15.89:0/1571060263 shutdown_connections 2023-10-18T10:47:00.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.511+0000 7f2fb297f640 1 --2- 172.21.15.89:0/1571060263 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2f900656b0 0x7f2f90067b70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:00.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.511+0000 7f2fb297f640 1 --2- 172.21.15.89:0/1571060263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fac15c710 0x7f2fac157890 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:00.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.511+0000 7f2fb297f640 1 --2- 172.21.15.89:0/1571060263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fac15c330 0x7f2fac159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:00.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.511+0000 7f2fb297f640 1 --2- 172.21.15.89:0/1571060263 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fac0b44f0 0x7f2fac1592e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:00.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.511+0000 7f2fb297f640 1 -- 172.21.15.89:0/1571060263 >> 172.21.15.89:0/1571060263 conn(0x7f2fac070320 msgr2=0x7f2fac0d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:00.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.511+0000 7f2fb297f640 1 -- 172.21.15.89:0/1571060263 shutdown_connections 2023-10-18T10:47:00.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.511+0000 7f2fb297f640 1 -- 172.21.15.89:0/1571060263 wait complete. 2023-10-18T10:47:00.534 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:47:00.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.675+0000 7fda50b37640 1 Processor -- start 2023-10-18T10:47:00.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.675+0000 7fda50b37640 1 -- start start 2023-10-18T10:47:00.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.679+0000 7fda50b37640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda4c069900 0x7fda4c069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:00.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.679+0000 7fda50b37640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda4c0c94b0 con 0x7fda4c0cc660 2023-10-18T10:47:00.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.679+0000 7fda50b37640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda4c15cc00 con 0x7fda4c071130 2023-10-18T10:47:00.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.679+0000 7fda50b37640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c0c9180 con 0x7fda4c069900 2023-10-18T10:47:00.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.679+0000 7fda49d9b640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fda4c0cc660 0x7fda4c070a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58204/0 (socket says 172.21.15.89:58204) 2023-10-18T10:47:00.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.679+0000 7fda49d9b640 1 -- 172.21.15.89:0/1587189178 learned_addr learned my addr 172.21.15.89:0/1587189178 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:47:00.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.679+0000 7fda4a59c640 1 --2- 172.21.15.89:0/1587189178 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda4c069900 0x7fda4c069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:00.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.679+0000 7fda4959a640 1 -- 172.21.15.89:0/1587189178 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3369196015 0 0) 0x7fda4c0c94b0 con 0x7fda4c0cc660 2023-10-18T10:47:00.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.679+0000 7fda4959a640 1 -- 172.21.15.89:0/1587189178 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda34003530 con 0x7fda4c0cc660 2023-10-18T10:47:00.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.679+0000 7fda4959a640 1 -- 172.21.15.89:0/1587189178 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2517288960 0 0) 0x7fda34003530 con 0x7fda4c0cc660 2023-10-18T10:47:00.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.679+0000 7fda4959a640 1 -- 172.21.15.89:0/1587189178 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda4c069900 msgr2=0x7fda4c069ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:00.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.679+0000 7fda4959a640 1 --2- 172.21.15.89:0/1587189178 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda4c069900 0x7fda4c069ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:00.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.679+0000 7fda4959a640 1 -- 172.21.15.89:0/1587189178 >> v1:172.21.15.154:6789/0 conn(0x7fda4c071130 legacy=0x7fda4c0c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:00.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.679+0000 7fda4959a640 1 -- 172.21.15.89:0/1587189178 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4c06a220 con 0x7fda4c0cc660 2023-10-18T10:47:00.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.679+0000 7fda4959a640 1 -- 172.21.15.89:0/1587189178 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7fda40001680 con 0x7fda4c0cc660 2023-10-18T10:47:00.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.679+0000 7fda4959a640 1 -- 172.21.15.89:0/1587189178 >> v1:172.21.15.89:6789/0 conn(0x7fda4c0cc660 legacy=0x7fda4c070a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:00.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.679+0000 7fda4959a640 1 --2- 172.21.15.89:0/1587189178 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda340039b0 0x7fda34003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:00.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.679+0000 7fda4959a640 1 --2- 172.21.15.89:0/1587189178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda34004360 0x7fda34004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:00.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.679+0000 7fda4959a640 1 --2- 172.21.15.89:0/1587189178 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda4c069900 0x7fda34005090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:00.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.679+0000 7fda4959a640 1 -- 172.21.15.89:0/1587189178 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c0c9180 con 0x7fda34004360 2023-10-18T10:47:00.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.679+0000 7fda4959a640 1 -- 172.21.15.89:0/1587189178 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3400b990 con 0x7fda340039b0 2023-10-18T10:47:00.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.679+0000 7fda4959a640 1 -- 172.21.15.89:0/1587189178 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda3400bb50 con 0x7fda4c069900 2023-10-18T10:47:00.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.679+0000 7fda49d9b640 1 --2- 172.21.15.89:0/1587189178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda34004360 0x7fda34004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:00.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.679+0000 7fda4a59c640 1 --2- 172.21.15.89:0/1587189178 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda340039b0 0x7fda34003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:00.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.679+0000 7fda4ad9d640 1 --2- 172.21.15.89:0/1587189178 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda4c069900 0x7fda34005090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:00.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.679+0000 7fda49d9b640 1 -- 172.21.15.89:0/1587189178 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda4c069900 msgr2=0x7fda34005090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:00.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.679+0000 7fda49d9b640 1 --2- 172.21.15.89:0/1587189178 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda4c069900 0x7fda34005090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:00.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.679+0000 7fda49d9b640 1 -- 172.21.15.89:0/1587189178 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda340039b0 msgr2=0x7fda34003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:00.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.679+0000 7fda49d9b640 1 --2- 172.21.15.89:0/1587189178 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda340039b0 0x7fda34003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:00.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.679+0000 7fda49d9b640 1 -- 172.21.15.89:0/1587189178 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda34003530 con 0x7fda34004360 2023-10-18T10:47:00.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.683+0000 7fda49d9b640 1 --2- 172.21.15.89:0/1587189178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda34004360 0x7fda34004830 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fda4000c190 tx=0x7fda4001cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1b03f268d708b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:00.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.683+0000 7fda4959a640 1 -- 172.21.15.89:0/1587189178 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda40005ae0 con 0x7fda34004360 2023-10-18T10:47:00.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.683+0000 7fda4959a640 1 -- 172.21.15.89:0/1587189178 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda40005c60 con 0x7fda34004360 2023-10-18T10:47:00.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.683+0000 7fda4959a640 1 -- 172.21.15.89:0/1587189178 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda400a5780 con 0x7fda34004360 2023-10-18T10:47:00.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.683+0000 7fda50b37640 1 -- 172.21.15.89:0/1587189178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda34004360 msgr2=0x7fda34004830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:00.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.683+0000 7fda50b37640 1 --2- 172.21.15.89:0/1587189178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda34004360 0x7fda34004830 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fda4000c190 tx=0x7fda4001cb90 comp rx=0 tx=0).stop 2023-10-18T10:47:00.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.683+0000 7fda4ad9d640 1 -- 172.21.15.89:0/1587189178 reap_dead start 2023-10-18T10:47:00.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.683+0000 7fda50b37640 1 -- 172.21.15.89:0/1587189178 shutdown_connections 2023-10-18T10:47:00.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.683+0000 7fda50b37640 1 -- 172.21.15.89:0/1587189178 >> 172.21.15.89:0/1587189178 conn(0x7fda4c0cb5a0 msgr2=0x7fda4c0cb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:00.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.683+0000 7fda50b37640 1 -- 172.21.15.89:0/1587189178 shutdown_connections 2023-10-18T10:47:00.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.683+0000 7fda50b37640 1 -- 172.21.15.89:0/1587189178 wait complete. 2023-10-18T10:47:00.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.687+0000 7fda50b37640 1 Processor -- start 2023-10-18T10:47:00.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.687+0000 7fda50b37640 1 -- start start 2023-10-18T10:47:00.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.687+0000 7fda50b37640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda34004360 0x7fda4c159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:00.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.687+0000 7fda50b37640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda4c0b1780 0x7fda4c0b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:00.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.687+0000 7fda50b37640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda4c0b3b40 0x7fda4c0b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:00.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.687+0000 7fda50b37640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c0b4500 con 0x7fda4c0b3b40 2023-10-18T10:47:00.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.687+0000 7fda50b37640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c0b21d0 con 0x7fda34004360 2023-10-18T10:47:00.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.687+0000 7fda50b37640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c0b24b0 con 0x7fda4c0b1780 2023-10-18T10:47:00.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.687+0000 7fda4ad9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda4c0b3b40 0x7fda4c0b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:00.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.687+0000 7fda4ad9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda4c0b3b40 0x7fda4c0b3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57608/0 (socket says 172.21.15.89:57608) 2023-10-18T10:47:00.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.687+0000 7fda4ad9d640 1 -- 172.21.15.89:0/1966024290 learned_addr learned my addr 172.21.15.89:0/1966024290 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:00.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.687+0000 7fda4a59c640 1 --2- 172.21.15.89:0/1966024290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda34004360 0x7fda4c159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:00.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.687+0000 7fda49d9b640 1 --2- 172.21.15.89:0/1966024290 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda4c0b1780 0x7fda4c0b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:00.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.687+0000 7fda4ad9d640 1 -- 172.21.15.89:0/1966024290 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda4c0b1780 msgr2=0x7fda4c0b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:00.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.687+0000 7fda4ad9d640 1 --2- 172.21.15.89:0/1966024290 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda4c0b1780 0x7fda4c0b1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:00.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.687+0000 7fda4ad9d640 1 -- 172.21.15.89:0/1966024290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda34004360 msgr2=0x7fda4c159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:00.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.687+0000 7fda4ad9d640 1 --2- 172.21.15.89:0/1966024290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda34004360 0x7fda4c159250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:00.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.687+0000 7fda4ad9d640 1 -- 172.21.15.89:0/1966024290 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4c0b2790 con 0x7fda4c0b3b40 2023-10-18T10:47:00.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.687+0000 7fda4a59c640 1 --2- 172.21.15.89:0/1966024290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda34004360 0x7fda4c159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:47:00.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.687+0000 7fda4ad9d640 1 --2- 172.21.15.89:0/1966024290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda4c0b3b40 0x7fda4c0b3fc0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fda4c0b4c70 tx=0x7fda4400ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:00.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.687+0000 7fda3affd640 1 -- 172.21.15.89:0/1966024290 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda44006d70 con 0x7fda4c0b3b40 2023-10-18T10:47:00.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.687+0000 7fda3affd640 1 -- 172.21.15.89:0/1966024290 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda44003980 con 0x7fda4c0b3b40 2023-10-18T10:47:00.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.687+0000 7fda3affd640 1 -- 172.21.15.89:0/1966024290 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda44003d30 con 0x7fda4c0b3b40 2023-10-18T10:47:00.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.687+0000 7fda50b37640 1 -- 172.21.15.89:0/1966024290 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda4c0b2a70 con 0x7fda4c0b3b40 2023-10-18T10:47:00.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.687+0000 7fda50b37640 1 -- 172.21.15.89:0/1966024290 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda4c159790 con 0x7fda4c0b3b40 2023-10-18T10:47:00.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.691+0000 7fda50b37640 1 -- 172.21.15.89:0/1966024290 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda4c072a80 con 0x7fda4c0b3b40 2023-10-18T10:47:00.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.691+0000 7fda3affd640 1 -- 172.21.15.89:0/1966024290 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fda4400d720 con 0x7fda4c0b3b40 2023-10-18T10:47:00.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.691+0000 7fda3affd640 1 --2- 172.21.15.89:0/1966024290 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fda2c0656b0 0x7fda2c067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:00.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.691+0000 7fda3affd640 1 -- 172.21.15.89:0/1966024290 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fda44082350 con 0x7fda4c0b3b40 2023-10-18T10:47:00.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.691+0000 7fda4a59c640 1 --2- 172.21.15.89:0/1966024290 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fda2c0656b0 0x7fda2c067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:00.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.691+0000 7fda4a59c640 1 --2- 172.21.15.89:0/1966024290 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fda2c0656b0 0x7fda2c067b70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fda3c009be0 tx=0x7fda3c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:00.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.695+0000 7fda3affd640 1 -- 172.21.15.89:0/1966024290 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fda4404fa40 con 0x7fda4c0b3b40 2023-10-18T10:47:00.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.843+0000 7fda50b37640 1 -- 172.21.15.89:0/1966024290 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fda4c04dbe0 con 0x7fda4c0b3b40 2023-10-18T10:47:00.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.843+0000 7fda3affd640 1 -- 172.21.15.89:0/1966024290 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7fda44053060 con 0x7fda4c0b3b40 2023-10-18T10:47:00.846 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-18T10:47:00.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.843+0000 7fda50b37640 1 -- 172.21.15.89:0/1966024290 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fda2c0656b0 msgr2=0x7fda2c067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:00.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.843+0000 7fda50b37640 1 --2- 172.21.15.89:0/1966024290 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fda2c0656b0 0x7fda2c067b70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fda3c009be0 tx=0x7fda3c008040 comp rx=0 tx=0).stop 2023-10-18T10:47:00.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.843+0000 7fda50b37640 1 -- 172.21.15.89:0/1966024290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda4c0b3b40 msgr2=0x7fda4c0b3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:00.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.843+0000 7fda50b37640 1 --2- 172.21.15.89:0/1966024290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda4c0b3b40 0x7fda4c0b3fc0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fda4c0b4c70 tx=0x7fda4400ff70 comp rx=0 tx=0).stop 2023-10-18T10:47:00.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.843+0000 7fda50b37640 1 -- 172.21.15.89:0/1966024290 shutdown_connections 2023-10-18T10:47:00.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.843+0000 7fda50b37640 1 --2- 172.21.15.89:0/1966024290 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fda2c0656b0 0x7fda2c067b70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:00.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.843+0000 7fda50b37640 1 --2- 172.21.15.89:0/1966024290 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda4c0b1780 0x7fda4c0b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:00.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.843+0000 7fda50b37640 1 --2- 172.21.15.89:0/1966024290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda4c0b3b40 0x7fda4c0b3fc0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:00.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.843+0000 7fda50b37640 1 --2- 172.21.15.89:0/1966024290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda34004360 0x7fda4c159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:00.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.843+0000 7fda50b37640 1 -- 172.21.15.89:0/1966024290 >> 172.21.15.89:0/1966024290 conn(0x7fda4c0cb5a0 msgr2=0x7fda4c0ad8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:00.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.843+0000 7fda50b37640 1 -- 172.21.15.89:0/1966024290 shutdown_connections 2023-10-18T10:47:00.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:00.843+0000 7fda50b37640 1 -- 172.21.15.89:0/1966024290 wait complete. 2023-10-18T10:47:00.867 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:47:01.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.007+0000 7fd096a88640 1 Processor -- start 2023-10-18T10:47:01.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.007+0000 7fd096a88640 1 -- start start 2023-10-18T10:47:01.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.007+0000 7fd096a88640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0900cd4f0 0x7fd090143470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:01.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.007+0000 7fd096a88640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd090145b70 0x7fd090160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:01.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.007+0000 7fd096a88640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0900cd8d0 con 0x7fd0900ce070 2023-10-18T10:47:01.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.007+0000 7fd096a88640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd090069900 con 0x7fd090145b70 2023-10-18T10:47:01.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.007+0000 7fd096a88640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd09015cbf0 con 0x7fd0900cd4f0 2023-10-18T10:47:01.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.007+0000 7fd087fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd090145b70 0x7fd090160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:01.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.007+0000 7fd087fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd090145b70 0x7fd090160cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57620/0 (socket says 172.21.15.89:57620) 2023-10-18T10:47:01.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.007+0000 7fd087fff640 1 -- 172.21.15.89:0/2248908444 learned_addr learned my addr 172.21.15.89:0/2248908444 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:01.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.007+0000 7fd094824640 1 --2- 172.21.15.89:0/2248908444 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0900cd4f0 0x7fd090143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:01.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.007+0000 7fd087fff640 1 -- 172.21.15.89:0/2248908444 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0900cd4f0 msgr2=0x7fd090143470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:01.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.007+0000 7fd087fff640 1 --2- 172.21.15.89:0/2248908444 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0900cd4f0 0x7fd090143470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:01.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.007+0000 7fd087fff640 1 -- 172.21.15.89:0/2248908444 >> v1:172.21.15.89:6789/0 conn(0x7fd0900ce070 legacy=0x7fd09006abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:01.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.007+0000 7fd087fff640 1 -- 172.21.15.89:0/2248908444 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd090161230 con 0x7fd090145b70 2023-10-18T10:47:01.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.011+0000 7fd087fff640 1 --2- 172.21.15.89:0/2248908444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd090145b70 0x7fd090160cf0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd080009870 tx=0x7fd08001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fffa6b7723650fc0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:01.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.011+0000 7fd0877fe640 1 -- 172.21.15.89:0/2248908444 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0800aa670 con 0x7fd090145b70 2023-10-18T10:47:01.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.011+0000 7fd0877fe640 1 -- 172.21.15.89:0/2248908444 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0800aa7d0 con 0x7fd090145b70 2023-10-18T10:47:01.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.011+0000 7fd0877fe640 1 -- 172.21.15.89:0/2248908444 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0800aab60 con 0x7fd090145b70 2023-10-18T10:47:01.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.011+0000 7fd096a88640 1 -- 172.21.15.89:0/2248908444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd090145b70 msgr2=0x7fd090160cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:01.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.011+0000 7fd096a88640 1 --2- 172.21.15.89:0/2248908444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd090145b70 0x7fd090160cf0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd080009870 tx=0x7fd08001c510 comp rx=0 tx=0).stop 2023-10-18T10:47:01.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.011+0000 7fd096a88640 1 -- 172.21.15.89:0/2248908444 shutdown_connections 2023-10-18T10:47:01.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.011+0000 7fd096a88640 1 --2- 172.21.15.89:0/2248908444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd090145b70 0x7fd090160cf0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:01.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.011+0000 7fd096a88640 1 --2- 172.21.15.89:0/2248908444 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0900cd4f0 0x7fd090143470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:01.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.011+0000 7fd096a88640 1 -- 172.21.15.89:0/2248908444 >> 172.21.15.89:0/2248908444 conn(0x7fd09006dad0 msgr2=0x7fd09006dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:01.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.011+0000 7fd096a88640 1 -- 172.21.15.89:0/2248908444 shutdown_connections 2023-10-18T10:47:01.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.011+0000 7fd096a88640 1 -- 172.21.15.89:0/2248908444 wait complete. 2023-10-18T10:47:01.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.015+0000 7fd096a88640 1 Processor -- start 2023-10-18T10:47:01.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.015+0000 7fd096a88640 1 -- start start 2023-10-18T10:47:01.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.015+0000 7fd096a88640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0900cd4f0 0x7fd0900c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:01.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.015+0000 7fd096a88640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0900ce070 0x7fd0900c2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:01.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.015+0000 7fd096a88640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd090145b70 0x7fd0900b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:01.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.015+0000 7fd096a88640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0900c87f0 con 0x7fd0900cd4f0 2023-10-18T10:47:01.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.015+0000 7fd096a88640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0900c7be0 con 0x7fd0900ce070 2023-10-18T10:47:01.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.015+0000 7fd096a88640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0900c7ec0 con 0x7fd090145b70 2023-10-18T10:47:01.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.015+0000 7fd094824640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0900cd4f0 0x7fd0900c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:01.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.015+0000 7fd094824640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0900cd4f0 0x7fd0900c7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57632/0 (socket says 172.21.15.89:57632) 2023-10-18T10:47:01.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.015+0000 7fd094824640 1 -- 172.21.15.89:0/2371020151 learned_addr learned my addr 172.21.15.89:0/2371020151 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:01.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.015+0000 7fd095025640 1 --2- 172.21.15.89:0/2371020151 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd090145b70 0x7fd0900b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:01.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.015+0000 7fd087fff640 1 --2- 172.21.15.89:0/2371020151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0900ce070 0x7fd0900c2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:01.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.015+0000 7fd094824640 1 -- 172.21.15.89:0/2371020151 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd090145b70 msgr2=0x7fd0900b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:01.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.015+0000 7fd094824640 1 --2- 172.21.15.89:0/2371020151 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd090145b70 0x7fd0900b34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:01.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.015+0000 7fd094824640 1 -- 172.21.15.89:0/2371020151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0900ce070 msgr2=0x7fd0900c2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:01.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.015+0000 7fd094824640 1 --2- 172.21.15.89:0/2371020151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0900ce070 0x7fd0900c2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:01.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.015+0000 7fd094824640 1 -- 172.21.15.89:0/2371020151 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0800094d0 con 0x7fd0900cd4f0 2023-10-18T10:47:01.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.015+0000 7fd087fff640 1 --2- 172.21.15.89:0/2371020151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0900ce070 0x7fd0900c2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:47:01.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.015+0000 7fd094824640 1 --2- 172.21.15.89:0/2371020151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0900cd4f0 0x7fd0900c7570 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd08c00e870 tx=0x7fd08c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:01.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.015+0000 7fd0857fa640 1 -- 172.21.15.89:0/2371020151 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c009510 con 0x7fd0900cd4f0 2023-10-18T10:47:01.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.015+0000 7fd0857fa640 1 -- 172.21.15.89:0/2371020151 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08c00eec0 con 0x7fd0900cd4f0 2023-10-18T10:47:01.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.015+0000 7fd096a88640 1 -- 172.21.15.89:0/2371020151 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0900c81a0 con 0x7fd0900cd4f0 2023-10-18T10:47:01.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.015+0000 7fd0857fa640 1 -- 172.21.15.89:0/2371020151 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c00cb40 con 0x7fd0900cd4f0 2023-10-18T10:47:01.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.015+0000 7fd096a88640 1 -- 172.21.15.89:0/2371020151 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0900c2e70 con 0x7fd0900cd4f0 2023-10-18T10:47:01.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.019+0000 7fd096a88640 1 -- 172.21.15.89:0/2371020151 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd09006a670 con 0x7fd0900cd4f0 2023-10-18T10:47:01.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.019+0000 7fd0857fa640 1 -- 172.21.15.89:0/2371020151 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fd08c00cca0 con 0x7fd0900cd4f0 2023-10-18T10:47:01.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.019+0000 7fd0857fa640 1 --2- 172.21.15.89:0/2371020151 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd07c065680 0x7fd07c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:01.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.019+0000 7fd0857fa640 1 -- 172.21.15.89:0/2371020151 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd08c0832d0 con 0x7fd0900cd4f0 2023-10-18T10:47:01.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.019+0000 7fd087fff640 1 --2- 172.21.15.89:0/2371020151 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd07c065680 0x7fd07c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:01.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.019+0000 7fd087fff640 1 --2- 172.21.15.89:0/2371020151 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd07c065680 0x7fd07c067b40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd0800ae000 tx=0x7fd0800a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:01.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.023+0000 7fd0857fa640 1 -- 172.21.15.89:0/2371020151 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fd09006a670 con 0x7fd0900cd4f0 2023-10-18T10:47:01.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.171+0000 7fd096a88640 1 -- 172.21.15.89:0/2371020151 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd09006a670 con 0x7fd0900cd4f0 2023-10-18T10:47:01.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.171+0000 7fd0857fa640 1 -- 172.21.15.89:0/2371020151 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd09006a670 con 0x7fd0900cd4f0 2023-10-18T10:47:01.175 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-18T10:47:01.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.175+0000 7fd096a88640 1 -- 172.21.15.89:0/2371020151 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd07c065680 msgr2=0x7fd07c067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:01.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.175+0000 7fd096a88640 1 --2- 172.21.15.89:0/2371020151 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd07c065680 0x7fd07c067b40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd0800ae000 tx=0x7fd0800a7000 comp rx=0 tx=0).stop 2023-10-18T10:47:01.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.175+0000 7fd096a88640 1 -- 172.21.15.89:0/2371020151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0900cd4f0 msgr2=0x7fd0900c7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:01.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.175+0000 7fd096a88640 1 --2- 172.21.15.89:0/2371020151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0900cd4f0 0x7fd0900c7570 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd08c00e870 tx=0x7fd08c00ec70 comp rx=0 tx=0).stop 2023-10-18T10:47:01.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.175+0000 7fd096a88640 1 -- 172.21.15.89:0/2371020151 shutdown_connections 2023-10-18T10:47:01.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.175+0000 7fd096a88640 1 --2- 172.21.15.89:0/2371020151 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd07c065680 0x7fd07c067b40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:01.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.175+0000 7fd096a88640 1 --2- 172.21.15.89:0/2371020151 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd090145b70 0x7fd0900b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:01.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.175+0000 7fd096a88640 1 --2- 172.21.15.89:0/2371020151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0900ce070 0x7fd0900c2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:01.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.175+0000 7fd096a88640 1 --2- 172.21.15.89:0/2371020151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0900cd4f0 0x7fd0900c7570 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:01.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.175+0000 7fd096a88640 1 -- 172.21.15.89:0/2371020151 >> 172.21.15.89:0/2371020151 conn(0x7fd09006dad0 msgr2=0x7fd090158330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:01.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.175+0000 7fd096a88640 1 -- 172.21.15.89:0/2371020151 shutdown_connections 2023-10-18T10:47:01.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.175+0000 7fd096a88640 1 -- 172.21.15.89:0/2371020151 wait complete. 2023-10-18T10:47:01.196 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T10:47:01.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.335+0000 7f1af7d52640 1 Processor -- start 2023-10-18T10:47:01.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.335+0000 7f1af7d52640 1 -- start start 2023-10-18T10:47:01.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.335+0000 7f1af7d52640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1af00b1860 0x7f1af00cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:01.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.335+0000 7f1af7d52640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1af00d1e50 con 0x7f1af015c550 2023-10-18T10:47:01.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.335+0000 7f1af7d52640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1af00d2130 con 0x7f1af00b1cc0 2023-10-18T10:47:01.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.335+0000 7f1af7d52640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af00d2410 con 0x7f1af00b1860 2023-10-18T10:47:01.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.335+0000 7f1af5aee640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1af015c550 0x7f1af00d2ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58222/0 (socket says 172.21.15.89:58222) 2023-10-18T10:47:01.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.335+0000 7f1af5aee640 1 -- 172.21.15.89:0/3060566687 learned_addr learned my addr 172.21.15.89:0/3060566687 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:47:01.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.335+0000 7f1af52ed640 1 --2- 172.21.15.89:0/3060566687 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1af00b1860 0x7f1af00cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:01.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.335+0000 7f1af4aec640 1 -- 172.21.15.89:0/3060566687 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2550889900 0 0) 0x7f1af00d1e50 con 0x7f1af015c550 2023-10-18T10:47:01.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.335+0000 7f1af4aec640 1 -- 172.21.15.89:0/3060566687 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1adc003530 con 0x7f1af015c550 2023-10-18T10:47:01.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.335+0000 7f1af4aec640 1 -- 172.21.15.89:0/3060566687 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 40061489 0 0) 0x7f1af00d2130 con 0x7f1af00b1cc0 2023-10-18T10:47:01.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.335+0000 7f1af4aec640 1 -- 172.21.15.89:0/3060566687 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1af00d1e50 con 0x7f1af00b1cc0 2023-10-18T10:47:01.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.335+0000 7f1af4aec640 1 -- 172.21.15.89:0/3060566687 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3246731641 0 0) 0x7f1adc003530 con 0x7f1af015c550 2023-10-18T10:47:01.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.335+0000 7f1af52ed640 1 -- 172.21.15.89:0/3060566687 >> v1:172.21.15.154:6789/0 conn(0x7f1af00b1cc0 legacy=0x7f1af00b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:01.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.335+0000 7f1af52ed640 1 -- 172.21.15.89:0/3060566687 >> v1:172.21.15.89:6789/0 conn(0x7f1af015c550 legacy=0x7f1af00d2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:01.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.335+0000 7f1af52ed640 1 -- 172.21.15.89:0/3060566687 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af00d26f0 con 0x7f1af00b1860 2023-10-18T10:47:01.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.339+0000 7f1af52ed640 1 --2- 172.21.15.89:0/3060566687 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1af00b1860 0x7f1af00cf610 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1ae00097f0 tx=0x7f1ae001c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79bb47f285f5a8b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:01.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.339+0000 7f1af4aec640 1 -- 172.21.15.89:0/3060566687 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aec001680 con 0x7f1af00b1860 2023-10-18T10:47:01.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.339+0000 7f1af4aec640 1 -- 172.21.15.89:0/3060566687 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae00a7050 con 0x7f1af00b1860 2023-10-18T10:47:01.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.339+0000 7f1af4aec640 1 -- 172.21.15.89:0/3060566687 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae00ab620 con 0x7f1af00b1860 2023-10-18T10:47:01.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.339+0000 7f1af7d52640 1 -- 172.21.15.89:0/3060566687 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1af00b1860 msgr2=0x7f1af00cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:01.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.339+0000 7f1af7d52640 1 --2- 172.21.15.89:0/3060566687 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1af00b1860 0x7f1af00cf610 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1ae00097f0 tx=0x7f1ae001c3b0 comp rx=0 tx=0).stop 2023-10-18T10:47:01.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.339+0000 7f1af7d52640 1 -- 172.21.15.89:0/3060566687 shutdown_connections 2023-10-18T10:47:01.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.339+0000 7f1af7d52640 1 --2- 172.21.15.89:0/3060566687 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1af00b1860 0x7f1af00cf610 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:01.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.339+0000 7f1af7d52640 1 -- 172.21.15.89:0/3060566687 >> 172.21.15.89:0/3060566687 conn(0x7f1af0069d00 msgr2=0x7f1af006a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:01.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.339+0000 7f1af7d52640 1 -- 172.21.15.89:0/3060566687 shutdown_connections 2023-10-18T10:47:01.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.339+0000 7f1af7d52640 1 -- 172.21.15.89:0/3060566687 wait complete. 2023-10-18T10:47:01.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.343+0000 7f1af7d52640 1 Processor -- start 2023-10-18T10:47:01.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.343+0000 7f1af7d52640 1 -- start start 2023-10-18T10:47:01.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.343+0000 7f1af7d52640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1af00b1860 0x7f1af0158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:01.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.343+0000 7f1af7d52640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1af00b1cc0 0x7f1af015e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:01.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.343+0000 7f1af7d52640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1af015c550 0x7f1af015e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:01.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.343+0000 7f1af7d52640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af015de90 con 0x7f1af00b1cc0 2023-10-18T10:47:01.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.343+0000 7f1af7d52640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af0154770 con 0x7f1af015c550 2023-10-18T10:47:01.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.343+0000 7f1af7d52640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af015d7d0 con 0x7f1af00b1860 2023-10-18T10:47:01.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.343+0000 7f1af52ed640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1af00b1cc0 0x7f1af015e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:01.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.343+0000 7f1af52ed640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1af00b1cc0 0x7f1af015e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57636/0 (socket says 172.21.15.89:57636) 2023-10-18T10:47:01.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.343+0000 7f1af52ed640 1 -- 172.21.15.89:0/2927459269 learned_addr learned my addr 172.21.15.89:0/2927459269 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:01.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.343+0000 7f1af5aee640 1 --2- 172.21.15.89:0/2927459269 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1af00b1860 0x7f1af0158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:01.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.343+0000 7f1af62ef640 1 --2- 172.21.15.89:0/2927459269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1af015c550 0x7f1af015e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:01.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.343+0000 7f1af52ed640 1 -- 172.21.15.89:0/2927459269 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1af00b1860 msgr2=0x7f1af0158990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:01.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.343+0000 7f1af52ed640 1 --2- 172.21.15.89:0/2927459269 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1af00b1860 0x7f1af0158990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:01.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.343+0000 7f1af52ed640 1 -- 172.21.15.89:0/2927459269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1af015c550 msgr2=0x7f1af015e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:01.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.343+0000 7f1af52ed640 1 --2- 172.21.15.89:0/2927459269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1af015c550 0x7f1af015e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:01.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.343+0000 7f1af52ed640 1 -- 172.21.15.89:0/2927459269 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae0009420 con 0x7f1af00b1cc0 2023-10-18T10:47:01.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.343+0000 7f1af52ed640 1 --2- 172.21.15.89:0/2927459269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1af00b1cc0 0x7f1af015e2b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1ae00097f0 tx=0x7f1ae0019150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:01.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.343+0000 7f1ae67fc640 1 -- 172.21.15.89:0/2927459269 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae00b4400 con 0x7f1af00b1cc0 2023-10-18T10:47:01.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.343+0000 7f1ae67fc640 1 -- 172.21.15.89:0/2927459269 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae00b4560 con 0x7f1af00b1cc0 2023-10-18T10:47:01.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.343+0000 7f1af7d52640 1 -- 172.21.15.89:0/2927459269 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1af015dab0 con 0x7f1af00b1cc0 2023-10-18T10:47:01.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.343+0000 7f1ae67fc640 1 -- 172.21.15.89:0/2927459269 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae00b4910 con 0x7f1af00b1cc0 2023-10-18T10:47:01.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.343+0000 7f1af7d52640 1 -- 172.21.15.89:0/2927459269 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1af0154a50 con 0x7f1af00b1cc0 2023-10-18T10:47:01.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.347+0000 7f1af7d52640 1 -- 172.21.15.89:0/2927459269 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1af015dfd0 con 0x7f1af00b1cc0 2023-10-18T10:47:01.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.347+0000 7f1ae67fc640 1 -- 172.21.15.89:0/2927459269 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f1ae00b2040 con 0x7f1af00b1cc0 2023-10-18T10:47:01.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.347+0000 7f1ae67fc640 1 --2- 172.21.15.89:0/2927459269 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1ad40656b0 0x7f1ad4067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:01.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.347+0000 7f1ae67fc640 1 -- 172.21.15.89:0/2927459269 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1ae011d160 con 0x7f1af00b1cc0 2023-10-18T10:47:01.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.347+0000 7f1af5aee640 1 --2- 172.21.15.89:0/2927459269 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1ad40656b0 0x7f1ad4067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:01.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.347+0000 7f1af5aee640 1 --2- 172.21.15.89:0/2927459269 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1ad40656b0 0x7f1ad4067b70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1aec005890 tx=0x7f1aec009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:01.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.351+0000 7f1ae67fc640 1 -- 172.21.15.89:0/2927459269 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f1af015dfd0 con 0x7f1af00b1cc0 2023-10-18T10:47:01.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.499+0000 7f1af7d52640 1 -- 172.21.15.89:0/2927459269 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1af015dfd0 con 0x7f1af00b1cc0 2023-10-18T10:47:01.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.499+0000 7f1ae67fc640 1 -- 172.21.15.89:0/2927459269 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1af015dfd0 con 0x7f1af00b1cc0 2023-10-18T10:47:01.503 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-18T10:47:01.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.503+0000 7f1af7d52640 1 -- 172.21.15.89:0/2927459269 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1ad40656b0 msgr2=0x7f1ad4067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:01.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.503+0000 7f1af7d52640 1 --2- 172.21.15.89:0/2927459269 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1ad40656b0 0x7f1ad4067b70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1aec005890 tx=0x7f1aec009040 comp rx=0 tx=0).stop 2023-10-18T10:47:01.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.503+0000 7f1af7d52640 1 -- 172.21.15.89:0/2927459269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1af00b1cc0 msgr2=0x7f1af015e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:01.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.503+0000 7f1af7d52640 1 --2- 172.21.15.89:0/2927459269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1af00b1cc0 0x7f1af015e2b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1ae00097f0 tx=0x7f1ae0019150 comp rx=0 tx=0).stop 2023-10-18T10:47:01.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.503+0000 7f1af7d52640 1 -- 172.21.15.89:0/2927459269 shutdown_connections 2023-10-18T10:47:01.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.503+0000 7f1af7d52640 1 --2- 172.21.15.89:0/2927459269 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1ad40656b0 0x7f1ad4067b70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:01.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.503+0000 7f1af7d52640 1 --2- 172.21.15.89:0/2927459269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1af015c550 0x7f1af015e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:01.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.503+0000 7f1af7d52640 1 --2- 172.21.15.89:0/2927459269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1af00b1cc0 0x7f1af015e2b0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:01.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.503+0000 7f1af7d52640 1 --2- 172.21.15.89:0/2927459269 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1af00b1860 0x7f1af0158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:01.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.503+0000 7f1af7d52640 1 -- 172.21.15.89:0/2927459269 >> 172.21.15.89:0/2927459269 conn(0x7f1af0069d00 msgr2=0x7f1af00d0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:01.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.503+0000 7f1af7d52640 1 -- 172.21.15.89:0/2927459269 shutdown_connections 2023-10-18T10:47:01.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.503+0000 7f1af7d52640 1 -- 172.21.15.89:0/2927459269 wait complete. 2023-10-18T10:47:01.524 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:47:01.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.659+0000 7f04d9a6a640 1 Processor -- start 2023-10-18T10:47:01.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.659+0000 7f04d9a6a640 1 -- start start 2023-10-18T10:47:01.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.659+0000 7f04d9a6a640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f04d415c940 0x7f04d40d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:01.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.659+0000 7f04d9a6a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f04d40d2f40 0x7f04d40d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:01.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.659+0000 7f04d9a6a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04d40d10f0 con 0x7f04d40d0ca0 2023-10-18T10:47:01.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.659+0000 7f04d9a6a640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04d415da50 con 0x7f04d415c940 2023-10-18T10:47:01.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.659+0000 7f04d9a6a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04d415ce40 con 0x7f04d40d2f40 2023-10-18T10:47:01.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.659+0000 7f04d37fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f04d415c940 0x7f04d40d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:01.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.659+0000 7f04d37fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f04d415c940 0x7f04d40d2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57646/0 (socket says 172.21.15.89:57646) 2023-10-18T10:47:01.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.659+0000 7f04d37fe640 1 -- 172.21.15.89:0/2786054397 learned_addr learned my addr 172.21.15.89:0/2786054397 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:01.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.659+0000 7f04caffd640 1 --2- 172.21.15.89:0/2786054397 >> v2:172.21.15.154:3300/0 conn(0x7f04d40d2f40 0x7f04d40d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:01.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.659+0000 7f04d37fe640 1 -- 172.21.15.89:0/2786054397 >> v2:172.21.15.154:3300/0 conn(0x7f04d40d2f40 msgr2=0x7f04d40d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:01.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.659+0000 7f04d37fe640 1 --2- 172.21.15.89:0/2786054397 >> v2:172.21.15.154:3300/0 conn(0x7f04d40d2f40 0x7f04d40d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:01.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.659+0000 7f04d37fe640 1 -- 172.21.15.89:0/2786054397 >> v1:172.21.15.154:6789/0 conn(0x7f04d40d0ca0 legacy=0x7f04d40cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:01.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.659+0000 7f04d37fe640 1 -- 172.21.15.89:0/2786054397 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d415d120 con 0x7f04d415c940 2023-10-18T10:47:01.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.659+0000 7f04caffd640 1 --2- 172.21.15.89:0/2786054397 >> v2:172.21.15.154:3300/0 conn(0x7f04d40d2f40 0x7f04d40d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:47:01.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.663+0000 7f04d37fe640 1 --2- 172.21.15.89:0/2786054397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04d415c940 0x7f04d40d2a00 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f04d40724f0 tx=0x7f04bc01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c5f1bcd7ee0a24 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:01.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.663+0000 7f04d2ffd640 1 -- 172.21.15.89:0/2786054397 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc0ac640 con 0x7f04d415c940 2023-10-18T10:47:01.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.663+0000 7f04d2ffd640 1 -- 172.21.15.89:0/2786054397 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04bc0ac7a0 con 0x7f04d415c940 2023-10-18T10:47:01.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.663+0000 7f04d2ffd640 1 -- 172.21.15.89:0/2786054397 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc0acb30 con 0x7f04d415c940 2023-10-18T10:47:01.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.663+0000 7f04d9a6a640 1 -- 172.21.15.89:0/2786054397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04d415c940 msgr2=0x7f04d40d2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:01.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.663+0000 7f04d9a6a640 1 --2- 172.21.15.89:0/2786054397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04d415c940 0x7f04d40d2a00 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f04d40724f0 tx=0x7f04bc01c4d0 comp rx=0 tx=0).stop 2023-10-18T10:47:01.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.663+0000 7f04d9a6a640 1 -- 172.21.15.89:0/2786054397 shutdown_connections 2023-10-18T10:47:01.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.663+0000 7f04d9a6a640 1 --2- 172.21.15.89:0/2786054397 >> v2:172.21.15.154:3300/0 conn(0x7f04d40d2f40 0x7f04d40d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:01.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.663+0000 7f04d9a6a640 1 --2- 172.21.15.89:0/2786054397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04d415c940 0x7f04d40d2a00 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:01.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.663+0000 7f04d9a6a640 1 -- 172.21.15.89:0/2786054397 >> 172.21.15.89:0/2786054397 conn(0x7f04d4089fd0 msgr2=0x7f04d409b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:01.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.663+0000 7f04d9a6a640 1 -- 172.21.15.89:0/2786054397 shutdown_connections 2023-10-18T10:47:01.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.663+0000 7f04d9a6a640 1 -- 172.21.15.89:0/2786054397 wait complete. 2023-10-18T10:47:01.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.667+0000 7f04d9a6a640 1 Processor -- start 2023-10-18T10:47:01.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.667+0000 7f04d9a6a640 1 -- start start 2023-10-18T10:47:01.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.667+0000 7f04d9a6a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04d40d0ca0 0x7f04d40b68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:01.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.667+0000 7f04d9a6a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f04d4155b90 0x7f04d4150050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:01.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.667+0000 7f04d9a6a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04d41508b0 0x7f04d415e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:01.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.667+0000 7f04d9a6a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d40b64c0 con 0x7f04d40d0ca0 2023-10-18T10:47:01.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.667+0000 7f04d9a6a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d4156b70 con 0x7f04d41508b0 2023-10-18T10:47:01.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.667+0000 7f04d9a6a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04d40b5e00 con 0x7f04d4155b90 2023-10-18T10:47:01.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.667+0000 7f04d37fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04d40d0ca0 0x7f04d40b68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:01.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.667+0000 7f04d37fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04d40d0ca0 0x7f04d40b68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57648/0 (socket says 172.21.15.89:57648) 2023-10-18T10:47:01.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.667+0000 7f04d37fe640 1 -- 172.21.15.89:0/1641626878 learned_addr learned my addr 172.21.15.89:0/1641626878 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:01.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.667+0000 7f04caffd640 1 --2- 172.21.15.89:0/1641626878 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f04d4155b90 0x7f04d4150050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:01.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.667+0000 7f04d3fff640 1 --2- 172.21.15.89:0/1641626878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04d41508b0 0x7f04d415e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:01.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.667+0000 7f04d37fe640 1 -- 172.21.15.89:0/1641626878 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f04d4155b90 msgr2=0x7f04d4150050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:01.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.667+0000 7f04d37fe640 1 --2- 172.21.15.89:0/1641626878 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f04d4155b90 0x7f04d4150050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:01.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.667+0000 7f04d37fe640 1 -- 172.21.15.89:0/1641626878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04d41508b0 msgr2=0x7f04d415e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:01.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.667+0000 7f04d37fe640 1 --2- 172.21.15.89:0/1641626878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04d41508b0 0x7f04d415e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:01.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.667+0000 7f04d37fe640 1 -- 172.21.15.89:0/1641626878 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04bc0094d0 con 0x7f04d40d0ca0 2023-10-18T10:47:01.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.667+0000 7f04caffd640 1 --2- 172.21.15.89:0/1641626878 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f04d4155b90 0x7f04d4150050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:47:01.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.667+0000 7f04d3fff640 1 --2- 172.21.15.89:0/1641626878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04d41508b0 0x7f04d415e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:47:01.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.667+0000 7f04d37fe640 1 --2- 172.21.15.89:0/1641626878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04d40d0ca0 0x7f04d40b68e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f04bc0190e0 tx=0x7f04bc01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:01.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.667+0000 7f04d0ff9640 1 -- 172.21.15.89:0/1641626878 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc0bd070 con 0x7f04d40d0ca0 2023-10-18T10:47:01.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.667+0000 7f04d0ff9640 1 -- 172.21.15.89:0/1641626878 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04bc0ac5e0 con 0x7f04d40d0ca0 2023-10-18T10:47:01.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.667+0000 7f04d9a6a640 1 -- 172.21.15.89:0/1641626878 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04d40b60e0 con 0x7f04d40d0ca0 2023-10-18T10:47:01.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.667+0000 7f04d0ff9640 1 -- 172.21.15.89:0/1641626878 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc0b7830 con 0x7f04d40d0ca0 2023-10-18T10:47:01.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.667+0000 7f04d9a6a640 1 -- 172.21.15.89:0/1641626878 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04d4150590 con 0x7f04d40d0ca0 2023-10-18T10:47:01.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.671+0000 7f04d9a6a640 1 -- 172.21.15.89:0/1641626878 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04d40c9570 con 0x7f04d40d0ca0 2023-10-18T10:47:01.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.671+0000 7f04d0ff9640 1 -- 172.21.15.89:0/1641626878 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f04bc0b3040 con 0x7f04d40d0ca0 2023-10-18T10:47:01.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.671+0000 7f04d0ff9640 1 --2- 172.21.15.89:0/1641626878 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f04b4065680 0x7f04b4067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:01.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.671+0000 7f04d0ff9640 1 -- 172.21.15.89:0/1641626878 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f04bc11cf70 con 0x7f04d40d0ca0 2023-10-18T10:47:01.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.671+0000 7f04caffd640 1 --2- 172.21.15.89:0/1641626878 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f04b4065680 0x7f04b4067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:01.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.671+0000 7f04caffd640 1 --2- 172.21.15.89:0/1641626878 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f04b4065680 0x7f04b4067b40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f04d415fe60 tx=0x7f04c4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:01.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.675+0000 7f04d0ff9640 1 -- 172.21.15.89:0/1641626878 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f04bc0ea790 con 0x7f04d40d0ca0 2023-10-18T10:47:01.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.855+0000 7f04d9a6a640 1 -- 172.21.15.89:0/1641626878 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04d404dbe0 con 0x7f04d40d0ca0 2023-10-18T10:47:01.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.855+0000 7f04d0ff9640 1 -- 172.21.15.89:0/1641626878 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4874 (secure 0 0 0) 0x7f04bc0eddb0 con 0x7f04d40d0ca0 2023-10-18T10:47:01.857 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:47:01.857 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/1247973498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1247973498},{"type":"v1","addr":"172.21.15.154:6836","nonce":1247973498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4284,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/516057316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":516057316},{"type":"v1","addr":"172.21.15.154:6837","nonce":516057316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4299,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6834/3957051407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3957051407},{"type":"v1","addr":"172.21.15.89:6834","nonce":3957051407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:45:55.655016+0000","modified":"2023-10-18T10:46:13.833215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4312":{"gid":4312,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6835/2906113039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6833","nonce":2906113039},{"type":"v1","addr":"172.21.15.89:6835","nonce":2906113039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T10:47:01.858 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 9 2023-10-18T10:47:01.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.855+0000 7f04d9a6a640 1 -- 172.21.15.89:0/1641626878 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f04b4065680 msgr2=0x7f04b4067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:01.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.855+0000 7f04d9a6a640 1 --2- 172.21.15.89:0/1641626878 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f04b4065680 0x7f04b4067b40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f04d415fe60 tx=0x7f04c4008040 comp rx=0 tx=0).stop 2023-10-18T10:47:01.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.855+0000 7f04d9a6a640 1 -- 172.21.15.89:0/1641626878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04d40d0ca0 msgr2=0x7f04d40b68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:01.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.855+0000 7f04d9a6a640 1 --2- 172.21.15.89:0/1641626878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04d40d0ca0 0x7f04d40b68e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f04bc0190e0 tx=0x7f04bc01ad10 comp rx=0 tx=0).stop 2023-10-18T10:47:01.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.855+0000 7f04d9a6a640 1 -- 172.21.15.89:0/1641626878 shutdown_connections 2023-10-18T10:47:01.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.855+0000 7f04d9a6a640 1 --2- 172.21.15.89:0/1641626878 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f04b4065680 0x7f04b4067b40 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:01.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.855+0000 7f04d9a6a640 1 --2- 172.21.15.89:0/1641626878 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f04d4155b90 0x7f04d4150050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:01.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.855+0000 7f04d9a6a640 1 --2- 172.21.15.89:0/1641626878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04d41508b0 0x7f04d415e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:01.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.855+0000 7f04d9a6a640 1 --2- 172.21.15.89:0/1641626878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04d40d0ca0 0x7f04d40b68e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:01.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.855+0000 7f04d9a6a640 1 -- 172.21.15.89:0/1641626878 >> 172.21.15.89:0/1641626878 conn(0x7f04d4089fd0 msgr2=0x7f04d40d2550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:01.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.855+0000 7f04d9a6a640 1 -- 172.21.15.89:0/1641626878 shutdown_connections 2023-10-18T10:47:01.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:01.855+0000 7f04d9a6a640 1 -- 172.21.15.89:0/1641626878 wait complete. 2023-10-18T10:47:01.878 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:47:02.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.011+0000 7eff7d7fa640 1 Processor -- start 2023-10-18T10:47:02.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.011+0000 7eff7d7fa640 1 -- start start 2023-10-18T10:47:02.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.015+0000 7eff7d7fa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff7815c940 0x7eff780d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:02.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.015+0000 7eff7d7fa640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7eff780d2f40 0x7eff780d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:02.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.015+0000 7eff7d7fa640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff7815da50 con 0x7eff780d0ca0 2023-10-18T10:47:02.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.015+0000 7eff7d7fa640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff780d1130 con 0x7eff780d2f40 2023-10-18T10:47:02.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.015+0000 7eff7d7fa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff7815ce40 con 0x7eff7815c940 2023-10-18T10:47:02.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.015+0000 7eff6e7fc640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7eff780d2f40 0x7eff780d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:02.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.015+0000 7eff6e7fc640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7eff780d2f40 0x7eff780d0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57650/0 (socket says 172.21.15.89:57650) 2023-10-18T10:47:02.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.015+0000 7eff6e7fc640 1 -- 172.21.15.89:0/3307486597 learned_addr learned my addr 172.21.15.89:0/3307486597 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:02.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.015+0000 7eff76ffd640 1 --2- 172.21.15.89:0/3307486597 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff7815c940 0x7eff780d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:02.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.015+0000 7eff6e7fc640 1 -- 172.21.15.89:0/3307486597 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff7815c940 msgr2=0x7eff780d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:02.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.015+0000 7eff6e7fc640 1 --2- 172.21.15.89:0/3307486597 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff7815c940 0x7eff780d2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:02.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.015+0000 7eff6e7fc640 1 -- 172.21.15.89:0/3307486597 >> v1:172.21.15.89:6789/0 conn(0x7eff780d0ca0 legacy=0x7eff780cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:02.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.015+0000 7eff6e7fc640 1 -- 172.21.15.89:0/3307486597 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff7815d120 con 0x7eff780d2f40 2023-10-18T10:47:02.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.019+0000 7eff6e7fc640 1 --2- 172.21.15.89:0/3307486597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff780d2f40 0x7eff780d0760 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7eff60009e50 tx=0x7eff6001c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c00d83e26e7f2f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:02.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.019+0000 7eff767fc640 1 -- 172.21.15.89:0/3307486597 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff600a57c0 con 0x7eff780d2f40 2023-10-18T10:47:02.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.019+0000 7eff767fc640 1 -- 172.21.15.89:0/3307486597 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff600a5920 con 0x7eff780d2f40 2023-10-18T10:47:02.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.019+0000 7eff767fc640 1 -- 172.21.15.89:0/3307486597 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff600a5cb0 con 0x7eff780d2f40 2023-10-18T10:47:02.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.019+0000 7eff7d7fa640 1 -- 172.21.15.89:0/3307486597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff780d2f40 msgr2=0x7eff780d0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:02.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.019+0000 7eff7d7fa640 1 --2- 172.21.15.89:0/3307486597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff780d2f40 0x7eff780d0760 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7eff60009e50 tx=0x7eff6001c9e0 comp rx=0 tx=0).stop 2023-10-18T10:47:02.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.019+0000 7eff7d7fa640 1 -- 172.21.15.89:0/3307486597 shutdown_connections 2023-10-18T10:47:02.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.019+0000 7eff7d7fa640 1 --2- 172.21.15.89:0/3307486597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff780d2f40 0x7eff780d0760 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:02.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.019+0000 7eff7d7fa640 1 --2- 172.21.15.89:0/3307486597 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff7815c940 0x7eff780d2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:02.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.019+0000 7eff7d7fa640 1 -- 172.21.15.89:0/3307486597 >> 172.21.15.89:0/3307486597 conn(0x7eff78089fd0 msgr2=0x7eff7809b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:02.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.019+0000 7eff7d7fa640 1 -- 172.21.15.89:0/3307486597 shutdown_connections 2023-10-18T10:47:02.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.019+0000 7eff7d7fa640 1 -- 172.21.15.89:0/3307486597 wait complete. 2023-10-18T10:47:02.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.019+0000 7eff7d7fa640 1 Processor -- start 2023-10-18T10:47:02.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.019+0000 7eff7d7fa640 1 -- start start 2023-10-18T10:47:02.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.023+0000 7eff7d7fa640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff780d0ca0 0x7eff78155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:02.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.023+0000 7eff7d7fa640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff7814ee40 0x7eff780b7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:02.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.023+0000 7eff7d7fa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff7814f2c0 0x7eff7815e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:02.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.023+0000 7eff7d7fa640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff780b80a0 con 0x7eff780d0ca0 2023-10-18T10:47:02.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.023+0000 7eff7d7fa640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff78154b30 con 0x7eff7814ee40 2023-10-18T10:47:02.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.023+0000 7eff7d7fa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff78155a80 con 0x7eff7814f2c0 2023-10-18T10:47:02.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.023+0000 7eff76ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff780d0ca0 0x7eff78155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:02.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.023+0000 7eff76ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff780d0ca0 0x7eff78155540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57654/0 (socket says 172.21.15.89:57654) 2023-10-18T10:47:02.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.023+0000 7eff76ffd640 1 -- 172.21.15.89:0/1534753218 learned_addr learned my addr 172.21.15.89:0/1534753218 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:02.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.023+0000 7eff777fe640 1 --2- 172.21.15.89:0/1534753218 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff7814f2c0 0x7eff7815e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:02.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.023+0000 7eff6e7fc640 1 --2- 172.21.15.89:0/1534753218 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff7814ee40 0x7eff780b7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:02.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.023+0000 7eff76ffd640 1 -- 172.21.15.89:0/1534753218 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff7814f2c0 msgr2=0x7eff7815e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:02.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.023+0000 7eff76ffd640 1 --2- 172.21.15.89:0/1534753218 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff7814f2c0 0x7eff7815e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:02.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.023+0000 7eff76ffd640 1 -- 172.21.15.89:0/1534753218 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff7814ee40 msgr2=0x7eff780b7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:02.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.023+0000 7eff76ffd640 1 --2- 172.21.15.89:0/1534753218 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff7814ee40 0x7eff780b7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:02.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.023+0000 7eff76ffd640 1 -- 172.21.15.89:0/1534753218 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff60009b30 con 0x7eff780d0ca0 2023-10-18T10:47:02.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.023+0000 7eff76ffd640 1 --2- 172.21.15.89:0/1534753218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff780d0ca0 0x7eff78155540 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7eff6800e870 tx=0x7eff6800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:02.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.023+0000 7eff6ffff640 1 -- 172.21.15.89:0/1534753218 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff68009510 con 0x7eff780d0ca0 2023-10-18T10:47:02.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.023+0000 7eff6ffff640 1 -- 172.21.15.89:0/1534753218 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff6800eec0 con 0x7eff780d0ca0 2023-10-18T10:47:02.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.023+0000 7eff7d7fa640 1 -- 172.21.15.89:0/1534753218 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff780b7ae0 con 0x7eff780d0ca0 2023-10-18T10:47:02.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.023+0000 7eff6ffff640 1 -- 172.21.15.89:0/1534753218 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6800cb40 con 0x7eff780d0ca0 2023-10-18T10:47:02.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.023+0000 7eff7d7fa640 1 -- 172.21.15.89:0/1534753218 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff78154d40 con 0x7eff780d0ca0 2023-10-18T10:47:02.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.023+0000 7eff7d7fa640 1 -- 172.21.15.89:0/1534753218 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff7804dbe0 con 0x7eff780d0ca0 2023-10-18T10:47:02.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.023+0000 7eff6ffff640 1 -- 172.21.15.89:0/1534753218 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7eff6800cca0 con 0x7eff780d0ca0 2023-10-18T10:47:02.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.027+0000 7eff6ffff640 1 --2- 172.21.15.89:0/1534753218 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7eff58065680 0x7eff58067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:02.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.027+0000 7eff6ffff640 1 -- 172.21.15.89:0/1534753218 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7eff68083260 con 0x7eff780d0ca0 2023-10-18T10:47:02.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.027+0000 7eff6e7fc640 1 --2- 172.21.15.89:0/1534753218 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7eff58065680 0x7eff58067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:02.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.027+0000 7eff6e7fc640 1 --2- 172.21.15.89:0/1534753218 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7eff58065680 0x7eff58067b40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7eff600ae000 tx=0x7eff600a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:02.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.031+0000 7eff6ffff640 1 -- 172.21.15.89:0/1534753218 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7eff680509e0 con 0x7eff780d0ca0 2023-10-18T10:47:02.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.179+0000 7eff7d7fa640 1 -- 172.21.15.89:0/1534753218 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff780b85d0 con 0x7eff780d0ca0 2023-10-18T10:47:02.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.179+0000 7eff6ffff640 1 -- 172.21.15.89:0/1534753218 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20192 (secure 0 0 0) 0x7eff68054000 con 0x7eff780d0ca0 2023-10-18T10:47:02.182 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:47:02.183 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":29,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:46:36.958604+0000","last_up_change":"2023-10-18T10:45:42.164688+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:45:53.309122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-18T10:45:54.659763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:45:57.796514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6824","nonce":773078364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6827","nonce":773078364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6831","nonce":773078364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6829","nonce":773078364}]},"public_addr":"172.21.15.154:6824/773078364","cluster_addr":"172.21.15.154:6827/773078364","heartbeat_back_addr":"172.21.15.154:6831/773078364","heartbeat_front_addr":"172.21.15.154:6829/773078364","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":1,"to":5}]},{"pgid":"4.1a","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:47:02.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.183+0000 7eff7d7fa640 1 -- 172.21.15.89:0/1534753218 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7eff58065680 msgr2=0x7eff58067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:02.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.183+0000 7eff7d7fa640 1 --2- 172.21.15.89:0/1534753218 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7eff58065680 0x7eff58067b40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7eff600ae000 tx=0x7eff600a7070 comp rx=0 tx=0).stop 2023-10-18T10:47:02.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.183+0000 7eff7d7fa640 1 -- 172.21.15.89:0/1534753218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff780d0ca0 msgr2=0x7eff78155540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:02.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.183+0000 7eff7d7fa640 1 --2- 172.21.15.89:0/1534753218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff780d0ca0 0x7eff78155540 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7eff6800e870 tx=0x7eff6800ec70 comp rx=0 tx=0).stop 2023-10-18T10:47:02.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.183+0000 7eff7d7fa640 1 -- 172.21.15.89:0/1534753218 shutdown_connections 2023-10-18T10:47:02.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.183+0000 7eff7d7fa640 1 --2- 172.21.15.89:0/1534753218 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7eff58065680 0x7eff58067b40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:02.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.183+0000 7eff7d7fa640 1 --2- 172.21.15.89:0/1534753218 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff7814f2c0 0x7eff7815e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:02.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.183+0000 7eff7d7fa640 1 --2- 172.21.15.89:0/1534753218 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff7814ee40 0x7eff780b7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:02.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.183+0000 7eff7d7fa640 1 --2- 172.21.15.89:0/1534753218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff780d0ca0 0x7eff78155540 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:02.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.183+0000 7eff7d7fa640 1 -- 172.21.15.89:0/1534753218 >> 172.21.15.89:0/1534753218 conn(0x7eff78089fd0 msgr2=0x7eff7815dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:02.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.183+0000 7eff7d7fa640 1 -- 172.21.15.89:0/1534753218 shutdown_connections 2023-10-18T10:47:02.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.183+0000 7eff7d7fa640 1 -- 172.21.15.89:0/1534753218 wait complete. 2023-10-18T10:47:02.204 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-18T10:47:02.204 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:47:02.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.339+0000 7f07ea9db640 1 Processor -- start 2023-10-18T10:47:02.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.339+0000 7f07ea9db640 1 -- start start 2023-10-18T10:47:02.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.343+0000 7f07ea9db640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f07e40cd4f0 0x7f07e4143470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:02.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.343+0000 7f07ea9db640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07e40ce070 0x7f07e40ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:02.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.343+0000 7f07ea9db640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e40cd8d0 con 0x7f07e4145b70 2023-10-18T10:47:02.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.343+0000 7f07ea9db640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e4069900 con 0x7f07e40cd4f0 2023-10-18T10:47:02.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.343+0000 7f07ea9db640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e40cea20 con 0x7f07e40ce070 2023-10-18T10:47:02.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.343+0000 7f07e3fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f07e40cd4f0 0x7f07e4143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:02.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.343+0000 7f07e3fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f07e40cd4f0 0x7f07e4143470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59742/0 (socket says 172.21.15.89:59742) 2023-10-18T10:47:02.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.343+0000 7f07e3fff640 1 -- 172.21.15.89:0/1974098958 learned_addr learned my addr 172.21.15.89:0/1974098958 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:02.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.343+0000 7f07e8f78640 1 --2- 172.21.15.89:0/1974098958 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07e40ce070 0x7f07e40ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:02.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.343+0000 7f07e3fff640 1 -- 172.21.15.89:0/1974098958 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07e40ce070 msgr2=0x7f07e40ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:02.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.343+0000 7f07e3fff640 1 --2- 172.21.15.89:0/1974098958 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07e40ce070 0x7f07e40ce4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:02.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.343+0000 7f07e3fff640 1 -- 172.21.15.89:0/1974098958 >> v1:172.21.15.154:6789/0 conn(0x7f07e4145b70 legacy=0x7f07e4160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:02.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.343+0000 7f07e3fff640 1 -- 172.21.15.89:0/1974098958 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e415cbf0 con 0x7f07e40cd4f0 2023-10-18T10:47:02.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.347+0000 7f07e3fff640 1 --2- 172.21.15.89:0/1974098958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07e40cd4f0 0x7f07e4143470 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f07e4069a40 tx=0x7f07d401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ca06fe95faffa4 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:02.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.347+0000 7f07e2ffd640 1 -- 172.21.15.89:0/1974098958 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d40aa650 con 0x7f07e40cd4f0 2023-10-18T10:47:02.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.347+0000 7f07e2ffd640 1 -- 172.21.15.89:0/1974098958 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d40aa7b0 con 0x7f07e40cd4f0 2023-10-18T10:47:02.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.347+0000 7f07e2ffd640 1 -- 172.21.15.89:0/1974098958 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d40aab40 con 0x7f07e40cd4f0 2023-10-18T10:47:02.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.347+0000 7f07ea9db640 1 -- 172.21.15.89:0/1974098958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07e40cd4f0 msgr2=0x7f07e4143470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:02.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.347+0000 7f07ea9db640 1 --2- 172.21.15.89:0/1974098958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07e40cd4f0 0x7f07e4143470 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f07e4069a40 tx=0x7f07d401c4f0 comp rx=0 tx=0).stop 2023-10-18T10:47:02.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.347+0000 7f07ea9db640 1 -- 172.21.15.89:0/1974098958 shutdown_connections 2023-10-18T10:47:02.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.347+0000 7f07ea9db640 1 --2- 172.21.15.89:0/1974098958 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07e40ce070 0x7f07e40ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:02.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.347+0000 7f07ea9db640 1 --2- 172.21.15.89:0/1974098958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07e40cd4f0 0x7f07e4143470 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:02.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.347+0000 7f07ea9db640 1 -- 172.21.15.89:0/1974098958 >> 172.21.15.89:0/1974098958 conn(0x7f07e406dad0 msgr2=0x7f07e406dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:02.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.347+0000 7f07ea9db640 1 -- 172.21.15.89:0/1974098958 shutdown_connections 2023-10-18T10:47:02.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.347+0000 7f07ea9db640 1 -- 172.21.15.89:0/1974098958 wait complete. 2023-10-18T10:47:02.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.347+0000 7f07ea9db640 1 Processor -- start 2023-10-18T10:47:02.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.347+0000 7f07ea9db640 1 -- start start 2023-10-18T10:47:02.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.351+0000 7f07ea9db640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07e40cd4f0 0x7f07e41446a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:02.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.351+0000 7f07ea9db640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07e40ce070 0x7f07e4144be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:02.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.351+0000 7f07ea9db640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07e4145b70 0x7f07e40c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:02.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.351+0000 7f07ea9db640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e40b0b80 con 0x7f07e40ce070 2023-10-18T10:47:02.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.351+0000 7f07ea9db640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e40c91a0 con 0x7f07e40cd4f0 2023-10-18T10:47:02.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.351+0000 7f07ea9db640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e40c9480 con 0x7f07e4145b70 2023-10-18T10:47:02.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.351+0000 7f07e37fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07e40ce070 0x7f07e4144be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:02.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.351+0000 7f07e37fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07e40ce070 0x7f07e4144be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59746/0 (socket says 172.21.15.89:59746) 2023-10-18T10:47:02.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.351+0000 7f07e37fe640 1 -- 172.21.15.89:0/4112929646 learned_addr learned my addr 172.21.15.89:0/4112929646 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:02.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.351+0000 7f07e8f78640 1 --2- 172.21.15.89:0/4112929646 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07e4145b70 0x7f07e40c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:02.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.351+0000 7f07e3fff640 1 --2- 172.21.15.89:0/4112929646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07e40cd4f0 0x7f07e41446a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:02.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.351+0000 7f07e37fe640 1 -- 172.21.15.89:0/4112929646 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07e4145b70 msgr2=0x7f07e40c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:02.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.351+0000 7f07e37fe640 1 --2- 172.21.15.89:0/4112929646 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07e4145b70 0x7f07e40c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:02.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.351+0000 7f07e37fe640 1 -- 172.21.15.89:0/4112929646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07e40cd4f0 msgr2=0x7f07e41446a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:02.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.351+0000 7f07e37fe640 1 --2- 172.21.15.89:0/4112929646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07e40cd4f0 0x7f07e41446a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:02.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.351+0000 7f07e37fe640 1 -- 172.21.15.89:0/4112929646 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d40094d0 con 0x7f07e40ce070 2023-10-18T10:47:02.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.351+0000 7f07e3fff640 1 --2- 172.21.15.89:0/4112929646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07e40cd4f0 0x7f07e41446a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:47:02.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.351+0000 7f07e37fe640 1 --2- 172.21.15.89:0/4112929646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07e40ce070 0x7f07e4144be0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f07d800c500 tx=0x7f07d800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:02.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.351+0000 7f07e0ff9640 1 -- 172.21.15.89:0/4112929646 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d8005220 con 0x7f07e40ce070 2023-10-18T10:47:02.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.351+0000 7f07e0ff9640 1 -- 172.21.15.89:0/4112929646 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d8005380 con 0x7f07e40ce070 2023-10-18T10:47:02.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.351+0000 7f07ea9db640 1 -- 172.21.15.89:0/4112929646 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e40c9760 con 0x7f07e40ce070 2023-10-18T10:47:02.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.351+0000 7f07e0ff9640 1 -- 172.21.15.89:0/4112929646 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d8007270 con 0x7f07e40ce070 2023-10-18T10:47:02.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.351+0000 7f07ea9db640 1 -- 172.21.15.89:0/4112929646 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e40c8de0 con 0x7f07e40ce070 2023-10-18T10:47:02.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.351+0000 7f07ea9db640 1 -- 172.21.15.89:0/4112929646 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e40b0d30 con 0x7f07e40ce070 2023-10-18T10:47:02.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.351+0000 7f07e0ff9640 1 -- 172.21.15.89:0/4112929646 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f07d80073d0 con 0x7f07e40ce070 2023-10-18T10:47:02.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.355+0000 7f07e0ff9640 1 --2- 172.21.15.89:0/4112929646 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f07d0065680 0x7f07d0067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:02.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.355+0000 7f07e0ff9640 1 -- 172.21.15.89:0/4112929646 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f07d8082aa0 con 0x7f07e40ce070 2023-10-18T10:47:02.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.355+0000 7f07e3fff640 1 --2- 172.21.15.89:0/4112929646 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f07d0065680 0x7f07d0067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:02.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.355+0000 7f07e3fff640 1 --2- 172.21.15.89:0/4112929646 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f07d0065680 0x7f07d0067b40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f07d40ae000 tx=0x7f07d40a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:02.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.359+0000 7f07e0ff9640 1 -- 172.21.15.89:0/4112929646 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f07d80501a0 con 0x7f07e40ce070 2023-10-18T10:47:02.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.539+0000 7f07ea9db640 1 -- 172.21.15.89:0/4112929646 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07e40b0970 con 0x7f07e40ce070 2023-10-18T10:47:02.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.539+0000 7f07e0ff9640 1 -- 172.21.15.89:0/4112929646 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4874 (secure 0 0 0) 0x7f07d80537c0 con 0x7f07e40ce070 2023-10-18T10:47:02.542 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:47:02.543 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/1247973498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1247973498},{"type":"v1","addr":"172.21.15.154:6836","nonce":1247973498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4284,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/516057316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":516057316},{"type":"v1","addr":"172.21.15.154:6837","nonce":516057316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4299,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6834/3957051407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3957051407},{"type":"v1","addr":"172.21.15.89:6834","nonce":3957051407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:45:55.655016+0000","modified":"2023-10-18T10:46:13.833215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4312":{"gid":4312,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6835/2906113039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6833","nonce":2906113039},{"type":"v1","addr":"172.21.15.89:6835","nonce":2906113039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T10:47:02.544 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 9 2023-10-18T10:47:02.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.543+0000 7f07ea9db640 1 -- 172.21.15.89:0/4112929646 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f07d0065680 msgr2=0x7f07d0067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:02.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.543+0000 7f07ea9db640 1 --2- 172.21.15.89:0/4112929646 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f07d0065680 0x7f07d0067b40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f07d40ae000 tx=0x7f07d40a7000 comp rx=0 tx=0).stop 2023-10-18T10:47:02.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.543+0000 7f07ea9db640 1 -- 172.21.15.89:0/4112929646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07e40ce070 msgr2=0x7f07e4144be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:02.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.543+0000 7f07ea9db640 1 --2- 172.21.15.89:0/4112929646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07e40ce070 0x7f07e4144be0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f07d800c500 tx=0x7f07d800c9c0 comp rx=0 tx=0).stop 2023-10-18T10:47:02.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.543+0000 7f07ea9db640 1 -- 172.21.15.89:0/4112929646 shutdown_connections 2023-10-18T10:47:02.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.543+0000 7f07ea9db640 1 --2- 172.21.15.89:0/4112929646 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f07d0065680 0x7f07d0067b40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:02.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.543+0000 7f07ea9db640 1 --2- 172.21.15.89:0/4112929646 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07e4145b70 0x7f07e40c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:02.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.543+0000 7f07ea9db640 1 --2- 172.21.15.89:0/4112929646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07e40ce070 0x7f07e4144be0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:02.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.543+0000 7f07ea9db640 1 --2- 172.21.15.89:0/4112929646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07e40cd4f0 0x7f07e41446a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:02.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.543+0000 7f07ea9db640 1 -- 172.21.15.89:0/4112929646 >> 172.21.15.89:0/4112929646 conn(0x7f07e406dad0 msgr2=0x7f07e4154510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:02.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.543+0000 7f07ea9db640 1 -- 172.21.15.89:0/4112929646 shutdown_connections 2023-10-18T10:47:02.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.543+0000 7f07ea9db640 1 -- 172.21.15.89:0/4112929646 wait complete. 2023-10-18T10:47:02.563 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:47:02.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.699+0000 7fd0574cb640 1 Processor -- start 2023-10-18T10:47:02.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.699+0000 7fd0574cb640 1 -- start start 2023-10-18T10:47:02.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.699+0000 7fd0574cb640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd0500d0bc0 0x7fd0500cd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:02.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.699+0000 7fd0574cb640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd05015da50 con 0x7fd05015c940 2023-10-18T10:47:02.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.699+0000 7fd0574cb640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0500d3110 con 0x7fd0500d0760 2023-10-18T10:47:02.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.699+0000 7fd0574cb640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd05015ce40 con 0x7fd0500d0bc0 2023-10-18T10:47:02.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.699+0000 7fd055267640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd05015c940 0x7fd0500d2a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37418/0 (socket says 172.21.15.89:37418) 2023-10-18T10:47:02.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.699+0000 7fd055267640 1 -- 172.21.15.89:0/2918415904 learned_addr learned my addr 172.21.15.89:0/2918415904 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:47:02.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.699+0000 7fd055a68640 1 --2- 172.21.15.89:0/2918415904 >> v2:172.21.15.89:3300/0 conn(0x7fd0500d0bc0 0x7fd0500cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:02.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.699+0000 7fd055a68640 1 -- 172.21.15.89:0/2918415904 >> v1:172.21.15.154:6789/0 conn(0x7fd0500d0760 legacy=0x7fd0500cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:02.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.699+0000 7fd055a68640 1 -- 172.21.15.89:0/2918415904 >> v1:172.21.15.89:6789/0 conn(0x7fd05015c940 legacy=0x7fd0500d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:02.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.699+0000 7fd055a68640 1 -- 172.21.15.89:0/2918415904 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd05015d120 con 0x7fd0500d0bc0 2023-10-18T10:47:02.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.703+0000 7fd055a68640 1 --2- 172.21.15.89:0/2918415904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0500d0bc0 0x7fd0500cd870 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd048009d90 tx=0x7fd04801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96781ed3f681cb2b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:02.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.703+0000 7fd047fff640 1 -- 172.21.15.89:0/2918415904 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0480a59a0 con 0x7fd0500d0bc0 2023-10-18T10:47:02.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.703+0000 7fd047fff640 1 -- 172.21.15.89:0/2918415904 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0500ce5c0 con 0x7fd0500d0bc0 2023-10-18T10:47:02.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.703+0000 7fd047fff640 1 -- 172.21.15.89:0/2918415904 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0480a5d20 con 0x7fd0500d0bc0 2023-10-18T10:47:02.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.703+0000 7fd0574cb640 1 -- 172.21.15.89:0/2918415904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0500d0bc0 msgr2=0x7fd0500cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:02.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.703+0000 7fd0574cb640 1 --2- 172.21.15.89:0/2918415904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0500d0bc0 0x7fd0500cd870 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd048009d90 tx=0x7fd04801c8e0 comp rx=0 tx=0).stop 2023-10-18T10:47:02.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.703+0000 7fd0574cb640 1 -- 172.21.15.89:0/2918415904 shutdown_connections 2023-10-18T10:47:02.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.703+0000 7fd0574cb640 1 --2- 172.21.15.89:0/2918415904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0500d0bc0 0x7fd0500cd870 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:02.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.703+0000 7fd0574cb640 1 -- 172.21.15.89:0/2918415904 >> 172.21.15.89:0/2918415904 conn(0x7fd050089fd0 msgr2=0x7fd05009b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:02.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.703+0000 7fd0574cb640 1 -- 172.21.15.89:0/2918415904 shutdown_connections 2023-10-18T10:47:02.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.703+0000 7fd0574cb640 1 -- 172.21.15.89:0/2918415904 wait complete. 2023-10-18T10:47:02.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.707+0000 7fd0574cb640 1 Processor -- start 2023-10-18T10:47:02.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.707+0000 7fd0574cb640 1 -- start start 2023-10-18T10:47:02.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.707+0000 7fd0574cb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0500d0760 0x7fd050150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:02.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.707+0000 7fd0574cb640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0500d0bc0 0x7fd0500ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:02.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.707+0000 7fd0574cb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0500cac00 0x7fd0500b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:02.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.707+0000 7fd0574cb640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0500b7b50 con 0x7fd0500d0bc0 2023-10-18T10:47:02.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.707+0000 7fd0574cb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0500b5d40 con 0x7fd0500d0760 2023-10-18T10:47:02.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.707+0000 7fd0574cb640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0500c89c0 con 0x7fd0500cac00 2023-10-18T10:47:02.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.707+0000 7fd054a66640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0500d0bc0 0x7fd0500ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:02.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.707+0000 7fd054a66640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0500d0bc0 0x7fd0500ca6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59774/0 (socket says 172.21.15.89:59774) 2023-10-18T10:47:02.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.707+0000 7fd054a66640 1 -- 172.21.15.89:0/2950138506 learned_addr learned my addr 172.21.15.89:0/2950138506 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:02.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.707+0000 7fd055267640 1 --2- 172.21.15.89:0/2950138506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0500d0760 0x7fd050150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:02.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.707+0000 7fd055a68640 1 --2- 172.21.15.89:0/2950138506 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0500cac00 0x7fd0500b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:02.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.707+0000 7fd054a66640 1 -- 172.21.15.89:0/2950138506 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0500cac00 msgr2=0x7fd0500b61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:02.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.707+0000 7fd054a66640 1 --2- 172.21.15.89:0/2950138506 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0500cac00 0x7fd0500b61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:02.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.707+0000 7fd054a66640 1 -- 172.21.15.89:0/2950138506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0500d0760 msgr2=0x7fd050150330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:02.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.707+0000 7fd054a66640 1 --2- 172.21.15.89:0/2950138506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0500d0760 0x7fd050150330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:02.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.707+0000 7fd054a66640 1 -- 172.21.15.89:0/2950138506 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd048009a70 con 0x7fd0500d0bc0 2023-10-18T10:47:02.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.707+0000 7fd054a66640 1 --2- 172.21.15.89:0/2950138506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0500d0bc0 0x7fd0500ca6c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd04000c490 tx=0x7fd040001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:02.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.707+0000 7fd045ffb640 1 -- 172.21.15.89:0/2950138506 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd040003600 con 0x7fd0500d0bc0 2023-10-18T10:47:02.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.707+0000 7fd045ffb640 1 -- 172.21.15.89:0/2950138506 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd040003780 con 0x7fd0500d0bc0 2023-10-18T10:47:02.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.707+0000 7fd045ffb640 1 -- 172.21.15.89:0/2950138506 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd040003b70 con 0x7fd0500d0bc0 2023-10-18T10:47:02.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.707+0000 7fd0574cb640 1 -- 172.21.15.89:0/2950138506 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0500b7da0 con 0x7fd0500d0bc0 2023-10-18T10:47:02.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.707+0000 7fd0574cb640 1 -- 172.21.15.89:0/2950138506 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd050150ae0 con 0x7fd0500d0bc0 2023-10-18T10:47:02.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.711+0000 7fd0574cb640 1 -- 172.21.15.89:0/2950138506 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0500b5eb0 con 0x7fd0500d0bc0 2023-10-18T10:47:02.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.711+0000 7fd045ffb640 1 -- 172.21.15.89:0/2950138506 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fd040003cd0 con 0x7fd0500d0bc0 2023-10-18T10:47:02.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.711+0000 7fd045ffb640 1 --2- 172.21.15.89:0/2950138506 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd0340656b0 0x7fd034067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:02.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.711+0000 7fd045ffb640 1 -- 172.21.15.89:0/2950138506 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd040082210 con 0x7fd0500d0bc0 2023-10-18T10:47:02.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.711+0000 7fd055267640 1 --2- 172.21.15.89:0/2950138506 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd0340656b0 0x7fd034067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:02.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.711+0000 7fd055267640 1 --2- 172.21.15.89:0/2950138506 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd0340656b0 0x7fd034067b70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd04c005890 tx=0x7fd04c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:02.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.715+0000 7fd045ffb640 1 -- 172.21.15.89:0/2950138506 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fd04004fa40 con 0x7fd0500d0bc0 2023-10-18T10:47:02.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.895+0000 7fd0574cb640 1 -- 172.21.15.89:0/2950138506 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd05004dbe0 con 0x7fd0500d0bc0 2023-10-18T10:47:02.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.895+0000 7fd045ffb640 1 -- 172.21.15.89:0/2950138506 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4874 (secure 0 0 0) 0x7fd040053060 con 0x7fd0500d0bc0 2023-10-18T10:47:02.899 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:47:02.899 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/1247973498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1247973498},{"type":"v1","addr":"172.21.15.154:6836","nonce":1247973498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4284,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/516057316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":516057316},{"type":"v1","addr":"172.21.15.154:6837","nonce":516057316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4299,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6834/3957051407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3957051407},{"type":"v1","addr":"172.21.15.89:6834","nonce":3957051407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:45:55.655016+0000","modified":"2023-10-18T10:46:13.833215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4312":{"gid":4312,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6835/2906113039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6833","nonce":2906113039},{"type":"v1","addr":"172.21.15.89:6835","nonce":2906113039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T10:47:02.899 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 9 2023-10-18T10:47:02.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.899+0000 7fd0574cb640 1 -- 172.21.15.89:0/2950138506 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd0340656b0 msgr2=0x7fd034067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:02.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.899+0000 7fd0574cb640 1 --2- 172.21.15.89:0/2950138506 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd0340656b0 0x7fd034067b70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd04c005890 tx=0x7fd04c009040 comp rx=0 tx=0).stop 2023-10-18T10:47:02.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.899+0000 7fd0574cb640 1 -- 172.21.15.89:0/2950138506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0500d0bc0 msgr2=0x7fd0500ca6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:02.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.899+0000 7fd0574cb640 1 --2- 172.21.15.89:0/2950138506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0500d0bc0 0x7fd0500ca6c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd04000c490 tx=0x7fd040001e80 comp rx=0 tx=0).stop 2023-10-18T10:47:02.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.899+0000 7fd0574cb640 1 -- 172.21.15.89:0/2950138506 shutdown_connections 2023-10-18T10:47:02.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.899+0000 7fd0574cb640 1 --2- 172.21.15.89:0/2950138506 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd0340656b0 0x7fd034067b70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:02.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.899+0000 7fd0574cb640 1 --2- 172.21.15.89:0/2950138506 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0500cac00 0x7fd0500b61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:02.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.899+0000 7fd0574cb640 1 --2- 172.21.15.89:0/2950138506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0500d0bc0 0x7fd0500ca6c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:02.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.899+0000 7fd0574cb640 1 --2- 172.21.15.89:0/2950138506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0500d0760 0x7fd050150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:02.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.899+0000 7fd0574cb640 1 -- 172.21.15.89:0/2950138506 >> 172.21.15.89:0/2950138506 conn(0x7fd050089fd0 msgr2=0x7fd050158650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:02.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.899+0000 7fd0574cb640 1 -- 172.21.15.89:0/2950138506 shutdown_connections 2023-10-18T10:47:02.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:02.899+0000 7fd0574cb640 1 -- 172.21.15.89:0/2950138506 wait complete. 2023-10-18T10:47:02.920 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:47:03.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.055+0000 7fa60ab40640 1 Processor -- start 2023-10-18T10:47:03.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.055+0000 7fa60ab40640 1 -- start start 2023-10-18T10:47:03.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.055+0000 7fa60ab40640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa60406ff00 0x7fa60415e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:03.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.055+0000 7fa60ab40640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa6040ca340 0x7fa60415b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:03.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.055+0000 7fa60ab40640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa60415acb0 con 0x7fa60415b680 2023-10-18T10:47:03.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.055+0000 7fa60ab40640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa60415ef30 con 0x7fa6040ca340 2023-10-18T10:47:03.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.055+0000 7fa60ab40640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa60415aa10 con 0x7fa60406ff00 2023-10-18T10:47:03.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.055+0000 7fa5fbfff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa6040ca340 0x7fa60415b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:03.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.059+0000 7fa5fbfff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa6040ca340 0x7fa60415b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59788/0 (socket says 172.21.15.89:59788) 2023-10-18T10:47:03.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.059+0000 7fa5fbfff640 1 -- 172.21.15.89:0/3440969869 learned_addr learned my addr 172.21.15.89:0/3440969869 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:03.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.059+0000 7fa6088dc640 1 --2- 172.21.15.89:0/3440969869 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa60406ff00 0x7fa60415e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:03.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.059+0000 7fa5fbfff640 1 -- 172.21.15.89:0/3440969869 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa60406ff00 msgr2=0x7fa60415e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:03.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.059+0000 7fa5fbfff640 1 --2- 172.21.15.89:0/3440969869 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa60406ff00 0x7fa60415e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:03.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.059+0000 7fa5fbfff640 1 -- 172.21.15.89:0/3440969869 >> v1:172.21.15.154:6789/0 conn(0x7fa60415b680 legacy=0x7fa6040c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:03.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.059+0000 7fa5fbfff640 1 -- 172.21.15.89:0/3440969869 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6040702e0 con 0x7fa6040ca340 2023-10-18T10:47:03.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.059+0000 7fa5fbfff640 1 --2- 172.21.15.89:0/3440969869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6040ca340 0x7fa60415b140 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4009870 tx=0x7fa5f401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e43b65fb5e5ea7cc server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:03.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.059+0000 7fa5fb7fe640 1 -- 172.21.15.89:0/3440969869 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f40ab600 con 0x7fa6040ca340 2023-10-18T10:47:03.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.059+0000 7fa5fb7fe640 1 -- 172.21.15.89:0/3440969869 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f40ab760 con 0x7fa6040ca340 2023-10-18T10:47:03.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.059+0000 7fa5fb7fe640 1 -- 172.21.15.89:0/3440969869 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f40abaf0 con 0x7fa6040ca340 2023-10-18T10:47:03.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.063+0000 7fa60ab40640 1 -- 172.21.15.89:0/3440969869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6040ca340 msgr2=0x7fa60415b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:03.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.063+0000 7fa60ab40640 1 --2- 172.21.15.89:0/3440969869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6040ca340 0x7fa60415b140 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4009870 tx=0x7fa5f401c440 comp rx=0 tx=0).stop 2023-10-18T10:47:03.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.063+0000 7fa60ab40640 1 -- 172.21.15.89:0/3440969869 shutdown_connections 2023-10-18T10:47:03.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.063+0000 7fa60ab40640 1 --2- 172.21.15.89:0/3440969869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6040ca340 0x7fa60415b140 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:03.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.063+0000 7fa60ab40640 1 --2- 172.21.15.89:0/3440969869 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa60406ff00 0x7fa60415e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:03.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.063+0000 7fa60ab40640 1 -- 172.21.15.89:0/3440969869 >> 172.21.15.89:0/3440969869 conn(0x7fa604069f00 msgr2=0x7fa6040cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:03.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.063+0000 7fa60ab40640 1 -- 172.21.15.89:0/3440969869 shutdown_connections 2023-10-18T10:47:03.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.063+0000 7fa60ab40640 1 -- 172.21.15.89:0/3440969869 wait complete. 2023-10-18T10:47:03.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.063+0000 7fa60ab40640 1 Processor -- start 2023-10-18T10:47:03.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.063+0000 7fa60ab40640 1 -- start start 2023-10-18T10:47:03.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.063+0000 7fa60ab40640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa60406ff00 0x7fa60406ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:03.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.063+0000 7fa60ab40640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6040ca340 0x7fa60414d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:03.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.063+0000 7fa60ab40640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa60414d8b0 0x7fa6040b2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:03.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.063+0000 7fa60ab40640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60414dd60 con 0x7fa60414d8b0 2023-10-18T10:47:03.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.063+0000 7fa60ab40640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6041525e0 con 0x7fa6040ca340 2023-10-18T10:47:03.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.063+0000 7fa60ab40640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa604152e80 con 0x7fa60406ff00 2023-10-18T10:47:03.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.063+0000 7fa6090dd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa60414d8b0 0x7fa6040b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:03.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.063+0000 7fa6088dc640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa60406ff00 0x7fa60406ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:03.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.063+0000 7fa6090dd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa60414d8b0 0x7fa6040b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59796/0 (socket says 172.21.15.89:59796) 2023-10-18T10:47:03.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.063+0000 7fa6090dd640 1 -- 172.21.15.89:0/805789175 learned_addr learned my addr 172.21.15.89:0/805789175 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:03.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.063+0000 7fa5fbfff640 1 --2- 172.21.15.89:0/805789175 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6040ca340 0x7fa60414d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:03.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.067+0000 7fa6090dd640 1 -- 172.21.15.89:0/805789175 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa60406ff00 msgr2=0x7fa60406ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:03.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.067+0000 7fa6090dd640 1 --2- 172.21.15.89:0/805789175 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa60406ff00 0x7fa60406ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:03.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.067+0000 7fa6090dd640 1 -- 172.21.15.89:0/805789175 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6040ca340 msgr2=0x7fa60414d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:03.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.067+0000 7fa6090dd640 1 --2- 172.21.15.89:0/805789175 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6040ca340 0x7fa60414d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:03.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.067+0000 7fa6090dd640 1 -- 172.21.15.89:0/805789175 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f40094d0 con 0x7fa60414d8b0 2023-10-18T10:47:03.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.067+0000 7fa6090dd640 1 --2- 172.21.15.89:0/805789175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa60414d8b0 0x7fa6040b2270 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00c7b0 tx=0x7fa5fc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:03.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.067+0000 7fa5f97fa640 1 -- 172.21.15.89:0/805789175 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc00aa50 con 0x7fa60414d8b0 2023-10-18T10:47:03.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.067+0000 7fa5f97fa640 1 -- 172.21.15.89:0/805789175 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5fc00abb0 con 0x7fa60414d8b0 2023-10-18T10:47:03.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.067+0000 7fa5f97fa640 1 -- 172.21.15.89:0/805789175 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc00ce60 con 0x7fa60414d8b0 2023-10-18T10:47:03.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.067+0000 7fa60ab40640 1 -- 172.21.15.89:0/805789175 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6040b2fa0 con 0x7fa60414d8b0 2023-10-18T10:47:03.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.067+0000 7fa60ab40640 1 -- 172.21.15.89:0/805789175 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa604152b10 con 0x7fa60414d8b0 2023-10-18T10:47:03.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.067+0000 7fa60ab40640 1 -- 172.21.15.89:0/805789175 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa60404dbe0 con 0x7fa60414d8b0 2023-10-18T10:47:03.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.067+0000 7fa5f97fa640 1 -- 172.21.15.89:0/805789175 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fa5fc012420 con 0x7fa60414d8b0 2023-10-18T10:47:03.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.067+0000 7fa5f97fa640 1 --2- 172.21.15.89:0/805789175 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa5f00656b0 0x7fa5f0067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:03.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.067+0000 7fa5f97fa640 1 -- 172.21.15.89:0/805789175 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa5fc082de0 con 0x7fa60414d8b0 2023-10-18T10:47:03.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.067+0000 7fa6088dc640 1 --2- 172.21.15.89:0/805789175 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa5f00656b0 0x7fa5f0067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:03.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.071+0000 7fa6088dc640 1 --2- 172.21.15.89:0/805789175 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa5f00656b0 0x7fa5f0067b70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa6000079c0 tx=0x7fa600008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:03.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.071+0000 7fa5f97fa640 1 -- 172.21.15.89:0/805789175 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fa5fc050610 con 0x7fa60414d8b0 2023-10-18T10:47:03.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.223+0000 7fa60ab40640 1 -- 172.21.15.89:0/805789175 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa604152fc0 con 0x7fa60414d8b0 2023-10-18T10:47:03.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.223+0000 7fa5f97fa640 1 -- 172.21.15.89:0/805789175 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20192 (secure 0 0 0) 0x7fa5fc016020 con 0x7fa60414d8b0 2023-10-18T10:47:03.227 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:47:03.228 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":29,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:46:36.958604+0000","last_up_change":"2023-10-18T10:45:42.164688+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:45:53.309122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-18T10:45:54.659763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:45:57.796514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6824","nonce":773078364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6827","nonce":773078364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6831","nonce":773078364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6829","nonce":773078364}]},"public_addr":"172.21.15.154:6824/773078364","cluster_addr":"172.21.15.154:6827/773078364","heartbeat_back_addr":"172.21.15.154:6831/773078364","heartbeat_front_addr":"172.21.15.154:6829/773078364","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":1,"to":5}]},{"pgid":"4.1a","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:47:03.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.227+0000 7fa60ab40640 1 -- 172.21.15.89:0/805789175 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa5f00656b0 msgr2=0x7fa5f0067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:03.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.227+0000 7fa60ab40640 1 --2- 172.21.15.89:0/805789175 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa5f00656b0 0x7fa5f0067b70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa6000079c0 tx=0x7fa600008040 comp rx=0 tx=0).stop 2023-10-18T10:47:03.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.227+0000 7fa60ab40640 1 -- 172.21.15.89:0/805789175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa60414d8b0 msgr2=0x7fa6040b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:03.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.227+0000 7fa60ab40640 1 --2- 172.21.15.89:0/805789175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa60414d8b0 0x7fa6040b2270 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00c7b0 tx=0x7fa5fc00cc70 comp rx=0 tx=0).stop 2023-10-18T10:47:03.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.227+0000 7fa60ab40640 1 -- 172.21.15.89:0/805789175 shutdown_connections 2023-10-18T10:47:03.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.227+0000 7fa60ab40640 1 --2- 172.21.15.89:0/805789175 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa5f00656b0 0x7fa5f0067b70 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:03.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.227+0000 7fa60ab40640 1 --2- 172.21.15.89:0/805789175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa60414d8b0 0x7fa6040b2270 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:03.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.227+0000 7fa60ab40640 1 --2- 172.21.15.89:0/805789175 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6040ca340 0x7fa60414d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:03.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.227+0000 7fa60ab40640 1 --2- 172.21.15.89:0/805789175 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa60406ff00 0x7fa60406ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:03.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.227+0000 7fa60ab40640 1 -- 172.21.15.89:0/805789175 >> 172.21.15.89:0/805789175 conn(0x7fa604069f00 msgr2=0x7fa60414e230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:03.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.227+0000 7fa60ab40640 1 -- 172.21.15.89:0/805789175 shutdown_connections 2023-10-18T10:47:03.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.227+0000 7fa60ab40640 1 -- 172.21.15.89:0/805789175 wait complete. 2023-10-18T10:47:03.248 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-18T10:47:03.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.387+0000 7f45fdda3640 1 Processor -- start 2023-10-18T10:47:03.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.387+0000 7f45fdda3640 1 -- start start 2023-10-18T10:47:03.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.387+0000 7f45fdda3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f45f806ff00 0x7f45f815e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:03.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.387+0000 7f45fdda3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45f80c8960 0x7f45f80c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:03.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.387+0000 7f45fdda3640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45f815acb0 con 0x7f45f80ca340 2023-10-18T10:47:03.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.387+0000 7f45fdda3640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45f815ef30 con 0x7f45f806ff00 2023-10-18T10:47:03.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.387+0000 7f45fdda3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f815aa10 con 0x7f45f80c8960 2023-10-18T10:47:03.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.387+0000 7f45f6ffd640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f45f80ca340 0x7f45f815b140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37434/0 (socket says 172.21.15.89:37434) 2023-10-18T10:47:03.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.387+0000 7f45f6ffd640 1 -- 172.21.15.89:0/3671072543 learned_addr learned my addr 172.21.15.89:0/3671072543 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:47:03.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.387+0000 7f45f77fe640 1 --2- 172.21.15.89:0/3671072543 >> v2:172.21.15.154:3300/0 conn(0x7f45f806ff00 0x7f45f815e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:03.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.387+0000 7f45f7fff640 1 --2- 172.21.15.89:0/3671072543 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45f80c8960 0x7f45f80c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:03.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.387+0000 7f45f67fc640 1 -- 172.21.15.89:0/3671072543 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 393104167 0 0) 0x7f45f815acb0 con 0x7f45f80ca340 2023-10-18T10:47:03.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.387+0000 7f45f67fc640 1 -- 172.21.15.89:0/3671072543 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45e0003530 con 0x7f45f80ca340 2023-10-18T10:47:03.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.387+0000 7f45f77fe640 1 -- 172.21.15.89:0/3671072543 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45f80c8960 msgr2=0x7f45f80c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:03.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.387+0000 7f45f77fe640 1 --2- 172.21.15.89:0/3671072543 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45f80c8960 0x7f45f80c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:03.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.387+0000 7f45f77fe640 1 -- 172.21.15.89:0/3671072543 >> v1:172.21.15.89:6789/0 conn(0x7f45f80ca340 legacy=0x7f45f815b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:03.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.387+0000 7f45f67fc640 1 -- 172.21.15.89:0/3671072543 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 901686389 0 0) 0x7f45e0003530 con 0x7f45f80ca340 2023-10-18T10:47:03.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.387+0000 7f45f77fe640 1 -- 172.21.15.89:0/3671072543 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f80702e0 con 0x7f45f806ff00 2023-10-18T10:47:03.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.391+0000 7f45f77fe640 1 --2- 172.21.15.89:0/3671072543 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45f806ff00 0x7f45f815e000 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f45e80097c0 tx=0x7f45e801c310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70145987cfb43486 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:03.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.391+0000 7f45f67fc640 1 -- 172.21.15.89:0/3671072543 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec001680 con 0x7f45f806ff00 2023-10-18T10:47:03.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.391+0000 7f45f67fc640 1 -- 172.21.15.89:0/3671072543 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45e80a7050 con 0x7f45f806ff00 2023-10-18T10:47:03.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.391+0000 7f45f67fc640 1 -- 172.21.15.89:0/3671072543 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e80ab600 con 0x7f45f806ff00 2023-10-18T10:47:03.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.391+0000 7f45fdda3640 1 -- 172.21.15.89:0/3671072543 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45f806ff00 msgr2=0x7f45f815e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:03.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.391+0000 7f45fdda3640 1 --2- 172.21.15.89:0/3671072543 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45f806ff00 0x7f45f815e000 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f45e80097c0 tx=0x7f45e801c310 comp rx=0 tx=0).stop 2023-10-18T10:47:03.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.391+0000 7f45fdda3640 1 -- 172.21.15.89:0/3671072543 shutdown_connections 2023-10-18T10:47:03.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.391+0000 7f45fdda3640 1 --2- 172.21.15.89:0/3671072543 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45f80c8960 0x7f45f80c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:03.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.391+0000 7f45fdda3640 1 --2- 172.21.15.89:0/3671072543 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45f806ff00 0x7f45f815e000 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:03.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.391+0000 7f45fdda3640 1 -- 172.21.15.89:0/3671072543 >> 172.21.15.89:0/3671072543 conn(0x7f45f8069f00 msgr2=0x7f45f80cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:03.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.391+0000 7f45fdda3640 1 -- 172.21.15.89:0/3671072543 shutdown_connections 2023-10-18T10:47:03.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.391+0000 7f45fdda3640 1 -- 172.21.15.89:0/3671072543 wait complete. 2023-10-18T10:47:03.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.395+0000 7f45fdda3640 1 Processor -- start 2023-10-18T10:47:03.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.395+0000 7f45fdda3640 1 -- start start 2023-10-18T10:47:03.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.395+0000 7f45fdda3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45f806ff00 0x7f45f81f46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:03.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.395+0000 7f45fdda3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45f80c8960 0x7f45f80b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:03.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.395+0000 7f45fdda3640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45f80ca340 0x7f45f8153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:03.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.395+0000 7f45fdda3640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f8152770 con 0x7f45f80ca340 2023-10-18T10:47:03.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.395+0000 7f45fdda3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f81532f0 con 0x7f45f80c8960 2023-10-18T10:47:03.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.395+0000 7f45fdda3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f81529c0 con 0x7f45f806ff00 2023-10-18T10:47:03.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.395+0000 7f45f7fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45f80ca340 0x7f45f8153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:03.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.395+0000 7f45f7fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45f80ca340 0x7f45f8153780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59798/0 (socket says 172.21.15.89:59798) 2023-10-18T10:47:03.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.395+0000 7f45f7fff640 1 -- 172.21.15.89:0/1261842295 learned_addr learned my addr 172.21.15.89:0/1261842295 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:03.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.395+0000 7f45f6ffd640 1 --2- 172.21.15.89:0/1261842295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45f80c8960 0x7f45f80b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:03.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.395+0000 7f45f77fe640 1 --2- 172.21.15.89:0/1261842295 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45f806ff00 0x7f45f81f46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:03.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.395+0000 7f45f7fff640 1 -- 172.21.15.89:0/1261842295 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45f806ff00 msgr2=0x7f45f81f46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:03.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.395+0000 7f45f7fff640 1 --2- 172.21.15.89:0/1261842295 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45f806ff00 0x7f45f81f46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:03.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.395+0000 7f45f7fff640 1 -- 172.21.15.89:0/1261842295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45f80c8960 msgr2=0x7f45f80b0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:03.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.395+0000 7f45f7fff640 1 --2- 172.21.15.89:0/1261842295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45f80c8960 0x7f45f80b0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:03.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.395+0000 7f45f7fff640 1 -- 172.21.15.89:0/1261842295 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e8009420 con 0x7f45f80ca340 2023-10-18T10:47:03.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.395+0000 7f45f6ffd640 1 --2- 172.21.15.89:0/1261842295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45f80c8960 0x7f45f80b0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:47:03.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.395+0000 7f45f77fe640 1 --2- 172.21.15.89:0/1261842295 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45f806ff00 0x7f45f81f46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:47:03.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.395+0000 7f45f7fff640 1 --2- 172.21.15.89:0/1261842295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45f80ca340 0x7f45f8153780 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f45f000ec10 tx=0x7f45f000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:03.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.395+0000 7f45dffff640 1 -- 172.21.15.89:0/1261842295 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f000cd20 con 0x7f45f80ca340 2023-10-18T10:47:03.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.395+0000 7f45dffff640 1 -- 172.21.15.89:0/1261842295 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f000f040 con 0x7f45f80ca340 2023-10-18T10:47:03.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.395+0000 7f45fdda3640 1 -- 172.21.15.89:0/1261842295 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45f8152ca0 con 0x7f45f80ca340 2023-10-18T10:47:03.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.395+0000 7f45dffff640 1 -- 172.21.15.89:0/1261842295 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f00146c0 con 0x7f45f80ca340 2023-10-18T10:47:03.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.395+0000 7f45fdda3640 1 -- 172.21.15.89:0/1261842295 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45f81f4c00 con 0x7f45f80ca340 2023-10-18T10:47:03.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.399+0000 7f45fdda3640 1 -- 172.21.15.89:0/1261842295 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45f8153460 con 0x7f45f80ca340 2023-10-18T10:47:03.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.399+0000 7f45dffff640 1 -- 172.21.15.89:0/1261842295 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f45f0014a30 con 0x7f45f80ca340 2023-10-18T10:47:03.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.399+0000 7f45dffff640 1 --2- 172.21.15.89:0/1261842295 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f45e4065680 0x7f45e4067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:03.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.399+0000 7f45dffff640 1 -- 172.21.15.89:0/1261842295 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f45f0083d90 con 0x7f45f80ca340 2023-10-18T10:47:03.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.399+0000 7f45f77fe640 1 --2- 172.21.15.89:0/1261842295 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f45e4065680 0x7f45e4067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:03.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.399+0000 7f45f77fe640 1 --2- 172.21.15.89:0/1261842295 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f45e4065680 0x7f45e4067b40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f45e80ad000 tx=0x7f45e80a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:03.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.403+0000 7f45dffff640 1 -- 172.21.15.89:0/1261842295 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f45f0051510 con 0x7f45f80ca340 2023-10-18T10:47:03.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.583+0000 7f45fdda3640 1 -- 172.21.15.89:0/1261842295 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f45f804dbe0 con 0x7f45f80ca340 2023-10-18T10:47:03.586 INFO:tasks.ceph.mon.a.smithi089.stderr:2023-10-18T10:47:03.583+0000 7f921e576640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T10:47:03.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.591+0000 7f45dffff640 1 -- 172.21.15.89:0/1261842295 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7f45f0018020 con 0x7f45f80ca340 2023-10-18T10:47:03.593 INFO:teuthology.orchestra.run.smithi089.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-18T10:47:03.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.591+0000 7f45fdda3640 1 -- 172.21.15.89:0/1261842295 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f45e4065680 msgr2=0x7f45e4067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:03.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.591+0000 7f45fdda3640 1 --2- 172.21.15.89:0/1261842295 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f45e4065680 0x7f45e4067b40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f45e80ad000 tx=0x7f45e80a6000 comp rx=0 tx=0).stop 2023-10-18T10:47:03.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.591+0000 7f45fdda3640 1 -- 172.21.15.89:0/1261842295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45f80ca340 msgr2=0x7f45f8153780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:03.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.591+0000 7f45fdda3640 1 --2- 172.21.15.89:0/1261842295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45f80ca340 0x7f45f8153780 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f45f000ec10 tx=0x7f45f000c490 comp rx=0 tx=0).stop 2023-10-18T10:47:03.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.591+0000 7f45fdda3640 1 -- 172.21.15.89:0/1261842295 shutdown_connections 2023-10-18T10:47:03.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.591+0000 7f45fdda3640 1 --2- 172.21.15.89:0/1261842295 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f45e4065680 0x7f45e4067b40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:03.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.591+0000 7f45fdda3640 1 --2- 172.21.15.89:0/1261842295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45f80ca340 0x7f45f8153780 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:03.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.591+0000 7f45fdda3640 1 --2- 172.21.15.89:0/1261842295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45f80c8960 0x7f45f80b0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:03.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.591+0000 7f45fdda3640 1 --2- 172.21.15.89:0/1261842295 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f45f806ff00 0x7f45f81f46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:03.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.591+0000 7f45fdda3640 1 -- 172.21.15.89:0/1261842295 >> 172.21.15.89:0/1261842295 conn(0x7f45f8069f00 msgr2=0x7f45f8159450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:03.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.591+0000 7f45fdda3640 1 -- 172.21.15.89:0/1261842295 shutdown_connections 2023-10-18T10:47:03.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.591+0000 7f45fdda3640 1 -- 172.21.15.89:0/1261842295 wait complete. 2023-10-18T10:47:03.603 INFO:tasks.ceph.mds.a.smithi089.stderr: -5342> 2023-10-18T10:45:57.455+0000 7fc15bd82640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T10:47:03.615 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-18T10:47:03.652 INFO:tasks.ceph.mds.a.smithi089.stdout:starting mds.a at 2023-10-18T10:47:03.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.755+0000 7f8f31c61640 1 Processor -- start 2023-10-18T10:47:03.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.755+0000 7f8f31c61640 1 -- start start 2023-10-18T10:47:03.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.755+0000 7f8f31c61640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8f2c143820 0x7f8f2c0cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:03.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.755+0000 7f8f31c61640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f2c15e0d0 con 0x7f8f2c143d70 2023-10-18T10:47:03.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.755+0000 7f8f31c61640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f2c15e320 con 0x7f8f2c0cfc50 2023-10-18T10:47:03.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.755+0000 7f8f31c61640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f2c15e600 con 0x7f8f2c143820 2023-10-18T10:47:03.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.755+0000 7f8f23fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8f2c143820 0x7f8f2c0cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:03.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.755+0000 7f8f23fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8f2c143820 0x7f8f2c0cd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59824/0 (socket says 172.21.15.89:59824) 2023-10-18T10:47:03.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.755+0000 7f8f23fff640 1 -- 172.21.15.89:0/1192243619 learned_addr learned my addr 172.21.15.89:0/1192243619 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:03.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.755+0000 7f8f2bfff640 1 --1- 172.21.15.89:0/1192243619 >> v1:172.21.15.89:6789/0 conn(0x7f8f2c143d70 0x7f8f2c0ce220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37462/0 (socket says 172.21.15.89:37462) 2023-10-18T10:47:03.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.755+0000 7f8f23fff640 1 -- 172.21.15.89:0/1192243619 >> v1:172.21.15.154:6789/0 conn(0x7f8f2c0cfc50 legacy=0x7f8f2c0ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:03.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.755+0000 7f8f23fff640 1 -- 172.21.15.89:0/1192243619 >> v1:172.21.15.89:6789/0 conn(0x7f8f2c143d70 legacy=0x7f8f2c0ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:03.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.755+0000 7f8f23fff640 1 -- 172.21.15.89:0/1192243619 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f2c15e8e0 con 0x7f8f2c143820 2023-10-18T10:47:03.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.759+0000 7f8f23fff640 1 --2- 172.21.15.89:0/1192243619 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f2c143820 0x7f8f2c0cd420 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8f14009870 tx=0x7f8f1401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878d220e673b9a47 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:03.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.759+0000 7f8f2affd640 1 -- 172.21.15.89:0/1192243619 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f140aa430 con 0x7f8f2c143820 2023-10-18T10:47:03.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.759+0000 7f8f2affd640 1 -- 172.21.15.89:0/1192243619 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f2c0d0240 con 0x7f8f2c143820 2023-10-18T10:47:03.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.759+0000 7f8f2affd640 1 -- 172.21.15.89:0/1192243619 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f140aa7b0 con 0x7f8f2c143820 2023-10-18T10:47:03.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.759+0000 7f8f31c61640 1 -- 172.21.15.89:0/1192243619 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f2c143820 msgr2=0x7f8f2c0cd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:03.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.759+0000 7f8f31c61640 1 --2- 172.21.15.89:0/1192243619 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f2c143820 0x7f8f2c0cd420 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8f14009870 tx=0x7f8f1401c4b0 comp rx=0 tx=0).stop 2023-10-18T10:47:03.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.759+0000 7f8f31c61640 1 -- 172.21.15.89:0/1192243619 shutdown_connections 2023-10-18T10:47:03.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.759+0000 7f8f31c61640 1 --2- 172.21.15.89:0/1192243619 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f2c143820 0x7f8f2c0cd420 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:03.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.759+0000 7f8f31c61640 1 -- 172.21.15.89:0/1192243619 >> 172.21.15.89:0/1192243619 conn(0x7f8f2c06dad0 msgr2=0x7f8f2c06dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:03.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.759+0000 7f8f31c61640 1 -- 172.21.15.89:0/1192243619 shutdown_connections 2023-10-18T10:47:03.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.759+0000 7f8f31c61640 1 -- 172.21.15.89:0/1192243619 wait complete. 2023-10-18T10:47:03.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.763+0000 7f8f31c61640 1 Processor -- start 2023-10-18T10:47:03.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.763+0000 7f8f31c61640 1 -- start start 2023-10-18T10:47:03.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.763+0000 7f8f31c61640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f2c0cfc50 0x7f8f2c0c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:03.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.763+0000 7f8f31c61640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f2c143820 0x7f8f2c06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:03.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.763+0000 7f8f31c61640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f2c143d70 0x7f8f2c06b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:03.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.763+0000 7f8f31c61640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2c06a720 con 0x7f8f2c0cfc50 2023-10-18T10:47:03.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.763+0000 7f8f31c61640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2c151ef0 con 0x7f8f2c143820 2023-10-18T10:47:03.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.763+0000 7f8f31c61640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2c155080 con 0x7f8f2c143d70 2023-10-18T10:47:03.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.763+0000 7f8f2b7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f2c0cfc50 0x7f8f2c0c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:03.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.763+0000 7f8f2b7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f2c0cfc50 0x7f8f2c0c81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59834/0 (socket says 172.21.15.89:59834) 2023-10-18T10:47:03.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.763+0000 7f8f2b7fe640 1 -- 172.21.15.89:0/43622703 learned_addr learned my addr 172.21.15.89:0/43622703 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:03.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.763+0000 7f8f2bfff640 1 --2- 172.21.15.89:0/43622703 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f2c143d70 0x7f8f2c06b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:03.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.763+0000 7f8f23fff640 1 --2- 172.21.15.89:0/43622703 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f2c143820 0x7f8f2c06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:03.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.763+0000 7f8f2b7fe640 1 -- 172.21.15.89:0/43622703 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f2c143d70 msgr2=0x7f8f2c06b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:03.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.763+0000 7f8f2b7fe640 1 --2- 172.21.15.89:0/43622703 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f2c143d70 0x7f8f2c06b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:03.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.763+0000 7f8f2b7fe640 1 -- 172.21.15.89:0/43622703 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f2c143820 msgr2=0x7f8f2c06ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:03.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.763+0000 7f8f2b7fe640 1 --2- 172.21.15.89:0/43622703 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f2c143820 0x7f8f2c06ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:03.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.763+0000 7f8f2b7fe640 1 -- 172.21.15.89:0/43622703 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f140094d0 con 0x7f8f2c0cfc50 2023-10-18T10:47:03.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.763+0000 7f8f2b7fe640 1 --2- 172.21.15.89:0/43622703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f2c0cfc50 0x7f8f2c0c81f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c00c3c0 tx=0x7f8f1c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:03.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.763+0000 7f8f28ff9640 1 -- 172.21.15.89:0/43622703 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c0052d0 con 0x7f8f2c0cfc50 2023-10-18T10:47:03.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.763+0000 7f8f28ff9640 1 -- 172.21.15.89:0/43622703 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f1c005430 con 0x7f8f2c0cfc50 2023-10-18T10:47:03.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.763+0000 7f8f31c61640 1 -- 172.21.15.89:0/43622703 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f2c06a3b0 con 0x7f8f2c0cfc50 2023-10-18T10:47:03.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.763+0000 7f8f28ff9640 1 -- 172.21.15.89:0/43622703 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c003840 con 0x7f8f2c0cfc50 2023-10-18T10:47:03.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.763+0000 7f8f31c61640 1 -- 172.21.15.89:0/43622703 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f2c165b20 con 0x7f8f2c0cfc50 2023-10-18T10:47:03.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.767+0000 7f8f31c61640 1 -- 172.21.15.89:0/43622703 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f2c04dbe0 con 0x7f8f2c0cfc50 2023-10-18T10:47:03.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.767+0000 7f8f28ff9640 1 -- 172.21.15.89:0/43622703 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f8f1c0039a0 con 0x7f8f2c0cfc50 2023-10-18T10:47:03.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.767+0000 7f8f28ff9640 1 --2- 172.21.15.89:0/43622703 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8f0c065680 0x7f8f0c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:03.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.767+0000 7f8f28ff9640 1 -- 172.21.15.89:0/43622703 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f8f1c082680 con 0x7f8f2c0cfc50 2023-10-18T10:47:03.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.767+0000 7f8f23fff640 1 --2- 172.21.15.89:0/43622703 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8f0c065680 0x7f8f0c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:03.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.767+0000 7f8f23fff640 1 --2- 172.21.15.89:0/43622703 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8f0c065680 0x7f8f0c067b40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8f140ae000 tx=0x7f8f140a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:03.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.771+0000 7f8f28ff9640 1 -- 172.21.15.89:0/43622703 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f8f1c04fdc0 con 0x7f8f2c0cfc50 2023-10-18T10:47:03.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:03.959+0000 7f8f31c61640 1 -- 172.21.15.89:0/43622703 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8f2c06e5f0 con 0x7f8f2c0cfc50 2023-10-18T10:47:04.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.591+0000 7f8f28ff9640 1 -- 172.21.15.89:0/43622703 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8f1c0533e0 con 0x7f8f2c0cfc50 2023-10-18T10:47:04.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.591+0000 7f8f31c61640 1 -- 172.21.15.89:0/43622703 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8f0c065680 msgr2=0x7f8f0c067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:04.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.591+0000 7f8f31c61640 1 --2- 172.21.15.89:0/43622703 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8f0c065680 0x7f8f0c067b40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8f140ae000 tx=0x7f8f140a7000 comp rx=0 tx=0).stop 2023-10-18T10:47:04.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.591+0000 7f8f31c61640 1 -- 172.21.15.89:0/43622703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f2c0cfc50 msgr2=0x7f8f2c0c81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:04.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.591+0000 7f8f31c61640 1 --2- 172.21.15.89:0/43622703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f2c0cfc50 0x7f8f2c0c81f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c00c3c0 tx=0x7f8f1c00c880 comp rx=0 tx=0).stop 2023-10-18T10:47:04.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.591+0000 7f8f31c61640 1 -- 172.21.15.89:0/43622703 shutdown_connections 2023-10-18T10:47:04.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.591+0000 7f8f31c61640 1 --2- 172.21.15.89:0/43622703 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8f0c065680 0x7f8f0c067b40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:04.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.591+0000 7f8f31c61640 1 --2- 172.21.15.89:0/43622703 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f2c143d70 0x7f8f2c06b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:04.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.591+0000 7f8f31c61640 1 --2- 172.21.15.89:0/43622703 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f2c143820 0x7f8f2c06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:04.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.591+0000 7f8f31c61640 1 --2- 172.21.15.89:0/43622703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f2c0cfc50 0x7f8f2c0c81f0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:04.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.591+0000 7f8f31c61640 1 -- 172.21.15.89:0/43622703 >> 172.21.15.89:0/43622703 conn(0x7f8f2c06dad0 msgr2=0x7f8f2c15ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:04.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.595+0000 7f8f31c61640 1 -- 172.21.15.89:0/43622703 shutdown_connections 2023-10-18T10:47:04.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.595+0000 7f8f31c61640 1 -- 172.21.15.89:0/43622703 wait complete. 2023-10-18T10:47:04.617 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-18T10:47:04.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.799+0000 7f90b9fa3640 1 Processor -- start 2023-10-18T10:47:04.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.799+0000 7f90b9fa3640 1 -- start start 2023-10-18T10:47:04.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.803+0000 7f90b9fa3640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f90b4158b80 0x7f90b415d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:04.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.803+0000 7f90b9fa3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90b406b7d0 0x7f90b406bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:04.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.803+0000 7f90b9fa3640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90b415d710 con 0x7f90b4071810 2023-10-18T10:47:04.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.803+0000 7f90b9fa3640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90b40b0d10 con 0x7f90b4158b80 2023-10-18T10:47:04.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.803+0000 7f90b9fa3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b406c180 con 0x7f90b406b7d0 2023-10-18T10:47:04.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.803+0000 7f90b37fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f90b4158b80 0x7f90b415d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:04.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.803+0000 7f90b37fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f90b4158b80 0x7f90b415d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59850/0 (socket says 172.21.15.89:59850) 2023-10-18T10:47:04.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.803+0000 7f90b37fe640 1 -- 172.21.15.89:0/3536732436 learned_addr learned my addr 172.21.15.89:0/3536732436 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:04.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.803+0000 7f90b3fff640 1 --2- 172.21.15.89:0/3536732436 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90b406b7d0 0x7f90b406bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:04.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.803+0000 7f90b37fe640 1 -- 172.21.15.89:0/3536732436 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90b406b7d0 msgr2=0x7f90b406bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:04.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.803+0000 7f90b37fe640 1 --2- 172.21.15.89:0/3536732436 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90b406b7d0 0x7f90b406bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:04.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.803+0000 7f90b37fe640 1 -- 172.21.15.89:0/3536732436 >> v1:172.21.15.154:6789/0 conn(0x7f90b4071810 legacy=0x7f90b40cbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:04.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.803+0000 7f90b37fe640 1 -- 172.21.15.89:0/3536732436 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b406ff30 con 0x7f90b4158b80 2023-10-18T10:47:04.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.807+0000 7f90b37fe640 1 --2- 172.21.15.89:0/3536732436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90b4158b80 0x7f90b415d1d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f90a4009870 tx=0x7f90a401ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b157107cd14221 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:04.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.807+0000 7f90b2ffd640 1 -- 172.21.15.89:0/3536732436 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a40a89b0 con 0x7f90b4158b80 2023-10-18T10:47:04.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.807+0000 7f90b2ffd640 1 -- 172.21.15.89:0/3536732436 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a40a8b10 con 0x7f90b4158b80 2023-10-18T10:47:04.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.807+0000 7f90b2ffd640 1 -- 172.21.15.89:0/3536732436 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a40a8ea0 con 0x7f90b4158b80 2023-10-18T10:47:04.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.807+0000 7f90b9fa3640 1 -- 172.21.15.89:0/3536732436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90b4158b80 msgr2=0x7f90b415d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:04.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.807+0000 7f90b9fa3640 1 --2- 172.21.15.89:0/3536732436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90b4158b80 0x7f90b415d1d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f90a4009870 tx=0x7f90a401ecf0 comp rx=0 tx=0).stop 2023-10-18T10:47:04.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.807+0000 7f90b9fa3640 1 -- 172.21.15.89:0/3536732436 shutdown_connections 2023-10-18T10:47:04.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.807+0000 7f90b9fa3640 1 --2- 172.21.15.89:0/3536732436 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90b406b7d0 0x7f90b406bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:04.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.807+0000 7f90b9fa3640 1 --2- 172.21.15.89:0/3536732436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90b4158b80 0x7f90b415d1d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:04.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.807+0000 7f90b9fa3640 1 -- 172.21.15.89:0/3536732436 >> 172.21.15.89:0/3536732436 conn(0x7f90b40cad40 msgr2=0x7f90b40cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:04.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.807+0000 7f90b9fa3640 1 -- 172.21.15.89:0/3536732436 shutdown_connections 2023-10-18T10:47:04.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.807+0000 7f90b9fa3640 1 -- 172.21.15.89:0/3536732436 wait complete. 2023-10-18T10:47:04.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.807+0000 7f90b9fa3640 1 Processor -- start 2023-10-18T10:47:04.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.807+0000 7f90b9fa3640 1 -- start start 2023-10-18T10:47:04.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.811+0000 7f90b9fa3640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90b406b7d0 0x7f90b41f1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:04.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.811+0000 7f90b9fa3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90b4071810 0x7f90b40b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:04.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.811+0000 7f90b9fa3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90b4158b80 0x7f90b40b1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:04.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.811+0000 7f90b9fa3640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b40c7dc0 con 0x7f90b406b7d0 2023-10-18T10:47:04.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.811+0000 7f90b9fa3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b40c3500 con 0x7f90b4158b80 2023-10-18T10:47:04.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.811+0000 7f90b9fa3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b40c7ae0 con 0x7f90b4071810 2023-10-18T10:47:04.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.811+0000 7f90b37fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90b406b7d0 0x7f90b41f1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:04.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.811+0000 7f90b37fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90b406b7d0 0x7f90b41f1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59858/0 (socket says 172.21.15.89:59858) 2023-10-18T10:47:04.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.811+0000 7f90b37fe640 1 -- 172.21.15.89:0/2669172341 learned_addr learned my addr 172.21.15.89:0/2669172341 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:04.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.811+0000 7f90aaffd640 1 --2- 172.21.15.89:0/2669172341 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90b4071810 0x7f90b40b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:04.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.811+0000 7f90b3fff640 1 --2- 172.21.15.89:0/2669172341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90b4158b80 0x7f90b40b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:04.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.811+0000 7f90b37fe640 1 -- 172.21.15.89:0/2669172341 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90b4071810 msgr2=0x7f90b40b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:04.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.811+0000 7f90b37fe640 1 --2- 172.21.15.89:0/2669172341 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90b4071810 0x7f90b40b0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:04.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.811+0000 7f90b37fe640 1 -- 172.21.15.89:0/2669172341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90b4158b80 msgr2=0x7f90b40b1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:04.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.811+0000 7f90b37fe640 1 --2- 172.21.15.89:0/2669172341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90b4158b80 0x7f90b40b1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:04.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.811+0000 7f90b37fe640 1 -- 172.21.15.89:0/2669172341 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a40094d0 con 0x7f90b406b7d0 2023-10-18T10:47:04.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.811+0000 7f90b37fe640 1 --2- 172.21.15.89:0/2669172341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90b406b7d0 0x7f90b41f1fc0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f90a40b3040 tx=0x7f90a401ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:04.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.811+0000 7f90b0ff9640 1 -- 172.21.15.89:0/2669172341 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a40a89b0 con 0x7f90b406b7d0 2023-10-18T10:47:04.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.811+0000 7f90b0ff9640 1 -- 172.21.15.89:0/2669172341 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a40bb480 con 0x7f90b406b7d0 2023-10-18T10:47:04.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.811+0000 7f90b0ff9640 1 -- 172.21.15.89:0/2669172341 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a40bb830 con 0x7f90b406b7d0 2023-10-18T10:47:04.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.811+0000 7f90b9fa3640 1 -- 172.21.15.89:0/2669172341 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90b40c2eb0 con 0x7f90b406b7d0 2023-10-18T10:47:04.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.811+0000 7f90b9fa3640 1 -- 172.21.15.89:0/2669172341 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b41f2500 con 0x7f90b406b7d0 2023-10-18T10:47:04.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.811+0000 7f90b9fa3640 1 -- 172.21.15.89:0/2669172341 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90b404dbe0 con 0x7f90b406b7d0 2023-10-18T10:47:04.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.811+0000 7f90b0ff9640 1 -- 172.21.15.89:0/2669172341 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f90a40bb990 con 0x7f90b406b7d0 2023-10-18T10:47:04.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.815+0000 7f90b0ff9640 1 --2- 172.21.15.89:0/2669172341 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f9098065680 0x7f9098067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:04.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.815+0000 7f90b0ff9640 1 -- 172.21.15.89:0/2669172341 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f90a4121600 con 0x7f90b406b7d0 2023-10-18T10:47:04.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.815+0000 7f90aaffd640 1 --2- 172.21.15.89:0/2669172341 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f9098065680 0x7f9098067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:04.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.815+0000 7f90aaffd640 1 --2- 172.21.15.89:0/2669172341 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f9098065680 0x7f9098067b40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f90b41f5c20 tx=0x7f909c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:04.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.819+0000 7f90b0ff9640 1 -- 172.21.15.89:0/2669172341 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f90a40eed40 con 0x7f90b406b7d0 2023-10-18T10:47:04.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:04.967+0000 7f90b9fa3640 1 -- 172.21.15.89:0/2669172341 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f90b415d710 con 0x7f90b406b7d0 2023-10-18T10:47:05.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.759+0000 7f90b0ff9640 1 -- 172.21.15.89:0/2669172341 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v31) v1 ==== 181+0+0 (secure 0 0 0) 0x7f90b415d710 con 0x7f90b406b7d0 2023-10-18T10:47:05.761 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' removed 2023-10-18T10:47:05.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.763+0000 7f90b9fa3640 1 -- 172.21.15.89:0/2669172341 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f9098065680 msgr2=0x7f9098067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:05.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.763+0000 7f90b9fa3640 1 --2- 172.21.15.89:0/2669172341 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f9098065680 0x7f9098067b40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f90b41f5c20 tx=0x7f909c009040 comp rx=0 tx=0).stop 2023-10-18T10:47:05.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.763+0000 7f90b9fa3640 1 -- 172.21.15.89:0/2669172341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90b406b7d0 msgr2=0x7f90b41f1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:05.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.763+0000 7f90b9fa3640 1 --2- 172.21.15.89:0/2669172341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90b406b7d0 0x7f90b41f1fc0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f90a40b3040 tx=0x7f90a401ba90 comp rx=0 tx=0).stop 2023-10-18T10:47:05.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.763+0000 7f90b9fa3640 1 -- 172.21.15.89:0/2669172341 shutdown_connections 2023-10-18T10:47:05.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.763+0000 7f90b9fa3640 1 --2- 172.21.15.89:0/2669172341 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f9098065680 0x7f9098067b40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:05.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.763+0000 7f90b9fa3640 1 --2- 172.21.15.89:0/2669172341 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90b4071810 0x7f90b40b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:05.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.763+0000 7f90b9fa3640 1 --2- 172.21.15.89:0/2669172341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90b4158b80 0x7f90b40b1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:05.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.763+0000 7f90b9fa3640 1 --2- 172.21.15.89:0/2669172341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90b406b7d0 0x7f90b41f1fc0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:05.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.763+0000 7f90b9fa3640 1 -- 172.21.15.89:0/2669172341 >> 172.21.15.89:0/2669172341 conn(0x7f90b40cad40 msgr2=0x7f90b40cbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:05.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.763+0000 7f90b9fa3640 1 -- 172.21.15.89:0/2669172341 shutdown_connections 2023-10-18T10:47:05.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.763+0000 7f90b9fa3640 1 -- 172.21.15.89:0/2669172341 wait complete. 2023-10-18T10:47:05.791 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-18T10:47:05.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.935+0000 7faff3ecb640 1 Processor -- start 2023-10-18T10:47:05.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.935+0000 7faff3ecb640 1 -- start start 2023-10-18T10:47:05.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.935+0000 7faff3ecb640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fafec1600b0 0x7fafec0b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:05.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.935+0000 7faff3ecb640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fafec069900 0x7fafec069d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:05.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.935+0000 7faff3ecb640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafec06a4c0 con 0x7fafec0d0c70 2023-10-18T10:47:05.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.935+0000 7faff3ecb640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafec0b1a80 con 0x7fafec1600b0 2023-10-18T10:47:05.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.935+0000 7faff3ecb640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafec06a2b0 con 0x7fafec069900 2023-10-18T10:47:05.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.935+0000 7faff1c67640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fafec1600b0 0x7fafec0b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:05.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.935+0000 7faff1c67640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fafec1600b0 0x7fafec0b12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59868/0 (socket says 172.21.15.89:59868) 2023-10-18T10:47:05.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.935+0000 7faff1c67640 1 -- 172.21.15.89:0/4068627289 learned_addr learned my addr 172.21.15.89:0/4068627289 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:05.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.935+0000 7faff1c67640 1 -- 172.21.15.89:0/4068627289 >> v2:172.21.15.154:3300/0 conn(0x7fafec069900 msgr2=0x7fafec069d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:05.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.935+0000 7faff1c67640 1 --2- 172.21.15.89:0/4068627289 >> v2:172.21.15.154:3300/0 conn(0x7fafec069900 0x7fafec069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:05.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.935+0000 7faff1c67640 1 -- 172.21.15.89:0/4068627289 >> v1:172.21.15.154:6789/0 conn(0x7fafec0d0c70 legacy=0x7fafec06a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:05.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.935+0000 7faff1c67640 1 -- 172.21.15.89:0/4068627289 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafec160490 con 0x7fafec1600b0 2023-10-18T10:47:05.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.939+0000 7faff1c67640 1 --2- 172.21.15.89:0/4068627289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fafec1600b0 0x7fafec0b12b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fafec06bac0 tx=0x7fafe801c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9430094d6aa7bc73 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:05.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.939+0000 7faff0c65640 1 -- 172.21.15.89:0/4068627289 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe80aa620 con 0x7fafec1600b0 2023-10-18T10:47:05.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.939+0000 7faff0c65640 1 -- 172.21.15.89:0/4068627289 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe80aa780 con 0x7fafec1600b0 2023-10-18T10:47:05.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.939+0000 7faff0c65640 1 -- 172.21.15.89:0/4068627289 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe80aab10 con 0x7fafec1600b0 2023-10-18T10:47:05.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.939+0000 7faff3ecb640 1 -- 172.21.15.89:0/4068627289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fafec1600b0 msgr2=0x7fafec0b12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:05.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.939+0000 7faff3ecb640 1 --2- 172.21.15.89:0/4068627289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fafec1600b0 0x7fafec0b12b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fafec06bac0 tx=0x7fafe801c4c0 comp rx=0 tx=0).stop 2023-10-18T10:47:05.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.939+0000 7faff3ecb640 1 -- 172.21.15.89:0/4068627289 shutdown_connections 2023-10-18T10:47:05.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.939+0000 7faff3ecb640 1 --2- 172.21.15.89:0/4068627289 >> v2:172.21.15.154:3300/0 conn(0x7fafec069900 0x7fafec069d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:05.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.939+0000 7faff3ecb640 1 --2- 172.21.15.89:0/4068627289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fafec1600b0 0x7fafec0b12b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:05.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.939+0000 7faff3ecb640 1 -- 172.21.15.89:0/4068627289 >> 172.21.15.89:0/4068627289 conn(0x7fafec09ba00 msgr2=0x7fafec0d2690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:05.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.939+0000 7faff3ecb640 1 -- 172.21.15.89:0/4068627289 shutdown_connections 2023-10-18T10:47:05.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.939+0000 7faff3ecb640 1 -- 172.21.15.89:0/4068627289 wait complete. 2023-10-18T10:47:05.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.943+0000 7faff3ecb640 1 Processor -- start 2023-10-18T10:47:05.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.943+0000 7faff3ecb640 1 -- start start 2023-10-18T10:47:05.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.943+0000 7faff3ecb640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fafec069900 0x7fafec152d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:05.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.943+0000 7faff3ecb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fafec0d0c70 0x7fafec150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:05.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.943+0000 7faff3ecb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fafec1600b0 0x7fafec0b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:05.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.943+0000 7faff3ecb640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafec150bd0 con 0x7fafec069900 2023-10-18T10:47:05.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.943+0000 7faff3ecb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafec151c70 con 0x7fafec1600b0 2023-10-18T10:47:05.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.943+0000 7faff3ecb640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafec151f50 con 0x7fafec0d0c70 2023-10-18T10:47:05.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.943+0000 7faff1c67640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fafec069900 0x7fafec152d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:05.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.943+0000 7faff1c67640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fafec069900 0x7fafec152d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59874/0 (socket says 172.21.15.89:59874) 2023-10-18T10:47:05.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.943+0000 7faff1c67640 1 -- 172.21.15.89:0/2710703626 learned_addr learned my addr 172.21.15.89:0/2710703626 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:05.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.943+0000 7faff2468640 1 --2- 172.21.15.89:0/2710703626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fafec1600b0 0x7fafec0b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:05.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.943+0000 7faff1466640 1 --2- 172.21.15.89:0/2710703626 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fafec0d0c70 0x7fafec150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:05.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.943+0000 7faff1c67640 1 -- 172.21.15.89:0/2710703626 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fafec0d0c70 msgr2=0x7fafec150690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:05.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.943+0000 7faff1c67640 1 --2- 172.21.15.89:0/2710703626 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fafec0d0c70 0x7fafec150690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:05.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.943+0000 7faff1c67640 1 -- 172.21.15.89:0/2710703626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fafec1600b0 msgr2=0x7fafec0b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:05.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.943+0000 7faff1c67640 1 --2- 172.21.15.89:0/2710703626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fafec1600b0 0x7fafec0b3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:05.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.943+0000 7faff1c67640 1 -- 172.21.15.89:0/2710703626 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe80094d0 con 0x7fafec069900 2023-10-18T10:47:05.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.943+0000 7faff1c67640 1 --2- 172.21.15.89:0/2710703626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fafec069900 0x7fafec152d10 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fafe801a0f0 tx=0x7fafe801ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:05.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.943+0000 7fafe27fc640 1 -- 172.21.15.89:0/2710703626 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe80bd5f0 con 0x7fafec069900 2023-10-18T10:47:05.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.943+0000 7fafe27fc640 1 -- 172.21.15.89:0/2710703626 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe80bd750 con 0x7fafec069900 2023-10-18T10:47:05.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.943+0000 7faff3ecb640 1 -- 172.21.15.89:0/2710703626 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafec152230 con 0x7fafec069900 2023-10-18T10:47:05.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.943+0000 7fafe27fc640 1 -- 172.21.15.89:0/2710703626 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe80bdb00 con 0x7fafec069900 2023-10-18T10:47:05.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.943+0000 7faff3ecb640 1 -- 172.21.15.89:0/2710703626 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafec1527f0 con 0x7fafec069900 2023-10-18T10:47:05.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.943+0000 7faff3ecb640 1 -- 172.21.15.89:0/2710703626 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafec04dbe0 con 0x7fafec069900 2023-10-18T10:47:05.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.947+0000 7fafe27fc640 1 -- 172.21.15.89:0/2710703626 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fafe80b3600 con 0x7fafec069900 2023-10-18T10:47:05.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.947+0000 7fafe27fc640 1 --2- 172.21.15.89:0/2710703626 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fafd0065680 0x7fafd0067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:05.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.947+0000 7fafe27fc640 1 -- 172.21.15.89:0/2710703626 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6519+0+0 (secure 0 0 0) 0x7fafe811dec0 con 0x7fafec069900 2023-10-18T10:47:05.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.947+0000 7faff1466640 1 --2- 172.21.15.89:0/2710703626 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fafd0065680 0x7fafd0067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:05.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.947+0000 7faff1466640 1 --2- 172.21.15.89:0/2710703626 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fafd0065680 0x7fafd0067b40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fafec0b3d30 tx=0x7fafdc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:05.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:05.951+0000 7fafe27fc640 1 -- 172.21.15.89:0/2710703626 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fafe80eb860 con 0x7fafec069900 2023-10-18T10:47:06.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.099+0000 7faff3ecb640 1 -- 172.21.15.89:0/2710703626 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fafec15ef20 con 0x7fafec069900 2023-10-18T10:47:06.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.771+0000 7fafe27fc640 1 -- 172.21.15.89:0/2710703626 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v32) v1 ==== 169+0+0 (secure 0 0 0) 0x7fafe80c30d0 con 0x7fafec069900 2023-10-18T10:47:06.772 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' removed 2023-10-18T10:47:06.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.771+0000 7faff3ecb640 1 -- 172.21.15.89:0/2710703626 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fafd0065680 msgr2=0x7fafd0067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:06.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.771+0000 7faff3ecb640 1 --2- 172.21.15.89:0/2710703626 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fafd0065680 0x7fafd0067b40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fafec0b3d30 tx=0x7fafdc009040 comp rx=0 tx=0).stop 2023-10-18T10:47:06.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.771+0000 7faff3ecb640 1 -- 172.21.15.89:0/2710703626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fafec069900 msgr2=0x7fafec152d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:06.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.771+0000 7faff3ecb640 1 --2- 172.21.15.89:0/2710703626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fafec069900 0x7fafec152d10 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fafe801a0f0 tx=0x7fafe801ad00 comp rx=0 tx=0).stop 2023-10-18T10:47:06.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.771+0000 7faff3ecb640 1 -- 172.21.15.89:0/2710703626 shutdown_connections 2023-10-18T10:47:06.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.771+0000 7faff3ecb640 1 --2- 172.21.15.89:0/2710703626 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fafd0065680 0x7fafd0067b40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:06.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.771+0000 7faff3ecb640 1 --2- 172.21.15.89:0/2710703626 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fafec0d0c70 0x7fafec150690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:06.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.771+0000 7faff3ecb640 1 --2- 172.21.15.89:0/2710703626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fafec1600b0 0x7fafec0b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:06.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.771+0000 7faff3ecb640 1 --2- 172.21.15.89:0/2710703626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fafec069900 0x7fafec152d10 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:06.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.771+0000 7faff3ecb640 1 -- 172.21.15.89:0/2710703626 >> 172.21.15.89:0/2710703626 conn(0x7fafec09ba00 msgr2=0x7fafec156020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:06.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.771+0000 7faff3ecb640 1 -- 172.21.15.89:0/2710703626 shutdown_connections 2023-10-18T10:47:06.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.775+0000 7faff3ecb640 1 -- 172.21.15.89:0/2710703626 wait complete. 2023-10-18T10:47:06.811 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-10-18T10:47:06.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.955+0000 7f2fd8a19640 1 Processor -- start 2023-10-18T10:47:06.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.955+0000 7f2fd8a19640 1 -- start start 2023-10-18T10:47:06.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.959+0000 7f2fd8a19640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2fd4158b80 0x7f2fd415d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:06.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.959+0000 7f2fd8a19640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2fd4071810 0x7f2fd40cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:06.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.959+0000 7f2fd8a19640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fd40cc3d0 0x7f2fd406b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:06.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.959+0000 7f2fd8a19640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd40b0d10 con 0x7f2fd4071810 2023-10-18T10:47:06.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.959+0000 7f2fd8a19640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd415d710 con 0x7f2fd4158b80 2023-10-18T10:47:06.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.959+0000 7f2fd8a19640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd406ff30 con 0x7f2fd40cc3d0 2023-10-18T10:47:06.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.959+0000 7f2fd1d9b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2fd4071810 0x7f2fd40cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:06.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.959+0000 7f2fd1d9b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2fd4071810 0x7f2fd40cbe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59888/0 (socket says 172.21.15.89:59888) 2023-10-18T10:47:06.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.959+0000 7f2fd1d9b640 1 -- 172.21.15.89:0/1344379967 learned_addr learned my addr 172.21.15.89:0/1344379967 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:06.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.959+0000 7f2fd259c640 1 --2- 172.21.15.89:0/1344379967 >> v2:172.21.15.154:3300/0 conn(0x7f2fd4158b80 0x7f2fd415d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:06.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.959+0000 7f2fd2d9d640 1 --2- 172.21.15.89:0/1344379967 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fd40cc3d0 0x7f2fd406b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:06.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.959+0000 7f2fd1d9b640 1 -- 172.21.15.89:0/1344379967 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fd40cc3d0 msgr2=0x7f2fd406b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:06.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.959+0000 7f2fd1d9b640 1 --2- 172.21.15.89:0/1344379967 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fd40cc3d0 0x7f2fd406b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:06.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.959+0000 7f2fd1d9b640 1 -- 172.21.15.89:0/1344379967 >> v2:172.21.15.154:3300/0 conn(0x7f2fd4158b80 msgr2=0x7f2fd415d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:06.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.959+0000 7f2fd1d9b640 1 --2- 172.21.15.89:0/1344379967 >> v2:172.21.15.154:3300/0 conn(0x7f2fd4158b80 0x7f2fd415d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:06.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.959+0000 7f2fd1d9b640 1 -- 172.21.15.89:0/1344379967 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd40b0750 con 0x7f2fd4071810 2023-10-18T10:47:06.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.959+0000 7f2fd1d9b640 1 --2- 172.21.15.89:0/1344379967 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fd4071810 0x7f2fd40cbe90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4072410 tx=0x7f2fc801ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86403fcd2f3aad82 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:06.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.963+0000 7f2fd159a640 1 -- 172.21.15.89:0/1344379967 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc80a89b0 con 0x7f2fd4071810 2023-10-18T10:47:06.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.963+0000 7f2fd159a640 1 -- 172.21.15.89:0/1344379967 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc80a8b10 con 0x7f2fd4071810 2023-10-18T10:47:06.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.963+0000 7f2fd159a640 1 -- 172.21.15.89:0/1344379967 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc80a8ea0 con 0x7f2fd4071810 2023-10-18T10:47:06.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.963+0000 7f2fd8a19640 1 -- 172.21.15.89:0/1344379967 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fd4071810 msgr2=0x7f2fd40cbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:06.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.963+0000 7f2fd8a19640 1 --2- 172.21.15.89:0/1344379967 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fd4071810 0x7f2fd40cbe90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4072410 tx=0x7f2fc801ecf0 comp rx=0 tx=0).stop 2023-10-18T10:47:06.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.963+0000 7f2fd8a19640 1 -- 172.21.15.89:0/1344379967 shutdown_connections 2023-10-18T10:47:06.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.963+0000 7f2fd8a19640 1 --2- 172.21.15.89:0/1344379967 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fd40cc3d0 0x7f2fd406b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:06.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.963+0000 7f2fd8a19640 1 --2- 172.21.15.89:0/1344379967 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fd4071810 0x7f2fd40cbe90 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:06.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.963+0000 7f2fd8a19640 1 --2- 172.21.15.89:0/1344379967 >> v2:172.21.15.154:3300/0 conn(0x7f2fd4158b80 0x7f2fd415d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:06.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.963+0000 7f2fd8a19640 1 -- 172.21.15.89:0/1344379967 >> 172.21.15.89:0/1344379967 conn(0x7f2fd40cad40 msgr2=0x7f2fd40cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:06.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.963+0000 7f2fd8a19640 1 -- 172.21.15.89:0/1344379967 shutdown_connections 2023-10-18T10:47:06.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.963+0000 7f2fd8a19640 1 -- 172.21.15.89:0/1344379967 wait complete. 2023-10-18T10:47:06.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.963+0000 7f2fd8a19640 1 Processor -- start 2023-10-18T10:47:06.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.963+0000 7f2fd8a19640 1 -- start start 2023-10-18T10:47:06.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.967+0000 7f2fd8a19640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fd4071810 0x7f2fd41f53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:06.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.967+0000 7f2fd8a19640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fd40cc3d0 0x7f2fd41f58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:06.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.967+0000 7f2fd8a19640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fd4158b80 0x7f2fd40b3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:06.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.967+0000 7f2fd8a19640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd4149400 con 0x7f2fd4158b80 2023-10-18T10:47:06.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.967+0000 7f2fd8a19640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd41487f0 con 0x7f2fd40cc3d0 2023-10-18T10:47:06.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.967+0000 7f2fd8a19640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd4148ad0 con 0x7f2fd4071810 2023-10-18T10:47:06.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.967+0000 7f2fd2d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fd4158b80 0x7f2fd40b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:06.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.967+0000 7f2fd2d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fd4158b80 0x7f2fd40b3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59894/0 (socket says 172.21.15.89:59894) 2023-10-18T10:47:06.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.967+0000 7f2fd2d9d640 1 -- 172.21.15.89:0/681042746 learned_addr learned my addr 172.21.15.89:0/681042746 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:06.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.967+0000 7f2fd259c640 1 --2- 172.21.15.89:0/681042746 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fd4071810 0x7f2fd41f53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:06.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.967+0000 7f2fd1d9b640 1 --2- 172.21.15.89:0/681042746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fd40cc3d0 0x7f2fd41f58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:06.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.967+0000 7f2fd2d9d640 1 -- 172.21.15.89:0/681042746 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fd4071810 msgr2=0x7f2fd41f53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:06.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.967+0000 7f2fd2d9d640 1 --2- 172.21.15.89:0/681042746 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fd4071810 0x7f2fd41f53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:06.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.967+0000 7f2fd2d9d640 1 -- 172.21.15.89:0/681042746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fd40cc3d0 msgr2=0x7f2fd41f58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:06.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.967+0000 7f2fd2d9d640 1 --2- 172.21.15.89:0/681042746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fd40cc3d0 0x7f2fd41f58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:06.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.967+0000 7f2fd2d9d640 1 -- 172.21.15.89:0/681042746 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc80094d0 con 0x7f2fd4158b80 2023-10-18T10:47:06.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.967+0000 7f2fd2d9d640 1 --2- 172.21.15.89:0/681042746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fd4158b80 0x7f2fd40b3250 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc007db0 tx=0x7f2fcc00d490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:06.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.967+0000 7f2fc2ffd640 1 -- 172.21.15.89:0/681042746 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc017070 con 0x7f2fd4158b80 2023-10-18T10:47:06.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.967+0000 7f2fc2ffd640 1 -- 172.21.15.89:0/681042746 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fcc00a2e0 con 0x7f2fd4158b80 2023-10-18T10:47:06.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.967+0000 7f2fc2ffd640 1 -- 172.21.15.89:0/681042746 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc00f040 con 0x7f2fd4158b80 2023-10-18T10:47:06.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.967+0000 7f2fd8a19640 1 -- 172.21.15.89:0/681042746 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd4148db0 con 0x7f2fd4158b80 2023-10-18T10:47:06.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.967+0000 7f2fd8a19640 1 -- 172.21.15.89:0/681042746 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd41482f0 con 0x7f2fd4158b80 2023-10-18T10:47:06.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.967+0000 7f2fd8a19640 1 -- 172.21.15.89:0/681042746 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd4149570 con 0x7f2fd4158b80 2023-10-18T10:47:06.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.967+0000 7f2fc2ffd640 1 -- 172.21.15.89:0/681042746 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f2fcc008440 con 0x7f2fd4158b80 2023-10-18T10:47:06.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.971+0000 7f2fc2ffd640 1 --2- 172.21.15.89:0/681042746 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2fb4065680 0x7f2fb4067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:06.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.971+0000 7f2fc2ffd640 1 -- 172.21.15.89:0/681042746 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6047+0+0 (secure 0 0 0) 0x7f2fcc082a20 con 0x7f2fd4158b80 2023-10-18T10:47:06.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.971+0000 7f2fd259c640 1 --2- 172.21.15.89:0/681042746 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2fb4065680 0x7f2fb4067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:06.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.971+0000 7f2fd259c640 1 --2- 172.21.15.89:0/681042746 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2fb4065680 0x7f2fb4067b40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4009a40 tx=0x7f2fc4009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:06.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:06.971+0000 7f2fc2ffd640 1 -- 172.21.15.89:0/681042746 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f2fcc054030 con 0x7f2fd4158b80 2023-10-18T10:47:07.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:07.123+0000 7f2fd8a19640 1 -- 172.21.15.89:0/681042746 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2fd404dbe0 con 0x7f2fd4158b80 2023-10-18T10:47:07.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:07.775+0000 7f2fc2ffd640 1 -- 172.21.15.89:0/681042746 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v33) v1 ==== 178+0+0 (secure 0 0 0) 0x7f2fcc053bc0 con 0x7f2fd4158b80 2023-10-18T10:47:07.779 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data_ec' removed 2023-10-18T10:47:07.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:07.779+0000 7f2fd8a19640 1 -- 172.21.15.89:0/681042746 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2fb4065680 msgr2=0x7f2fb4067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:07.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:07.779+0000 7f2fd8a19640 1 --2- 172.21.15.89:0/681042746 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2fb4065680 0x7f2fb4067b40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4009a40 tx=0x7f2fc4009190 comp rx=0 tx=0).stop 2023-10-18T10:47:07.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:07.779+0000 7f2fd8a19640 1 -- 172.21.15.89:0/681042746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fd4158b80 msgr2=0x7f2fd40b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:07.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:07.779+0000 7f2fd8a19640 1 --2- 172.21.15.89:0/681042746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fd4158b80 0x7f2fd40b3250 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc007db0 tx=0x7f2fcc00d490 comp rx=0 tx=0).stop 2023-10-18T10:47:07.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:07.779+0000 7f2fd8a19640 1 -- 172.21.15.89:0/681042746 shutdown_connections 2023-10-18T10:47:07.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:07.779+0000 7f2fd8a19640 1 --2- 172.21.15.89:0/681042746 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2fb4065680 0x7f2fb4067b40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:07.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:07.779+0000 7f2fd8a19640 1 --2- 172.21.15.89:0/681042746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fd4158b80 0x7f2fd40b3250 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:07.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:07.779+0000 7f2fd8a19640 1 --2- 172.21.15.89:0/681042746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fd40cc3d0 0x7f2fd41f58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:07.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:07.779+0000 7f2fd8a19640 1 --2- 172.21.15.89:0/681042746 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2fd4071810 0x7f2fd41f53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:07.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:07.779+0000 7f2fd8a19640 1 -- 172.21.15.89:0/681042746 >> 172.21.15.89:0/681042746 conn(0x7f2fd40cad40 msgr2=0x7f2fd40cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:07.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:07.779+0000 7f2fd8a19640 1 -- 172.21.15.89:0/681042746 shutdown_connections 2023-10-18T10:47:07.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:07.779+0000 7f2fd8a19640 1 -- 172.21.15.89:0/681042746 wait complete. 2023-10-18T10:47:07.805 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-18T10:47:07.805 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-18T10:47:07.806 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-18T10:47:07.806 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:47:07.806 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-18T10:47:07.806 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-18T10:47:07.807 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-18T10:47:07.807 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:47:07.807 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-18T10:47:07.808 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-18T10:47:07.808 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-18T10:47:07.808 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:47:07.808 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-18T10:47:07.809 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-18T10:47:07.809 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-18T10:47:07.809 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:47:07.820 INFO:tasks.ceph.mds.a:Stopped 2023-10-18T10:47:07.820 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-18T10:47:07.821 INFO:tasks.ceph.mds.c:Stopped 2023-10-18T10:47:07.822 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-18T10:47:07.823 INFO:tasks.ceph.mds.b:Stopped 2023-10-18T10:47:07.823 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-18T10:47:07.824 INFO:tasks.ceph.mds.d:Stopped 2023-10-18T10:47:07.824 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-18T10:47:07.868 INFO:tasks.ceph.mds.b:Started 2023-10-18T10:47:07.868 DEBUG:teuthology.parallel:result is None 2023-10-18T10:47:07.869 INFO:tasks.ceph.mds.a:Started 2023-10-18T10:47:07.870 DEBUG:teuthology.parallel:result is None 2023-10-18T10:47:07.871 INFO:tasks.ceph.mds.d:Started 2023-10-18T10:47:07.871 DEBUG:teuthology.parallel:result is None 2023-10-18T10:47:07.872 INFO:tasks.ceph.mds.c:Started 2023-10-18T10:47:07.873 DEBUG:teuthology.parallel:result is None 2023-10-18T10:47:07.873 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-18T10:47:07.955 INFO:tasks.ceph.mds.b.smithi154.stdout:starting mds.b at 2023-10-18T10:47:07.956 INFO:tasks.ceph.mds.d.smithi154.stdout:starting mds.d at 2023-10-18T10:47:07.956 INFO:tasks.ceph.mds.d.smithi154.stderr:2023-10-18T10:47:07.952+0000 7f50bb546640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-18T10:47:07.959 INFO:tasks.ceph.mds.a.smithi089.stdout:starting mds.a at 2023-10-18T10:47:07.961 INFO:tasks.ceph.mds.c.smithi089.stdout:starting mds.c at 2023-10-18T10:47:08.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.031+0000 7fc3f3d03640 1 Processor -- start 2023-10-18T10:47:08.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.031+0000 7fc3f3d03640 1 -- start start 2023-10-18T10:47:08.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.031+0000 7fc3f3d03640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc3ec15ca60 0x7fc3ec0ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:08.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.031+0000 7fc3f3d03640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc3ec0b17f0 0x7fc3ec0d04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:08.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.031+0000 7fc3f3d03640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3ec0d0e50 con 0x7fc3ec0d0a00 2023-10-18T10:47:08.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.031+0000 7fc3f3d03640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3ec0d1340 con 0x7fc3ec0b17f0 2023-10-18T10:47:08.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.031+0000 7fc3f3d03640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3ec15ce40 con 0x7fc3ec15ca60 2023-10-18T10:47:08.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.031+0000 7fc3f129e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc3ec0b17f0 0x7fc3ec0d04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:08.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.031+0000 7fc3f129e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc3ec0b17f0 0x7fc3ec0d04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59936/0 (socket says 172.21.15.89:59936) 2023-10-18T10:47:08.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.031+0000 7fc3f129e640 1 -- 172.21.15.89:0/1239005041 learned_addr learned my addr 172.21.15.89:0/1239005041 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:08.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.031+0000 7fc3f1a9f640 1 --2- 172.21.15.89:0/1239005041 >> v2:172.21.15.154:3300/0 conn(0x7fc3ec15ca60 0x7fc3ec0ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:08.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.031+0000 7fc3f129e640 1 -- 172.21.15.89:0/1239005041 >> v2:172.21.15.154:3300/0 conn(0x7fc3ec15ca60 msgr2=0x7fc3ec0ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:08.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.031+0000 7fc3f129e640 1 --2- 172.21.15.89:0/1239005041 >> v2:172.21.15.154:3300/0 conn(0x7fc3ec15ca60 0x7fc3ec0ccec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:08.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.031+0000 7fc3f129e640 1 -- 172.21.15.89:0/1239005041 >> v1:172.21.15.154:6789/0 conn(0x7fc3ec0d0a00 legacy=0x7fc3ec0cf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:08.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.031+0000 7fc3f129e640 1 -- 172.21.15.89:0/1239005041 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3ec0cd730 con 0x7fc3ec0b17f0 2023-10-18T10:47:08.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.031+0000 7fc3f1a9f640 1 --2- 172.21.15.89:0/1239005041 >> v2:172.21.15.154:3300/0 conn(0x7fc3ec15ca60 0x7fc3ec0ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:47:08.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.035+0000 7fc3f129e640 1 --2- 172.21.15.89:0/1239005041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3ec0b17f0 0x7fc3ec0d04c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec0b52d0 tx=0x7fc3dc01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c4b54857a07803c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:08.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.035+0000 7fc3f0a9d640 1 -- 172.21.15.89:0/1239005041 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc0ab640 con 0x7fc3ec0b17f0 2023-10-18T10:47:08.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.035+0000 7fc3f0a9d640 1 -- 172.21.15.89:0/1239005041 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3dc0a7070 con 0x7fc3ec0b17f0 2023-10-18T10:47:08.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.035+0000 7fc3f0a9d640 1 -- 172.21.15.89:0/1239005041 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc0ab9c0 con 0x7fc3ec0b17f0 2023-10-18T10:47:08.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.035+0000 7fc3f3d03640 1 -- 172.21.15.89:0/1239005041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3ec0b17f0 msgr2=0x7fc3ec0d04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:08.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.035+0000 7fc3f3d03640 1 --2- 172.21.15.89:0/1239005041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3ec0b17f0 0x7fc3ec0d04c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec0b52d0 tx=0x7fc3dc01c440 comp rx=0 tx=0).stop 2023-10-18T10:47:08.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.035+0000 7fc3f3d03640 1 -- 172.21.15.89:0/1239005041 shutdown_connections 2023-10-18T10:47:08.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.035+0000 7fc3f3d03640 1 --2- 172.21.15.89:0/1239005041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3ec0b17f0 0x7fc3ec0d04c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:08.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.035+0000 7fc3f3d03640 1 --2- 172.21.15.89:0/1239005041 >> v2:172.21.15.154:3300/0 conn(0x7fc3ec15ca60 0x7fc3ec0ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:08.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.035+0000 7fc3f3d03640 1 -- 172.21.15.89:0/1239005041 >> 172.21.15.89:0/1239005041 conn(0x7fc3ec09b990 msgr2=0x7fc3ec0d26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:08.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.035+0000 7fc3f3d03640 1 -- 172.21.15.89:0/1239005041 shutdown_connections 2023-10-18T10:47:08.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.035+0000 7fc3f3d03640 1 -- 172.21.15.89:0/1239005041 wait complete. 2023-10-18T10:47:08.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.039+0000 7fc3f3d03640 1 Processor -- start 2023-10-18T10:47:08.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.039+0000 7fc3f3d03640 1 -- start start 2023-10-18T10:47:08.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.039+0000 7fc3f3d03640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3ec0b17f0 0x7fc3ec0c7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:08.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.039+0000 7fc3f3d03640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc3ec0d0a00 0x7fc3ec0b7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:08.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.039+0000 7fc3f3d03640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3ec15ca60 0x7fc3ec1f82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:08.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.039+0000 7fc3f3d03640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec0b7560 con 0x7fc3ec15ca60 2023-10-18T10:47:08.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.039+0000 7fc3f3d03640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec156150 con 0x7fc3ec0b17f0 2023-10-18T10:47:08.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.039+0000 7fc3f3d03640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec0b77e0 con 0x7fc3ec0d0a00 2023-10-18T10:47:08.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.039+0000 7fc3f22a0640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3ec15ca60 0x7fc3ec1f82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:08.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.039+0000 7fc3f129e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc3ec0d0a00 0x7fc3ec0b7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:08.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.039+0000 7fc3f22a0640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3ec15ca60 0x7fc3ec1f82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59952/0 (socket says 172.21.15.89:59952) 2023-10-18T10:47:08.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.039+0000 7fc3f22a0640 1 -- 172.21.15.89:0/1347762550 learned_addr learned my addr 172.21.15.89:0/1347762550 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:08.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.039+0000 7fc3f1a9f640 1 --2- 172.21.15.89:0/1347762550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3ec0b17f0 0x7fc3ec0c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:08.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.039+0000 7fc3f22a0640 1 -- 172.21.15.89:0/1347762550 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc3ec0d0a00 msgr2=0x7fc3ec0b7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:08.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.039+0000 7fc3f22a0640 1 --2- 172.21.15.89:0/1347762550 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc3ec0d0a00 0x7fc3ec0b7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:08.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.039+0000 7fc3f22a0640 1 -- 172.21.15.89:0/1347762550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3ec0b17f0 msgr2=0x7fc3ec0c7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:08.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.039+0000 7fc3f22a0640 1 --2- 172.21.15.89:0/1347762550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3ec0b17f0 0x7fc3ec0c7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:08.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.039+0000 7fc3f22a0640 1 -- 172.21.15.89:0/1347762550 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3dc0094d0 con 0x7fc3ec15ca60 2023-10-18T10:47:08.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.039+0000 7fc3f22a0640 1 --2- 172.21.15.89:0/1347762550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3ec15ca60 0x7fc3ec1f82c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc3e400c770 tx=0x7fc3e400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:08.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.039+0000 7fc3e27fc640 1 -- 172.21.15.89:0/1347762550 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4003b80 con 0x7fc3ec15ca60 2023-10-18T10:47:08.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.039+0000 7fc3e27fc640 1 -- 172.21.15.89:0/1347762550 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e4003ce0 con 0x7fc3ec15ca60 2023-10-18T10:47:08.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.039+0000 7fc3e27fc640 1 -- 172.21.15.89:0/1347762550 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e400aa90 con 0x7fc3ec15ca60 2023-10-18T10:47:08.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.039+0000 7fc3f3d03640 1 -- 172.21.15.89:0/1347762550 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3ec0b7ac0 con 0x7fc3ec15ca60 2023-10-18T10:47:08.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.039+0000 7fc3f3d03640 1 -- 172.21.15.89:0/1347762550 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3ec0b8080 con 0x7fc3ec15ca60 2023-10-18T10:47:08.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.039+0000 7fc3f3d03640 1 -- 172.21.15.89:0/1347762550 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3ec1563b0 con 0x7fc3ec15ca60 2023-10-18T10:47:08.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.043+0000 7fc3e27fc640 1 -- 172.21.15.89:0/1347762550 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fc3e400abf0 con 0x7fc3ec15ca60 2023-10-18T10:47:08.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.043+0000 7fc3e27fc640 1 --2- 172.21.15.89:0/1347762550 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc3d0065680 0x7fc3d0067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:08.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.043+0000 7fc3e27fc640 1 -- 172.21.15.89:0/1347762550 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fc3e4082830 con 0x7fc3ec15ca60 2023-10-18T10:47:08.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.043+0000 7fc3f1a9f640 1 --2- 172.21.15.89:0/1347762550 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc3d0065680 0x7fc3d0067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:08.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.043+0000 7fc3f1a9f640 1 --2- 172.21.15.89:0/1347762550 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc3d0065680 0x7fc3d0067b40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8006fd0 tx=0x7fc3e8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:08.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.047+0000 7fc3e27fc640 1 -- 172.21.15.89:0/1347762550 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fc3e4050560 con 0x7fc3ec15ca60 2023-10-18T10:47:08.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.207+0000 7fc3f3d03640 1 -- 172.21.15.89:0/1347762550 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc3ec0d0e50 con 0x7fc3ec15ca60 2023-10-18T10:47:08.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.787+0000 7fc3e27fc640 1 -- 172.21.15.89:0/1347762550 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v35) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc3ec0d0e50 con 0x7fc3ec15ca60 2023-10-18T10:47:08.789 INFO:teuthology.orchestra.run.smithi089.stderr: removed all blocklist entries 2023-10-18T10:47:08.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.787+0000 7fc3f3d03640 1 -- 172.21.15.89:0/1347762550 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc3d0065680 msgr2=0x7fc3d0067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:08.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.787+0000 7fc3f3d03640 1 --2- 172.21.15.89:0/1347762550 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc3d0065680 0x7fc3d0067b40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8006fd0 tx=0x7fc3e8008040 comp rx=0 tx=0).stop 2023-10-18T10:47:08.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.787+0000 7fc3f3d03640 1 -- 172.21.15.89:0/1347762550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3ec15ca60 msgr2=0x7fc3ec1f82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:08.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.787+0000 7fc3f3d03640 1 --2- 172.21.15.89:0/1347762550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3ec15ca60 0x7fc3ec1f82c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc3e400c770 tx=0x7fc3e400cc30 comp rx=0 tx=0).stop 2023-10-18T10:47:08.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.787+0000 7fc3f3d03640 1 -- 172.21.15.89:0/1347762550 shutdown_connections 2023-10-18T10:47:08.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.787+0000 7fc3f3d03640 1 --2- 172.21.15.89:0/1347762550 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc3d0065680 0x7fc3d0067b40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:08.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.787+0000 7fc3f3d03640 1 --2- 172.21.15.89:0/1347762550 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc3ec0d0a00 0x7fc3ec0b7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:08.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.787+0000 7fc3f3d03640 1 --2- 172.21.15.89:0/1347762550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3ec15ca60 0x7fc3ec1f82c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:08.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.787+0000 7fc3f3d03640 1 --2- 172.21.15.89:0/1347762550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3ec0b17f0 0x7fc3ec0c7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:08.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.787+0000 7fc3f3d03640 1 -- 172.21.15.89:0/1347762550 >> 172.21.15.89:0/1347762550 conn(0x7fc3ec09b990 msgr2=0x7fc3ec1591c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:08.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.787+0000 7fc3f3d03640 1 -- 172.21.15.89:0/1347762550 shutdown_connections 2023-10-18T10:47:08.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.787+0000 7fc3f3d03640 1 -- 172.21.15.89:0/1347762550 wait complete. 2023-10-18T10:47:08.814 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-18T10:47:08.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.947+0000 7f5fdbf00640 1 Processor -- start 2023-10-18T10:47:08.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.947+0000 7f5fdbf00640 1 -- start start 2023-10-18T10:47:08.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.951+0000 7f5fdbf00640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5fd40c9970 0x7f5fd4164230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:08.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.951+0000 7f5fdbf00640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fd4070b90 con 0x7f5fd4159490 2023-10-18T10:47:08.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.951+0000 7f5fdbf00640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fd4070e70 con 0x7f5fd4070600 2023-10-18T10:47:08.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.951+0000 7f5fdbf00640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd4071150 con 0x7f5fd40c9970 2023-10-18T10:47:08.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.951+0000 7f5fd9c9c640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5fd4159490 0x7f5fd415d8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37492/0 (socket says 172.21.15.89:37492) 2023-10-18T10:47:08.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.951+0000 7f5fd9c9c640 1 -- 172.21.15.89:0/448169530 learned_addr learned my addr 172.21.15.89:0/448169530 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:47:08.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.951+0000 7f5fd8c9a640 1 -- 172.21.15.89:0/448169530 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3556390515 0 0) 0x7f5fd4070b90 con 0x7f5fd4159490 2023-10-18T10:47:08.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.951+0000 7f5fd8c9a640 1 -- 172.21.15.89:0/448169530 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fc0003530 con 0x7f5fd4159490 2023-10-18T10:47:08.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.951+0000 7f5fda49d640 1 --2- 172.21.15.89:0/448169530 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5fd40c9970 0x7f5fd4164230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:08.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.951+0000 7f5fd8c9a640 1 -- 172.21.15.89:0/448169530 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 958459078 0 0) 0x7f5fc0003530 con 0x7f5fd4159490 2023-10-18T10:47:08.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.951+0000 7f5fd8c9a640 1 -- 172.21.15.89:0/448169530 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5fd40c9970 msgr2=0x7f5fd4164230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:08.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.951+0000 7f5fd8c9a640 1 --2- 172.21.15.89:0/448169530 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5fd40c9970 0x7f5fd4164230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:08.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.951+0000 7f5fd8c9a640 1 -- 172.21.15.89:0/448169530 >> v1:172.21.15.154:6789/0 conn(0x7f5fd4070600 legacy=0x7f5fd406b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:08.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.951+0000 7f5fd8c9a640 1 -- 172.21.15.89:0/448169530 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd4071430 con 0x7f5fd4159490 2023-10-18T10:47:08.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.951+0000 7f5fd8c9a640 1 -- 172.21.15.89:0/448169530 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7f5fd0001a00 con 0x7f5fd4159490 2023-10-18T10:47:08.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.951+0000 7f5fd8c9a640 1 -- 172.21.15.89:0/448169530 >> v1:172.21.15.89:6789/0 conn(0x7f5fd4159490 legacy=0x7f5fd415d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:08.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.951+0000 7f5fd8c9a640 1 --2- 172.21.15.89:0/448169530 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5fd40c9970 0x7f5fc00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:08.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.951+0000 7f5fd8c9a640 1 --2- 172.21.15.89:0/448169530 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fc0003fe0 0x7f5fc00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:08.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.951+0000 7f5fd8c9a640 1 --2- 172.21.15.89:0/448169530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fc0004d10 0x7f5fc0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:08.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.951+0000 7f5fd8c9a640 1 -- 172.21.15.89:0/448169530 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd4071150 con 0x7f5fc0004d10 2023-10-18T10:47:08.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.951+0000 7f5fd8c9a640 1 -- 172.21.15.89:0/448169530 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc000bbd0 con 0x7f5fc0003fe0 2023-10-18T10:47:08.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.951+0000 7f5fd8c9a640 1 -- 172.21.15.89:0/448169530 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc000bd90 con 0x7f5fd40c9970 2023-10-18T10:47:08.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.951+0000 7f5fda49d640 1 --2- 172.21.15.89:0/448169530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fc0004d10 0x7f5fc0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:08.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.951+0000 7f5fd9c9c640 1 --2- 172.21.15.89:0/448169530 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5fd40c9970 0x7f5fc00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:08.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.951+0000 7f5fd949b640 1 --2- 172.21.15.89:0/448169530 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fc0003fe0 0x7f5fc00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:08.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.951+0000 7f5fda49d640 1 -- 172.21.15.89:0/448169530 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5fd40c9970 msgr2=0x7f5fc00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:08.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.951+0000 7f5fda49d640 1 --2- 172.21.15.89:0/448169530 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5fd40c9970 0x7f5fc00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:08.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.951+0000 7f5fda49d640 1 -- 172.21.15.89:0/448169530 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fc0003fe0 msgr2=0x7f5fc00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:08.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.951+0000 7f5fda49d640 1 --2- 172.21.15.89:0/448169530 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fc0003fe0 0x7f5fc00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:08.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.951+0000 7f5fda49d640 1 -- 172.21.15.89:0/448169530 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc0003530 con 0x7f5fc0004d10 2023-10-18T10:47:08.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.955+0000 7f5fda49d640 1 --2- 172.21.15.89:0/448169530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fc0004d10 0x7f5fc0005190 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc00ba30 tx=0x7f5fcc01c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=742e17b2ab74f083 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:08.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.955+0000 7f5fd8c9a640 1 -- 172.21.15.89:0/448169530 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc0ac620 con 0x7f5fc0004d10 2023-10-18T10:47:08.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.955+0000 7f5fd8c9a640 1 -- 172.21.15.89:0/448169530 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fcc0a6040 con 0x7f5fc0004d10 2023-10-18T10:47:08.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.955+0000 7f5fd8c9a640 1 -- 172.21.15.89:0/448169530 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc0ac9c0 con 0x7f5fc0004d10 2023-10-18T10:47:08.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.955+0000 7f5fdbf00640 1 -- 172.21.15.89:0/448169530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fc0004d10 msgr2=0x7f5fc0005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:08.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.955+0000 7f5fdbf00640 1 --2- 172.21.15.89:0/448169530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fc0004d10 0x7f5fc0005190 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc00ba30 tx=0x7f5fcc01c680 comp rx=0 tx=0).stop 2023-10-18T10:47:08.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.955+0000 7f5fda49d640 1 -- 172.21.15.89:0/448169530 reap_dead start 2023-10-18T10:47:08.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.955+0000 7f5fdbf00640 1 -- 172.21.15.89:0/448169530 shutdown_connections 2023-10-18T10:47:08.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.955+0000 7f5fdbf00640 1 -- 172.21.15.89:0/448169530 >> 172.21.15.89:0/448169530 conn(0x7f5fd4069f00 msgr2=0x7f5fd40cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:08.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.955+0000 7f5fdbf00640 1 -- 172.21.15.89:0/448169530 shutdown_connections 2023-10-18T10:47:08.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.955+0000 7f5fdbf00640 1 -- 172.21.15.89:0/448169530 wait complete. 2023-10-18T10:47:08.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.959+0000 7f5fdbf00640 1 Processor -- start 2023-10-18T10:47:08.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.959+0000 7f5fdbf00640 1 -- start start 2023-10-18T10:47:08.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.959+0000 7f5fdbf00640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5fc0004d10 0x7f5fd40b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:08.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.959+0000 7f5fdbf00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fd4157930 0x7f5fd41e5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:08.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.959+0000 7f5fdbf00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fd40c4320 0x7f5fd40c47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:08.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.959+0000 7f5fdbf00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd40c4d10 con 0x7f5fd4157930 2023-10-18T10:47:08.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.959+0000 7f5fdbf00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd4157db0 con 0x7f5fd40c4320 2023-10-18T10:47:08.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.959+0000 7f5fdbf00640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd40aef90 con 0x7f5fc0004d10 2023-10-18T10:47:08.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.959+0000 7f5fd949b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fd4157930 0x7f5fd41e5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:08.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.959+0000 7f5fd9c9c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5fc0004d10 0x7f5fd40b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:08.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.959+0000 7f5fd949b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fd4157930 0x7f5fd41e5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59970/0 (socket says 172.21.15.89:59970) 2023-10-18T10:47:08.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.959+0000 7f5fd949b640 1 -- 172.21.15.89:0/768887820 learned_addr learned my addr 172.21.15.89:0/768887820 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:08.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.959+0000 7f5fda49d640 1 --2- 172.21.15.89:0/768887820 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fd40c4320 0x7f5fd40c47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:08.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.959+0000 7f5fd949b640 1 -- 172.21.15.89:0/768887820 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5fc0004d10 msgr2=0x7f5fd40b0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:08.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.959+0000 7f5fd949b640 1 --2- 172.21.15.89:0/768887820 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5fc0004d10 0x7f5fd40b0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:08.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.959+0000 7f5fd949b640 1 -- 172.21.15.89:0/768887820 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fd40c4320 msgr2=0x7f5fd40c47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:08.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.959+0000 7f5fd949b640 1 --2- 172.21.15.89:0/768887820 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fd40c4320 0x7f5fd40c47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:08.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.959+0000 7f5fd949b640 1 -- 172.21.15.89:0/768887820 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd40af1e0 con 0x7f5fd4157930 2023-10-18T10:47:08.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.959+0000 7f5fd949b640 1 --2- 172.21.15.89:0/768887820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fd4157930 0x7f5fd41e5ae0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5fc400f7f0 tx=0x7f5fc400fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:08.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.959+0000 7f5fca7fc640 1 -- 172.21.15.89:0/768887820 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc40050d0 con 0x7f5fd4157930 2023-10-18T10:47:08.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.959+0000 7f5fca7fc640 1 -- 172.21.15.89:0/768887820 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc4005230 con 0x7f5fd4157930 2023-10-18T10:47:08.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.959+0000 7f5fdbf00640 1 -- 172.21.15.89:0/768887820 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd40af4c0 con 0x7f5fd4157930 2023-10-18T10:47:08.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.959+0000 7f5fca7fc640 1 -- 172.21.15.89:0/768887820 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc400d6d0 con 0x7f5fd4157930 2023-10-18T10:47:08.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.959+0000 7f5fdbf00640 1 -- 172.21.15.89:0/768887820 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd40afa80 con 0x7f5fd4157930 2023-10-18T10:47:08.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.963+0000 7f5fdbf00640 1 -- 172.21.15.89:0/768887820 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd404dbe0 con 0x7f5fd4157930 2023-10-18T10:47:08.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.963+0000 7f5fca7fc640 1 -- 172.21.15.89:0/768887820 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f5fc400d830 con 0x7f5fd4157930 2023-10-18T10:47:08.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.963+0000 7f5fca7fc640 1 --2- 172.21.15.89:0/768887820 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5fb80656b0 0x7f5fb8067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:08.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.963+0000 7f5fca7fc640 1 -- 172.21.15.89:0/768887820 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f5fc40829a0 con 0x7f5fd4157930 2023-10-18T10:47:08.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.963+0000 7f5fd9c9c640 1 --2- 172.21.15.89:0/768887820 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5fb80656b0 0x7f5fb8067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:08.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.963+0000 7f5fd9c9c640 1 --2- 172.21.15.89:0/768887820 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5fb80656b0 0x7f5fb8067b70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5fd40ce930 tx=0x7f5fd0001800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:08.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:08.967+0000 7f5fca7fc640 1 -- 172.21.15.89:0/768887820 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f5fc4050710 con 0x7f5fd4157930 2023-10-18T10:47:09.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.163+0000 7f5fdbf00640 1 -- 172.21.15.89:0/768887820 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5fd40cbb60 con 0x7f5fd4157930 2023-10-18T10:47:09.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.167+0000 7f5fca7fc640 1 -- 172.21.15.89:0/768887820 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f5fc4053d30 con 0x7f5fd4157930 2023-10-18T10:47:09.168 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:47:09.169 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-18T10:47:09.169 INFO:teuthology.orchestra.run.smithi089.stdout: "auth_dump": [ 2023-10-18T10:47:09.169 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-18T10:47:09.170 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.a", 2023-10-18T10:47:09.170 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAity9l3YjmDhAApyQSVPKVH9p2kMmY8L12sw==", 2023-10-18T10:47:09.170 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-18T10:47:09.170 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-18T10:47:09.171 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-18T10:47:09.171 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-18T10:47:09.171 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-18T10:47:09.171 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-18T10:47:09.172 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-18T10:47:09.172 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-18T10:47:09.172 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.b", 2023-10-18T10:47:09.172 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAity9llE78GhAAZCzWHVS+Cy8PoENP2E7RbQ==", 2023-10-18T10:47:09.173 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-18T10:47:09.173 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-18T10:47:09.173 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-18T10:47:09.173 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-18T10:47:09.173 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-18T10:47:09.174 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-18T10:47:09.174 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-18T10:47:09.176 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-18T10:47:09.176 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.c", 2023-10-18T10:47:09.176 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAity9lJQIVFhAAqPlG3bmFHpxQZlY4J0Kk3Q==", 2023-10-18T10:47:09.177 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-18T10:47:09.177 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-18T10:47:09.177 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-18T10:47:09.177 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-18T10:47:09.177 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-18T10:47:09.178 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-18T10:47:09.178 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-18T10:47:09.178 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-18T10:47:09.178 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mds.d", 2023-10-18T10:47:09.178 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAity9laNByIhAAa8uuDEq77DKA67YeY+7a4A==", 2023-10-18T10:47:09.179 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-18T10:47:09.179 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-18T10:47:09.179 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-18T10:47:09.179 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-18T10:47:09.180 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-18T10:47:09.180 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-18T10:47:09.180 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-18T10:47:09.180 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-18T10:47:09.181 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.0", 2023-10-18T10:47:09.181 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAmty9lMm5nARAA0jHQUL2hduxEzO1ImPmP1g==", 2023-10-18T10:47:09.181 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-18T10:47:09.181 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-18T10:47:09.181 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-18T10:47:09.182 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-18T10:47:09.182 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-18T10:47:09.183 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-18T10:47:09.183 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-18T10:47:09.183 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.1", 2023-10-18T10:47:09.183 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQApty9lN0ZtChAAmkzQ69dcnGUj1jCMv/tksQ==", 2023-10-18T10:47:09.184 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-18T10:47:09.184 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-18T10:47:09.184 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-18T10:47:09.184 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-18T10:47:09.184 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-18T10:47:09.185 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-18T10:47:09.185 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-18T10:47:09.185 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.2", 2023-10-18T10:47:09.185 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAsty9l07p8EBAAPJVssCSLmw5sG9oM2OzJSQ==", 2023-10-18T10:47:09.186 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-18T10:47:09.186 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-18T10:47:09.186 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-18T10:47:09.186 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-18T10:47:09.186 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-18T10:47:09.187 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-18T10:47:09.187 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-18T10:47:09.187 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.3", 2023-10-18T10:47:09.187 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAvty9lnCP3GBAAfRyC++rI5Ha+iVN0wSMcNQ==", 2023-10-18T10:47:09.187 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-18T10:47:09.188 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-18T10:47:09.188 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-18T10:47:09.188 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-18T10:47:09.188 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-18T10:47:09.188 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-18T10:47:09.189 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-18T10:47:09.189 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.4", 2023-10-18T10:47:09.189 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA1ty9lwM0aHxAAXuBTefDP1w4rxukHFgqCqQ==", 2023-10-18T10:47:09.189 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-18T10:47:09.189 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-18T10:47:09.190 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-18T10:47:09.190 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-18T10:47:09.190 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-18T10:47:09.190 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-18T10:47:09.190 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-18T10:47:09.191 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.5", 2023-10-18T10:47:09.191 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA4ty9lalrVJhAAHaF+tMmtF8KN9Z4eFZrI8A==", 2023-10-18T10:47:09.191 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-18T10:47:09.191 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-18T10:47:09.191 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-18T10:47:09.191 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-18T10:47:09.192 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-18T10:47:09.192 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-18T10:47:09.192 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-18T10:47:09.192 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.6", 2023-10-18T10:47:09.192 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA7ty9lZT6iLRAApQbbomgjYQZrjfG0Jbj7dg==", 2023-10-18T10:47:09.193 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-18T10:47:09.193 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-18T10:47:09.193 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-18T10:47:09.193 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-18T10:47:09.193 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-18T10:47:09.194 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-18T10:47:09.194 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-18T10:47:09.194 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "osd.7", 2023-10-18T10:47:09.194 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQA+ty9lZlwFNBAAEt+82Hbl4mIWuvT/L8A/OA==", 2023-10-18T10:47:09.194 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-18T10:47:09.195 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow profile osd", 2023-10-18T10:47:09.195 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile osd", 2023-10-18T10:47:09.195 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-18T10:47:09.195 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-18T10:47:09.195 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-18T10:47:09.196 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-18T10:47:09.196 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.0", 2023-10-18T10:47:09.196 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAity9lApuiJhAAUNpuF1S7ZLrsIuRG0mrVNw==", 2023-10-18T10:47:09.196 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-18T10:47:09.196 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-18T10:47:09.197 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-18T10:47:09.197 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-18T10:47:09.197 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-18T10:47:09.197 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-18T10:47:09.197 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-18T10:47:09.197 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-18T10:47:09.198 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.1", 2023-10-18T10:47:09.198 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAity9lyrouMBAAOsZT3MhcFH5guAKKbmJFuw==", 2023-10-18T10:47:09.198 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-18T10:47:09.198 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-18T10:47:09.198 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-18T10:47:09.199 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-18T10:47:09.199 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-18T10:47:09.199 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-18T10:47:09.199 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-18T10:47:09.199 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-18T10:47:09.200 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.2", 2023-10-18T10:47:09.200 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAity9l7HqvLBAAYrtttxbjU+BBFgtOl3bvjg==", 2023-10-18T10:47:09.200 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-18T10:47:09.200 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-18T10:47:09.200 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-18T10:47:09.201 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-18T10:47:09.201 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-18T10:47:09.201 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-18T10:47:09.201 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-18T10:47:09.201 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-18T10:47:09.201 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.3", 2023-10-18T10:47:09.202 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAity9l/VAfNhAA6jHkCCY8phqDIvJLuRu/Rw==", 2023-10-18T10:47:09.202 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-18T10:47:09.202 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow", 2023-10-18T10:47:09.202 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow r", 2023-10-18T10:47:09.202 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow rw", 2023-10-18T10:47:09.203 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow rwx" 2023-10-18T10:47:09.203 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-18T10:47:09.203 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-18T10:47:09.203 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-18T10:47:09.203 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.admin", 2023-10-18T10:47:09.204 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAhty9lellhKxAAGcHg+Uv43ccMjR2dD2pvSg==", 2023-10-18T10:47:09.204 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-18T10:47:09.204 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-18T10:47:09.204 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": "allow *", 2023-10-18T10:47:09.204 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow *", 2023-10-18T10:47:09.205 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-18T10:47:09.205 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-18T10:47:09.205 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-18T10:47:09.205 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-18T10:47:09.205 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.bootstrap-mds", 2023-10-18T10:47:09.206 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBJty9l3cXMOBAAinX7K/C3CjxmCeCZe9Ptqg==", 2023-10-18T10:47:09.206 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-18T10:47:09.206 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile bootstrap-mds" 2023-10-18T10:47:09.206 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-18T10:47:09.206 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-18T10:47:09.206 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-18T10:47:09.207 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.bootstrap-mgr", 2023-10-18T10:47:09.207 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBJty9lONHMOBAA6qqdSS0h5qRJkwcHT7QQvw==", 2023-10-18T10:47:09.207 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-18T10:47:09.207 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-18T10:47:09.207 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-18T10:47:09.208 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-18T10:47:09.208 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-18T10:47:09.208 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.bootstrap-osd", 2023-10-18T10:47:09.208 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBJty9lLCvNOBAA6vX2QXP+800Ax9amta4wrg==", 2023-10-18T10:47:09.208 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-18T10:47:09.209 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile bootstrap-osd" 2023-10-18T10:47:09.209 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-18T10:47:09.209 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-18T10:47:09.209 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-18T10:47:09.209 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.bootstrap-rbd", 2023-10-18T10:47:09.209 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBJty9lATjNOBAA1WhuCU7DHnKWEXVjC1c6GQ==", 2023-10-18T10:47:09.210 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-18T10:47:09.210 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-18T10:47:09.210 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-18T10:47:09.210 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-18T10:47:09.210 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-18T10:47:09.211 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-18T10:47:09.211 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBJty9lNkTNOBAADwQ5wFC/NyPfmeHWZHTSSg==", 2023-10-18T10:47:09.211 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-18T10:47:09.211 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-18T10:47:09.211 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-18T10:47:09.212 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-18T10:47:09.212 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-18T10:47:09.212 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "client.bootstrap-rgw", 2023-10-18T10:47:09.212 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQBJty9l6E3NOBAAzSB0tDpp1B5Yvo38UzzfBA==", 2023-10-18T10:47:09.212 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-18T10:47:09.212 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-18T10:47:09.213 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-18T10:47:09.213 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-18T10:47:09.213 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-18T10:47:09.213 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mgr.x", 2023-10-18T10:47:09.213 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAity9liyzXBxAAuc/Y/Gay+OYIj9TFZ6q2Dw==", 2023-10-18T10:47:09.214 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-18T10:47:09.214 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-18T10:47:09.214 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile mgr", 2023-10-18T10:47:09.214 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-18T10:47:09.214 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-18T10:47:09.215 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-18T10:47:09.215 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-18T10:47:09.215 INFO:teuthology.orchestra.run.smithi089.stdout: "entity": "mgr.y", 2023-10-18T10:47:09.215 INFO:teuthology.orchestra.run.smithi089.stdout: "key": "AQAity9la0P9AxAAGS2Tg2NOUwGXRgWNRpU1Xg==", 2023-10-18T10:47:09.215 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": { 2023-10-18T10:47:09.216 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": "allow *", 2023-10-18T10:47:09.216 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": "allow profile mgr", 2023-10-18T10:47:09.216 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": "allow *" 2023-10-18T10:47:09.216 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-18T10:47:09.216 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-18T10:47:09.216 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-10-18T10:47:09.217 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-18T10:47:09.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.167+0000 7f5fdbf00640 1 -- 172.21.15.89:0/768887820 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5fb80656b0 msgr2=0x7f5fb8067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:09.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.167+0000 7f5fdbf00640 1 --2- 172.21.15.89:0/768887820 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5fb80656b0 0x7f5fb8067b70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5fd40ce930 tx=0x7f5fd0001800 comp rx=0 tx=0).stop 2023-10-18T10:47:09.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.167+0000 7f5fdbf00640 1 -- 172.21.15.89:0/768887820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fd4157930 msgr2=0x7f5fd41e5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:09.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.167+0000 7f5fdbf00640 1 --2- 172.21.15.89:0/768887820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fd4157930 0x7f5fd41e5ae0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5fc400f7f0 tx=0x7f5fc400fcb0 comp rx=0 tx=0).stop 2023-10-18T10:47:09.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.167+0000 7f5fdbf00640 1 -- 172.21.15.89:0/768887820 shutdown_connections 2023-10-18T10:47:09.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.167+0000 7f5fdbf00640 1 --2- 172.21.15.89:0/768887820 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5fb80656b0 0x7f5fb8067b70 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:09.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.167+0000 7f5fdbf00640 1 --2- 172.21.15.89:0/768887820 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fd40c4320 0x7f5fd40c47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:09.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.167+0000 7f5fdbf00640 1 --2- 172.21.15.89:0/768887820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fd4157930 0x7f5fd41e5ae0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:09.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.167+0000 7f5fdbf00640 1 --2- 172.21.15.89:0/768887820 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5fc0004d10 0x7f5fd40b0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:09.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.167+0000 7f5fdbf00640 1 -- 172.21.15.89:0/768887820 >> 172.21.15.89:0/768887820 conn(0x7f5fd4069f00 msgr2=0x7f5fd41587b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:09.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.167+0000 7f5fdbf00640 1 -- 172.21.15.89:0/768887820 shutdown_connections 2023-10-18T10:47:09.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.167+0000 7f5fdbf00640 1 -- 172.21.15.89:0/768887820 wait complete. 2023-10-18T10:47:09.221 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-18T10:47:09.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.355+0000 7fd5d945f640 1 Processor -- start 2023-10-18T10:47:09.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.355+0000 7fd5d945f640 1 -- start start 2023-10-18T10:47:09.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.359+0000 7fd5d945f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd5d415c940 0x7fd5d40d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:09.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.359+0000 7fd5d945f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5d40d0e70 0x7fd5d40cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:09.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.359+0000 7fd5d945f640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5d415da50 con 0x7fd5d40d2f40 2023-10-18T10:47:09.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.359+0000 7fd5d945f640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5d415ce40 con 0x7fd5d415c940 2023-10-18T10:47:09.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.359+0000 7fd5d945f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d415d120 con 0x7fd5d40d0e70 2023-10-18T10:47:09.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.359+0000 7fd5d27fc640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd5d40d2f40 0x7fd5d40d0760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37500/0 (socket says 172.21.15.89:37500) 2023-10-18T10:47:09.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.359+0000 7fd5d27fc640 1 -- 172.21.15.89:0/2548035793 learned_addr learned my addr 172.21.15.89:0/2548035793 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:47:09.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.359+0000 7fd5d2ffd640 1 --2- 172.21.15.89:0/2548035793 >> v2:172.21.15.89:3300/0 conn(0x7fd5d415c940 0x7fd5d40d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:09.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.359+0000 7fd5d37fe640 1 --2- 172.21.15.89:0/2548035793 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5d40d0e70 0x7fd5d40cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:09.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.359+0000 7fd5d2ffd640 1 -- 172.21.15.89:0/2548035793 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5d40d0e70 msgr2=0x7fd5d40cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:09.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.359+0000 7fd5d2ffd640 1 --2- 172.21.15.89:0/2548035793 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5d40d0e70 0x7fd5d40cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:09.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.359+0000 7fd5d2ffd640 1 -- 172.21.15.89:0/2548035793 >> v1:172.21.15.89:6789/0 conn(0x7fd5d40d2f40 legacy=0x7fd5d40d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:09.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.359+0000 7fd5d2ffd640 1 -- 172.21.15.89:0/2548035793 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d415d400 con 0x7fd5d415c940 2023-10-18T10:47:09.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.359+0000 7fd5d1ffb640 1 -- 172.21.15.89:0/2548035793 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4084861617 0 0) 0x7fd5d415da50 con 0x7fd5d40d2f40 2023-10-18T10:47:09.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.359+0000 7fd5d2ffd640 1 --2- 172.21.15.89:0/2548035793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5d415c940 0x7fd5d40d2a00 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd5c40097c0 tx=0x7fd5c401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f5575b189f7d97 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:09.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.363+0000 7fd5d1ffb640 1 -- 172.21.15.89:0/2548035793 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c40ab5f0 con 0x7fd5d415c940 2023-10-18T10:47:09.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.363+0000 7fd5d1ffb640 1 -- 172.21.15.89:0/2548035793 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5c40ab750 con 0x7fd5d415c940 2023-10-18T10:47:09.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.363+0000 7fd5d1ffb640 1 -- 172.21.15.89:0/2548035793 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c40abae0 con 0x7fd5d415c940 2023-10-18T10:47:09.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.363+0000 7fd5d945f640 1 -- 172.21.15.89:0/2548035793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5d415c940 msgr2=0x7fd5d40d2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:09.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.363+0000 7fd5d945f640 1 --2- 172.21.15.89:0/2548035793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5d415c940 0x7fd5d40d2a00 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd5c40097c0 tx=0x7fd5c401c450 comp rx=0 tx=0).stop 2023-10-18T10:47:09.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.363+0000 7fd5d945f640 1 -- 172.21.15.89:0/2548035793 shutdown_connections 2023-10-18T10:47:09.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.363+0000 7fd5d945f640 1 --2- 172.21.15.89:0/2548035793 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5d40d0e70 0x7fd5d40cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:09.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.363+0000 7fd5d945f640 1 --2- 172.21.15.89:0/2548035793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5d415c940 0x7fd5d40d2a00 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:09.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.363+0000 7fd5d945f640 1 -- 172.21.15.89:0/2548035793 >> 172.21.15.89:0/2548035793 conn(0x7fd5d4089fd0 msgr2=0x7fd5d409b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:09.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.363+0000 7fd5d945f640 1 -- 172.21.15.89:0/2548035793 shutdown_connections 2023-10-18T10:47:09.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.363+0000 7fd5d945f640 1 -- 172.21.15.89:0/2548035793 wait complete. 2023-10-18T10:47:09.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.363+0000 7fd5d945f640 1 Processor -- start 2023-10-18T10:47:09.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.363+0000 7fd5d945f640 1 -- start start 2023-10-18T10:47:09.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.363+0000 7fd5d945f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5d40d0e70 0x7fd5d40c6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:09.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.363+0000 7fd5d945f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5d4152cc0 0x7fd5d40c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:09.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.363+0000 7fd5d945f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5d4153520 0x7fd5d41539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:09.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.363+0000 7fd5d945f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d4153ee0 con 0x7fd5d40d0e70 2023-10-18T10:47:09.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.363+0000 7fd5d945f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d40c66c0 con 0x7fd5d4153520 2023-10-18T10:47:09.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.363+0000 7fd5d945f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d40b6460 con 0x7fd5d4152cc0 2023-10-18T10:47:09.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.363+0000 7fd5d2ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5d40d0e70 0x7fd5d40c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:09.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.363+0000 7fd5d2ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5d40d0e70 0x7fd5d40c6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59976/0 (socket says 172.21.15.89:59976) 2023-10-18T10:47:09.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.363+0000 7fd5d2ffd640 1 -- 172.21.15.89:0/96646945 learned_addr learned my addr 172.21.15.89:0/96646945 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:09.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.367+0000 7fd5d27fc640 1 --2- 172.21.15.89:0/96646945 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5d4152cc0 0x7fd5d40c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:09.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.367+0000 7fd5d37fe640 1 --2- 172.21.15.89:0/96646945 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5d4153520 0x7fd5d41539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:09.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.367+0000 7fd5d2ffd640 1 -- 172.21.15.89:0/96646945 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5d4152cc0 msgr2=0x7fd5d40c99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:09.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.367+0000 7fd5d2ffd640 1 --2- 172.21.15.89:0/96646945 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5d4152cc0 0x7fd5d40c99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:09.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.367+0000 7fd5d2ffd640 1 -- 172.21.15.89:0/96646945 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5d4153520 msgr2=0x7fd5d41539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:09.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.367+0000 7fd5d2ffd640 1 --2- 172.21.15.89:0/96646945 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5d4153520 0x7fd5d41539a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:09.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.367+0000 7fd5d2ffd640 1 -- 172.21.15.89:0/96646945 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c4009420 con 0x7fd5d40d0e70 2023-10-18T10:47:09.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.367+0000 7fd5d27fc640 1 --2- 172.21.15.89:0/96646945 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5d4152cc0 0x7fd5d40c99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:47:09.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.367+0000 7fd5d2ffd640 1 --2- 172.21.15.89:0/96646945 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5d40d0e70 0x7fd5d40c6180 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd5c40ad040 tx=0x7fd5c401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:09.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.367+0000 7fd5bb7fe640 1 -- 172.21.15.89:0/96646945 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c40bf610 con 0x7fd5d40d0e70 2023-10-18T10:47:09.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.367+0000 7fd5bb7fe640 1 -- 172.21.15.89:0/96646945 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5c40bf790 con 0x7fd5d40d0e70 2023-10-18T10:47:09.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.367+0000 7fd5bb7fe640 1 -- 172.21.15.89:0/96646945 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c40bfb40 con 0x7fd5d40d0e70 2023-10-18T10:47:09.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.367+0000 7fd5d945f640 1 -- 172.21.15.89:0/96646945 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5d40b6740 con 0x7fd5d40d0e70 2023-10-18T10:47:09.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.367+0000 7fd5d945f640 1 -- 172.21.15.89:0/96646945 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5d40b6d00 con 0x7fd5d40d0e70 2023-10-18T10:47:09.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.367+0000 7fd5d945f640 1 -- 172.21.15.89:0/96646945 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5d40b65a0 con 0x7fd5d40d0e70 2023-10-18T10:47:09.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.367+0000 7fd5bb7fe640 1 -- 172.21.15.89:0/96646945 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fd5c40b5400 con 0x7fd5d40d0e70 2023-10-18T10:47:09.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.367+0000 7fd5bb7fe640 1 --2- 172.21.15.89:0/96646945 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd5c00656b0 0x7fd5c0067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:09.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.367+0000 7fd5bb7fe640 1 -- 172.21.15.89:0/96646945 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fd5c40a8050 con 0x7fd5d40d0e70 2023-10-18T10:47:09.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.367+0000 7fd5d27fc640 1 --2- 172.21.15.89:0/96646945 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd5c00656b0 0x7fd5c0067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:09.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.371+0000 7fd5d27fc640 1 --2- 172.21.15.89:0/96646945 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd5c00656b0 0x7fd5c0067b70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8000f70 tx=0x7fd5c8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:09.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.371+0000 7fd5bb7fe640 1 -- 172.21.15.89:0/96646945 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fd5d40b65a0 con 0x7fd5d40d0e70 2023-10-18T10:47:09.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.571+0000 7fd5d945f640 1 -- 172.21.15.89:0/96646945 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fd5d40b65a0 con 0x7fd5d40d0e70 2023-10-18T10:47:09.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.575+0000 7fd5bb7fe640 1 -- 172.21.15.89:0/96646945 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd5d40b65a0 con 0x7fd5d40d0e70 2023-10-18T10:47:09.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.575+0000 7fd5d945f640 1 -- 172.21.15.89:0/96646945 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd5c00656b0 msgr2=0x7fd5c0067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:09.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.575+0000 7fd5d945f640 1 --2- 172.21.15.89:0/96646945 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd5c00656b0 0x7fd5c0067b70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8000f70 tx=0x7fd5c8009040 comp rx=0 tx=0).stop 2023-10-18T10:47:09.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.575+0000 7fd5d945f640 1 -- 172.21.15.89:0/96646945 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5d40d0e70 msgr2=0x7fd5d40c6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:09.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.575+0000 7fd5d945f640 1 --2- 172.21.15.89:0/96646945 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5d40d0e70 0x7fd5d40c6180 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd5c40ad040 tx=0x7fd5c401ac90 comp rx=0 tx=0).stop 2023-10-18T10:47:09.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.575+0000 7fd5d945f640 1 -- 172.21.15.89:0/96646945 shutdown_connections 2023-10-18T10:47:09.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.575+0000 7fd5d945f640 1 --2- 172.21.15.89:0/96646945 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd5c00656b0 0x7fd5c0067b70 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:09.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.575+0000 7fd5d945f640 1 --2- 172.21.15.89:0/96646945 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5d4152cc0 0x7fd5d40c99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:09.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.575+0000 7fd5d945f640 1 --2- 172.21.15.89:0/96646945 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5d4153520 0x7fd5d41539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:09.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.575+0000 7fd5d945f640 1 --2- 172.21.15.89:0/96646945 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5d40d0e70 0x7fd5d40c6180 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:09.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.575+0000 7fd5d945f640 1 -- 172.21.15.89:0/96646945 >> 172.21.15.89:0/96646945 conn(0x7fd5d4089fd0 msgr2=0x7fd5d415bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:09.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.575+0000 7fd5d945f640 1 -- 172.21.15.89:0/96646945 shutdown_connections 2023-10-18T10:47:09.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.575+0000 7fd5d945f640 1 -- 172.21.15.89:0/96646945 wait complete. 2023-10-18T10:47:09.598 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-18T10:47:09.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.735+0000 7f5bfcfb7640 1 Processor -- start 2023-10-18T10:47:09.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.735+0000 7f5bfcfb7640 1 -- start start 2023-10-18T10:47:09.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.735+0000 7f5bfcfb7640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5bf80d0050 0x7f5bf80d1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:09.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.735+0000 7f5bfcfb7640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5bf80d1e70 0x7f5bf80b5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:09.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.735+0000 7f5bfcfb7640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bf80d2290 con 0x7f5bf815c560 2023-10-18T10:47:09.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.735+0000 7f5bfcfb7640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bf80b4a60 con 0x7f5bf80d1e70 2023-10-18T10:47:09.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.735+0000 7f5bfcfb7640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf815c940 con 0x7f5bf80d0050 2023-10-18T10:47:09.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.735+0000 7f5bf7fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5bf80d1e70 0x7f5bf80b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:09.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.735+0000 7f5bf6ffd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5bf80d0050 0x7f5bf80d1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:09.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.735+0000 7f5bf77fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f5bf815c560 0x7f5bf80b4350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.89:43498/0 (socket says 172.21.15.89:43498) 2023-10-18T10:47:09.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.735+0000 7f5bf77fe640 1 -- 172.21.15.89:0/4130791156 learned_addr learned my addr 172.21.15.89:0/4130791156 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:47:09.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.735+0000 7f5bf67fc640 1 -- 172.21.15.89:0/4130791156 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2632242196 0 0) 0x7f5bf80d2290 con 0x7f5bf815c560 2023-10-18T10:47:09.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.735+0000 7f5bf7fff640 1 -- 172.21.15.89:0/4130791156 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5bf80d0050 msgr2=0x7f5bf80d1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:09.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.735+0000 7f5bf7fff640 1 --2- 172.21.15.89:0/4130791156 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5bf80d0050 0x7f5bf80d1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:09.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.735+0000 7f5bf7fff640 1 -- 172.21.15.89:0/4130791156 >> v1:172.21.15.154:6789/0 conn(0x7f5bf815c560 legacy=0x7f5bf80b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:09.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.735+0000 7f5bf7fff640 1 -- 172.21.15.89:0/4130791156 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf80d0550 con 0x7f5bf80d1e70 2023-10-18T10:47:09.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.735+0000 7f5bf6ffd640 1 --2- 172.21.15.89:0/4130791156 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5bf80d0050 0x7f5bf80d1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:47:09.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.739+0000 7f5bf7fff640 1 --2- 172.21.15.89:0/4130791156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bf80d1e70 0x7f5bf80b5460 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0009d90 tx=0x7f5bf001c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=677ecc03cd0f7c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:09.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.739+0000 7f5bf67fc640 1 -- 172.21.15.89:0/4130791156 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf00a5900 con 0x7f5bf80d1e70 2023-10-18T10:47:09.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.739+0000 7f5bf67fc640 1 -- 172.21.15.89:0/4130791156 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf00a5a60 con 0x7f5bf80d1e70 2023-10-18T10:47:09.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.739+0000 7f5bf67fc640 1 -- 172.21.15.89:0/4130791156 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf00a5df0 con 0x7f5bf80d1e70 2023-10-18T10:47:09.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.739+0000 7f5bfcfb7640 1 -- 172.21.15.89:0/4130791156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bf80d1e70 msgr2=0x7f5bf80b5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:09.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.739+0000 7f5bfcfb7640 1 --2- 172.21.15.89:0/4130791156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bf80d1e70 0x7f5bf80b5460 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0009d90 tx=0x7f5bf001c950 comp rx=0 tx=0).stop 2023-10-18T10:47:09.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.739+0000 7f5bfcfb7640 1 -- 172.21.15.89:0/4130791156 shutdown_connections 2023-10-18T10:47:09.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.739+0000 7f5bfcfb7640 1 --2- 172.21.15.89:0/4130791156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bf80d1e70 0x7f5bf80b5460 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:09.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.739+0000 7f5bfcfb7640 1 --2- 172.21.15.89:0/4130791156 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5bf80d0050 0x7f5bf80d1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:09.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.739+0000 7f5bfcfb7640 1 -- 172.21.15.89:0/4130791156 >> 172.21.15.89:0/4130791156 conn(0x7f5bf80d2510 msgr2=0x7f5bf80d2920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:09.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.739+0000 7f5bfcfb7640 1 -- 172.21.15.89:0/4130791156 shutdown_connections 2023-10-18T10:47:09.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.739+0000 7f5bfcfb7640 1 -- 172.21.15.89:0/4130791156 wait complete. 2023-10-18T10:47:09.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.743+0000 7f5bfcfb7640 1 Processor -- start 2023-10-18T10:47:09.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.743+0000 7f5bfcfb7640 1 -- start start 2023-10-18T10:47:09.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.743+0000 7f5bfcfb7640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5bf80d0050 0x7f5bf8154e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:09.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.743+0000 7f5bfcfb7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bf80d1e70 0x7f5bf815b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:09.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.743+0000 7f5bfcfb7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bf815c560 0x7f5bf814d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:09.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.743+0000 7f5bfcfb7640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf8159be0 con 0x7f5bf80d1e70 2023-10-18T10:47:09.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.743+0000 7f5bfcfb7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf815b7e0 con 0x7f5bf815c560 2023-10-18T10:47:09.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.743+0000 7f5bfcfb7640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf8155520 con 0x7f5bf80d0050 2023-10-18T10:47:09.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.743+0000 7f5bf6ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bf80d1e70 0x7f5bf815b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:09.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.743+0000 7f5bf6ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bf80d1e70 0x7f5bf815b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59982/0 (socket says 172.21.15.89:59982) 2023-10-18T10:47:09.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.743+0000 7f5bf6ffd640 1 -- 172.21.15.89:0/2325634171 learned_addr learned my addr 172.21.15.89:0/2325634171 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:09.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.743+0000 7f5bf77fe640 1 --2- 172.21.15.89:0/2325634171 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5bf80d0050 0x7f5bf8154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:09.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.743+0000 7f5bf7fff640 1 --2- 172.21.15.89:0/2325634171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bf815c560 0x7f5bf814d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:09.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.743+0000 7f5bf6ffd640 1 -- 172.21.15.89:0/2325634171 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5bf80d0050 msgr2=0x7f5bf8154e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:09.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.743+0000 7f5bf6ffd640 1 --2- 172.21.15.89:0/2325634171 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5bf80d0050 0x7f5bf8154e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:09.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.743+0000 7f5bf6ffd640 1 -- 172.21.15.89:0/2325634171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bf815c560 msgr2=0x7f5bf814d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:09.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.743+0000 7f5bf6ffd640 1 --2- 172.21.15.89:0/2325634171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bf815c560 0x7f5bf814d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:09.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.743+0000 7f5bf6ffd640 1 -- 172.21.15.89:0/2325634171 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bec009420 con 0x7f5bf80d1e70 2023-10-18T10:47:09.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.743+0000 7f5bf7fff640 1 --2- 172.21.15.89:0/2325634171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bf815c560 0x7f5bf814d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:47:09.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.743+0000 7f5bf77fe640 1 --2- 172.21.15.89:0/2325634171 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5bf80d0050 0x7f5bf8154e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:47:09.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.743+0000 7f5bf6ffd640 1 --2- 172.21.15.89:0/2325634171 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bf80d1e70 0x7f5bf815b2a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5bec00ec10 tx=0x7f5bec00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:09.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.743+0000 7f5bdffff640 1 -- 172.21.15.89:0/2325634171 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bec00cc30 con 0x7f5bf80d1e70 2023-10-18T10:47:09.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.743+0000 7f5bdffff640 1 -- 172.21.15.89:0/2325634171 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bec00cd90 con 0x7f5bf80d1e70 2023-10-18T10:47:09.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.743+0000 7f5bfcfb7640 1 -- 172.21.15.89:0/2325634171 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bf0009a70 con 0x7f5bf80d1e70 2023-10-18T10:47:09.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.743+0000 7f5bdffff640 1 -- 172.21.15.89:0/2325634171 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bec013690 con 0x7f5bf80d1e70 2023-10-18T10:47:09.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.743+0000 7f5bfcfb7640 1 -- 172.21.15.89:0/2325634171 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf8159870 con 0x7f5bf80d1e70 2023-10-18T10:47:09.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.747+0000 7f5bfcfb7640 1 -- 172.21.15.89:0/2325634171 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bf8073b50 con 0x7f5bf80d1e70 2023-10-18T10:47:09.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.747+0000 7f5bdffff640 1 -- 172.21.15.89:0/2325634171 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f5bec0137f0 con 0x7f5bf80d1e70 2023-10-18T10:47:09.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.747+0000 7f5bdffff640 1 --2- 172.21.15.89:0/2325634171 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5be4065680 0x7f5be4067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:09.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.747+0000 7f5bdffff640 1 -- 172.21.15.89:0/2325634171 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f5bec082ff0 con 0x7f5bf80d1e70 2023-10-18T10:47:09.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.747+0000 7f5bf77fe640 1 --2- 172.21.15.89:0/2325634171 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5be4065680 0x7f5be4067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:09.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.747+0000 7f5bf77fe640 1 --2- 172.21.15.89:0/2325634171 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5be4065680 0x7f5be4067b40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5be8000fc0 tx=0x7f5be8007d20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:09.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.751+0000 7f5bdffff640 1 -- 172.21.15.89:0/2325634171 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f5bec050c20 con 0x7f5bf80d1e70 2023-10-18T10:47:09.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.955+0000 7f5bfcfb7640 1 -- 172.21.15.89:0/2325634171 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f5bf814dad0 con 0x7f5bf80d1e70 2023-10-18T10:47:09.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.963+0000 7f5bdffff640 1 -- 172.21.15.89:0/2325634171 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f5bec054240 con 0x7f5bf80d1e70 2023-10-18T10:47:09.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.963+0000 7f5bfcfb7640 1 -- 172.21.15.89:0/2325634171 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5be4065680 msgr2=0x7f5be4067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:09.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.963+0000 7f5bfcfb7640 1 --2- 172.21.15.89:0/2325634171 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5be4065680 0x7f5be4067b40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5be8000fc0 tx=0x7f5be8007d20 comp rx=0 tx=0).stop 2023-10-18T10:47:09.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.963+0000 7f5bfcfb7640 1 -- 172.21.15.89:0/2325634171 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bf80d1e70 msgr2=0x7f5bf815b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:09.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.963+0000 7f5bfcfb7640 1 --2- 172.21.15.89:0/2325634171 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bf80d1e70 0x7f5bf815b2a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5bec00ec10 tx=0x7f5bec00c520 comp rx=0 tx=0).stop 2023-10-18T10:47:09.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.963+0000 7f5bfcfb7640 1 -- 172.21.15.89:0/2325634171 shutdown_connections 2023-10-18T10:47:09.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.963+0000 7f5bfcfb7640 1 --2- 172.21.15.89:0/2325634171 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5be4065680 0x7f5be4067b40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:09.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.963+0000 7f5bfcfb7640 1 --2- 172.21.15.89:0/2325634171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bf815c560 0x7f5bf814d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:09.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.963+0000 7f5bfcfb7640 1 --2- 172.21.15.89:0/2325634171 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5bf80d1e70 0x7f5bf815b2a0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:09.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.963+0000 7f5bfcfb7640 1 --2- 172.21.15.89:0/2325634171 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5bf80d0050 0x7f5bf8154e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:09.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.963+0000 7f5bfcfb7640 1 -- 172.21.15.89:0/2325634171 >> 172.21.15.89:0/2325634171 conn(0x7f5bf80d2510 msgr2=0x7f5bf80b5290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:09.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.963+0000 7f5bfcfb7640 1 -- 172.21.15.89:0/2325634171 shutdown_connections 2023-10-18T10:47:09.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:09.963+0000 7f5bfcfb7640 1 -- 172.21.15.89:0/2325634171 wait complete. 2023-10-18T10:47:09.985 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-18T10:47:10.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.123+0000 7fe432a69640 1 Processor -- start 2023-10-18T10:47:10.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.123+0000 7fe432a69640 1 -- start start 2023-10-18T10:47:10.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.123+0000 7fe432a69640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe42c15c570 0x7fe42c0d18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:10.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.123+0000 7fe432a69640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe42c0d1e30 0x7fe42c0b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:10.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.123+0000 7fe432a69640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe42c0d12f0 con 0x7fe42c0b4450 2023-10-18T10:47:10.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.123+0000 7fe432a69640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe42c0b48e0 con 0x7fe42c15c570 2023-10-18T10:47:10.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.123+0000 7fe432a69640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c0d04d0 con 0x7fe42c0d1e30 2023-10-18T10:47:10.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.123+0000 7fe431a67640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe42c15c570 0x7fe42c0d18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:10.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.123+0000 7fe431a67640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe42c15c570 0x7fe42c0d18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59988/0 (socket says 172.21.15.89:59988) 2023-10-18T10:47:10.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.123+0000 7fe431a67640 1 -- 172.21.15.89:0/3726736606 learned_addr learned my addr 172.21.15.89:0/3726736606 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:10.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.123+0000 7fe431266640 1 --2- 172.21.15.89:0/3726736606 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe42c0d1e30 0x7fe42c0b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:10.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.123+0000 7fe431a67640 1 -- 172.21.15.89:0/3726736606 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe42c0d1e30 msgr2=0x7fe42c0b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:10.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.123+0000 7fe431a67640 1 --2- 172.21.15.89:0/3726736606 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe42c0d1e30 0x7fe42c0b3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:10.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.123+0000 7fe431a67640 1 -- 172.21.15.89:0/3726736606 >> v1:172.21.15.154:6789/0 conn(0x7fe42c0b4450 legacy=0x7fe42c069900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:10.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.123+0000 7fe431a67640 1 -- 172.21.15.89:0/3726736606 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe42c15c950 con 0x7fe42c15c570 2023-10-18T10:47:10.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.123+0000 7fe431266640 1 --2- 172.21.15.89:0/3726736606 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe42c0d1e30 0x7fe42c0b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:47:10.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.127+0000 7fe431a67640 1 --2- 172.21.15.89:0/3726736606 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe42c15c570 0x7fe42c0d18f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe428009870 tx=0x7fe42801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5421262148533a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:10.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.127+0000 7fe430a65640 1 -- 172.21.15.89:0/3726736606 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4280ac410 con 0x7fe42c15c570 2023-10-18T10:47:10.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.127+0000 7fe430a65640 1 -- 172.21.15.89:0/3726736606 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4280ac570 con 0x7fe42c15c570 2023-10-18T10:47:10.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.127+0000 7fe430a65640 1 -- 172.21.15.89:0/3726736606 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4280ac900 con 0x7fe42c15c570 2023-10-18T10:47:10.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.127+0000 7fe432a69640 1 -- 172.21.15.89:0/3726736606 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe42c15c570 msgr2=0x7fe42c0d18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:10.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.127+0000 7fe432a69640 1 --2- 172.21.15.89:0/3726736606 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe42c15c570 0x7fe42c0d18f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe428009870 tx=0x7fe42801c500 comp rx=0 tx=0).stop 2023-10-18T10:47:10.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.127+0000 7fe432a69640 1 -- 172.21.15.89:0/3726736606 shutdown_connections 2023-10-18T10:47:10.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.127+0000 7fe432a69640 1 --2- 172.21.15.89:0/3726736606 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe42c0d1e30 0x7fe42c0b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:10.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.127+0000 7fe432a69640 1 --2- 172.21.15.89:0/3726736606 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe42c15c570 0x7fe42c0d18f0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:10.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.127+0000 7fe432a69640 1 -- 172.21.15.89:0/3726736606 >> 172.21.15.89:0/3726736606 conn(0x7fe42c0cef10 msgr2=0x7fe42c0cf320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:10.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.127+0000 7fe432a69640 1 -- 172.21.15.89:0/3726736606 shutdown_connections 2023-10-18T10:47:10.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.127+0000 7fe432a69640 1 -- 172.21.15.89:0/3726736606 wait complete. 2023-10-18T10:47:10.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.131+0000 7fe432a69640 1 Processor -- start 2023-10-18T10:47:10.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.131+0000 7fe432a69640 1 -- start start 2023-10-18T10:47:10.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.131+0000 7fe432a69640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe42c0b4450 0x7fe42c15bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:10.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.131+0000 7fe432a69640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe42c0d1e30 0x7fe42c1578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:10.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.131+0000 7fe432a69640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe42c15c570 0x7fe42c157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:10.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.131+0000 7fe432a69640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c157590 con 0x7fe42c15c570 2023-10-18T10:47:10.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.131+0000 7fe432a69640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c157f60 con 0x7fe42c0d1e30 2023-10-18T10:47:10.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.131+0000 7fe432a69640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c15afb0 con 0x7fe42c0b4450 2023-10-18T10:47:10.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.131+0000 7fe432268640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe42c15c570 0x7fe42c157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:10.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.131+0000 7fe432268640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe42c15c570 0x7fe42c157050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59996/0 (socket says 172.21.15.89:59996) 2023-10-18T10:47:10.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.131+0000 7fe432268640 1 -- 172.21.15.89:0/1911761482 learned_addr learned my addr 172.21.15.89:0/1911761482 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:10.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.131+0000 7fe431a67640 1 --2- 172.21.15.89:0/1911761482 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe42c0b4450 0x7fe42c15bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:10.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.131+0000 7fe431266640 1 --2- 172.21.15.89:0/1911761482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe42c0d1e30 0x7fe42c1578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:10.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.131+0000 7fe432268640 1 -- 172.21.15.89:0/1911761482 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe42c0b4450 msgr2=0x7fe42c15bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:10.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.131+0000 7fe432268640 1 --2- 172.21.15.89:0/1911761482 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe42c0b4450 0x7fe42c15bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:10.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.131+0000 7fe432268640 1 -- 172.21.15.89:0/1911761482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe42c0d1e30 msgr2=0x7fe42c1578d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:10.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.131+0000 7fe432268640 1 --2- 172.21.15.89:0/1911761482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe42c0d1e30 0x7fe42c1578d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:10.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.131+0000 7fe432268640 1 -- 172.21.15.89:0/1911761482 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4280094d0 con 0x7fe42c15c570 2023-10-18T10:47:10.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.131+0000 7fe432268640 1 --2- 172.21.15.89:0/1911761482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe42c15c570 0x7fe42c157050 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe42400c780 tx=0x7fe42400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:10.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.131+0000 7fe4227fc640 1 -- 172.21.15.89:0/1911761482 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe424004b20 con 0x7fe42c15c570 2023-10-18T10:47:10.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.131+0000 7fe4227fc640 1 -- 172.21.15.89:0/1911761482 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe424004c80 con 0x7fe42c15c570 2023-10-18T10:47:10.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.131+0000 7fe4227fc640 1 -- 172.21.15.89:0/1911761482 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42400aa90 con 0x7fe42c15c570 2023-10-18T10:47:10.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.131+0000 7fe432a69640 1 -- 172.21.15.89:0/1911761482 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe42c15b290 con 0x7fe42c15c570 2023-10-18T10:47:10.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.131+0000 7fe432a69640 1 -- 172.21.15.89:0/1911761482 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe42c15b850 con 0x7fe42c15c570 2023-10-18T10:47:10.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.135+0000 7fe432a69640 1 -- 172.21.15.89:0/1911761482 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe42c15ba70 con 0x7fe42c15c570 2023-10-18T10:47:10.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.135+0000 7fe4227fc640 1 -- 172.21.15.89:0/1911761482 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fe42400abf0 con 0x7fe42c15c570 2023-10-18T10:47:10.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.135+0000 7fe4227fc640 1 --2- 172.21.15.89:0/1911761482 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe410065680 0x7fe410067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:10.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.135+0000 7fe4227fc640 1 -- 172.21.15.89:0/1911761482 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fe424082ca0 con 0x7fe42c15c570 2023-10-18T10:47:10.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.135+0000 7fe431a67640 1 --2- 172.21.15.89:0/1911761482 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe410065680 0x7fe410067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:10.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.139+0000 7fe431a67640 1 --2- 172.21.15.89:0/1911761482 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe410065680 0x7fe410067b40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe4280ae000 tx=0x7fe428009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:10.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.139+0000 7fe4227fc640 1 -- 172.21.15.89:0/1911761482 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fe424050980 con 0x7fe42c15c570 2023-10-18T10:47:10.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.343+0000 7fe432a69640 1 -- 172.21.15.89:0/1911761482 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fe42c073b50 con 0x7fe42c15c570 2023-10-18T10:47:10.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.347+0000 7fe4227fc640 1 -- 172.21.15.89:0/1911761482 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe424053fa0 con 0x7fe42c15c570 2023-10-18T10:47:10.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.351+0000 7fe432a69640 1 -- 172.21.15.89:0/1911761482 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe410065680 msgr2=0x7fe410067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:10.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.351+0000 7fe432a69640 1 --2- 172.21.15.89:0/1911761482 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe410065680 0x7fe410067b40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe4280ae000 tx=0x7fe428009040 comp rx=0 tx=0).stop 2023-10-18T10:47:10.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.351+0000 7fe432a69640 1 -- 172.21.15.89:0/1911761482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe42c15c570 msgr2=0x7fe42c157050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:10.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.351+0000 7fe432a69640 1 --2- 172.21.15.89:0/1911761482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe42c15c570 0x7fe42c157050 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe42400c780 tx=0x7fe42400cc40 comp rx=0 tx=0).stop 2023-10-18T10:47:10.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.351+0000 7fe432a69640 1 -- 172.21.15.89:0/1911761482 shutdown_connections 2023-10-18T10:47:10.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.351+0000 7fe432a69640 1 --2- 172.21.15.89:0/1911761482 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe410065680 0x7fe410067b40 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:10.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.351+0000 7fe432a69640 1 --2- 172.21.15.89:0/1911761482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe42c15c570 0x7fe42c157050 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:10.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.351+0000 7fe432a69640 1 --2- 172.21.15.89:0/1911761482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe42c0d1e30 0x7fe42c1578d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:10.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.351+0000 7fe432a69640 1 --2- 172.21.15.89:0/1911761482 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe42c0b4450 0x7fe42c15bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:10.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.351+0000 7fe432a69640 1 -- 172.21.15.89:0/1911761482 >> 172.21.15.89:0/1911761482 conn(0x7fe42c0cef10 msgr2=0x7fe42c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:10.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.351+0000 7fe432a69640 1 -- 172.21.15.89:0/1911761482 shutdown_connections 2023-10-18T10:47:10.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.351+0000 7fe432a69640 1 -- 172.21.15.89:0/1911761482 wait complete. 2023-10-18T10:47:10.371 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-18T10:47:10.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.507+0000 7fda8de88640 1 Processor -- start 2023-10-18T10:47:10.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.507+0000 7fda8de88640 1 -- start start 2023-10-18T10:47:10.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.507+0000 7fda8de88640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fda88159490 0x7fda8815d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:10.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.507+0000 7fda8de88640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fda88070600 0x7fda8806b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:10.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.507+0000 7fda8de88640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda8806bd10 0x7fda880cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:10.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.507+0000 7fda8de88640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda8806c1a0 con 0x7fda88159490 2023-10-18T10:47:10.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.507+0000 7fda8de88640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda8815ddf0 con 0x7fda88070600 2023-10-18T10:47:10.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.507+0000 7fda8de88640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda88070b90 con 0x7fda8806bd10 2023-10-18T10:47:10.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.507+0000 7fda877fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fda88159490 0x7fda8815d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:10.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.507+0000 7fda877fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fda88159490 0x7fda8815d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60008/0 (socket says 172.21.15.89:60008) 2023-10-18T10:47:10.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.507+0000 7fda877fe640 1 -- 172.21.15.89:0/2713160747 learned_addr learned my addr 172.21.15.89:0/2713160747 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:10.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.507+0000 7fda86ffd640 1 --2- 172.21.15.89:0/2713160747 >> v2:172.21.15.154:3300/0 conn(0x7fda88070600 0x7fda8806b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:10.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.507+0000 7fda87fff640 1 --2- 172.21.15.89:0/2713160747 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda8806bd10 0x7fda880cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:10.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.511+0000 7fda877fe640 1 -- 172.21.15.89:0/2713160747 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda8806bd10 msgr2=0x7fda880cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:10.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.511+0000 7fda877fe640 1 --2- 172.21.15.89:0/2713160747 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda8806bd10 0x7fda880cbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:10.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.511+0000 7fda877fe640 1 -- 172.21.15.89:0/2713160747 >> v2:172.21.15.154:3300/0 conn(0x7fda88070600 msgr2=0x7fda8806b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:10.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.511+0000 7fda877fe640 1 --2- 172.21.15.89:0/2713160747 >> v2:172.21.15.154:3300/0 conn(0x7fda88070600 0x7fda8806b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:10.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.511+0000 7fda877fe640 1 -- 172.21.15.89:0/2713160747 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda88070e70 con 0x7fda88159490 2023-10-18T10:47:10.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.511+0000 7fda877fe640 1 --2- 172.21.15.89:0/2713160747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda88159490 0x7fda8815d8b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fda8806dd60 tx=0x7fda7801c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857f444fb6c0c1f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:10.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.511+0000 7fda867fc640 1 -- 172.21.15.89:0/2713160747 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda780ab600 con 0x7fda88159490 2023-10-18T10:47:10.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.511+0000 7fda867fc640 1 -- 172.21.15.89:0/2713160747 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda780a7050 con 0x7fda88159490 2023-10-18T10:47:10.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.511+0000 7fda867fc640 1 -- 172.21.15.89:0/2713160747 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda780ab980 con 0x7fda88159490 2023-10-18T10:47:10.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.511+0000 7fda8de88640 1 -- 172.21.15.89:0/2713160747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda88159490 msgr2=0x7fda8815d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:10.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.511+0000 7fda8de88640 1 --2- 172.21.15.89:0/2713160747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda88159490 0x7fda8815d8b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fda8806dd60 tx=0x7fda7801c180 comp rx=0 tx=0).stop 2023-10-18T10:47:10.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.511+0000 7fda8de88640 1 -- 172.21.15.89:0/2713160747 shutdown_connections 2023-10-18T10:47:10.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.511+0000 7fda8de88640 1 --2- 172.21.15.89:0/2713160747 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda8806bd10 0x7fda880cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:10.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.511+0000 7fda8de88640 1 --2- 172.21.15.89:0/2713160747 >> v2:172.21.15.154:3300/0 conn(0x7fda88070600 0x7fda8806b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:10.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.511+0000 7fda8de88640 1 --2- 172.21.15.89:0/2713160747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda88159490 0x7fda8815d8b0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:10.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.511+0000 7fda8de88640 1 -- 172.21.15.89:0/2713160747 >> 172.21.15.89:0/2713160747 conn(0x7fda88069f00 msgr2=0x7fda880cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:10.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.511+0000 7fda8de88640 1 -- 172.21.15.89:0/2713160747 shutdown_connections 2023-10-18T10:47:10.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.515+0000 7fda8de88640 1 -- 172.21.15.89:0/2713160747 wait complete. 2023-10-18T10:47:10.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.515+0000 7fda8de88640 1 Processor -- start 2023-10-18T10:47:10.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.515+0000 7fda8de88640 1 -- start start 2023-10-18T10:47:10.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.515+0000 7fda8de88640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda8806bd10 0x7fda880c7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:10.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.515+0000 7fda8de88640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda88070600 0x7fda8815a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:10.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.515+0000 7fda8de88640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda88159490 0x7fda8814fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:10.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.515+0000 7fda8de88640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda880c71f0 con 0x7fda8806bd10 2023-10-18T10:47:10.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.515+0000 7fda8de88640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda880c65e0 con 0x7fda88070600 2023-10-18T10:47:10.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.515+0000 7fda8de88640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda880c68c0 con 0x7fda88159490 2023-10-18T10:47:10.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.515+0000 7fda877fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda8806bd10 0x7fda880c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:10.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.515+0000 7fda877fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda8806bd10 0x7fda880c7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60010/0 (socket says 172.21.15.89:60010) 2023-10-18T10:47:10.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.515+0000 7fda877fe640 1 -- 172.21.15.89:0/2297318306 learned_addr learned my addr 172.21.15.89:0/2297318306 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:10.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.515+0000 7fda87fff640 1 --2- 172.21.15.89:0/2297318306 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda88159490 0x7fda8814fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:10.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.515+0000 7fda86ffd640 1 --2- 172.21.15.89:0/2297318306 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda88070600 0x7fda8815a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:10.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.515+0000 7fda877fe640 1 -- 172.21.15.89:0/2297318306 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda88159490 msgr2=0x7fda8814fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:10.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.515+0000 7fda877fe640 1 --2- 172.21.15.89:0/2297318306 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda88159490 0x7fda8814fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:10.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.515+0000 7fda877fe640 1 -- 172.21.15.89:0/2297318306 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda88070600 msgr2=0x7fda8815a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:10.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.515+0000 7fda877fe640 1 --2- 172.21.15.89:0/2297318306 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda88070600 0x7fda8815a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:10.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.515+0000 7fda877fe640 1 -- 172.21.15.89:0/2297318306 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda780094d0 con 0x7fda8806bd10 2023-10-18T10:47:10.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.515+0000 7fda877fe640 1 --2- 172.21.15.89:0/2297318306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda8806bd10 0x7fda880c7680 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fda7801a830 tx=0x7fda78018f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:10.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.515+0000 7fda6ffff640 1 -- 172.21.15.89:0/2297318306 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda780b4420 con 0x7fda8806bd10 2023-10-18T10:47:10.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.515+0000 7fda6ffff640 1 -- 172.21.15.89:0/2297318306 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda780b4580 con 0x7fda8806bd10 2023-10-18T10:47:10.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.515+0000 7fda6ffff640 1 -- 172.21.15.89:0/2297318306 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda780b4930 con 0x7fda8806bd10 2023-10-18T10:47:10.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.519+0000 7fda8de88640 1 -- 172.21.15.89:0/2297318306 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda880c6ba0 con 0x7fda8806bd10 2023-10-18T10:47:10.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.519+0000 7fda8de88640 1 -- 172.21.15.89:0/2297318306 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda880c7bc0 con 0x7fda8806bd10 2023-10-18T10:47:10.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.519+0000 7fda8de88640 1 -- 172.21.15.89:0/2297318306 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda8804dbe0 con 0x7fda8806bd10 2023-10-18T10:47:10.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.519+0000 7fda6ffff640 1 -- 172.21.15.89:0/2297318306 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fda780b2070 con 0x7fda8806bd10 2023-10-18T10:47:10.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.519+0000 7fda6ffff640 1 --2- 172.21.15.89:0/2297318306 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fda74065680 0x7fda74067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:10.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.519+0000 7fda6ffff640 1 -- 172.21.15.89:0/2297318306 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fda7811bdd0 con 0x7fda8806bd10 2023-10-18T10:47:10.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.519+0000 7fda86ffd640 1 --2- 172.21.15.89:0/2297318306 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fda74065680 0x7fda74067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:10.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.519+0000 7fda86ffd640 1 --2- 172.21.15.89:0/2297318306 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fda74065680 0x7fda74067b40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fda88150ae0 tx=0x7fda7c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:10.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.523+0000 7fda6ffff640 1 -- 172.21.15.89:0/2297318306 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fda780e9a00 con 0x7fda8806bd10 2023-10-18T10:47:10.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.723+0000 7fda8de88640 1 -- 172.21.15.89:0/2297318306 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fda8814af70 con 0x7fda8806bd10 2023-10-18T10:47:10.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.727+0000 7fda6ffff640 1 -- 172.21.15.89:0/2297318306 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fda780ed020 con 0x7fda8806bd10 2023-10-18T10:47:10.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.731+0000 7fda8de88640 1 -- 172.21.15.89:0/2297318306 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fda74065680 msgr2=0x7fda74067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:10.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.731+0000 7fda8de88640 1 --2- 172.21.15.89:0/2297318306 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fda74065680 0x7fda74067b40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fda88150ae0 tx=0x7fda7c008040 comp rx=0 tx=0).stop 2023-10-18T10:47:10.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.731+0000 7fda8de88640 1 -- 172.21.15.89:0/2297318306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda8806bd10 msgr2=0x7fda880c7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:10.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.731+0000 7fda8de88640 1 --2- 172.21.15.89:0/2297318306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda8806bd10 0x7fda880c7680 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fda7801a830 tx=0x7fda78018f20 comp rx=0 tx=0).stop 2023-10-18T10:47:10.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.731+0000 7fda8de88640 1 -- 172.21.15.89:0/2297318306 shutdown_connections 2023-10-18T10:47:10.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.731+0000 7fda8de88640 1 --2- 172.21.15.89:0/2297318306 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fda74065680 0x7fda74067b40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:10.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.731+0000 7fda8de88640 1 --2- 172.21.15.89:0/2297318306 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda88159490 0x7fda8814fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:10.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.731+0000 7fda8de88640 1 --2- 172.21.15.89:0/2297318306 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda88070600 0x7fda8815a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:10.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.731+0000 7fda8de88640 1 --2- 172.21.15.89:0/2297318306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda8806bd10 0x7fda880c7680 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:10.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.731+0000 7fda8de88640 1 -- 172.21.15.89:0/2297318306 >> 172.21.15.89:0/2297318306 conn(0x7fda88069f00 msgr2=0x7fda880c8940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:10.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.731+0000 7fda8de88640 1 -- 172.21.15.89:0/2297318306 shutdown_connections 2023-10-18T10:47:10.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.731+0000 7fda8de88640 1 -- 172.21.15.89:0/2297318306 wait complete. 2023-10-18T10:47:10.752 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-18T10:47:10.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.891+0000 7f2329c6d640 1 Processor -- start 2023-10-18T10:47:10.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.891+0000 7f2329c6d640 1 -- start start 2023-10-18T10:47:10.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.891+0000 7f2329c6d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f23240b1860 0x7f23240cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:10.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.891+0000 7f2329c6d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f23240b1cc0 0x7f23240cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:10.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.891+0000 7f2329c6d640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23240d1e50 con 0x7f232415c550 2023-10-18T10:47:10.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.891+0000 7f2329c6d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23240d2130 con 0x7f23240b1cc0 2023-10-18T10:47:10.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.891+0000 7f2329c6d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23240d2410 con 0x7f23240b1860 2023-10-18T10:47:10.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.891+0000 7f23237fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f232415c550 0x7f23240d2ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37504/0 (socket says 172.21.15.89:37504) 2023-10-18T10:47:10.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.891+0000 7f23237fe640 1 -- 172.21.15.89:0/2406132127 learned_addr learned my addr 172.21.15.89:0/2406132127 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:47:10.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.891+0000 7f2323fff640 1 --2- 172.21.15.89:0/2406132127 >> v2:172.21.15.154:3300/0 conn(0x7f23240b1cc0 0x7f23240cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:10.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.891+0000 7f231affd640 1 --2- 172.21.15.89:0/2406132127 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f23240b1860 0x7f23240cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:10.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.891+0000 7f2322ffd640 1 -- 172.21.15.89:0/2406132127 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 660151244 0 0) 0x7f23240d1e50 con 0x7f232415c550 2023-10-18T10:47:10.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.891+0000 7f2322ffd640 1 -- 172.21.15.89:0/2406132127 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2310003530 con 0x7f232415c550 2023-10-18T10:47:10.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.891+0000 7f2322ffd640 1 -- 172.21.15.89:0/2406132127 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 557788792 0 0) 0x7f2310003530 con 0x7f232415c550 2023-10-18T10:47:10.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.891+0000 7f2322ffd640 1 -- 172.21.15.89:0/2406132127 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f23240b1860 msgr2=0x7f23240cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:10.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.891+0000 7f2322ffd640 1 --2- 172.21.15.89:0/2406132127 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f23240b1860 0x7f23240cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:10.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.891+0000 7f2322ffd640 1 -- 172.21.15.89:0/2406132127 >> v2:172.21.15.154:3300/0 conn(0x7f23240b1cc0 msgr2=0x7f23240cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:10.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.891+0000 7f2322ffd640 1 --2- 172.21.15.89:0/2406132127 >> v2:172.21.15.154:3300/0 conn(0x7f23240b1cc0 0x7f23240cfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:10.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.891+0000 7f2322ffd640 1 -- 172.21.15.89:0/2406132127 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23240d26f0 con 0x7f232415c550 2023-10-18T10:47:10.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.891+0000 7f2322ffd640 1 -- 172.21.15.89:0/2406132127 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7f2314001680 con 0x7f232415c550 2023-10-18T10:47:10.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.891+0000 7f2322ffd640 1 -- 172.21.15.89:0/2406132127 >> v1:172.21.15.89:6789/0 conn(0x7f232415c550 legacy=0x7f23240d2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:10.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.891+0000 7f2322ffd640 1 --2- 172.21.15.89:0/2406132127 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2310003980 0x7f2310003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:10.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.891+0000 7f2322ffd640 1 --2- 172.21.15.89:0/2406132127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2310004370 0x7f2310004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:10.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.891+0000 7f2322ffd640 1 --2- 172.21.15.89:0/2406132127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2310004ec0 0x7f231000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:10.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.891+0000 7f2322ffd640 1 -- 172.21.15.89:0/2406132127 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23240d2130 con 0x7f2310004370 2023-10-18T10:47:10.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.891+0000 7f2322ffd640 1 -- 172.21.15.89:0/2406132127 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23240d2410 con 0x7f2310004ec0 2023-10-18T10:47:10.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.891+0000 7f2322ffd640 1 -- 172.21.15.89:0/2406132127 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f231000bed0 con 0x7f2310003980 2023-10-18T10:47:10.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.891+0000 7f231affd640 1 --2- 172.21.15.89:0/2406132127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2310004370 0x7f2310004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:10.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.891+0000 7f23237fe640 1 --2- 172.21.15.89:0/2406132127 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2310003980 0x7f2310003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:10.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.891+0000 7f2323fff640 1 --2- 172.21.15.89:0/2406132127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2310004ec0 0x7f231000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:10.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.891+0000 7f231affd640 1 -- 172.21.15.89:0/2406132127 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2310003980 msgr2=0x7f2310003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:10.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.891+0000 7f231affd640 1 --2- 172.21.15.89:0/2406132127 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2310003980 0x7f2310003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:10.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.891+0000 7f231affd640 1 -- 172.21.15.89:0/2406132127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2310004ec0 msgr2=0x7f231000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:10.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.891+0000 7f231affd640 1 --2- 172.21.15.89:0/2406132127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2310004ec0 0x7f231000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:10.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.891+0000 7f231affd640 1 -- 172.21.15.89:0/2406132127 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2310003530 con 0x7f2310004370 2023-10-18T10:47:10.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.895+0000 7f231affd640 1 --2- 172.21.15.89:0/2406132127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2310004370 0x7f2310004850 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f23240b5290 tx=0x7f230c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90be11dd1a17a0e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:10.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.895+0000 7f2322ffd640 1 -- 172.21.15.89:0/2406132127 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230c0a5520 con 0x7f2310004370 2023-10-18T10:47:10.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.895+0000 7f2322ffd640 1 -- 172.21.15.89:0/2406132127 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f230c0a5680 con 0x7f2310004370 2023-10-18T10:47:10.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.895+0000 7f2322ffd640 1 -- 172.21.15.89:0/2406132127 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230c0a5a30 con 0x7f2310004370 2023-10-18T10:47:10.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.895+0000 7f2329c6d640 1 -- 172.21.15.89:0/2406132127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2310004370 msgr2=0x7f2310004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:10.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.895+0000 7f2329c6d640 1 --2- 172.21.15.89:0/2406132127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2310004370 0x7f2310004850 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f23240b5290 tx=0x7f230c01c930 comp rx=0 tx=0).stop 2023-10-18T10:47:10.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.895+0000 7f2323fff640 1 -- 172.21.15.89:0/2406132127 reap_dead start 2023-10-18T10:47:10.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.895+0000 7f2329c6d640 1 -- 172.21.15.89:0/2406132127 shutdown_connections 2023-10-18T10:47:10.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.895+0000 7f2329c6d640 1 -- 172.21.15.89:0/2406132127 >> 172.21.15.89:0/2406132127 conn(0x7f2324069d00 msgr2=0x7f232406a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:10.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.895+0000 7f2329c6d640 1 -- 172.21.15.89:0/2406132127 shutdown_connections 2023-10-18T10:47:10.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.895+0000 7f2329c6d640 1 -- 172.21.15.89:0/2406132127 wait complete. 2023-10-18T10:47:10.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.899+0000 7f2329c6d640 1 Processor -- start 2023-10-18T10:47:10.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.899+0000 7f2329c6d640 1 -- start start 2023-10-18T10:47:10.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.899+0000 7f2329c6d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2310004370 0x7f23240cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:10.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.899+0000 7f2329c6d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f232415c930 0x7f2324153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:10.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.899+0000 7f2329c6d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2324154cc0 0x7f2324155140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:10.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.899+0000 7f2329c6d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2324154830 con 0x7f232415c930 2023-10-18T10:47:10.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.899+0000 7f2329c6d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2324153c20 con 0x7f2310004370 2023-10-18T10:47:10.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.899+0000 7f2329c6d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2324153f00 con 0x7f2324154cc0 2023-10-18T10:47:10.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.899+0000 7f231affd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f232415c930 0x7f2324153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:10.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.899+0000 7f231affd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f232415c930 0x7f2324153340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60034/0 (socket says 172.21.15.89:60034) 2023-10-18T10:47:10.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.899+0000 7f231affd640 1 -- 172.21.15.89:0/3241846449 learned_addr learned my addr 172.21.15.89:0/3241846449 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:10.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.899+0000 7f23237fe640 1 --2- 172.21.15.89:0/3241846449 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2310004370 0x7f23240cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:10.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.899+0000 7f2323fff640 1 --2- 172.21.15.89:0/3241846449 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2324154cc0 0x7f2324155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:10.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.899+0000 7f231affd640 1 -- 172.21.15.89:0/3241846449 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2324154cc0 msgr2=0x7f2324155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:10.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.899+0000 7f231affd640 1 --2- 172.21.15.89:0/3241846449 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2324154cc0 0x7f2324155140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:10.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.899+0000 7f231affd640 1 -- 172.21.15.89:0/3241846449 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2310004370 msgr2=0x7f23240cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:10.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.899+0000 7f231affd640 1 --2- 172.21.15.89:0/3241846449 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2310004370 0x7f23240cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:10.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.899+0000 7f231affd640 1 -- 172.21.15.89:0/3241846449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23241541e0 con 0x7f232415c930 2023-10-18T10:47:10.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.899+0000 7f2323fff640 1 --2- 172.21.15.89:0/3241846449 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2324154cc0 0x7f2324155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:47:10.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.899+0000 7f231affd640 1 --2- 172.21.15.89:0/3241846449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f232415c930 0x7f2324153340 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f230c0b0040 tx=0x7f2310005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:10.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.899+0000 7f2320ff9640 1 -- 172.21.15.89:0/3241846449 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230c0a59f0 con 0x7f232415c930 2023-10-18T10:47:10.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.899+0000 7f2320ff9640 1 -- 172.21.15.89:0/3241846449 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f230c0a5b90 con 0x7f232415c930 2023-10-18T10:47:10.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.899+0000 7f2329c6d640 1 -- 172.21.15.89:0/3241846449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23241544c0 con 0x7f232415c930 2023-10-18T10:47:10.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.899+0000 7f2320ff9640 1 -- 172.21.15.89:0/3241846449 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230c003620 con 0x7f232415c930 2023-10-18T10:47:10.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.899+0000 7f2329c6d640 1 -- 172.21.15.89:0/3241846449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2324153880 con 0x7f232415c930 2023-10-18T10:47:10.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.903+0000 7f2329c6d640 1 -- 172.21.15.89:0/3241846449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23241549a0 con 0x7f232415c930 2023-10-18T10:47:10.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.903+0000 7f2320ff9640 1 -- 172.21.15.89:0/3241846449 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f230c0b4070 con 0x7f232415c930 2023-10-18T10:47:10.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.903+0000 7f2320ff9640 1 --2- 172.21.15.89:0/3241846449 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f23040656b0 0x7f2304067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:10.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.903+0000 7f2320ff9640 1 -- 172.21.15.89:0/3241846449 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f230c11c7f0 con 0x7f232415c930 2023-10-18T10:47:10.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.903+0000 7f23237fe640 1 --2- 172.21.15.89:0/3241846449 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f23040656b0 0x7f2304067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:10.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.903+0000 7f23237fe640 1 --2- 172.21.15.89:0/3241846449 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f23040656b0 0x7f2304067b70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2314001800 tx=0x7f2314007900 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:10.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:10.907+0000 7f2320ff9640 1 -- 172.21.15.89:0/3241846449 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f230c0ea420 con 0x7f232415c930 2023-10-18T10:47:11.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.111+0000 7f2329c6d640 1 -- 172.21.15.89:0/3241846449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f232404dbe0 con 0x7f232415c930 2023-10-18T10:47:11.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.115+0000 7f2320ff9640 1 -- 172.21.15.89:0/3241846449 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f230c0b20a0 con 0x7f232415c930 2023-10-18T10:47:11.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.115+0000 7f2329c6d640 1 -- 172.21.15.89:0/3241846449 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f23040656b0 msgr2=0x7f2304067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:11.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.115+0000 7f2329c6d640 1 --2- 172.21.15.89:0/3241846449 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f23040656b0 0x7f2304067b70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2314001800 tx=0x7f2314007900 comp rx=0 tx=0).stop 2023-10-18T10:47:11.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.115+0000 7f2329c6d640 1 -- 172.21.15.89:0/3241846449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f232415c930 msgr2=0x7f2324153340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:11.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.115+0000 7f2329c6d640 1 --2- 172.21.15.89:0/3241846449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f232415c930 0x7f2324153340 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f230c0b0040 tx=0x7f2310005800 comp rx=0 tx=0).stop 2023-10-18T10:47:11.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.119+0000 7f2329c6d640 1 -- 172.21.15.89:0/3241846449 shutdown_connections 2023-10-18T10:47:11.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.119+0000 7f2329c6d640 1 --2- 172.21.15.89:0/3241846449 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f23040656b0 0x7f2304067b70 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:11.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.119+0000 7f2329c6d640 1 --2- 172.21.15.89:0/3241846449 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2324154cc0 0x7f2324155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:11.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.119+0000 7f2329c6d640 1 --2- 172.21.15.89:0/3241846449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f232415c930 0x7f2324153340 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:11.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.119+0000 7f2329c6d640 1 --2- 172.21.15.89:0/3241846449 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2310004370 0x7f23240cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:11.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.119+0000 7f2329c6d640 1 -- 172.21.15.89:0/3241846449 >> 172.21.15.89:0/3241846449 conn(0x7f2324069d00 msgr2=0x7f23240b6ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:11.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.119+0000 7f2329c6d640 1 -- 172.21.15.89:0/3241846449 shutdown_connections 2023-10-18T10:47:11.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.119+0000 7f2329c6d640 1 -- 172.21.15.89:0/3241846449 wait complete. 2023-10-18T10:47:11.139 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-18T10:47:11.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.275+0000 7f28ce373640 1 Processor -- start 2023-10-18T10:47:11.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.275+0000 7f28ce373640 1 -- start start 2023-10-18T10:47:11.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.275+0000 7f28ce373640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28c80b27d0 0x7f28c80ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:11.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.275+0000 7f28ce373640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f28c80cfe60 0x7f28c815de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:11.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.275+0000 7f28ce373640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c80caa30 con 0x7f28c8159fd0 2023-10-18T10:47:11.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.275+0000 7f28ce373640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28c80cee60 con 0x7f28c80cfe60 2023-10-18T10:47:11.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.275+0000 7f28ce373640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c80cf4a0 con 0x7f28c80b27d0 2023-10-18T10:47:11.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.275+0000 7f28cdb72640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f28c80cfe60 0x7f28c815de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:11.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.275+0000 7f28cdb72640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f28c80cfe60 0x7f28c815de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60040/0 (socket says 172.21.15.89:60040) 2023-10-18T10:47:11.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.275+0000 7f28cdb72640 1 -- 172.21.15.89:0/219065581 learned_addr learned my addr 172.21.15.89:0/219065581 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:11.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.275+0000 7f28ccb70640 1 --2- 172.21.15.89:0/219065581 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28c80b27d0 0x7f28c80ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:11.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.275+0000 7f28cdb72640 1 -- 172.21.15.89:0/219065581 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28c80b27d0 msgr2=0x7f28c80ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:11.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.275+0000 7f28cdb72640 1 --2- 172.21.15.89:0/219065581 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28c80b27d0 0x7f28c80ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:11.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.275+0000 7f28cdb72640 1 -- 172.21.15.89:0/219065581 >> v1:172.21.15.154:6789/0 conn(0x7f28c8159fd0 legacy=0x7f28c80cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:11.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.275+0000 7f28cdb72640 1 -- 172.21.15.89:0/219065581 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c80caf80 con 0x7f28c80cfe60 2023-10-18T10:47:11.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.275+0000 7f28ccb70640 1 --2- 172.21.15.89:0/219065581 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28c80b27d0 0x7f28c80ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:47:11.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.279+0000 7f28cdb72640 1 --2- 172.21.15.89:0/219065581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28c80cfe60 0x7f28c815de60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f28c0009e50 tx=0x7f28c001c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a4777540ad164c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:11.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.279+0000 7f28bbfff640 1 -- 172.21.15.89:0/219065581 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c00a58a0 con 0x7f28c80cfe60 2023-10-18T10:47:11.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.279+0000 7f28bbfff640 1 -- 172.21.15.89:0/219065581 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c00a5a00 con 0x7f28c80cfe60 2023-10-18T10:47:11.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.279+0000 7f28bbfff640 1 -- 172.21.15.89:0/219065581 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c00a5d90 con 0x7f28c80cfe60 2023-10-18T10:47:11.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.279+0000 7f28ce373640 1 -- 172.21.15.89:0/219065581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28c80cfe60 msgr2=0x7f28c815de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:11.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.279+0000 7f28ce373640 1 --2- 172.21.15.89:0/219065581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28c80cfe60 0x7f28c815de60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f28c0009e50 tx=0x7f28c001c9e0 comp rx=0 tx=0).stop 2023-10-18T10:47:11.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.279+0000 7f28ce373640 1 -- 172.21.15.89:0/219065581 shutdown_connections 2023-10-18T10:47:11.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.279+0000 7f28ce373640 1 --2- 172.21.15.89:0/219065581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28c80cfe60 0x7f28c815de60 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:11.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.279+0000 7f28ce373640 1 --2- 172.21.15.89:0/219065581 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28c80b27d0 0x7f28c80ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:11.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.279+0000 7f28ce373640 1 -- 172.21.15.89:0/219065581 >> 172.21.15.89:0/219065581 conn(0x7f28c806bb10 msgr2=0x7f28c806bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:11.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.279+0000 7f28ce373640 1 -- 172.21.15.89:0/219065581 shutdown_connections 2023-10-18T10:47:11.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.279+0000 7f28ce373640 1 -- 172.21.15.89:0/219065581 wait complete. 2023-10-18T10:47:11.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.283+0000 7f28ce373640 1 Processor -- start 2023-10-18T10:47:11.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.283+0000 7f28ce373640 1 -- start start 2023-10-18T10:47:11.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.283+0000 7f28ce373640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28c80b27d0 0x7f28c80c3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:11.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.283+0000 7f28ce373640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28c80cfe60 0x7f28c814d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:11.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.283+0000 7f28ce373640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28c8159fd0 0x7f28c80d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:11.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.283+0000 7f28ce373640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c80c43e0 con 0x7f28c80cfe60 2023-10-18T10:47:11.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.283+0000 7f28ce373640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c80c46c0 con 0x7f28c80b27d0 2023-10-18T10:47:11.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.283+0000 7f28ce373640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c80c49a0 con 0x7f28c8159fd0 2023-10-18T10:47:11.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.283+0000 7f28ccb70640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28c80cfe60 0x7f28c814d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:11.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.283+0000 7f28ccb70640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28c80cfe60 0x7f28c814d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60050/0 (socket says 172.21.15.89:60050) 2023-10-18T10:47:11.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.283+0000 7f28ccb70640 1 -- 172.21.15.89:0/767282876 learned_addr learned my addr 172.21.15.89:0/767282876 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:11.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.283+0000 7f28cdb72640 1 --2- 172.21.15.89:0/767282876 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28c8159fd0 0x7f28c80d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:11.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.283+0000 7f28cd371640 1 --2- 172.21.15.89:0/767282876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28c80b27d0 0x7f28c80c3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:11.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.283+0000 7f28ccb70640 1 -- 172.21.15.89:0/767282876 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28c8159fd0 msgr2=0x7f28c80d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:11.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.283+0000 7f28ccb70640 1 --2- 172.21.15.89:0/767282876 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28c8159fd0 0x7f28c80d02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:11.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.283+0000 7f28ccb70640 1 -- 172.21.15.89:0/767282876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28c80b27d0 msgr2=0x7f28c80c3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:11.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.283+0000 7f28ccb70640 1 --2- 172.21.15.89:0/767282876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28c80b27d0 0x7f28c80c3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:11.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.283+0000 7f28ccb70640 1 -- 172.21.15.89:0/767282876 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c0009b30 con 0x7f28c80cfe60 2023-10-18T10:47:11.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.283+0000 7f28cdb72640 1 --2- 172.21.15.89:0/767282876 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28c8159fd0 0x7f28c80d02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:47:11.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.283+0000 7f28cd371640 1 --2- 172.21.15.89:0/767282876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28c80b27d0 0x7f28c80c3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:47:11.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.283+0000 7f28ccb70640 1 --2- 172.21.15.89:0/767282876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28c80cfe60 0x7f28c814d3c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f28c400e810 tx=0x7f28c400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:11.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.283+0000 7f28b9ffb640 1 -- 172.21.15.89:0/767282876 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c400c8d0 con 0x7f28c80cfe60 2023-10-18T10:47:11.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.283+0000 7f28b9ffb640 1 -- 172.21.15.89:0/767282876 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c400ca30 con 0x7f28c80cfe60 2023-10-18T10:47:11.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.283+0000 7f28b9ffb640 1 -- 172.21.15.89:0/767282876 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c400cde0 con 0x7f28c80cfe60 2023-10-18T10:47:11.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.283+0000 7f28ce373640 1 -- 172.21.15.89:0/767282876 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c80c4c80 con 0x7f28c80cfe60 2023-10-18T10:47:11.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.283+0000 7f28ce373640 1 -- 172.21.15.89:0/767282876 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c814de90 con 0x7f28c80cfe60 2023-10-18T10:47:11.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.287+0000 7f28ce373640 1 -- 172.21.15.89:0/767282876 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c8073b50 con 0x7f28c80cfe60 2023-10-18T10:47:11.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.287+0000 7f28b9ffb640 1 -- 172.21.15.89:0/767282876 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f28c4013410 con 0x7f28c80cfe60 2023-10-18T10:47:11.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.287+0000 7f28b9ffb640 1 --2- 172.21.15.89:0/767282876 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f28ac065680 0x7f28ac067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:11.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.287+0000 7f28b9ffb640 1 -- 172.21.15.89:0/767282876 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f28c4082ae0 con 0x7f28c80cfe60 2023-10-18T10:47:11.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.287+0000 7f28cd371640 1 --2- 172.21.15.89:0/767282876 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f28ac065680 0x7f28ac067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:11.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.287+0000 7f28cd371640 1 --2- 172.21.15.89:0/767282876 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f28ac065680 0x7f28ac067b40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f28bc000f70 tx=0x7f28bc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:11.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.291+0000 7f28b9ffb640 1 -- 172.21.15.89:0/767282876 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f28c4050710 con 0x7f28c80cfe60 2023-10-18T10:47:11.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.495+0000 7f28ce373640 1 -- 172.21.15.89:0/767282876 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f28c80b25b0 con 0x7f28c80cfe60 2023-10-18T10:47:11.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.499+0000 7f28b9ffb640 1 -- 172.21.15.89:0/767282876 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f28c4053d30 con 0x7f28c80cfe60 2023-10-18T10:47:11.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.503+0000 7f28ce373640 1 -- 172.21.15.89:0/767282876 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f28ac065680 msgr2=0x7f28ac067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:11.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.503+0000 7f28ce373640 1 --2- 172.21.15.89:0/767282876 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f28ac065680 0x7f28ac067b40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f28bc000f70 tx=0x7f28bc009040 comp rx=0 tx=0).stop 2023-10-18T10:47:11.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.503+0000 7f28ce373640 1 -- 172.21.15.89:0/767282876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28c80cfe60 msgr2=0x7f28c814d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:11.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.503+0000 7f28ce373640 1 --2- 172.21.15.89:0/767282876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28c80cfe60 0x7f28c814d3c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f28c400e810 tx=0x7f28c400ecd0 comp rx=0 tx=0).stop 2023-10-18T10:47:11.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.503+0000 7f28ce373640 1 -- 172.21.15.89:0/767282876 shutdown_connections 2023-10-18T10:47:11.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.503+0000 7f28ce373640 1 --2- 172.21.15.89:0/767282876 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f28ac065680 0x7f28ac067b40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:11.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.503+0000 7f28ce373640 1 --2- 172.21.15.89:0/767282876 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f28c8159fd0 0x7f28c80d02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:11.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.503+0000 7f28ce373640 1 --2- 172.21.15.89:0/767282876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28c80cfe60 0x7f28c814d3c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:11.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.503+0000 7f28ce373640 1 --2- 172.21.15.89:0/767282876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28c80b27d0 0x7f28c80c3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:11.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.503+0000 7f28ce373640 1 -- 172.21.15.89:0/767282876 >> 172.21.15.89:0/767282876 conn(0x7f28c806bb10 msgr2=0x7f28c806a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:11.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.503+0000 7f28ce373640 1 -- 172.21.15.89:0/767282876 shutdown_connections 2023-10-18T10:47:11.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.503+0000 7f28ce373640 1 -- 172.21.15.89:0/767282876 wait complete. 2023-10-18T10:47:11.525 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:47:11.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.659+0000 7f42b0f96640 1 Processor -- start 2023-10-18T10:47:11.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.659+0000 7f42b0f96640 1 -- start start 2023-10-18T10:47:11.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.659+0000 7f42b0f96640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f42ac159490 0x7f42ac15d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:11.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.663+0000 7f42b0f96640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42ac15ddf0 con 0x7f42ac06bee0 2023-10-18T10:47:11.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.663+0000 7f42b0f96640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42ac070b90 con 0x7f42ac070600 2023-10-18T10:47:11.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.663+0000 7f42b0f96640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42ac070e70 con 0x7f42ac159490 2023-10-18T10:47:11.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.663+0000 7f42aad9d640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f42ac06bee0 0x7f42ac0c8960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37506/0 (socket says 172.21.15.89:37506) 2023-10-18T10:47:11.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.663+0000 7f42aad9d640 1 -- 172.21.15.89:0/753807198 learned_addr learned my addr 172.21.15.89:0/753807198 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:47:11.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.663+0000 7f42aa59c640 1 --2- 172.21.15.89:0/753807198 >> v2:172.21.15.154:3300/0 conn(0x7f42ac159490 0x7f42ac15d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:11.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.663+0000 7f42a959a640 1 -- 172.21.15.89:0/753807198 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3607818326 0 0) 0x7f42ac15ddf0 con 0x7f42ac06bee0 2023-10-18T10:47:11.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.663+0000 7f42a959a640 1 -- 172.21.15.89:0/753807198 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4294003530 con 0x7f42ac06bee0 2023-10-18T10:47:11.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.663+0000 7f42a959a640 1 -- 172.21.15.89:0/753807198 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 180609042 0 0) 0x7f4294003530 con 0x7f42ac06bee0 2023-10-18T10:47:11.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.663+0000 7f42a959a640 1 -- 172.21.15.89:0/753807198 >> v2:172.21.15.154:3300/0 conn(0x7f42ac159490 msgr2=0x7f42ac15d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:11.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.663+0000 7f42a959a640 1 --2- 172.21.15.89:0/753807198 >> v2:172.21.15.154:3300/0 conn(0x7f42ac159490 0x7f42ac15d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:11.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.663+0000 7f42a959a640 1 -- 172.21.15.89:0/753807198 >> v1:172.21.15.154:6789/0 conn(0x7f42ac070600 legacy=0x7f42ac06b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:11.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.663+0000 7f42a959a640 1 -- 172.21.15.89:0/753807198 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ac071150 con 0x7f42ac06bee0 2023-10-18T10:47:11.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.663+0000 7f42a959a640 1 -- 172.21.15.89:0/753807198 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7f42a4001680 con 0x7f42ac06bee0 2023-10-18T10:47:11.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.663+0000 7f42a959a640 1 -- 172.21.15.89:0/753807198 >> v1:172.21.15.89:6789/0 conn(0x7f42ac06bee0 legacy=0x7f42ac0c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:11.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.663+0000 7f42a959a640 1 --2- 172.21.15.89:0/753807198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4294003980 0x7f4294003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:11.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.663+0000 7f42a959a640 1 --2- 172.21.15.89:0/753807198 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42940047d0 0x7f4294004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:11.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.663+0000 7f42a959a640 1 --2- 172.21.15.89:0/753807198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4294005190 0x7f429400dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:11.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.663+0000 7f42a959a640 1 -- 172.21.15.89:0/753807198 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac070e70 con 0x7f4294005190 2023-10-18T10:47:11.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.663+0000 7f42a959a640 1 -- 172.21.15.89:0/753807198 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429400e340 con 0x7f4294003980 2023-10-18T10:47:11.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.663+0000 7f42a959a640 1 -- 172.21.15.89:0/753807198 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f429400e500 con 0x7f42940047d0 2023-10-18T10:47:11.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.663+0000 7f42aad9d640 1 --2- 172.21.15.89:0/753807198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4294005190 0x7f429400dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:11.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.663+0000 7f42aa59c640 1 --2- 172.21.15.89:0/753807198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4294003980 0x7f4294003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:11.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.663+0000 7f42a9d9b640 1 --2- 172.21.15.89:0/753807198 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42940047d0 0x7f4294004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:11.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.663+0000 7f42aad9d640 1 -- 172.21.15.89:0/753807198 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42940047d0 msgr2=0x7f4294004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:11.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.663+0000 7f42aad9d640 1 --2- 172.21.15.89:0/753807198 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42940047d0 0x7f4294004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:11.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.663+0000 7f42aad9d640 1 -- 172.21.15.89:0/753807198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4294003980 msgr2=0x7f4294003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:11.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.663+0000 7f42aad9d640 1 --2- 172.21.15.89:0/753807198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4294003980 0x7f4294003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:11.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.663+0000 7f42aad9d640 1 -- 172.21.15.89:0/753807198 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4294003530 con 0x7f4294005190 2023-10-18T10:47:11.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.663+0000 7f42aa59c640 1 --2- 172.21.15.89:0/753807198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4294003980 0x7f4294003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:47:11.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.667+0000 7f42aad9d640 1 -- 172.21.15.89:0/753807198 reap_dead start 2023-10-18T10:47:11.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.667+0000 7f42aad9d640 1 --2- 172.21.15.89:0/753807198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4294005190 0x7f429400dba0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f42ac06f630 tx=0x7f42a401f350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886cb8c511fc8f6d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:11.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.667+0000 7f42a959a640 1 -- 172.21.15.89:0/753807198 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a4005ae0 con 0x7f4294005190 2023-10-18T10:47:11.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.667+0000 7f42a959a640 1 -- 172.21.15.89:0/753807198 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a40ab070 con 0x7f4294005190 2023-10-18T10:47:11.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.667+0000 7f42a959a640 1 -- 172.21.15.89:0/753807198 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a4005e80 con 0x7f4294005190 2023-10-18T10:47:11.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.667+0000 7f42b0f96640 1 -- 172.21.15.89:0/753807198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4294005190 msgr2=0x7f429400dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:11.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.667+0000 7f42b0f96640 1 --2- 172.21.15.89:0/753807198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4294005190 0x7f429400dba0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f42ac06f630 tx=0x7f42a401f350 comp rx=0 tx=0).stop 2023-10-18T10:47:11.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.667+0000 7f42b0f96640 1 -- 172.21.15.89:0/753807198 shutdown_connections 2023-10-18T10:47:11.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.667+0000 7f42b0f96640 1 --2- 172.21.15.89:0/753807198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4294005190 0x7f429400dba0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:11.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.667+0000 7f42b0f96640 1 -- 172.21.15.89:0/753807198 >> 172.21.15.89:0/753807198 conn(0x7f42ac069f00 msgr2=0x7f42ac0cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:11.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.667+0000 7f42b0f96640 1 -- 172.21.15.89:0/753807198 shutdown_connections 2023-10-18T10:47:11.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.667+0000 7f42b0f96640 1 -- 172.21.15.89:0/753807198 wait complete. 2023-10-18T10:47:11.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.667+0000 7f42b0f96640 1 Processor -- start 2023-10-18T10:47:11.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.667+0000 7f42b0f96640 1 -- start start 2023-10-18T10:47:11.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.671+0000 7f42b0f96640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4294005190 0x7f42ac14b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:11.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.671+0000 7f42b0f96640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42ac14fec0 0x7f42ac0c2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:11.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.671+0000 7f42b0f96640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42ac0ae740 0x7f42ac0aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:11.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.671+0000 7f42b0f96640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac0af110 con 0x7f42ac0ae740 2023-10-18T10:47:11.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.671+0000 7f42b0f96640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac14b6f0 con 0x7f42ac14fec0 2023-10-18T10:47:11.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.671+0000 7f42b0f96640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac14e350 con 0x7f4294005190 2023-10-18T10:47:11.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.671+0000 7f42aad9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42ac0ae740 0x7f42ac0aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:11.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.671+0000 7f42aad9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42ac0ae740 0x7f42ac0aebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60078/0 (socket says 172.21.15.89:60078) 2023-10-18T10:47:11.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.671+0000 7f42aad9d640 1 -- 172.21.15.89:0/1969838026 learned_addr learned my addr 172.21.15.89:0/1969838026 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:11.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.671+0000 7f42aa59c640 1 --2- 172.21.15.89:0/1969838026 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4294005190 0x7f42ac14b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:11.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.671+0000 7f42a9d9b640 1 --2- 172.21.15.89:0/1969838026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42ac14fec0 0x7f42ac0c2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:11.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.671+0000 7f42aad9d640 1 -- 172.21.15.89:0/1969838026 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4294005190 msgr2=0x7f42ac14b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:11.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.671+0000 7f42aad9d640 1 --2- 172.21.15.89:0/1969838026 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4294005190 0x7f42ac14b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:11.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.671+0000 7f42aad9d640 1 -- 172.21.15.89:0/1969838026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42ac14fec0 msgr2=0x7f42ac0c2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:11.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.671+0000 7f42aad9d640 1 --2- 172.21.15.89:0/1969838026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42ac14fec0 0x7f42ac0c2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:11.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.671+0000 7f42aad9d640 1 -- 172.21.15.89:0/1969838026 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ac14e630 con 0x7f42ac0ae740 2023-10-18T10:47:11.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.671+0000 7f42aad9d640 1 --2- 172.21.15.89:0/1969838026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42ac0ae740 0x7f42ac0aebd0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f42a401f180 tx=0x7f4294005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:11.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.671+0000 7f429affd640 1 -- 172.21.15.89:0/1969838026 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a4001640 con 0x7f42ac0ae740 2023-10-18T10:47:11.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.671+0000 7f429affd640 1 -- 172.21.15.89:0/1969838026 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a40b6040 con 0x7f42ac0ae740 2023-10-18T10:47:11.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.671+0000 7f429affd640 1 -- 172.21.15.89:0/1969838026 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a40049f0 con 0x7f42ac0ae740 2023-10-18T10:47:11.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.671+0000 7f42b0f96640 1 -- 172.21.15.89:0/1969838026 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42ac14e910 con 0x7f42ac0ae740 2023-10-18T10:47:11.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.671+0000 7f42b0f96640 1 -- 172.21.15.89:0/1969838026 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42ac14eed0 con 0x7f42ac0ae740 2023-10-18T10:47:11.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.671+0000 7f42b0f96640 1 -- 172.21.15.89:0/1969838026 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42ac14f0c0 con 0x7f42ac0ae740 2023-10-18T10:47:11.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.671+0000 7f429affd640 1 -- 172.21.15.89:0/1969838026 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f42a40ad030 con 0x7f42ac0ae740 2023-10-18T10:47:11.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.675+0000 7f429affd640 1 --2- 172.21.15.89:0/1969838026 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f428c0656b0 0x7f428c067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:11.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.675+0000 7f429affd640 1 -- 172.21.15.89:0/1969838026 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f42a411f050 con 0x7f42ac0ae740 2023-10-18T10:47:11.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.675+0000 7f42aa59c640 1 --2- 172.21.15.89:0/1969838026 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f428c0656b0 0x7f428c067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:11.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.675+0000 7f42aa59c640 1 --2- 172.21.15.89:0/1969838026 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f428c0656b0 0x7f428c067b70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f429c009fd0 tx=0x7f429c00b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:11.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.679+0000 7f429affd640 1 -- 172.21.15.89:0/1969838026 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f42a40ecdd0 con 0x7f42ac0ae740 2023-10-18T10:47:11.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.831+0000 7f42b0f96640 1 -- 172.21.15.89:0/1969838026 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42ac0cbb60 con 0x7f42ac0ae740 2023-10-18T10:47:11.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.831+0000 7f429affd640 1 -- 172.21.15.89:0/1969838026 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+14820 (secure 0 0 0) 0x7f42a40c0090 con 0x7f42ac0ae740 2023-10-18T10:47:11.833 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:47:11.833 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":37,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:47:08.798707+0000","last_up_change":"2023-10-18T10:45:42.164688+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6824","nonce":773078364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6827","nonce":773078364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6831","nonce":773078364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6829","nonce":773078364}]},"public_addr":"172.21.15.154:6824/773078364","cluster_addr":"172.21.15.154:6827/773078364","heartbeat_back_addr":"172.21.15.154:6831/773078364","heartbeat_front_addr":"172.21.15.154:6829/773078364","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:47:11.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.831+0000 7f42b0f96640 1 -- 172.21.15.89:0/1969838026 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f428c0656b0 msgr2=0x7f428c067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:11.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.831+0000 7f42b0f96640 1 --2- 172.21.15.89:0/1969838026 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f428c0656b0 0x7f428c067b70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f429c009fd0 tx=0x7f429c00b040 comp rx=0 tx=0).stop 2023-10-18T10:47:11.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.831+0000 7f42b0f96640 1 -- 172.21.15.89:0/1969838026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42ac0ae740 msgr2=0x7f42ac0aebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:11.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.831+0000 7f42b0f96640 1 --2- 172.21.15.89:0/1969838026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42ac0ae740 0x7f42ac0aebd0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f42a401f180 tx=0x7f4294005800 comp rx=0 tx=0).stop 2023-10-18T10:47:11.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.831+0000 7f42b0f96640 1 -- 172.21.15.89:0/1969838026 shutdown_connections 2023-10-18T10:47:11.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.831+0000 7f42b0f96640 1 --2- 172.21.15.89:0/1969838026 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f428c0656b0 0x7f428c067b70 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:11.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.831+0000 7f42b0f96640 1 --2- 172.21.15.89:0/1969838026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42ac0ae740 0x7f42ac0aebd0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:11.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.831+0000 7f42b0f96640 1 --2- 172.21.15.89:0/1969838026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42ac14fec0 0x7f42ac0c2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:11.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.831+0000 7f42b0f96640 1 --2- 172.21.15.89:0/1969838026 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4294005190 0x7f42ac14b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:11.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.831+0000 7f42b0f96640 1 -- 172.21.15.89:0/1969838026 >> 172.21.15.89:0/1969838026 conn(0x7f42ac069f00 msgr2=0x7f42ac0cace0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:11.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.831+0000 7f42b0f96640 1 -- 172.21.15.89:0/1969838026 shutdown_connections 2023-10-18T10:47:11.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.831+0000 7f42b0f96640 1 -- 172.21.15.89:0/1969838026 wait complete. 2023-10-18T10:47:11.854 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:45:43.114184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-18T10:45:48.730968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T10:47:11.855 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:47:11.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.991+0000 7fbbdf493640 1 Processor -- start 2023-10-18T10:47:11.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.995+0000 7fbbdf493640 1 -- start start 2023-10-18T10:47:11.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.995+0000 7fbbdf493640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbbd815e600 0x7fbbd80c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:11.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.995+0000 7fbbdf493640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbd81614e0 con 0x7fbbd80afd90 2023-10-18T10:47:11.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.995+0000 7fbbdf493640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbd815e9e0 con 0x7fbbd806c1b0 2023-10-18T10:47:11.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.995+0000 7fbbdf493640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd8160f80 con 0x7fbbd815e600 2023-10-18T10:47:11.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.995+0000 7fbbdda30640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fbbd80afd90 0x7fbbd80c1140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:37516/0 (socket says 172.21.15.89:37516) 2023-10-18T10:47:11.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.995+0000 7fbbdda30640 1 -- 172.21.15.89:0/540509578 learned_addr learned my addr 172.21.15.89:0/540509578 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:47:11.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.995+0000 7fbbdd22f640 1 --2- 172.21.15.89:0/540509578 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbbd815e600 0x7fbbd80c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:11.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.995+0000 7fbbcffff640 1 -- 172.21.15.89:0/540509578 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3897831717 0 0) 0x7fbbd81614e0 con 0x7fbbd80afd90 2023-10-18T10:47:12.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.995+0000 7fbbcffff640 1 -- 172.21.15.89:0/540509578 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbc4003530 con 0x7fbbd80afd90 2023-10-18T10:47:12.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.995+0000 7fbbcffff640 1 -- 172.21.15.89:0/540509578 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1180889391 0 0) 0x7fbbd815e9e0 con 0x7fbbd806c1b0 2023-10-18T10:47:12.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.995+0000 7fbbdd22f640 1 -- 172.21.15.89:0/540509578 >> v1:172.21.15.154:6789/0 conn(0x7fbbd806c1b0 legacy=0x7fbbd8161970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:12.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.995+0000 7fbbdd22f640 1 -- 172.21.15.89:0/540509578 >> v1:172.21.15.89:6789/0 conn(0x7fbbd80afd90 legacy=0x7fbbd80c1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:12.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.995+0000 7fbbdd22f640 1 -- 172.21.15.89:0/540509578 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd806c610 con 0x7fbbd815e600 2023-10-18T10:47:12.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.999+0000 7fbbdd22f640 1 --2- 172.21.15.89:0/540509578 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbbd815e600 0x7fbbd80c8960 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbbd806c800 tx=0x7fbbd401c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dc45687b3090be9 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:12.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.999+0000 7fbbcffff640 1 -- 172.21.15.89:0/540509578 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd40ad680 con 0x7fbbd815e600 2023-10-18T10:47:12.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.999+0000 7fbbcffff640 1 -- 172.21.15.89:0/540509578 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd40ad7e0 con 0x7fbbd815e600 2023-10-18T10:47:12.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.999+0000 7fbbcffff640 1 -- 172.21.15.89:0/540509578 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd40adb70 con 0x7fbbd815e600 2023-10-18T10:47:12.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.999+0000 7fbbdf493640 1 -- 172.21.15.89:0/540509578 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbbd815e600 msgr2=0x7fbbd80c8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:12.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.999+0000 7fbbdf493640 1 --2- 172.21.15.89:0/540509578 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbbd815e600 0x7fbbd80c8960 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbbd806c800 tx=0x7fbbd401c470 comp rx=0 tx=0).stop 2023-10-18T10:47:12.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.999+0000 7fbbdf493640 1 -- 172.21.15.89:0/540509578 shutdown_connections 2023-10-18T10:47:12.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.999+0000 7fbbdf493640 1 --2- 172.21.15.89:0/540509578 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbbd815e600 0x7fbbd80c8960 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:12.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.999+0000 7fbbdf493640 1 -- 172.21.15.89:0/540509578 >> 172.21.15.89:0/540509578 conn(0x7fbbd8069f00 msgr2=0x7fbbd80cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:12.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.999+0000 7fbbdf493640 1 -- 172.21.15.89:0/540509578 shutdown_connections 2023-10-18T10:47:12.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:11.999+0000 7fbbdf493640 1 -- 172.21.15.89:0/540509578 wait complete. 2023-10-18T10:47:12.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.003+0000 7fbbdf493640 1 Processor -- start 2023-10-18T10:47:12.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.003+0000 7fbbdf493640 1 -- start start 2023-10-18T10:47:12.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.003+0000 7fbbdf493640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbbd806c1b0 0x7fbbd80ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:12.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.003+0000 7fbbdf493640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbd80afd90 0x7fbbd8157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:12.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.003+0000 7fbbdf493640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbbd815e600 0x7fbbd8158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:12.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.003+0000 7fbbdf493640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd8154590 con 0x7fbbd80afd90 2023-10-18T10:47:12.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.003+0000 7fbbdf493640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd815b130 con 0x7fbbd806c1b0 2023-10-18T10:47:12.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.003+0000 7fbbdf493640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd815b410 con 0x7fbbd815e600 2023-10-18T10:47:12.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.003+0000 7fbbdca2e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbd80afd90 0x7fbbd8157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:12.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.003+0000 7fbbdca2e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbd80afd90 0x7fbbd8157b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60092/0 (socket says 172.21.15.89:60092) 2023-10-18T10:47:12.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.003+0000 7fbbdca2e640 1 -- 172.21.15.89:0/72463721 learned_addr learned my addr 172.21.15.89:0/72463721 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:12.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.003+0000 7fbbdd22f640 1 --2- 172.21.15.89:0/72463721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbbd806c1b0 0x7fbbd80ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:12.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.003+0000 7fbbdda30640 1 --2- 172.21.15.89:0/72463721 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbbd815e600 0x7fbbd8158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:12.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.003+0000 7fbbdca2e640 1 -- 172.21.15.89:0/72463721 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbbd815e600 msgr2=0x7fbbd8158050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:12.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.003+0000 7fbbdca2e640 1 --2- 172.21.15.89:0/72463721 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbbd815e600 0x7fbbd8158050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:12.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.003+0000 7fbbdca2e640 1 -- 172.21.15.89:0/72463721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbbd806c1b0 msgr2=0x7fbbd80ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:12.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.003+0000 7fbbdca2e640 1 --2- 172.21.15.89:0/72463721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbbd806c1b0 0x7fbbd80ccd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:12.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.003+0000 7fbbdca2e640 1 -- 172.21.15.89:0/72463721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd4009420 con 0x7fbbd80afd90 2023-10-18T10:47:12.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.003+0000 7fbbdd22f640 1 --2- 172.21.15.89:0/72463721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbbd806c1b0 0x7fbbd80ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:47:12.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.003+0000 7fbbdca2e640 1 --2- 172.21.15.89:0/72463721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbd80afd90 0x7fbbd8157b10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fbbc800c4d0 tx=0x7fbbc800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:12.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.003+0000 7fbbcdffb640 1 -- 172.21.15.89:0/72463721 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc8005480 con 0x7fbbd80afd90 2023-10-18T10:47:12.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.003+0000 7fbbcdffb640 1 -- 172.21.15.89:0/72463721 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc8003600 con 0x7fbbd80afd90 2023-10-18T10:47:12.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.003+0000 7fbbdf493640 1 -- 172.21.15.89:0/72463721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbd815b6f0 con 0x7fbbd80afd90 2023-10-18T10:47:12.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.003+0000 7fbbcdffb640 1 -- 172.21.15.89:0/72463721 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc80039b0 con 0x7fbbd80afd90 2023-10-18T10:47:12.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.003+0000 7fbbdf493640 1 -- 172.21.15.89:0/72463721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbd8159cc0 con 0x7fbbd80afd90 2023-10-18T10:47:12.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.003+0000 7fbbdf493640 1 -- 172.21.15.89:0/72463721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbd804dbe0 con 0x7fbbd80afd90 2023-10-18T10:47:12.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.007+0000 7fbbcdffb640 1 -- 172.21.15.89:0/72463721 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fbbc8003b10 con 0x7fbbd80afd90 2023-10-18T10:47:12.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.007+0000 7fbbcdffb640 1 --2- 172.21.15.89:0/72463721 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fbbbc0656b0 0x7fbbbc067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:12.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.007+0000 7fbbcdffb640 1 -- 172.21.15.89:0/72463721 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fbbc8082290 con 0x7fbbd80afd90 2023-10-18T10:47:12.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.007+0000 7fbbdd22f640 1 --2- 172.21.15.89:0/72463721 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fbbbc0656b0 0x7fbbbc067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:12.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.007+0000 7fbbdd22f640 1 --2- 172.21.15.89:0/72463721 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fbbbc0656b0 0x7fbbbc067b70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbbd40af000 tx=0x7fbbd40a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:12.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.011+0000 7fbbcdffb640 1 -- 172.21.15.89:0/72463721 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fbbc8053a10 con 0x7fbbd80afd90 2023-10-18T10:47:12.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.159+0000 7fbbdf493640 1 -- 172.21.15.89:0/72463721 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbbd815bba0 con 0x7fbbd80afd90 2023-10-18T10:47:12.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.159+0000 7fbbcdffb640 1 -- 172.21.15.89:0/72463721 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbbc8017020 con 0x7fbbd80afd90 2023-10-18T10:47:12.162 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-18T10:47:12.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.159+0000 7fbbdf493640 1 -- 172.21.15.89:0/72463721 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fbbbc0656b0 msgr2=0x7fbbbc067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:12.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.159+0000 7fbbdf493640 1 --2- 172.21.15.89:0/72463721 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fbbbc0656b0 0x7fbbbc067b70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbbd40af000 tx=0x7fbbd40a7000 comp rx=0 tx=0).stop 2023-10-18T10:47:12.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.159+0000 7fbbdf493640 1 -- 172.21.15.89:0/72463721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbd80afd90 msgr2=0x7fbbd8157b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:12.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.159+0000 7fbbdf493640 1 --2- 172.21.15.89:0/72463721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbd80afd90 0x7fbbd8157b10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fbbc800c4d0 tx=0x7fbbc800c990 comp rx=0 tx=0).stop 2023-10-18T10:47:12.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.159+0000 7fbbdf493640 1 -- 172.21.15.89:0/72463721 shutdown_connections 2023-10-18T10:47:12.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.159+0000 7fbbdf493640 1 --2- 172.21.15.89:0/72463721 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fbbbc0656b0 0x7fbbbc067b70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:12.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.159+0000 7fbbdf493640 1 --2- 172.21.15.89:0/72463721 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbbd815e600 0x7fbbd8158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:12.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.159+0000 7fbbdf493640 1 --2- 172.21.15.89:0/72463721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbd80afd90 0x7fbbd8157b10 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:12.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.159+0000 7fbbdf493640 1 --2- 172.21.15.89:0/72463721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbbd806c1b0 0x7fbbd80ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:12.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.159+0000 7fbbdf493640 1 -- 172.21.15.89:0/72463721 >> 172.21.15.89:0/72463721 conn(0x7fbbd8069f00 msgr2=0x7fbbd815fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:12.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.159+0000 7fbbdf493640 1 -- 172.21.15.89:0/72463721 shutdown_connections 2023-10-18T10:47:12.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.159+0000 7fbbdf493640 1 -- 172.21.15.89:0/72463721 wait complete. 2023-10-18T10:47:12.182 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-18T10:47:12.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.319+0000 7f2959796640 1 Processor -- start 2023-10-18T10:47:12.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.319+0000 7f2959796640 1 -- start start 2023-10-18T10:47:12.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.319+0000 7f2959796640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f295415c550 0x7f29540d2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:12.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.319+0000 7f2959796640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f29540b1860 0x7f29540cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:12.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.319+0000 7f2959796640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f29540b1cc0 0x7f29540cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:12.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.319+0000 7f2959796640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29540d1e50 con 0x7f29540b1860 2023-10-18T10:47:12.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.319+0000 7f2959796640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29540d2130 con 0x7f29540b1cc0 2023-10-18T10:47:12.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.319+0000 7f2959796640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29540d2410 con 0x7f295415c550 2023-10-18T10:47:12.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.323+0000 7f29527fc640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f29540b1860 0x7f29540cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:12.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.323+0000 7f29527fc640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f29540b1860 0x7f29540cf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37792/0 (socket says 172.21.15.89:37792) 2023-10-18T10:47:12.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.323+0000 7f29527fc640 1 -- 172.21.15.89:0/1035450605 learned_addr learned my addr 172.21.15.89:0/1035450605 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:12.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.323+0000 7f29537fe640 1 --2- 172.21.15.89:0/1035450605 >> v2:172.21.15.154:3300/0 conn(0x7f29540b1cc0 0x7f29540cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:12.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.323+0000 7f2952ffd640 1 --2- 172.21.15.89:0/1035450605 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f295415c550 0x7f29540d2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:12.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.323+0000 7f29527fc640 1 -- 172.21.15.89:0/1035450605 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f295415c550 msgr2=0x7f29540d2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:12.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.323+0000 7f29527fc640 1 --2- 172.21.15.89:0/1035450605 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f295415c550 0x7f29540d2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:12.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.323+0000 7f29527fc640 1 -- 172.21.15.89:0/1035450605 >> v2:172.21.15.154:3300/0 conn(0x7f29540b1cc0 msgr2=0x7f29540cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:12.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.323+0000 7f29527fc640 1 --2- 172.21.15.89:0/1035450605 >> v2:172.21.15.154:3300/0 conn(0x7f29540b1cc0 0x7f29540cfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:12.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.323+0000 7f29527fc640 1 -- 172.21.15.89:0/1035450605 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29540d26f0 con 0x7f29540b1860 2023-10-18T10:47:12.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.323+0000 7f2952ffd640 1 --2- 172.21.15.89:0/1035450605 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f295415c550 0x7f29540d2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:47:12.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.323+0000 7f29527fc640 1 --2- 172.21.15.89:0/1035450605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29540b1860 0x7f29540cf610 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f29540ce8b0 tx=0x7f294801cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b7cae0abf5b30 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:12.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.323+0000 7f2951ffb640 1 -- 172.21.15.89:0/1035450605 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29480aa610 con 0x7f29540b1860 2023-10-18T10:47:12.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.323+0000 7f2951ffb640 1 -- 172.21.15.89:0/1035450605 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29480a6040 con 0x7f29540b1860 2023-10-18T10:47:12.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.323+0000 7f2951ffb640 1 -- 172.21.15.89:0/1035450605 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29480aa990 con 0x7f29540b1860 2023-10-18T10:47:12.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.327+0000 7f2959796640 1 -- 172.21.15.89:0/1035450605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29540b1860 msgr2=0x7f29540cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:12.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.327+0000 7f2959796640 1 --2- 172.21.15.89:0/1035450605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29540b1860 0x7f29540cf610 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f29540ce8b0 tx=0x7f294801cf30 comp rx=0 tx=0).stop 2023-10-18T10:47:12.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.327+0000 7f2959796640 1 -- 172.21.15.89:0/1035450605 shutdown_connections 2023-10-18T10:47:12.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.327+0000 7f2959796640 1 --2- 172.21.15.89:0/1035450605 >> v2:172.21.15.154:3300/0 conn(0x7f29540b1cc0 0x7f29540cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:12.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.327+0000 7f2959796640 1 --2- 172.21.15.89:0/1035450605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29540b1860 0x7f29540cf610 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:12.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.327+0000 7f2959796640 1 --2- 172.21.15.89:0/1035450605 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f295415c550 0x7f29540d2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:12.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.327+0000 7f2959796640 1 -- 172.21.15.89:0/1035450605 >> 172.21.15.89:0/1035450605 conn(0x7f2954069d00 msgr2=0x7f295406a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:12.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.327+0000 7f2959796640 1 -- 172.21.15.89:0/1035450605 shutdown_connections 2023-10-18T10:47:12.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.327+0000 7f2959796640 1 -- 172.21.15.89:0/1035450605 wait complete. 2023-10-18T10:47:12.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.327+0000 7f2959796640 1 Processor -- start 2023-10-18T10:47:12.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.327+0000 7f2959796640 1 -- start start 2023-10-18T10:47:12.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.327+0000 7f2959796640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29540b1860 0x7f2954156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:12.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.327+0000 7f2959796640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29540b1cc0 0x7f2954157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:12.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.327+0000 7f2959796640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f295415c550 0x7f2954152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:12.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.327+0000 7f2959796640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2954158050 con 0x7f295415c550 2023-10-18T10:47:12.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.327+0000 7f2959796640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2954157d70 con 0x7f29540b1cc0 2023-10-18T10:47:12.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.327+0000 7f2959796640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2954153e40 con 0x7f29540b1860 2023-10-18T10:47:12.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.327+0000 7f29537fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f295415c550 0x7f2954152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:12.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.327+0000 7f29537fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f295415c550 0x7f2954152100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37804/0 (socket says 172.21.15.89:37804) 2023-10-18T10:47:12.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.327+0000 7f29537fe640 1 -- 172.21.15.89:0/970692377 learned_addr learned my addr 172.21.15.89:0/970692377 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:12.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.327+0000 7f2952ffd640 1 --2- 172.21.15.89:0/970692377 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29540b1860 0x7f2954156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:12.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.327+0000 7f29527fc640 1 --2- 172.21.15.89:0/970692377 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29540b1cc0 0x7f2954157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:12.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.331+0000 7f29537fe640 1 -- 172.21.15.89:0/970692377 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29540b1860 msgr2=0x7f2954156fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:12.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.331+0000 7f29537fe640 1 --2- 172.21.15.89:0/970692377 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29540b1860 0x7f2954156fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:12.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.331+0000 7f29537fe640 1 -- 172.21.15.89:0/970692377 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29540b1cc0 msgr2=0x7f2954157510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:12.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.331+0000 7f29537fe640 1 --2- 172.21.15.89:0/970692377 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29540b1cc0 0x7f2954157510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:12.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.331+0000 7f29537fe640 1 -- 172.21.15.89:0/970692377 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29480094d0 con 0x7f295415c550 2023-10-18T10:47:12.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.331+0000 7f29537fe640 1 --2- 172.21.15.89:0/970692377 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f295415c550 0x7f2954152100 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f294c00d5a0 tx=0x7f294c00da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:12.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.331+0000 7f293b7fe640 1 -- 172.21.15.89:0/970692377 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c0125f0 con 0x7f295415c550 2023-10-18T10:47:12.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.331+0000 7f293b7fe640 1 -- 172.21.15.89:0/970692377 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294c012770 con 0x7f295415c550 2023-10-18T10:47:12.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.331+0000 7f293b7fe640 1 -- 172.21.15.89:0/970692377 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c012b20 con 0x7f295415c550 2023-10-18T10:47:12.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.331+0000 7f2959796640 1 -- 172.21.15.89:0/970692377 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2954154120 con 0x7f295415c550 2023-10-18T10:47:12.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.331+0000 7f2959796640 1 -- 172.21.15.89:0/970692377 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29541f5940 con 0x7f295415c550 2023-10-18T10:47:12.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.331+0000 7f2959796640 1 -- 172.21.15.89:0/970692377 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f295404dbe0 con 0x7f295415c550 2023-10-18T10:47:12.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.331+0000 7f293b7fe640 1 -- 172.21.15.89:0/970692377 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f294c012c80 con 0x7f295415c550 2023-10-18T10:47:12.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.331+0000 7f293b7fe640 1 --2- 172.21.15.89:0/970692377 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2940065680 0x7f2940067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:12.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.331+0000 7f293b7fe640 1 -- 172.21.15.89:0/970692377 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f294c082fa0 con 0x7f295415c550 2023-10-18T10:47:12.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.331+0000 7f2952ffd640 1 --2- 172.21.15.89:0/970692377 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2940065680 0x7f2940067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:12.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.335+0000 7f2952ffd640 1 --2- 172.21.15.89:0/970692377 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2940065680 0x7f2940067b40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f29540cce90 tx=0x7f2944007770 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:12.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.335+0000 7f293b7fe640 1 -- 172.21.15.89:0/970692377 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f294c050d10 con 0x7f295415c550 2023-10-18T10:47:12.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.483+0000 7f2959796640 1 -- 172.21.15.89:0/970692377 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f29541545d0 con 0x7f295415c550 2023-10-18T10:47:12.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.483+0000 7f293b7fe640 1 -- 172.21.15.89:0/970692377 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v37) v1 ==== 92+0+10 (secure 0 0 0) 0x7f294c054330 con 0x7f295415c550 2023-10-18T10:47:12.486 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-10-18T10:47:12.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.487+0000 7f2959796640 1 -- 172.21.15.89:0/970692377 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2940065680 msgr2=0x7f2940067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:12.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.487+0000 7f2959796640 1 --2- 172.21.15.89:0/970692377 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2940065680 0x7f2940067b40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f29540cce90 tx=0x7f2944007770 comp rx=0 tx=0).stop 2023-10-18T10:47:12.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.487+0000 7f2959796640 1 -- 172.21.15.89:0/970692377 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f295415c550 msgr2=0x7f2954152100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:12.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.487+0000 7f2959796640 1 --2- 172.21.15.89:0/970692377 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f295415c550 0x7f2954152100 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f294c00d5a0 tx=0x7f294c00da60 comp rx=0 tx=0).stop 2023-10-18T10:47:12.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.487+0000 7f2959796640 1 -- 172.21.15.89:0/970692377 shutdown_connections 2023-10-18T10:47:12.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.487+0000 7f2959796640 1 --2- 172.21.15.89:0/970692377 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2940065680 0x7f2940067b40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:12.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.487+0000 7f2959796640 1 --2- 172.21.15.89:0/970692377 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f295415c550 0x7f2954152100 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:12.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.487+0000 7f2959796640 1 --2- 172.21.15.89:0/970692377 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29540b1cc0 0x7f2954157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:12.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.487+0000 7f2959796640 1 --2- 172.21.15.89:0/970692377 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29540b1860 0x7f2954156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:12.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.487+0000 7f2959796640 1 -- 172.21.15.89:0/970692377 >> 172.21.15.89:0/970692377 conn(0x7f2954069d00 msgr2=0x7f29540b5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:12.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.487+0000 7f2959796640 1 -- 172.21.15.89:0/970692377 shutdown_connections 2023-10-18T10:47:12.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.487+0000 7f2959796640 1 -- 172.21.15.89:0/970692377 wait complete. 2023-10-18T10:47:12.507 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-18T10:47:12.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.643+0000 7f3da9826640 1 Processor -- start 2023-10-18T10:47:12.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.643+0000 7f3da9826640 1 -- start start 2023-10-18T10:47:12.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.643+0000 7f3da9826640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3da415c940 0x7f3da40d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:12.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.643+0000 7f3da9826640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3da40d0e70 0x7f3da40cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:12.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.643+0000 7f3da9826640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3da415da50 con 0x7f3da40d2f40 2023-10-18T10:47:12.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.643+0000 7f3da9826640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3da415ce40 con 0x7f3da415c940 2023-10-18T10:47:12.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.643+0000 7f3da9826640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3da415d120 con 0x7f3da40d0e70 2023-10-18T10:47:12.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.643+0000 7f3da2ffd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3da415c940 0x7f3da40d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:12.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.643+0000 7f3d9a7fc640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f3da40d2f40 0x7f3da40d0760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40606/0 (socket says 172.21.15.89:40606) 2023-10-18T10:47:12.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.643+0000 7f3d9a7fc640 1 -- 172.21.15.89:0/398863783 learned_addr learned my addr 172.21.15.89:0/398863783 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:47:12.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.643+0000 7f3da37fe640 1 --2- 172.21.15.89:0/398863783 >> v2:172.21.15.154:3300/0 conn(0x7f3da40d0e70 0x7f3da40cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:12.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.643+0000 7f3da2ffd640 1 -- 172.21.15.89:0/398863783 >> v2:172.21.15.154:3300/0 conn(0x7f3da40d0e70 msgr2=0x7f3da40cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:12.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.643+0000 7f3da2ffd640 1 --2- 172.21.15.89:0/398863783 >> v2:172.21.15.154:3300/0 conn(0x7f3da40d0e70 0x7f3da40cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:12.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.643+0000 7f3da2ffd640 1 -- 172.21.15.89:0/398863783 >> v1:172.21.15.89:6789/0 conn(0x7f3da40d2f40 legacy=0x7f3da40d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:12.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.643+0000 7f3da2ffd640 1 -- 172.21.15.89:0/398863783 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da415d400 con 0x7f3da415c940 2023-10-18T10:47:12.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.643+0000 7f3da27fc640 1 -- 172.21.15.89:0/398863783 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3868323121 0 0) 0x7f3da415da50 con 0x7f3da40d2f40 2023-10-18T10:47:12.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.647+0000 7f3da2ffd640 1 --2- 172.21.15.89:0/398863783 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3da415c940 0x7f3da40d2a00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3da408cb20 tx=0x7f3d9401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55dbd20bce083a23 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:12.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.647+0000 7f3da27fc640 1 -- 172.21.15.89:0/398863783 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d940a5940 con 0x7f3da415c940 2023-10-18T10:47:12.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.647+0000 7f3da27fc640 1 -- 172.21.15.89:0/398863783 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d940a5aa0 con 0x7f3da415c940 2023-10-18T10:47:12.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.647+0000 7f3da27fc640 1 -- 172.21.15.89:0/398863783 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d940a5e30 con 0x7f3da415c940 2023-10-18T10:47:12.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.647+0000 7f3da9826640 1 -- 172.21.15.89:0/398863783 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3da415c940 msgr2=0x7f3da40d2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:12.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.647+0000 7f3da9826640 1 --2- 172.21.15.89:0/398863783 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3da415c940 0x7f3da40d2a00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3da408cb20 tx=0x7f3d9401c8e0 comp rx=0 tx=0).stop 2023-10-18T10:47:12.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.647+0000 7f3da9826640 1 -- 172.21.15.89:0/398863783 shutdown_connections 2023-10-18T10:47:12.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.647+0000 7f3da9826640 1 --2- 172.21.15.89:0/398863783 >> v2:172.21.15.154:3300/0 conn(0x7f3da40d0e70 0x7f3da40cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:12.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.647+0000 7f3da9826640 1 --2- 172.21.15.89:0/398863783 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3da415c940 0x7f3da40d2a00 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:12.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.647+0000 7f3da9826640 1 -- 172.21.15.89:0/398863783 >> 172.21.15.89:0/398863783 conn(0x7f3da4089fd0 msgr2=0x7f3da409b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:12.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.647+0000 7f3da9826640 1 -- 172.21.15.89:0/398863783 shutdown_connections 2023-10-18T10:47:12.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.647+0000 7f3da9826640 1 -- 172.21.15.89:0/398863783 wait complete. 2023-10-18T10:47:12.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.651+0000 7f3da9826640 1 Processor -- start 2023-10-18T10:47:12.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.651+0000 7f3da9826640 1 -- start start 2023-10-18T10:47:12.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.651+0000 7f3da9826640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3da40d0e70 0x7f3da4159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:12.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.651+0000 7f3da9826640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3da414ee90 0x7f3da414cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:12.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.651+0000 7f3da9826640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3da4156910 0x7f3da40c9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:12.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.651+0000 7f3da9826640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da4157280 con 0x7f3da40d0e70 2023-10-18T10:47:12.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.651+0000 7f3da9826640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da4157560 con 0x7f3da414ee90 2023-10-18T10:47:12.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.651+0000 7f3da9826640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da4157840 con 0x7f3da4156910 2023-10-18T10:47:12.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.651+0000 7f3da2ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3da40d0e70 0x7f3da4159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:12.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.651+0000 7f3da2ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3da40d0e70 0x7f3da4159e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37826/0 (socket says 172.21.15.89:37826) 2023-10-18T10:47:12.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.651+0000 7f3da2ffd640 1 -- 172.21.15.89:0/1475687002 learned_addr learned my addr 172.21.15.89:0/1475687002 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:12.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.651+0000 7f3d9a7fc640 1 --2- 172.21.15.89:0/1475687002 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3da414ee90 0x7f3da414cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:12.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.651+0000 7f3da37fe640 1 --2- 172.21.15.89:0/1475687002 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3da4156910 0x7f3da40c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:12.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.651+0000 7f3da2ffd640 1 -- 172.21.15.89:0/1475687002 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3da4156910 msgr2=0x7f3da40c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:12.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.651+0000 7f3da2ffd640 1 --2- 172.21.15.89:0/1475687002 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3da4156910 0x7f3da40c9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:12.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.651+0000 7f3da2ffd640 1 -- 172.21.15.89:0/1475687002 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3da414ee90 msgr2=0x7f3da414cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:12.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.651+0000 7f3da2ffd640 1 --2- 172.21.15.89:0/1475687002 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3da414ee90 0x7f3da414cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:12.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.651+0000 7f3da2ffd640 1 -- 172.21.15.89:0/1475687002 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d94009a70 con 0x7f3da40d0e70 2023-10-18T10:47:12.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.651+0000 7f3da2ffd640 1 --2- 172.21.15.89:0/1475687002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3da40d0e70 0x7f3da4159e90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3d94000f90 tx=0x7f3d94019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:12.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.651+0000 7f3da0ff9640 1 -- 172.21.15.89:0/1475687002 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d940aee00 con 0x7f3da40d0e70 2023-10-18T10:47:12.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.651+0000 7f3da0ff9640 1 -- 172.21.15.89:0/1475687002 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d94009450 con 0x7f3da40d0e70 2023-10-18T10:47:12.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.651+0000 7f3da0ff9640 1 -- 172.21.15.89:0/1475687002 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d940b7610 con 0x7f3da40d0e70 2023-10-18T10:47:12.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.651+0000 7f3da9826640 1 -- 172.21.15.89:0/1475687002 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3da414d9d0 con 0x7f3da40d0e70 2023-10-18T10:47:12.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.651+0000 7f3da9826640 1 -- 172.21.15.89:0/1475687002 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3da414d6a0 con 0x7f3da40d0e70 2023-10-18T10:47:12.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.655+0000 7f3da9826640 1 -- 172.21.15.89:0/1475687002 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3da404dbe0 con 0x7f3da40d0e70 2023-10-18T10:47:12.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.655+0000 7f3da0ff9640 1 -- 172.21.15.89:0/1475687002 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f3d940b4050 con 0x7f3da40d0e70 2023-10-18T10:47:12.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.655+0000 7f3da0ff9640 1 --2- 172.21.15.89:0/1475687002 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3d84065680 0x7f3d84067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:12.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.655+0000 7f3da0ff9640 1 -- 172.21.15.89:0/1475687002 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f3d9411df40 con 0x7f3da40d0e70 2023-10-18T10:47:12.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.655+0000 7f3d9a7fc640 1 --2- 172.21.15.89:0/1475687002 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3d84065680 0x7f3d84067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:12.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.659+0000 7f3d9a7fc640 1 --2- 172.21.15.89:0/1475687002 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3d84065680 0x7f3d84067b40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3da40c95c0 tx=0x7f3d8c007b60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:12.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.659+0000 7f3da0ff9640 1 -- 172.21.15.89:0/1475687002 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f3d940ebc90 con 0x7f3da40d0e70 2023-10-18T10:47:12.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.815+0000 7f3da9826640 1 -- 172.21.15.89:0/1475687002 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3da40c56a0 con 0x7f3da40d0e70 2023-10-18T10:47:12.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.815+0000 7f3da0ff9640 1 -- 172.21.15.89:0/1475687002 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v37) v1 ==== 84+0+126 (secure 0 0 0) 0x7f3d940b2090 con 0x7f3da40d0e70 2023-10-18T10:47:12.818 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:47:12.819 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-10-18T10:47:12.819 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-18T10:47:12.819 INFO:teuthology.orchestra.run.smithi089.stdout: "poolnum": 1, 2023-10-18T10:47:12.819 INFO:teuthology.orchestra.run.smithi089.stdout: "poolname": ".mgr" 2023-10-18T10:47:12.819 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-18T10:47:12.820 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-18T10:47:12.820 INFO:teuthology.orchestra.run.smithi089.stdout: "poolnum": 2, 2023-10-18T10:47:12.820 INFO:teuthology.orchestra.run.smithi089.stdout: "poolname": "rbd" 2023-10-18T10:47:12.820 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-18T10:47:12.820 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-10-18T10:47:12.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.819+0000 7f3da9826640 1 -- 172.21.15.89:0/1475687002 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3d84065680 msgr2=0x7f3d84067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:12.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.819+0000 7f3da9826640 1 --2- 172.21.15.89:0/1475687002 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3d84065680 0x7f3d84067b40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3da40c95c0 tx=0x7f3d8c007b60 comp rx=0 tx=0).stop 2023-10-18T10:47:12.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.819+0000 7f3da9826640 1 -- 172.21.15.89:0/1475687002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3da40d0e70 msgr2=0x7f3da4159e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:12.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.819+0000 7f3da9826640 1 --2- 172.21.15.89:0/1475687002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3da40d0e70 0x7f3da4159e90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3d94000f90 tx=0x7f3d94019680 comp rx=0 tx=0).stop 2023-10-18T10:47:12.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.819+0000 7f3da9826640 1 -- 172.21.15.89:0/1475687002 shutdown_connections 2023-10-18T10:47:12.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.819+0000 7f3da9826640 1 --2- 172.21.15.89:0/1475687002 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3d84065680 0x7f3d84067b40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:12.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.819+0000 7f3da9826640 1 --2- 172.21.15.89:0/1475687002 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3da4156910 0x7f3da40c9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:12.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.819+0000 7f3da9826640 1 --2- 172.21.15.89:0/1475687002 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3da414ee90 0x7f3da414cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:12.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.819+0000 7f3da9826640 1 --2- 172.21.15.89:0/1475687002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3da40d0e70 0x7f3da4159e90 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:12.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.819+0000 7f3da9826640 1 -- 172.21.15.89:0/1475687002 >> 172.21.15.89:0/1475687002 conn(0x7f3da4089fd0 msgr2=0x7f3da415f130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:12.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.819+0000 7f3da9826640 1 -- 172.21.15.89:0/1475687002 shutdown_connections 2023-10-18T10:47:12.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.819+0000 7f3da9826640 1 -- 172.21.15.89:0/1475687002 wait complete. 2023-10-18T10:47:12.839 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-18T10:47:12.839 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-18T10:47:12.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.975+0000 7f25b7137640 1 Processor -- start 2023-10-18T10:47:12.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.979+0000 7f25b7137640 1 -- start start 2023-10-18T10:47:12.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.979+0000 7f25b7137640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f25b00d0cd0 0x7f25b00d25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:12.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.979+0000 7f25b7137640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f25b00d2ae0 0x7f25b00cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:12.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.979+0000 7f25b7137640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b0069900 con 0x7f25b015c9e0 2023-10-18T10:47:12.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.979+0000 7f25b7137640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25b00d1160 con 0x7f25b00d2ae0 2023-10-18T10:47:12.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.979+0000 7f25b7137640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25b00d1440 con 0x7f25b00d0cd0 2023-10-18T10:47:12.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.979+0000 7f25b56d4640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f25b00d2ae0 0x7f25b00cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:12.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.979+0000 7f25b4ed3640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f25b015c9e0 0x7f25b0160dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40620/0 (socket says 172.21.15.89:40620) 2023-10-18T10:47:12.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.979+0000 7f25b4ed3640 1 -- 172.21.15.89:0/1358348450 learned_addr learned my addr 172.21.15.89:0/1358348450 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:47:12.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.979+0000 7f25a7fff640 1 --2- 172.21.15.89:0/1358348450 >> v2:172.21.15.154:3300/0 conn(0x7f25b00d0cd0 0x7f25b00d25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:12.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.979+0000 7f25b56d4640 1 -- 172.21.15.89:0/1358348450 >> v2:172.21.15.154:3300/0 conn(0x7f25b00d0cd0 msgr2=0x7f25b00d25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:12.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.979+0000 7f25b56d4640 1 --2- 172.21.15.89:0/1358348450 >> v2:172.21.15.154:3300/0 conn(0x7f25b00d0cd0 0x7f25b00d25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:12.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.979+0000 7f25b56d4640 1 -- 172.21.15.89:0/1358348450 >> v1:172.21.15.89:6789/0 conn(0x7f25b015c9e0 legacy=0x7f25b0160dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:12.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.979+0000 7f25b56d4640 1 -- 172.21.15.89:0/1358348450 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b00cd4c0 con 0x7f25b00d2ae0 2023-10-18T10:47:12.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.979+0000 7f25a7fff640 1 --2- 172.21.15.89:0/1358348450 >> v2:172.21.15.154:3300/0 conn(0x7f25b00d0cd0 0x7f25b00d25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:47:12.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.983+0000 7f25b56d4640 1 --2- 172.21.15.89:0/1358348450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25b00d2ae0 0x7f25b00cdcc0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f25b00cccf0 tx=0x7f25a801c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38bdf4299b0e9cb server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:12.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.983+0000 7f25a77fe640 1 -- 172.21.15.89:0/1358348450 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a80a5830 con 0x7f25b00d2ae0 2023-10-18T10:47:12.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.983+0000 7f25a77fe640 1 -- 172.21.15.89:0/1358348450 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a80a5990 con 0x7f25b00d2ae0 2023-10-18T10:47:12.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.983+0000 7f25a77fe640 1 -- 172.21.15.89:0/1358348450 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a80a5d20 con 0x7f25b00d2ae0 2023-10-18T10:47:12.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.983+0000 7f25b7137640 1 -- 172.21.15.89:0/1358348450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25b00d2ae0 msgr2=0x7f25b00cdcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:12.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.983+0000 7f25b7137640 1 --2- 172.21.15.89:0/1358348450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25b00d2ae0 0x7f25b00cdcc0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f25b00cccf0 tx=0x7f25a801c880 comp rx=0 tx=0).stop 2023-10-18T10:47:12.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.983+0000 7f25b7137640 1 -- 172.21.15.89:0/1358348450 shutdown_connections 2023-10-18T10:47:12.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.983+0000 7f25b7137640 1 --2- 172.21.15.89:0/1358348450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25b00d2ae0 0x7f25b00cdcc0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:12.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.983+0000 7f25b7137640 1 --2- 172.21.15.89:0/1358348450 >> v2:172.21.15.154:3300/0 conn(0x7f25b00d0cd0 0x7f25b00d25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:12.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.983+0000 7f25b7137640 1 -- 172.21.15.89:0/1358348450 >> 172.21.15.89:0/1358348450 conn(0x7f25b008c0b0 msgr2=0x7f25b009b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:12.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.983+0000 7f25b7137640 1 -- 172.21.15.89:0/1358348450 shutdown_connections 2023-10-18T10:47:12.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.983+0000 7f25b7137640 1 -- 172.21.15.89:0/1358348450 wait complete. 2023-10-18T10:47:12.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.983+0000 7f25b7137640 1 Processor -- start 2023-10-18T10:47:12.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.983+0000 7f25b7137640 1 -- start start 2023-10-18T10:47:12.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.987+0000 7f25b7137640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25b00d2ae0 0x7f25b0154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:12.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.987+0000 7f25b7137640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25b015c9e0 0x7f25b00cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:12.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.987+0000 7f25b7137640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25b00ca720 0x7f25b00cb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:12.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.987+0000 7f25b7137640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b01552f0 con 0x7f25b00d2ae0 2023-10-18T10:47:12.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.987+0000 7f25b7137640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b00cb7a0 con 0x7f25b00ca720 2023-10-18T10:47:12.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.987+0000 7f25b7137640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b00c9960 con 0x7f25b015c9e0 2023-10-18T10:47:12.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.987+0000 7f25b4ed3640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25b00d2ae0 0x7f25b0154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:12.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.987+0000 7f25b4ed3640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25b00d2ae0 0x7f25b0154db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37840/0 (socket says 172.21.15.89:37840) 2023-10-18T10:47:12.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.987+0000 7f25b4ed3640 1 -- 172.21.15.89:0/4040048435 learned_addr learned my addr 172.21.15.89:0/4040048435 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:12.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.987+0000 7f25b56d4640 1 --2- 172.21.15.89:0/4040048435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25b00ca720 0x7f25b00cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:12.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.987+0000 7f25a7fff640 1 --2- 172.21.15.89:0/4040048435 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25b015c9e0 0x7f25b00cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:12.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.987+0000 7f25b4ed3640 1 -- 172.21.15.89:0/4040048435 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25b015c9e0 msgr2=0x7f25b00cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:12.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.987+0000 7f25b4ed3640 1 --2- 172.21.15.89:0/4040048435 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25b015c9e0 0x7f25b00cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:12.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.987+0000 7f25b4ed3640 1 -- 172.21.15.89:0/4040048435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25b00ca720 msgr2=0x7f25b00cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:12.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.987+0000 7f25b4ed3640 1 --2- 172.21.15.89:0/4040048435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25b00ca720 0x7f25b00cb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:12.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.987+0000 7f25b4ed3640 1 -- 172.21.15.89:0/4040048435 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a8009a70 con 0x7f25b00d2ae0 2023-10-18T10:47:12.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.987+0000 7f25b4ed3640 1 --2- 172.21.15.89:0/4040048435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25b00d2ae0 0x7f25b0154db0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f25ac007fd0 tx=0x7f25ac00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:12.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.987+0000 7f25a57fa640 1 -- 172.21.15.89:0/4040048435 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac0074e0 con 0x7f25b00d2ae0 2023-10-18T10:47:12.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.987+0000 7f25a57fa640 1 -- 172.21.15.89:0/4040048435 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25ac007640 con 0x7f25b00d2ae0 2023-10-18T10:47:12.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.987+0000 7f25b7137640 1 -- 172.21.15.89:0/4040048435 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b00c9c40 con 0x7f25b00d2ae0 2023-10-18T10:47:12.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.987+0000 7f25a57fa640 1 -- 172.21.15.89:0/4040048435 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac0059b0 con 0x7f25b00d2ae0 2023-10-18T10:47:12.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.987+0000 7f25b7137640 1 -- 172.21.15.89:0/4040048435 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b00ca200 con 0x7f25b00d2ae0 2023-10-18T10:47:12.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.987+0000 7f25b7137640 1 -- 172.21.15.89:0/4040048435 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25b004dbe0 con 0x7f25b00d2ae0 2023-10-18T10:47:12.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.987+0000 7f25a57fa640 1 -- 172.21.15.89:0/4040048435 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f25ac005b10 con 0x7f25b00d2ae0 2023-10-18T10:47:12.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.991+0000 7f25a57fa640 1 --2- 172.21.15.89:0/4040048435 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f259c065680 0x7f259c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:12.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.991+0000 7f25a57fa640 1 -- 172.21.15.89:0/4040048435 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f25ac081ff0 con 0x7f25b00d2ae0 2023-10-18T10:47:12.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.991+0000 7f25a7fff640 1 --2- 172.21.15.89:0/4040048435 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f259c065680 0x7f259c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:12.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.991+0000 7f25a7fff640 1 --2- 172.21.15.89:0/4040048435 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f259c065680 0x7f259c067b40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f25a0006fd0 tx=0x7f25a0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:12.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:12.995+0000 7f25a57fa640 1 -- 172.21.15.89:0/4040048435 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f25ac04fdc0 con 0x7f25b00d2ae0 2023-10-18T10:47:13.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.143+0000 7f25b7137640 1 -- 172.21.15.89:0/4040048435 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f25b00a0380 con 0x7f25b00d2ae0 2023-10-18T10:47:13.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.807+0000 7f25a57fa640 1 -- 172.21.15.89:0/4040048435 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v38) v1 ==== 138+0+0 (secure 0 0 0) 0x7f25ac0533e0 con 0x7f25b00d2ae0 2023-10-18T10:47:13.812 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' created 2023-10-18T10:47:13.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.811+0000 7f25b7137640 1 -- 172.21.15.89:0/4040048435 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f259c065680 msgr2=0x7f259c067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:13.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.811+0000 7f25b7137640 1 --2- 172.21.15.89:0/4040048435 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f259c065680 0x7f259c067b40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f25a0006fd0 tx=0x7f25a0008040 comp rx=0 tx=0).stop 2023-10-18T10:47:13.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.811+0000 7f25b7137640 1 -- 172.21.15.89:0/4040048435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25b00d2ae0 msgr2=0x7f25b0154db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:13.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.811+0000 7f25b7137640 1 --2- 172.21.15.89:0/4040048435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25b00d2ae0 0x7f25b0154db0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f25ac007fd0 tx=0x7f25ac00c890 comp rx=0 tx=0).stop 2023-10-18T10:47:13.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.811+0000 7f25b7137640 1 -- 172.21.15.89:0/4040048435 shutdown_connections 2023-10-18T10:47:13.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.811+0000 7f25b7137640 1 --2- 172.21.15.89:0/4040048435 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f259c065680 0x7f259c067b40 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:13.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.811+0000 7f25b7137640 1 --2- 172.21.15.89:0/4040048435 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25b015c9e0 0x7f25b00cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:13.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.811+0000 7f25b7137640 1 --2- 172.21.15.89:0/4040048435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25b00ca720 0x7f25b00cb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:13.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.811+0000 7f25b7137640 1 --2- 172.21.15.89:0/4040048435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25b00d2ae0 0x7f25b0154db0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:13.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.811+0000 7f25b7137640 1 -- 172.21.15.89:0/4040048435 >> 172.21.15.89:0/4040048435 conn(0x7f25b008c0b0 msgr2=0x7f25b006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:13.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.811+0000 7f25b7137640 1 -- 172.21.15.89:0/4040048435 shutdown_connections 2023-10-18T10:47:13.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.811+0000 7f25b7137640 1 -- 172.21.15.89:0/4040048435 wait complete. 2023-10-18T10:47:13.840 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-18T10:47:13.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.983+0000 7f5a97e6d640 1 Processor -- start 2023-10-18T10:47:13.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.983+0000 7f5a97e6d640 1 -- start start 2023-10-18T10:47:13.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.983+0000 7f5a97e6d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a9015c940 0x7f5a900d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:13.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.983+0000 7f5a97e6d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5a900d0e70 0x7f5a900cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:13.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.983+0000 7f5a97e6d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a9015da50 con 0x7f5a900d2f40 2023-10-18T10:47:13.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.983+0000 7f5a97e6d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a9015ce40 con 0x7f5a900d0e70 2023-10-18T10:47:13.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.983+0000 7f5a97e6d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9015d120 con 0x7f5a9015c940 2023-10-18T10:47:13.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.983+0000 7f5a95c09640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a9015c940 0x7f5a900d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:13.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.983+0000 7f5a9640a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5a900d0e70 0x7f5a900cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:13.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.983+0000 7f5a95c09640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a9015c940 0x7f5a900d2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:50450/0 (socket says 172.21.15.89:50450) 2023-10-18T10:47:13.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.983+0000 7f5a95c09640 1 -- 172.21.15.89:0/3128060841 learned_addr learned my addr 172.21.15.89:0/3128060841 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:13.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.983+0000 7f5a9640a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5a900d0e70 0x7f5a900cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:45448/0 (socket says 172.21.15.89:45448) 2023-10-18T10:47:13.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.983+0000 7f5a95408640 1 --1- 172.21.15.89:0/3128060841 >> v1:172.21.15.154:6789/0 conn(0x7f5a900d2f40 0x7f5a900d0760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.89:52146/0 (socket says 172.21.15.89:52146) 2023-10-18T10:47:13.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.983+0000 7f5a95c09640 1 -- 172.21.15.89:0/3128060841 >> v2:172.21.15.154:3300/0 conn(0x7f5a900d0e70 msgr2=0x7f5a900cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:13.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.983+0000 7f5a95c09640 1 --2- 172.21.15.89:0/3128060841 >> v2:172.21.15.154:3300/0 conn(0x7f5a900d0e70 0x7f5a900cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:13.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.983+0000 7f5a95c09640 1 -- 172.21.15.89:0/3128060841 >> v1:172.21.15.154:6789/0 conn(0x7f5a900d2f40 legacy=0x7f5a900d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:13.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.987+0000 7f5a95c09640 1 -- 172.21.15.89:0/3128060841 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a9015d400 con 0x7f5a9015c940 2023-10-18T10:47:13.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.987+0000 7f5a9640a640 1 --2- 172.21.15.89:0/3128060841 >> v2:172.21.15.154:3300/0 conn(0x7f5a900d0e70 0x7f5a900cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:47:13.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.987+0000 7f5a94c07640 1 -- 172.21.15.89:0/3128060841 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2996714948 0 0) 0x7f5a9015da50 con 0x7f5a900d2f40 2023-10-18T10:47:13.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.987+0000 7f5a95c09640 1 --2- 172.21.15.89:0/3128060841 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a9015c940 0x7f5a900d2a00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c0098a0 tx=0x7f5a8c01c4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c89e1f89e7dc1a2a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:13.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.987+0000 7f5a94c07640 1 -- 172.21.15.89:0/3128060841 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c0aa5f0 con 0x7f5a9015c940 2023-10-18T10:47:13.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.987+0000 7f5a94c07640 1 -- 172.21.15.89:0/3128060841 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8c0aa750 con 0x7f5a9015c940 2023-10-18T10:47:13.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.987+0000 7f5a94c07640 1 -- 172.21.15.89:0/3128060841 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c0aaae0 con 0x7f5a9015c940 2023-10-18T10:47:13.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.987+0000 7f5a97e6d640 1 -- 172.21.15.89:0/3128060841 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a9015c940 msgr2=0x7f5a900d2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:13.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.987+0000 7f5a97e6d640 1 --2- 172.21.15.89:0/3128060841 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a9015c940 0x7f5a900d2a00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c0098a0 tx=0x7f5a8c01c4f0 comp rx=0 tx=0).stop 2023-10-18T10:47:13.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.987+0000 7f5a97e6d640 1 -- 172.21.15.89:0/3128060841 shutdown_connections 2023-10-18T10:47:13.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.987+0000 7f5a97e6d640 1 --2- 172.21.15.89:0/3128060841 >> v2:172.21.15.154:3300/0 conn(0x7f5a900d0e70 0x7f5a900cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:13.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.987+0000 7f5a97e6d640 1 --2- 172.21.15.89:0/3128060841 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a9015c940 0x7f5a900d2a00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:13.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.987+0000 7f5a97e6d640 1 -- 172.21.15.89:0/3128060841 >> 172.21.15.89:0/3128060841 conn(0x7f5a90089fd0 msgr2=0x7f5a9009b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:13.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.991+0000 7f5a97e6d640 1 -- 172.21.15.89:0/3128060841 shutdown_connections 2023-10-18T10:47:13.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.991+0000 7f5a97e6d640 1 -- 172.21.15.89:0/3128060841 wait complete. 2023-10-18T10:47:13.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.991+0000 7f5a97e6d640 1 Processor -- start 2023-10-18T10:47:13.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.991+0000 7f5a97e6d640 1 -- start start 2023-10-18T10:47:13.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.991+0000 7f5a97e6d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a900d0e70 0x7f5a900c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:13.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.991+0000 7f5a97e6d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a900c7b30 0x7f5a90154120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:13.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.991+0000 7f5a97e6d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a9015ebe0 0x7f5a9015f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:13.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.991+0000 7f5a97e6d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9015f5d0 con 0x7f5a900c7b30 2023-10-18T10:47:13.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.991+0000 7f5a97e6d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a900c8380 con 0x7f5a9015ebe0 2023-10-18T10:47:13.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.991+0000 7f5a97e6d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a901554c0 con 0x7f5a900d0e70 2023-10-18T10:47:13.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.991+0000 7f5a95408640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a900c7b30 0x7f5a90154120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:13.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.991+0000 7f5a95408640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a900c7b30 0x7f5a90154120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37850/0 (socket says 172.21.15.89:37850) 2023-10-18T10:47:13.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.991+0000 7f5a95408640 1 -- 172.21.15.89:0/2214049405 learned_addr learned my addr 172.21.15.89:0/2214049405 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:13.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.991+0000 7f5a9640a640 1 --2- 172.21.15.89:0/2214049405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a9015ebe0 0x7f5a9015f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:13.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.991+0000 7f5a95c09640 1 --2- 172.21.15.89:0/2214049405 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a900d0e70 0x7f5a900c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:13.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.991+0000 7f5a95c09640 1 -- 172.21.15.89:0/2214049405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a9015ebe0 msgr2=0x7f5a9015f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:14.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.991+0000 7f5a95c09640 1 --2- 172.21.15.89:0/2214049405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a9015ebe0 0x7f5a9015f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:14.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.991+0000 7f5a95c09640 1 -- 172.21.15.89:0/2214049405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a900c7b30 msgr2=0x7f5a90154120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:14.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.991+0000 7f5a95c09640 1 --2- 172.21.15.89:0/2214049405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a900c7b30 0x7f5a90154120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:14.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.991+0000 7f5a95c09640 1 -- 172.21.15.89:0/2214049405 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a88009a70 con 0x7f5a900d0e70 2023-10-18T10:47:14.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.991+0000 7f5a9640a640 1 --2- 172.21.15.89:0/2214049405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a9015ebe0 0x7f5a9015f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:47:14.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.991+0000 7f5a95408640 1 --2- 172.21.15.89:0/2214049405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a900c7b30 0x7f5a90154120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:47:14.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.995+0000 7f5a95c09640 1 --2- 172.21.15.89:0/2214049405 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a900d0e70 0x7f5a900c8c60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c0ab000 tx=0x7f5a8c01ad30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:14.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.995+0000 7f5a867fc640 1 -- 172.21.15.89:0/2214049405 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c0bd5f0 con 0x7f5a900d0e70 2023-10-18T10:47:14.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.995+0000 7f5a867fc640 1 -- 172.21.15.89:0/2214049405 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8c0bd770 con 0x7f5a900d0e70 2023-10-18T10:47:14.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.995+0000 7f5a97e6d640 1 -- 172.21.15.89:0/2214049405 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a8c0094d0 con 0x7f5a900d0e70 2023-10-18T10:47:14.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.995+0000 7f5a867fc640 1 -- 172.21.15.89:0/2214049405 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c0bdb20 con 0x7f5a900d0e70 2023-10-18T10:47:14.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.995+0000 7f5a97e6d640 1 -- 172.21.15.89:0/2214049405 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a90155760 con 0x7f5a900d0e70 2023-10-18T10:47:14.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.995+0000 7f5a97e6d640 1 -- 172.21.15.89:0/2214049405 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a90155320 con 0x7f5a900d0e70 2023-10-18T10:47:14.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.995+0000 7f5a867fc640 1 -- 172.21.15.89:0/2214049405 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f5a8c0059a0 con 0x7f5a900d0e70 2023-10-18T10:47:14.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.995+0000 7f5a867fc640 1 --2- 172.21.15.89:0/2214049405 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5a74065680 0x7f5a74067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:14.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.995+0000 7f5a867fc640 1 -- 172.21.15.89:0/2214049405 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5900+0+0 (secure 0 0 0) 0x7f5a8c0a7070 con 0x7f5a900d0e70 2023-10-18T10:47:14.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.995+0000 7f5a95408640 1 --2- 172.21.15.89:0/2214049405 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5a74065680 0x7f5a74067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:14.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:13.999+0000 7f5a95408640 1 --2- 172.21.15.89:0/2214049405 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5a74065680 0x7f5a74067b40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5a80000f70 tx=0x7f5a80007e20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:14.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:14.003+0000 7f5a867fc640 1 -- 172.21.15.89:0/2214049405 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f5a8c0ea9e0 con 0x7f5a900d0e70 2023-10-18T10:47:14.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:14.151+0000 7f5a97e6d640 1 -- 172.21.15.89:0/2214049405 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5a900c95f0 con 0x7f5a900d0e70 2023-10-18T10:47:14.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:14.823+0000 7f5a867fc640 1 -- 172.21.15.89:0/2214049405 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v39) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5a900c95f0 con 0x7f5a900d0e70 2023-10-18T10:47:14.828 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' created 2023-10-18T10:47:14.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:14.827+0000 7f5a97e6d640 1 -- 172.21.15.89:0/2214049405 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5a74065680 msgr2=0x7f5a74067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:14.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:14.827+0000 7f5a97e6d640 1 --2- 172.21.15.89:0/2214049405 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5a74065680 0x7f5a74067b40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5a80000f70 tx=0x7f5a80007e20 comp rx=0 tx=0).stop 2023-10-18T10:47:14.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:14.827+0000 7f5a97e6d640 1 -- 172.21.15.89:0/2214049405 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a900d0e70 msgr2=0x7f5a900c8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:14.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:14.827+0000 7f5a97e6d640 1 --2- 172.21.15.89:0/2214049405 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a900d0e70 0x7f5a900c8c60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c0ab000 tx=0x7f5a8c01ad30 comp rx=0 tx=0).stop 2023-10-18T10:47:14.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:14.831+0000 7f5a97e6d640 1 -- 172.21.15.89:0/2214049405 shutdown_connections 2023-10-18T10:47:14.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:14.831+0000 7f5a97e6d640 1 --2- 172.21.15.89:0/2214049405 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5a74065680 0x7f5a74067b40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:14.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:14.831+0000 7f5a97e6d640 1 --2- 172.21.15.89:0/2214049405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a9015ebe0 0x7f5a9015f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:14.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:14.831+0000 7f5a97e6d640 1 --2- 172.21.15.89:0/2214049405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a900c7b30 0x7f5a90154120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:14.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:14.831+0000 7f5a97e6d640 1 --2- 172.21.15.89:0/2214049405 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5a900d0e70 0x7f5a900c8c60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:14.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:14.831+0000 7f5a97e6d640 1 -- 172.21.15.89:0/2214049405 >> 172.21.15.89:0/2214049405 conn(0x7f5a90089fd0 msgr2=0x7f5a9015fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:14.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:14.831+0000 7f5a97e6d640 1 -- 172.21.15.89:0/2214049405 shutdown_connections 2023-10-18T10:47:14.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:14.831+0000 7f5a97e6d640 1 -- 172.21.15.89:0/2214049405 wait complete. 2023-10-18T10:47:14.861 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-18T10:47:15.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:14.999+0000 7fb11c884640 1 Processor -- start 2023-10-18T10:47:15.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:14.999+0000 7fb11c884640 1 -- start start 2023-10-18T10:47:15.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.003+0000 7fb11c884640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb1180cd860 0x7fb1180cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:15.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.003+0000 7fb11c884640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1180d0520 con 0x7fb1180cf3b0 2023-10-18T10:47:15.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.003+0000 7fb11c884640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1180d0800 con 0x7fb11815c5d0 2023-10-18T10:47:15.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.003+0000 7fb11c884640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1180d15c0 con 0x7fb1180cd860 2023-10-18T10:47:15.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.003+0000 7fb10effd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb1180cd860 0x7fb1180cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:15.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.003+0000 7fb117fff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb1180cf3b0 0x7fb1180b5460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40626/0 (socket says 172.21.15.89:40626) 2023-10-18T10:47:15.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.003+0000 7fb117fff640 1 -- 172.21.15.89:0/3274596953 learned_addr learned my addr 172.21.15.89:0/3274596953 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:47:15.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.003+0000 7fb10effd640 1 -- 172.21.15.89:0/3274596953 >> v1:172.21.15.154:6789/0 conn(0x7fb11815c5d0 legacy=0x7fb1180d1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:15.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.003+0000 7fb10effd640 1 -- 172.21.15.89:0/3274596953 >> v1:172.21.15.89:6789/0 conn(0x7fb1180cf3b0 legacy=0x7fb1180b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:15.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.003+0000 7fb116ffd640 1 -- 172.21.15.89:0/3274596953 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3477756132 0 0) 0x7fb1180d0520 con 0x7fb1180cf3b0 2023-10-18T10:47:15.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.003+0000 7fb10effd640 1 -- 172.21.15.89:0/3274596953 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1180d0ae0 con 0x7fb1180cd860 2023-10-18T10:47:15.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.003+0000 7fb10effd640 1 --2- 172.21.15.89:0/3274596953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1180cd860 0x7fb1180cee70 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb1000097c0 tx=0x7fb10001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1815d706a3591762 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:15.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.007+0000 7fb116ffd640 1 -- 172.21.15.89:0/3274596953 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1000ab600 con 0x7fb1180cd860 2023-10-18T10:47:15.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.007+0000 7fb116ffd640 1 -- 172.21.15.89:0/3274596953 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1000a7050 con 0x7fb1180cd860 2023-10-18T10:47:15.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.007+0000 7fb116ffd640 1 -- 172.21.15.89:0/3274596953 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1000ab980 con 0x7fb1180cd860 2023-10-18T10:47:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.007+0000 7fb11c884640 1 -- 172.21.15.89:0/3274596953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1180cd860 msgr2=0x7fb1180cee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.007+0000 7fb11c884640 1 --2- 172.21.15.89:0/3274596953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1180cd860 0x7fb1180cee70 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb1000097c0 tx=0x7fb10001c480 comp rx=0 tx=0).stop 2023-10-18T10:47:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.007+0000 7fb11c884640 1 -- 172.21.15.89:0/3274596953 shutdown_connections 2023-10-18T10:47:15.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.007+0000 7fb11c884640 1 --2- 172.21.15.89:0/3274596953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1180cd860 0x7fb1180cee70 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:15.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.007+0000 7fb11c884640 1 -- 172.21.15.89:0/3274596953 >> 172.21.15.89:0/3274596953 conn(0x7fb1180b3f50 msgr2=0x7fb1180b4360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:15.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.007+0000 7fb11c884640 1 -- 172.21.15.89:0/3274596953 shutdown_connections 2023-10-18T10:47:15.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.007+0000 7fb11c884640 1 -- 172.21.15.89:0/3274596953 wait complete. 2023-10-18T10:47:15.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.007+0000 7fb11c884640 1 Processor -- start 2023-10-18T10:47:15.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.007+0000 7fb11c884640 1 -- start start 2023-10-18T10:47:15.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.007+0000 7fb11c884640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1180cd860 0x7fb1180ca790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:15.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.007+0000 7fb11c884640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1180cf3b0 0x7fb11815b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:15.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.007+0000 7fb11c884640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb11815c5d0 0x7fb1180b6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:15.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.007+0000 7fb11c884640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1180cacd0 con 0x7fb1180cd860 2023-10-18T10:47:15.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.007+0000 7fb11c884640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11815b6f0 con 0x7fb1180cf3b0 2023-10-18T10:47:15.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.007+0000 7fb11c884640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb11815cac0 con 0x7fb11815c5d0 2023-10-18T10:47:15.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.007+0000 7fb1177fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1180cd860 0x7fb1180ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:15.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.007+0000 7fb1177fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1180cd860 0x7fb1180ca790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37880/0 (socket says 172.21.15.89:37880) 2023-10-18T10:47:15.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.007+0000 7fb1177fe640 1 -- 172.21.15.89:0/947473708 learned_addr learned my addr 172.21.15.89:0/947473708 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:15.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.011+0000 7fb10effd640 1 --2- 172.21.15.89:0/947473708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1180cf3b0 0x7fb11815b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:15.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.011+0000 7fb117fff640 1 --2- 172.21.15.89:0/947473708 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb11815c5d0 0x7fb1180b6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:15.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.011+0000 7fb1177fe640 1 -- 172.21.15.89:0/947473708 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb11815c5d0 msgr2=0x7fb1180b6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:15.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.011+0000 7fb1177fe640 1 --2- 172.21.15.89:0/947473708 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb11815c5d0 0x7fb1180b6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:15.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.011+0000 7fb1177fe640 1 -- 172.21.15.89:0/947473708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1180cf3b0 msgr2=0x7fb11815b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:15.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.011+0000 7fb1177fe640 1 --2- 172.21.15.89:0/947473708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1180cf3b0 0x7fb11815b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:15.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.011+0000 7fb1177fe640 1 -- 172.21.15.89:0/947473708 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb100009420 con 0x7fb1180cd860 2023-10-18T10:47:15.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.011+0000 7fb1177fe640 1 --2- 172.21.15.89:0/947473708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1180cd860 0x7fb1180ca790 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb10800c3c0 tx=0x7fb10800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:15.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.011+0000 7fb114ff9640 1 -- 172.21.15.89:0/947473708 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1080052d0 con 0x7fb1180cd860 2023-10-18T10:47:15.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.011+0000 7fb114ff9640 1 -- 172.21.15.89:0/947473708 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb108005430 con 0x7fb1180cd860 2023-10-18T10:47:15.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.011+0000 7fb11c884640 1 -- 172.21.15.89:0/947473708 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb11815f240 con 0x7fb1180cd860 2023-10-18T10:47:15.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.011+0000 7fb114ff9640 1 -- 172.21.15.89:0/947473708 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb108003840 con 0x7fb1180cd860 2023-10-18T10:47:15.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.011+0000 7fb11c884640 1 -- 172.21.15.89:0/947473708 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb11815cd10 con 0x7fb1180cd860 2023-10-18T10:47:15.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.011+0000 7fb11c884640 1 -- 172.21.15.89:0/947473708 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb118073b50 con 0x7fb1180cd860 2023-10-18T10:47:15.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.011+0000 7fb114ff9640 1 -- 172.21.15.89:0/947473708 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fb1080039a0 con 0x7fb1180cd860 2023-10-18T10:47:15.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.015+0000 7fb114ff9640 1 --2- 172.21.15.89:0/947473708 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb0f8065680 0x7fb0f8067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:15.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.015+0000 7fb114ff9640 1 -- 172.21.15.89:0/947473708 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6282+0+0 (secure 0 0 0) 0x7fb108082820 con 0x7fb1180cd860 2023-10-18T10:47:15.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.015+0000 7fb10effd640 1 --2- 172.21.15.89:0/947473708 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb0f8065680 0x7fb0f8067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:15.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.015+0000 7fb10effd640 1 --2- 172.21.15.89:0/947473708 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb0f8065680 0x7fb0f8067b40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb1000ad000 tx=0x7fb100008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:15.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.015+0000 7fb114ff9640 1 -- 172.21.15.89:0/947473708 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fb108050290 con 0x7fb1180cd860 2023-10-18T10:47:15.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.195+0000 7fb11c884640 1 -- 172.21.15.89:0/947473708 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb1180b3b00 con 0x7fb1180cd860 2023-10-18T10:47:15.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.839+0000 7fb114ff9640 1 -- 172.21.15.89:0/947473708 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb1080538b0 con 0x7fb1180cd860 2023-10-18T10:47:15.844 INFO:teuthology.orchestra.run.smithi089.stderr:new fs with metadata pool 6 and data pool 7 2023-10-18T10:47:15.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.851+0000 7fb11c884640 1 -- 172.21.15.89:0/947473708 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb0f8065680 msgr2=0x7fb0f8067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:15.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.851+0000 7fb11c884640 1 --2- 172.21.15.89:0/947473708 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb0f8065680 0x7fb0f8067b40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb1000ad000 tx=0x7fb100008040 comp rx=0 tx=0).stop 2023-10-18T10:47:15.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.851+0000 7fb11c884640 1 -- 172.21.15.89:0/947473708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1180cd860 msgr2=0x7fb1180ca790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:15.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.851+0000 7fb11c884640 1 --2- 172.21.15.89:0/947473708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1180cd860 0x7fb1180ca790 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb10800c3c0 tx=0x7fb10800c880 comp rx=0 tx=0).stop 2023-10-18T10:47:15.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.851+0000 7fb11c884640 1 -- 172.21.15.89:0/947473708 shutdown_connections 2023-10-18T10:47:15.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.851+0000 7fb11c884640 1 --2- 172.21.15.89:0/947473708 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb0f8065680 0x7fb0f8067b40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:15.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.851+0000 7fb11c884640 1 --2- 172.21.15.89:0/947473708 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb11815c5d0 0x7fb1180b6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:15.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.851+0000 7fb11c884640 1 --2- 172.21.15.89:0/947473708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1180cf3b0 0x7fb11815b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:15.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.851+0000 7fb11c884640 1 --2- 172.21.15.89:0/947473708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1180cd860 0x7fb1180ca790 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:15.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.851+0000 7fb11c884640 1 -- 172.21.15.89:0/947473708 >> 172.21.15.89:0/947473708 conn(0x7fb1180b3f50 msgr2=0x7fb1180b4330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:15.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.855+0000 7fb11c884640 1 -- 172.21.15.89:0/947473708 shutdown_connections 2023-10-18T10:47:15.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:15.855+0000 7fb11c884640 1 -- 172.21.15.89:0/947473708 wait complete. 2023-10-18T10:47:15.890 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:47:16.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.027+0000 7f8ba9648640 1 Processor -- start 2023-10-18T10:47:16.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.027+0000 7f8ba9648640 1 -- start start 2023-10-18T10:47:16.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.031+0000 7f8ba9648640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8ba406ff00 0x7f8ba415e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:16.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.031+0000 7f8ba9648640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ba40ca340 0x7f8ba415b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:16.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.031+0000 7f8ba9648640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ba415acb0 con 0x7f8ba415b680 2023-10-18T10:47:16.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.031+0000 7f8ba9648640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ba415ef30 con 0x7f8ba406ff00 2023-10-18T10:47:16.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.031+0000 7f8ba9648640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba415aa10 con 0x7f8ba40ca340 2023-10-18T10:47:16.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.031+0000 7f8ba2ffd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8ba406ff00 0x7f8ba415e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:16.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.031+0000 7f8ba2ffd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8ba406ff00 0x7f8ba415e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37882/0 (socket says 172.21.15.89:37882) 2023-10-18T10:47:16.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.031+0000 7f8ba2ffd640 1 -- 172.21.15.89:0/966672523 learned_addr learned my addr 172.21.15.89:0/966672523 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:16.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.031+0000 7f8ba2ffd640 1 -- 172.21.15.89:0/966672523 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ba40ca340 msgr2=0x7f8ba415b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:16.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.031+0000 7f8ba2ffd640 1 --2- 172.21.15.89:0/966672523 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ba40ca340 0x7f8ba415b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:16.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.031+0000 7f8ba2ffd640 1 -- 172.21.15.89:0/966672523 >> v1:172.21.15.154:6789/0 conn(0x7f8ba415b680 legacy=0x7f8ba40c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:16.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.031+0000 7f8ba2ffd640 1 -- 172.21.15.89:0/966672523 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba40702e0 con 0x7f8ba406ff00 2023-10-18T10:47:16.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.031+0000 7f8ba2ffd640 1 --2- 172.21.15.89:0/966672523 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ba406ff00 0x7f8ba415e000 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c009870 tx=0x7f8b8c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe0ae8c70472c1f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:16.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.035+0000 7f8ba1ffb640 1 -- 172.21.15.89:0/966672523 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c0ab600 con 0x7f8ba406ff00 2023-10-18T10:47:16.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.035+0000 7f8ba1ffb640 1 -- 172.21.15.89:0/966672523 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8c0ab760 con 0x7f8ba406ff00 2023-10-18T10:47:16.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.035+0000 7f8ba1ffb640 1 -- 172.21.15.89:0/966672523 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c0abaf0 con 0x7f8ba406ff00 2023-10-18T10:47:16.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.035+0000 7f8ba9648640 1 -- 172.21.15.89:0/966672523 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ba406ff00 msgr2=0x7f8ba415e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:16.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.035+0000 7f8ba9648640 1 --2- 172.21.15.89:0/966672523 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ba406ff00 0x7f8ba415e000 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c009870 tx=0x7f8b8c01c440 comp rx=0 tx=0).stop 2023-10-18T10:47:16.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.035+0000 7f8ba9648640 1 -- 172.21.15.89:0/966672523 shutdown_connections 2023-10-18T10:47:16.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.035+0000 7f8ba9648640 1 --2- 172.21.15.89:0/966672523 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ba40ca340 0x7f8ba415b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:16.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.035+0000 7f8ba9648640 1 --2- 172.21.15.89:0/966672523 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ba406ff00 0x7f8ba415e000 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:16.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.035+0000 7f8ba9648640 1 -- 172.21.15.89:0/966672523 >> 172.21.15.89:0/966672523 conn(0x7f8ba4069f00 msgr2=0x7f8ba40cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:16.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.035+0000 7f8ba9648640 1 -- 172.21.15.89:0/966672523 shutdown_connections 2023-10-18T10:47:16.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.035+0000 7f8ba9648640 1 -- 172.21.15.89:0/966672523 wait complete. 2023-10-18T10:47:16.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.035+0000 7f8ba9648640 1 Processor -- start 2023-10-18T10:47:16.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.035+0000 7f8ba9648640 1 -- start start 2023-10-18T10:47:16.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.035+0000 7f8ba9648640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ba406ff00 0x7f8ba40c0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:16.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.035+0000 7f8ba9648640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ba40ca340 0x7f8ba406b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:16.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.035+0000 7f8ba9648640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ba406bd10 0x7f8ba40c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:16.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.035+0000 7f8ba9648640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba406c1c0 con 0x7f8ba40ca340 2023-10-18T10:47:16.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.035+0000 7f8ba9648640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba41f82e0 con 0x7f8ba406ff00 2023-10-18T10:47:16.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.035+0000 7f8ba9648640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba41f84f0 con 0x7f8ba406bd10 2023-10-18T10:47:16.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.035+0000 7f8ba27fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ba40ca340 0x7f8ba406b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:16.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.039+0000 7f8ba27fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ba40ca340 0x7f8ba406b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37896/0 (socket says 172.21.15.89:37896) 2023-10-18T10:47:16.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.039+0000 7f8ba27fc640 1 -- 172.21.15.89:0/1504359553 learned_addr learned my addr 172.21.15.89:0/1504359553 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:16.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.039+0000 7f8ba2ffd640 1 --2- 172.21.15.89:0/1504359553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ba406ff00 0x7f8ba40c0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:16.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.039+0000 7f8ba37fe640 1 --2- 172.21.15.89:0/1504359553 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ba406bd10 0x7f8ba40c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:16.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.039+0000 7f8ba27fc640 1 -- 172.21.15.89:0/1504359553 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ba406bd10 msgr2=0x7f8ba40c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:16.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.039+0000 7f8ba27fc640 1 --2- 172.21.15.89:0/1504359553 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ba406bd10 0x7f8ba40c7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:16.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.039+0000 7f8ba27fc640 1 -- 172.21.15.89:0/1504359553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ba406ff00 msgr2=0x7f8ba40c0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:16.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.039+0000 7f8ba27fc640 1 --2- 172.21.15.89:0/1504359553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ba406ff00 0x7f8ba40c0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:16.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.039+0000 7f8ba27fc640 1 -- 172.21.15.89:0/1504359553 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8c0094d0 con 0x7f8ba40ca340 2023-10-18T10:47:16.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.039+0000 7f8ba37fe640 1 --2- 172.21.15.89:0/1504359553 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ba406bd10 0x7f8ba40c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:47:16.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.039+0000 7f8ba27fc640 1 --2- 172.21.15.89:0/1504359553 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ba40ca340 0x7f8ba406b7d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f8b9400b550 tx=0x7f8b9400ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:16.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.039+0000 7f8b9b7fe640 1 -- 172.21.15.89:0/1504359553 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b94010870 con 0x7f8ba40ca340 2023-10-18T10:47:16.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.039+0000 7f8b9b7fe640 1 -- 172.21.15.89:0/1504359553 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b940109f0 con 0x7f8ba40ca340 2023-10-18T10:47:16.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.039+0000 7f8b9b7fe640 1 -- 172.21.15.89:0/1504359553 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b94010da0 con 0x7f8ba40ca340 2023-10-18T10:47:16.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.039+0000 7f8ba9648640 1 -- 172.21.15.89:0/1504359553 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ba41f87d0 con 0x7f8ba40ca340 2023-10-18T10:47:16.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.039+0000 7f8ba9648640 1 -- 172.21.15.89:0/1504359553 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ba41f8d90 con 0x7f8ba40ca340 2023-10-18T10:47:16.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.039+0000 7f8ba9648640 1 -- 172.21.15.89:0/1504359553 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ba404dbe0 con 0x7f8ba40ca340 2023-10-18T10:47:16.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.039+0000 7f8b9b7fe640 1 -- 172.21.15.89:0/1504359553 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f8b9400f3c0 con 0x7f8ba40ca340 2023-10-18T10:47:16.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.043+0000 7f8b9b7fe640 1 --2- 172.21.15.89:0/1504359553 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8b84065680 0x7f8b84067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:16.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.043+0000 7f8b9b7fe640 1 -- 172.21.15.89:0/1504359553 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f8b94082f10 con 0x7f8ba40ca340 2023-10-18T10:47:16.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.043+0000 7f8ba2ffd640 1 --2- 172.21.15.89:0/1504359553 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8b84065680 0x7f8b84067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:16.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.043+0000 7f8ba2ffd640 1 --2- 172.21.15.89:0/1504359553 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8b84065680 0x7f8b84067b40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0af000 tx=0x7f8b8c0b6040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:16.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.047+0000 7f8b9b7fe640 1 -- 172.21.15.89:0/1504359553 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f8b940508a0 con 0x7f8ba40ca340 2023-10-18T10:47:16.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.195+0000 7f8ba9648640 1 -- 172.21.15.89:0/1504359553 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ba4165170 con 0x7f8ba40ca340 2023-10-18T10:47:16.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.195+0000 7f8b9b7fe640 1 -- 172.21.15.89:0/1504359553 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+18489 (secure 0 0 0) 0x7f8b94053ec0 con 0x7f8ba40ca340 2023-10-18T10:47:16.198 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:47:16.198 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":40,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:47:15.199201+0000","last_up_change":"2023-10-18T10:45:42.164688+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:47:13.188701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-18T10:47:14.198224+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6824","nonce":773078364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6827","nonce":773078364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6831","nonce":773078364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6829","nonce":773078364}]},"public_addr":"172.21.15.154:6824/773078364","cluster_addr":"172.21.15.154:6827/773078364","heartbeat_back_addr":"172.21.15.154:6831/773078364","heartbeat_front_addr":"172.21.15.154:6829/773078364","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:47:16.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.199+0000 7f8ba9648640 1 -- 172.21.15.89:0/1504359553 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8b84065680 msgr2=0x7f8b84067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:16.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.199+0000 7f8ba9648640 1 --2- 172.21.15.89:0/1504359553 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8b84065680 0x7f8b84067b40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0af000 tx=0x7f8b8c0b6040 comp rx=0 tx=0).stop 2023-10-18T10:47:16.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.199+0000 7f8ba9648640 1 -- 172.21.15.89:0/1504359553 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ba40ca340 msgr2=0x7f8ba406b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:16.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.199+0000 7f8ba9648640 1 --2- 172.21.15.89:0/1504359553 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ba40ca340 0x7f8ba406b7d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f8b9400b550 tx=0x7f8b9400ba10 comp rx=0 tx=0).stop 2023-10-18T10:47:16.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.199+0000 7f8ba9648640 1 -- 172.21.15.89:0/1504359553 shutdown_connections 2023-10-18T10:47:16.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.199+0000 7f8ba9648640 1 --2- 172.21.15.89:0/1504359553 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8b84065680 0x7f8b84067b40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:16.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.199+0000 7f8ba9648640 1 --2- 172.21.15.89:0/1504359553 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ba406bd10 0x7f8ba40c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:16.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.199+0000 7f8ba9648640 1 --2- 172.21.15.89:0/1504359553 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ba40ca340 0x7f8ba406b7d0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:16.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.199+0000 7f8ba9648640 1 --2- 172.21.15.89:0/1504359553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ba406ff00 0x7f8ba40c0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:16.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.199+0000 7f8ba9648640 1 -- 172.21.15.89:0/1504359553 >> 172.21.15.89:0/1504359553 conn(0x7f8ba4069f00 msgr2=0x7f8ba40af170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:16.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.199+0000 7f8ba9648640 1 -- 172.21.15.89:0/1504359553 shutdown_connections 2023-10-18T10:47:16.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.199+0000 7f8ba9648640 1 -- 172.21.15.89:0/1504359553 wait complete. 2023-10-18T10:47:16.220 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:47:16.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.355+0000 7ff0eb5a3640 1 Processor -- start 2023-10-18T10:47:16.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.355+0000 7ff0eb5a3640 1 -- start start 2023-10-18T10:47:16.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.355+0000 7ff0eb5a3640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff0e415c550 0x7ff0e40d2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:16.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.355+0000 7ff0eb5a3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0e40cfd20 0x7ff0e40b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:16.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.355+0000 7ff0eb5a3640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0e40d1e50 con 0x7ff0e40b1860 2023-10-18T10:47:16.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.355+0000 7ff0eb5a3640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0e40d2130 con 0x7ff0e415c550 2023-10-18T10:47:16.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.355+0000 7ff0eb5a3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e40d2410 con 0x7ff0e40cfd20 2023-10-18T10:47:16.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.355+0000 7ff0e933f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff0e415c550 0x7ff0e40d2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:16.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.355+0000 7ff0e8b3e640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff0e40b1860 0x7ff0e40cf610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40630/0 (socket says 172.21.15.89:40630) 2023-10-18T10:47:16.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.355+0000 7ff0e8b3e640 1 -- 172.21.15.89:0/2054358273 learned_addr learned my addr 172.21.15.89:0/2054358273 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:47:16.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.355+0000 7ff0e9b40640 1 --2- 172.21.15.89:0/2054358273 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0e40cfd20 0x7ff0e40b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:16.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.355+0000 7ff0e933f640 1 -- 172.21.15.89:0/2054358273 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0e40cfd20 msgr2=0x7ff0e40b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:16.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.355+0000 7ff0e933f640 1 --2- 172.21.15.89:0/2054358273 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0e40cfd20 0x7ff0e40b55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:16.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.355+0000 7ff0e933f640 1 -- 172.21.15.89:0/2054358273 >> v1:172.21.15.89:6789/0 conn(0x7ff0e40b1860 legacy=0x7ff0e40cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:16.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.355+0000 7ff0e933f640 1 -- 172.21.15.89:0/2054358273 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e40d26f0 con 0x7ff0e415c550 2023-10-18T10:47:16.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.359+0000 7ff0e933f640 1 --2- 172.21.15.89:0/2054358273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0e415c550 0x7ff0e40d2ef0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff0e00097c0 tx=0x7ff0e001ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51bdf915d797fa5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:16.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.359+0000 7ff0dbfff640 1 -- 172.21.15.89:0/2054358273 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e00ac410 con 0x7ff0e415c550 2023-10-18T10:47:16.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.359+0000 7ff0dbfff640 1 -- 172.21.15.89:0/2054358273 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e00ac570 con 0x7ff0e415c550 2023-10-18T10:47:16.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.359+0000 7ff0dbfff640 1 -- 172.21.15.89:0/2054358273 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e00ac900 con 0x7ff0e415c550 2023-10-18T10:47:16.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.359+0000 7ff0eb5a3640 1 -- 172.21.15.89:0/2054358273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0e415c550 msgr2=0x7ff0e40d2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:16.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.359+0000 7ff0eb5a3640 1 --2- 172.21.15.89:0/2054358273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0e415c550 0x7ff0e40d2ef0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff0e00097c0 tx=0x7ff0e001ceb0 comp rx=0 tx=0).stop 2023-10-18T10:47:16.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.359+0000 7ff0eb5a3640 1 -- 172.21.15.89:0/2054358273 shutdown_connections 2023-10-18T10:47:16.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.359+0000 7ff0eb5a3640 1 --2- 172.21.15.89:0/2054358273 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0e40cfd20 0x7ff0e40b55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:16.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.359+0000 7ff0eb5a3640 1 --2- 172.21.15.89:0/2054358273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0e415c550 0x7ff0e40d2ef0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:16.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.359+0000 7ff0eb5a3640 1 -- 172.21.15.89:0/2054358273 >> 172.21.15.89:0/2054358273 conn(0x7ff0e4069d00 msgr2=0x7ff0e406a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:16.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.359+0000 7ff0eb5a3640 1 -- 172.21.15.89:0/2054358273 shutdown_connections 2023-10-18T10:47:16.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.359+0000 7ff0eb5a3640 1 -- 172.21.15.89:0/2054358273 wait complete. 2023-10-18T10:47:16.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.363+0000 7ff0eb5a3640 1 Processor -- start 2023-10-18T10:47:16.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.363+0000 7ff0eb5a3640 1 -- start start 2023-10-18T10:47:16.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.363+0000 7ff0eb5a3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0e40b1860 0x7ff0e4152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:16.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.363+0000 7ff0eb5a3640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0e40cfd20 0x7ff0e4154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:16.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.363+0000 7ff0eb5a3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0e415c550 0x7ff0e4154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:16.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.363+0000 7ff0eb5a3640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e415f710 con 0x7ff0e40cfd20 2023-10-18T10:47:16.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.363+0000 7ff0eb5a3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e415eb00 con 0x7ff0e415c550 2023-10-18T10:47:16.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.363+0000 7ff0eb5a3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e415ede0 con 0x7ff0e40b1860 2023-10-18T10:47:16.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.363+0000 7ff0e8b3e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0e40cfd20 0x7ff0e4154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:16.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.363+0000 7ff0e8b3e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0e40cfd20 0x7ff0e4154590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37910/0 (socket says 172.21.15.89:37910) 2023-10-18T10:47:16.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.363+0000 7ff0e8b3e640 1 -- 172.21.15.89:0/3078144180 learned_addr learned my addr 172.21.15.89:0/3078144180 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:16.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.363+0000 7ff0e933f640 1 --2- 172.21.15.89:0/3078144180 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0e40b1860 0x7ff0e4152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:16.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.363+0000 7ff0e9b40640 1 --2- 172.21.15.89:0/3078144180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0e415c550 0x7ff0e4154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:16.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.363+0000 7ff0e8b3e640 1 -- 172.21.15.89:0/3078144180 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0e40b1860 msgr2=0x7ff0e4152c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:16.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.363+0000 7ff0e8b3e640 1 --2- 172.21.15.89:0/3078144180 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0e40b1860 0x7ff0e4152c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:16.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.363+0000 7ff0e8b3e640 1 -- 172.21.15.89:0/3078144180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0e415c550 msgr2=0x7ff0e4154ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:16.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.363+0000 7ff0e8b3e640 1 --2- 172.21.15.89:0/3078144180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0e415c550 0x7ff0e4154ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:16.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.363+0000 7ff0e8b3e640 1 -- 172.21.15.89:0/3078144180 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e0009420 con 0x7ff0e40cfd20 2023-10-18T10:47:16.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.363+0000 7ff0e933f640 1 --2- 172.21.15.89:0/3078144180 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0e40b1860 0x7ff0e4152c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:47:16.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.363+0000 7ff0e9b40640 1 --2- 172.21.15.89:0/3078144180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0e415c550 0x7ff0e4154ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:47:16.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.363+0000 7ff0e8b3e640 1 --2- 172.21.15.89:0/3078144180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0e40cfd20 0x7ff0e4154590 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400c500 tx=0x7ff0d400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:16.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.363+0000 7ff0d9ffb640 1 -- 172.21.15.89:0/3078144180 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4005cf0 con 0x7ff0e40cfd20 2023-10-18T10:47:16.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.363+0000 7ff0d9ffb640 1 -- 172.21.15.89:0/3078144180 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d4005e50 con 0x7ff0e40cfd20 2023-10-18T10:47:16.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.363+0000 7ff0d9ffb640 1 -- 172.21.15.89:0/3078144180 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d40072f0 con 0x7ff0e40cfd20 2023-10-18T10:47:16.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.363+0000 7ff0eb5a3640 1 -- 172.21.15.89:0/3078144180 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e415f0c0 con 0x7ff0e40cfd20 2023-10-18T10:47:16.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.363+0000 7ff0eb5a3640 1 -- 172.21.15.89:0/3078144180 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e4153190 con 0x7ff0e40cfd20 2023-10-18T10:47:16.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.367+0000 7ff0eb5a3640 1 -- 172.21.15.89:0/3078144180 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0e404dbe0 con 0x7ff0e40cfd20 2023-10-18T10:47:16.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.367+0000 7ff0d9ffb640 1 -- 172.21.15.89:0/3078144180 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7ff0d4007450 con 0x7ff0e40cfd20 2023-10-18T10:47:16.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.367+0000 7ff0d9ffb640 1 --2- 172.21.15.89:0/3078144180 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff0c8065680 0x7ff0c8067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:16.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.367+0000 7ff0d9ffb640 1 -- 172.21.15.89:0/3078144180 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff0d4082450 con 0x7ff0e40cfd20 2023-10-18T10:47:16.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.367+0000 7ff0e933f640 1 --2- 172.21.15.89:0/3078144180 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff0c8065680 0x7ff0c8067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:16.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.367+0000 7ff0e933f640 1 --2- 172.21.15.89:0/3078144180 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff0c8065680 0x7ff0c8067b40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff0e00ae000 tx=0x7ff0e0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:16.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.371+0000 7ff0d9ffb640 1 -- 172.21.15.89:0/3078144180 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7ff0d404fe70 con 0x7ff0e40cfd20 2023-10-18T10:47:16.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.523+0000 7ff0eb5a3640 1 -- 172.21.15.89:0/3078144180 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0e40b8040 con 0x7ff0e40cfd20 2023-10-18T10:47:16.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.523+0000 7ff0d9ffb640 1 -- 172.21.15.89:0/3078144180 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+18489 (secure 0 0 0) 0x7ff0d4053490 con 0x7ff0e40cfd20 2023-10-18T10:47:16.525 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:47:16.525 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":40,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:47:15.199201+0000","last_up_change":"2023-10-18T10:45:42.164688+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:47:13.188701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-18T10:47:14.198224+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6824","nonce":773078364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6827","nonce":773078364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6831","nonce":773078364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6829","nonce":773078364}]},"public_addr":"172.21.15.154:6824/773078364","cluster_addr":"172.21.15.154:6827/773078364","heartbeat_back_addr":"172.21.15.154:6831/773078364","heartbeat_front_addr":"172.21.15.154:6829/773078364","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:47:16.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.523+0000 7ff0eb5a3640 1 -- 172.21.15.89:0/3078144180 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff0c8065680 msgr2=0x7ff0c8067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:16.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.523+0000 7ff0eb5a3640 1 --2- 172.21.15.89:0/3078144180 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff0c8065680 0x7ff0c8067b40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff0e00ae000 tx=0x7ff0e0008040 comp rx=0 tx=0).stop 2023-10-18T10:47:16.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.523+0000 7ff0eb5a3640 1 -- 172.21.15.89:0/3078144180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0e40cfd20 msgr2=0x7ff0e4154590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:16.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.523+0000 7ff0eb5a3640 1 --2- 172.21.15.89:0/3078144180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0e40cfd20 0x7ff0e4154590 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400c500 tx=0x7ff0d400c9c0 comp rx=0 tx=0).stop 2023-10-18T10:47:16.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.523+0000 7ff0eb5a3640 1 -- 172.21.15.89:0/3078144180 shutdown_connections 2023-10-18T10:47:16.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.523+0000 7ff0eb5a3640 1 --2- 172.21.15.89:0/3078144180 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff0c8065680 0x7ff0c8067b40 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:16.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.523+0000 7ff0eb5a3640 1 --2- 172.21.15.89:0/3078144180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0e415c550 0x7ff0e4154ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:16.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.523+0000 7ff0eb5a3640 1 --2- 172.21.15.89:0/3078144180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0e40cfd20 0x7ff0e4154590 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:16.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.523+0000 7ff0eb5a3640 1 --2- 172.21.15.89:0/3078144180 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0e40b1860 0x7ff0e4152c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:16.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.523+0000 7ff0eb5a3640 1 -- 172.21.15.89:0/3078144180 >> 172.21.15.89:0/3078144180 conn(0x7ff0e4069d00 msgr2=0x7ff0e4155520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:16.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.523+0000 7ff0eb5a3640 1 -- 172.21.15.89:0/3078144180 shutdown_connections 2023-10-18T10:47:16.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.523+0000 7ff0eb5a3640 1 -- 172.21.15.89:0/3078144180 wait complete. 2023-10-18T10:47:16.547 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-18T10:47:16.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.683+0000 7f16689ba640 1 Processor -- start 2023-10-18T10:47:16.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.683+0000 7f16689ba640 1 -- start start 2023-10-18T10:47:16.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.687+0000 7f16689ba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16640cd4f0 0x7f1664143470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:16.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.687+0000 7f16689ba640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16640cd8d0 con 0x7f1664145b70 2023-10-18T10:47:16.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.687+0000 7f16689ba640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f166415cbf0 con 0x7f16640ce070 2023-10-18T10:47:16.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.687+0000 7f16689ba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1664069900 con 0x7f16640cd4f0 2023-10-18T10:47:16.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.687+0000 7f1661d9b640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1664145b70 0x7f1664160cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40640/0 (socket says 172.21.15.89:40640) 2023-10-18T10:47:16.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.687+0000 7f1661d9b640 1 -- 172.21.15.89:0/3821519599 learned_addr learned my addr 172.21.15.89:0/3821519599 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:47:16.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.687+0000 7f166259c640 1 --2- 172.21.15.89:0/3821519599 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16640cd4f0 0x7f1664143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:16.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.687+0000 7f166159a640 1 -- 172.21.15.89:0/3821519599 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3763072892 0 0) 0x7f16640cd8d0 con 0x7f1664145b70 2023-10-18T10:47:16.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.687+0000 7f166159a640 1 -- 172.21.15.89:0/3821519599 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f164c003530 con 0x7f1664145b70 2023-10-18T10:47:16.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.687+0000 7f166159a640 1 -- 172.21.15.89:0/3821519599 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 181981431 0 0) 0x7f164c003530 con 0x7f1664145b70 2023-10-18T10:47:16.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.687+0000 7f166159a640 1 -- 172.21.15.89:0/3821519599 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16640cd4f0 msgr2=0x7f1664143470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:16.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.687+0000 7f166159a640 1 --2- 172.21.15.89:0/3821519599 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16640cd4f0 0x7f1664143470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:16.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.687+0000 7f166159a640 1 -- 172.21.15.89:0/3821519599 >> v1:172.21.15.154:6789/0 conn(0x7f16640ce070 legacy=0x7f166406abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:16.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.687+0000 7f166159a640 1 -- 172.21.15.89:0/3821519599 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1664069b10 con 0x7f1664145b70 2023-10-18T10:47:16.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.687+0000 7f166159a640 1 -- 172.21.15.89:0/3821519599 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7f1658001680 con 0x7f1664145b70 2023-10-18T10:47:16.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.687+0000 7f166159a640 1 -- 172.21.15.89:0/3821519599 >> v1:172.21.15.89:6789/0 conn(0x7f1664145b70 legacy=0x7f1664160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:16.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.687+0000 7f166159a640 1 --2- 172.21.15.89:0/3821519599 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f164c0039b0 0x7f164c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:16.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.687+0000 7f166159a640 1 --2- 172.21.15.89:0/3821519599 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16640cd4f0 0x7f164c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:16.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.687+0000 7f166159a640 1 --2- 172.21.15.89:0/3821519599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f164c004d10 0x7f164c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:16.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.687+0000 7f166159a640 1 -- 172.21.15.89:0/3821519599 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1664069900 con 0x7f164c004d10 2023-10-18T10:47:16.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.687+0000 7f166159a640 1 -- 172.21.15.89:0/3821519599 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164c00b990 con 0x7f164c0039b0 2023-10-18T10:47:16.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.687+0000 7f166159a640 1 -- 172.21.15.89:0/3821519599 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f164c00bb50 con 0x7f16640cd4f0 2023-10-18T10:47:16.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.687+0000 7f1662d9d640 1 --2- 172.21.15.89:0/3821519599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f164c004d10 0x7f164c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:16.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.687+0000 7f166259c640 1 --2- 172.21.15.89:0/3821519599 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f164c0039b0 0x7f164c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:16.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.687+0000 7f1661d9b640 1 --2- 172.21.15.89:0/3821519599 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16640cd4f0 0x7f164c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:16.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.687+0000 7f166259c640 1 -- 172.21.15.89:0/3821519599 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16640cd4f0 msgr2=0x7f164c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:16.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.687+0000 7f166259c640 1 --2- 172.21.15.89:0/3821519599 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16640cd4f0 0x7f164c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:16.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.687+0000 7f166259c640 1 -- 172.21.15.89:0/3821519599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f164c004d10 msgr2=0x7f164c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:16.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.687+0000 7f166259c640 1 --2- 172.21.15.89:0/3821519599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f164c004d10 0x7f164c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:16.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.687+0000 7f166259c640 1 -- 172.21.15.89:0/3821519599 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f164c003530 con 0x7f164c0039b0 2023-10-18T10:47:16.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.687+0000 7f1662d9d640 1 --2- 172.21.15.89:0/3821519599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f164c004d10 0x7f164c005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:47:16.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.691+0000 7f166259c640 1 --2- 172.21.15.89:0/3821519599 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f164c0039b0 0x7f164c003d90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1654009ae0 tx=0x7f165401c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=774f94d4362b858e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:16.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.691+0000 7f166159a640 1 -- 172.21.15.89:0/3821519599 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16540a6480 con 0x7f164c0039b0 2023-10-18T10:47:16.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.691+0000 7f166159a640 1 -- 172.21.15.89:0/3821519599 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16540a65e0 con 0x7f164c0039b0 2023-10-18T10:47:16.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.691+0000 7f166159a640 1 -- 172.21.15.89:0/3821519599 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16540a69d0 con 0x7f164c0039b0 2023-10-18T10:47:16.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.691+0000 7f16689ba640 1 -- 172.21.15.89:0/3821519599 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f164c0039b0 msgr2=0x7f164c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:16.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.691+0000 7f16689ba640 1 --2- 172.21.15.89:0/3821519599 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f164c0039b0 0x7f164c003d90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1654009ae0 tx=0x7f165401c990 comp rx=0 tx=0).stop 2023-10-18T10:47:16.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.691+0000 7f1662d9d640 1 -- 172.21.15.89:0/3821519599 reap_dead start 2023-10-18T10:47:16.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.691+0000 7f16689ba640 1 -- 172.21.15.89:0/3821519599 shutdown_connections 2023-10-18T10:47:16.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.691+0000 7f16689ba640 1 -- 172.21.15.89:0/3821519599 >> 172.21.15.89:0/3821519599 conn(0x7f166406dad0 msgr2=0x7f166406dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:16.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.691+0000 7f16689ba640 1 -- 172.21.15.89:0/3821519599 shutdown_connections 2023-10-18T10:47:16.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.691+0000 7f16689ba640 1 -- 172.21.15.89:0/3821519599 wait complete. 2023-10-18T10:47:16.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.695+0000 7f16689ba640 1 Processor -- start 2023-10-18T10:47:16.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.695+0000 7f16689ba640 1 -- start start 2023-10-18T10:47:16.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.695+0000 7f16689ba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f164c0039b0 0x7f1664159030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:16.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.695+0000 7f16689ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f166415bb30 0x7f166415cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:16.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.695+0000 7f16689ba640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f166415d400 0x7f1664156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:16.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.695+0000 7f16689ba640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16640c7e10 con 0x7f166415d400 2023-10-18T10:47:16.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.695+0000 7f16689ba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16640c3490 con 0x7f166415bb30 2023-10-18T10:47:16.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.695+0000 7f16689ba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16640c7570 con 0x7f164c0039b0 2023-10-18T10:47:16.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.695+0000 7f1662d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f166415d400 0x7f1664156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:16.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.695+0000 7f1662d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f166415d400 0x7f1664156630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37932/0 (socket says 172.21.15.89:37932) 2023-10-18T10:47:16.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.695+0000 7f1662d9d640 1 -- 172.21.15.89:0/3838292072 learned_addr learned my addr 172.21.15.89:0/3838292072 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:16.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.695+0000 7f166259c640 1 --2- 172.21.15.89:0/3838292072 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f164c0039b0 0x7f1664159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:16.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.695+0000 7f1661d9b640 1 --2- 172.21.15.89:0/3838292072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f166415bb30 0x7f166415cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:16.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.695+0000 7f1662d9d640 1 -- 172.21.15.89:0/3838292072 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f164c0039b0 msgr2=0x7f1664159030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:16.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.695+0000 7f1662d9d640 1 --2- 172.21.15.89:0/3838292072 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f164c0039b0 0x7f1664159030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:16.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.695+0000 7f1662d9d640 1 -- 172.21.15.89:0/3838292072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f166415bb30 msgr2=0x7f166415cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:16.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.695+0000 7f1662d9d640 1 --2- 172.21.15.89:0/3838292072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f166415bb30 0x7f166415cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:16.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.695+0000 7f1662d9d640 1 -- 172.21.15.89:0/3838292072 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16640c7850 con 0x7f166415d400 2023-10-18T10:47:16.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.695+0000 7f1661d9b640 1 --2- 172.21.15.89:0/3838292072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f166415bb30 0x7f166415cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:47:16.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.695+0000 7f166259c640 1 --2- 172.21.15.89:0/3838292072 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f164c0039b0 0x7f1664159030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:47:16.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.695+0000 7f1662d9d640 1 --2- 172.21.15.89:0/3838292072 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f166415d400 0x7f1664156630 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f16640b63d0 tx=0x7f165c00d150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:16.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.695+0000 7f1652ffd640 1 -- 172.21.15.89:0/3838292072 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165c003b90 con 0x7f166415d400 2023-10-18T10:47:16.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.695+0000 7f1652ffd640 1 -- 172.21.15.89:0/3838292072 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f165c003cf0 con 0x7f166415d400 2023-10-18T10:47:16.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.695+0000 7f1652ffd640 1 -- 172.21.15.89:0/3838292072 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165c00daf0 con 0x7f166415d400 2023-10-18T10:47:16.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.695+0000 7f16689ba640 1 -- 172.21.15.89:0/3838292072 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16640c7b30 con 0x7f166415d400 2023-10-18T10:47:16.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.695+0000 7f16689ba640 1 -- 172.21.15.89:0/3838292072 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1664156110 con 0x7f166415d400 2023-10-18T10:47:16.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.695+0000 7f16689ba640 1 -- 172.21.15.89:0/3838292072 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1664155c90 con 0x7f166415d400 2023-10-18T10:47:16.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.699+0000 7f1652ffd640 1 -- 172.21.15.89:0/3838292072 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f165c00dc50 con 0x7f166415d400 2023-10-18T10:47:16.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.699+0000 7f1652ffd640 1 --2- 172.21.15.89:0/3838292072 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f16440656b0 0x7f1644067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:16.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.699+0000 7f1652ffd640 1 -- 172.21.15.89:0/3838292072 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f165c083130 con 0x7f166415d400 2023-10-18T10:47:16.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.699+0000 7f166259c640 1 --2- 172.21.15.89:0/3838292072 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f16440656b0 0x7f1644067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:16.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.699+0000 7f166259c640 1 --2- 172.21.15.89:0/3838292072 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f16440656b0 0x7f1644067b70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f16540ad000 tx=0x7f16540046d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:16.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.703+0000 7f1652ffd640 1 -- 172.21.15.89:0/3838292072 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f1664155c90 con 0x7f166415d400 2023-10-18T10:47:16.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:16.883+0000 7f16689ba640 1 -- 172.21.15.89:0/3838292072 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1664155c90 con 0x7f166415d400 2023-10-18T10:47:17.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:17.847+0000 7f1652ffd640 1 -- 172.21.15.89:0/3838292072 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1664155c90 con 0x7f166415d400 2023-10-18T10:47:17.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:17.847+0000 7f16689ba640 1 -- 172.21.15.89:0/3838292072 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f16440656b0 msgr2=0x7f1644067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:17.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:17.847+0000 7f16689ba640 1 --2- 172.21.15.89:0/3838292072 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f16440656b0 0x7f1644067b70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f16540ad000 tx=0x7f16540046d0 comp rx=0 tx=0).stop 2023-10-18T10:47:17.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:17.847+0000 7f16689ba640 1 -- 172.21.15.89:0/3838292072 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f166415d400 msgr2=0x7f1664156630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:17.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:17.847+0000 7f16689ba640 1 --2- 172.21.15.89:0/3838292072 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f166415d400 0x7f1664156630 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f16640b63d0 tx=0x7f165c00d150 comp rx=0 tx=0).stop 2023-10-18T10:47:17.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:17.847+0000 7f16689ba640 1 -- 172.21.15.89:0/3838292072 shutdown_connections 2023-10-18T10:47:17.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:17.847+0000 7f16689ba640 1 --2- 172.21.15.89:0/3838292072 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f16440656b0 0x7f1644067b70 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:17.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:17.847+0000 7f16689ba640 1 --2- 172.21.15.89:0/3838292072 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f166415d400 0x7f1664156630 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:17.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:17.847+0000 7f16689ba640 1 --2- 172.21.15.89:0/3838292072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f166415bb30 0x7f166415cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:17.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:17.847+0000 7f16689ba640 1 --2- 172.21.15.89:0/3838292072 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f164c0039b0 0x7f1664159030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:17.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:17.847+0000 7f16689ba640 1 -- 172.21.15.89:0/3838292072 >> 172.21.15.89:0/3838292072 conn(0x7f166406dad0 msgr2=0x7f166415b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:17.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:17.847+0000 7f16689ba640 1 -- 172.21.15.89:0/3838292072 shutdown_connections 2023-10-18T10:47:17.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:17.847+0000 7f16689ba640 1 -- 172.21.15.89:0/3838292072 wait complete. 2023-10-18T10:47:17.873 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-18T10:47:17.874 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-18T10:47:18.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.015+0000 7fdc6a103640 1 Processor -- start 2023-10-18T10:47:18.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.015+0000 7fdc6a103640 1 -- start start 2023-10-18T10:47:18.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.019+0000 7fdc6a103640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdc640c9970 0x7fdc64164230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:18.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.019+0000 7fdc6a103640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc64070b90 con 0x7fdc64159490 2023-10-18T10:47:18.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.019+0000 7fdc6a103640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc64070e70 con 0x7fdc64070600 2023-10-18T10:47:18.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.019+0000 7fdc6a103640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc64071150 con 0x7fdc640c9970 2023-10-18T10:47:18.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.019+0000 7fdc637fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fdc64159490 0x7fdc6415d8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40650/0 (socket says 172.21.15.89:40650) 2023-10-18T10:47:18.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.019+0000 7fdc637fe640 1 -- 172.21.15.89:0/222072604 learned_addr learned my addr 172.21.15.89:0/222072604 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:47:18.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.019+0000 7fdc62ffd640 1 -- 172.21.15.89:0/222072604 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 619630449 0 0) 0x7fdc64070b90 con 0x7fdc64159490 2023-10-18T10:47:18.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.019+0000 7fdc62ffd640 1 -- 172.21.15.89:0/222072604 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc50003530 con 0x7fdc64159490 2023-10-18T10:47:18.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.019+0000 7fdc63fff640 1 --2- 172.21.15.89:0/222072604 >> v2:172.21.15.154:3300/0 conn(0x7fdc640c9970 0x7fdc64164230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:18.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.019+0000 7fdc62ffd640 1 -- 172.21.15.89:0/222072604 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4090887517 0 0) 0x7fdc50003530 con 0x7fdc64159490 2023-10-18T10:47:18.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.019+0000 7fdc62ffd640 1 -- 172.21.15.89:0/222072604 >> v2:172.21.15.154:3300/0 conn(0x7fdc640c9970 msgr2=0x7fdc64164230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:18.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.019+0000 7fdc62ffd640 1 --2- 172.21.15.89:0/222072604 >> v2:172.21.15.154:3300/0 conn(0x7fdc640c9970 0x7fdc64164230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:18.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.019+0000 7fdc62ffd640 1 -- 172.21.15.89:0/222072604 >> v1:172.21.15.154:6789/0 conn(0x7fdc64070600 legacy=0x7fdc6406b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:18.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.019+0000 7fdc62ffd640 1 -- 172.21.15.89:0/222072604 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc64071430 con 0x7fdc64159490 2023-10-18T10:47:18.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.019+0000 7fdc62ffd640 1 -- 172.21.15.89:0/222072604 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7fdc54001680 con 0x7fdc64159490 2023-10-18T10:47:18.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.019+0000 7fdc62ffd640 1 -- 172.21.15.89:0/222072604 >> v1:172.21.15.89:6789/0 conn(0x7fdc64159490 legacy=0x7fdc6415d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:18.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.019+0000 7fdc62ffd640 1 --2- 172.21.15.89:0/222072604 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc50003980 0x7fdc50003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:18.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.019+0000 7fdc62ffd640 1 --2- 172.21.15.89:0/222072604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc500047d0 0x7fdc50004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:18.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.019+0000 7fdc62ffd640 1 --2- 172.21.15.89:0/222072604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc50005190 0x7fdc5000dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:18.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.019+0000 7fdc62ffd640 1 -- 172.21.15.89:0/222072604 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc64071150 con 0x7fdc500047d0 2023-10-18T10:47:18.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.019+0000 7fdc62ffd640 1 -- 172.21.15.89:0/222072604 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5000e340 con 0x7fdc50005190 2023-10-18T10:47:18.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.019+0000 7fdc62ffd640 1 -- 172.21.15.89:0/222072604 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5000e500 con 0x7fdc50003980 2023-10-18T10:47:18.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.019+0000 7fdc5bfff640 1 --2- 172.21.15.89:0/222072604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc500047d0 0x7fdc50004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:18.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.019+0000 7fdc63fff640 1 --2- 172.21.15.89:0/222072604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc50005190 0x7fdc5000dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:18.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.019+0000 7fdc637fe640 1 --2- 172.21.15.89:0/222072604 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc50003980 0x7fdc50003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:18.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.019+0000 7fdc5bfff640 1 -- 172.21.15.89:0/222072604 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc50003980 msgr2=0x7fdc50003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:18.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.019+0000 7fdc5bfff640 1 --2- 172.21.15.89:0/222072604 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc50003980 0x7fdc50003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:18.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.019+0000 7fdc5bfff640 1 -- 172.21.15.89:0/222072604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc50005190 msgr2=0x7fdc5000dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:18.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.019+0000 7fdc5bfff640 1 --2- 172.21.15.89:0/222072604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc50005190 0x7fdc5000dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:18.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.019+0000 7fdc5bfff640 1 -- 172.21.15.89:0/222072604 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc50003530 con 0x7fdc500047d0 2023-10-18T10:47:18.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.019+0000 7fdc63fff640 1 -- 172.21.15.89:0/222072604 reap_dead start 2023-10-18T10:47:18.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.023+0000 7fdc5bfff640 1 --2- 172.21.15.89:0/222072604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc500047d0 0x7fdc50004c50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c001780 tx=0x7fdc4c01c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e12aaeba6dc368dd server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:18.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.023+0000 7fdc62ffd640 1 -- 172.21.15.89:0/222072604 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c0052d0 con 0x7fdc500047d0 2023-10-18T10:47:18.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.023+0000 7fdc62ffd640 1 -- 172.21.15.89:0/222072604 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc4c0a7050 con 0x7fdc500047d0 2023-10-18T10:47:18.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.023+0000 7fdc62ffd640 1 -- 172.21.15.89:0/222072604 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c003600 con 0x7fdc500047d0 2023-10-18T10:47:18.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.023+0000 7fdc6a103640 1 -- 172.21.15.89:0/222072604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc500047d0 msgr2=0x7fdc50004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:18.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.023+0000 7fdc6a103640 1 --2- 172.21.15.89:0/222072604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc500047d0 0x7fdc50004c50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c001780 tx=0x7fdc4c01c3b0 comp rx=0 tx=0).stop 2023-10-18T10:47:18.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.023+0000 7fdc6a103640 1 -- 172.21.15.89:0/222072604 shutdown_connections 2023-10-18T10:47:18.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.023+0000 7fdc6a103640 1 --2- 172.21.15.89:0/222072604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc500047d0 0x7fdc50004c50 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:18.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.023+0000 7fdc6a103640 1 -- 172.21.15.89:0/222072604 >> 172.21.15.89:0/222072604 conn(0x7fdc64069f00 msgr2=0x7fdc640cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:18.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.023+0000 7fdc6a103640 1 -- 172.21.15.89:0/222072604 shutdown_connections 2023-10-18T10:47:18.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.023+0000 7fdc6a103640 1 -- 172.21.15.89:0/222072604 wait complete. 2023-10-18T10:47:18.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.027+0000 7fdc6a103640 1 Processor -- start 2023-10-18T10:47:18.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.027+0000 7fdc6a103640 1 -- start start 2023-10-18T10:47:18.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.027+0000 7fdc6a103640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc500047d0 0x7fdc641537d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:18.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.027+0000 7fdc6a103640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc640aed80 0x7fdc640af200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:18.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.027+0000 7fdc6a103640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc6415b070 0x7fdc6415b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:18.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.027+0000 7fdc6a103640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc640c2020 con 0x7fdc640aed80 2023-10-18T10:47:18.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.027+0000 7fdc6a103640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc640af740 con 0x7fdc500047d0 2023-10-18T10:47:18.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.027+0000 7fdc6a103640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc64153d10 con 0x7fdc6415b070 2023-10-18T10:47:18.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.027+0000 7fdc637fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc500047d0 0x7fdc641537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:18.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.027+0000 7fdc5bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc640aed80 0x7fdc640af200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:18.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.027+0000 7fdc637fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc500047d0 0x7fdc641537d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:45544/0 (socket says 172.21.15.89:45544) 2023-10-18T10:47:18.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.027+0000 7fdc637fe640 1 -- 172.21.15.89:0/2588231912 learned_addr learned my addr 172.21.15.89:0/2588231912 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:18.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.027+0000 7fdc63fff640 1 --2- 172.21.15.89:0/2588231912 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc6415b070 0x7fdc6415b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:18.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.027+0000 7fdc5bfff640 1 -- 172.21.15.89:0/2588231912 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc6415b070 msgr2=0x7fdc6415b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:18.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.027+0000 7fdc5bfff640 1 --2- 172.21.15.89:0/2588231912 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc6415b070 0x7fdc6415b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:18.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.027+0000 7fdc5bfff640 1 -- 172.21.15.89:0/2588231912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc500047d0 msgr2=0x7fdc641537d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:18.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.027+0000 7fdc5bfff640 1 --2- 172.21.15.89:0/2588231912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc500047d0 0x7fdc641537d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:18.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.027+0000 7fdc5bfff640 1 -- 172.21.15.89:0/2588231912 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc640ac400 con 0x7fdc640aed80 2023-10-18T10:47:18.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.027+0000 7fdc5bfff640 1 --2- 172.21.15.89:0/2588231912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc640aed80 0x7fdc640af200 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000e310 tx=0x7fdc50005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:18.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.027+0000 7fdc60ff9640 1 -- 172.21.15.89:0/2588231912 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c0072f0 con 0x7fdc640aed80 2023-10-18T10:47:18.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.027+0000 7fdc60ff9640 1 -- 172.21.15.89:0/2588231912 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc4c007450 con 0x7fdc640aed80 2023-10-18T10:47:18.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.027+0000 7fdc6a103640 1 -- 172.21.15.89:0/2588231912 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc640ac6e0 con 0x7fdc640aed80 2023-10-18T10:47:18.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.027+0000 7fdc60ff9640 1 -- 172.21.15.89:0/2588231912 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c0067e0 con 0x7fdc640aed80 2023-10-18T10:47:18.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.027+0000 7fdc6a103640 1 -- 172.21.15.89:0/2588231912 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc640acca0 con 0x7fdc640aed80 2023-10-18T10:47:18.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.031+0000 7fdc6a103640 1 -- 172.21.15.89:0/2588231912 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc6404dbe0 con 0x7fdc640aed80 2023-10-18T10:47:18.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.031+0000 7fdc60ff9640 1 -- 172.21.15.89:0/2588231912 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fdc4c006940 con 0x7fdc640aed80 2023-10-18T10:47:18.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.031+0000 7fdc60ff9640 1 --2- 172.21.15.89:0/2588231912 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdc44065680 0x7fdc44067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:18.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.031+0000 7fdc60ff9640 1 -- 172.21.15.89:0/2588231912 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fdc4c11c590 con 0x7fdc640aed80 2023-10-18T10:47:18.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.031+0000 7fdc637fe640 1 --2- 172.21.15.89:0/2588231912 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdc44065680 0x7fdc44067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:18.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.031+0000 7fdc637fe640 1 --2- 172.21.15.89:0/2588231912 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdc44065680 0x7fdc44067b40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdc54000c10 tx=0x7fdc54000f90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:18.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.035+0000 7fdc60ff9640 1 -- 172.21.15.89:0/2588231912 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fdc4c0edae0 con 0x7fdc640aed80 2023-10-18T10:47:18.118 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:47:18.121 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:47:18.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.215+0000 7fdc6a103640 1 -- 172.21.15.89:0/2588231912 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fdc640cbb60 con 0x7fdc640aed80 2023-10-18T10:47:18.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.851+0000 7fdc60ff9640 1 -- 172.21.15.89:0/2588231912 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdc4c0ed670 con 0x7fdc640aed80 2023-10-18T10:47:18.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.855+0000 7fdc6a103640 1 -- 172.21.15.89:0/2588231912 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdc44065680 msgr2=0x7fdc44067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:18.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.855+0000 7fdc6a103640 1 --2- 172.21.15.89:0/2588231912 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdc44065680 0x7fdc44067b40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdc54000c10 tx=0x7fdc54000f90 comp rx=0 tx=0).stop 2023-10-18T10:47:18.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.855+0000 7fdc6a103640 1 -- 172.21.15.89:0/2588231912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc640aed80 msgr2=0x7fdc640af200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:18.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.855+0000 7fdc6a103640 1 --2- 172.21.15.89:0/2588231912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc640aed80 0x7fdc640af200 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000e310 tx=0x7fdc50005800 comp rx=0 tx=0).stop 2023-10-18T10:47:18.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.855+0000 7fdc6a103640 1 -- 172.21.15.89:0/2588231912 shutdown_connections 2023-10-18T10:47:18.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.855+0000 7fdc6a103640 1 --2- 172.21.15.89:0/2588231912 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdc44065680 0x7fdc44067b40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:18.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.855+0000 7fdc6a103640 1 --2- 172.21.15.89:0/2588231912 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc6415b070 0x7fdc6415b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:18.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.855+0000 7fdc6a103640 1 --2- 172.21.15.89:0/2588231912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc640aed80 0x7fdc640af200 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:18.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.855+0000 7fdc6a103640 1 --2- 172.21.15.89:0/2588231912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc500047d0 0x7fdc641537d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:18.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.855+0000 7fdc6a103640 1 -- 172.21.15.89:0/2588231912 >> 172.21.15.89:0/2588231912 conn(0x7fdc64069f00 msgr2=0x7fdc641571d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:18.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.855+0000 7fdc6a103640 1 -- 172.21.15.89:0/2588231912 shutdown_connections 2023-10-18T10:47:18.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:18.855+0000 7fdc6a103640 1 -- 172.21.15.89:0/2588231912 wait complete. 2023-10-18T10:47:18.877 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-18T10:47:18.878 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:47:19.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.059+0000 7f5c0fe6d640 1 Processor -- start 2023-10-18T10:47:19.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.059+0000 7f5c0fe6d640 1 -- start start 2023-10-18T10:47:19.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.059+0000 7f5c0fe6d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5c0815c2f0 0x7f5c080b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:19.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.059+0000 7f5c0fe6d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5c080d0270 0x7f5c080d0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:19.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.059+0000 7f5c0fe6d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c0815c6d0 0x7f5c0815cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:19.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.059+0000 7f5c0fe6d640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c0815d1b0 con 0x7f5c0815c2f0 2023-10-18T10:47:19.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.059+0000 7f5c0fe6d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c0815d490 con 0x7f5c080d0270 2023-10-18T10:47:19.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.059+0000 7f5c0fe6d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0815d770 con 0x7f5c0815c6d0 2023-10-18T10:47:19.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.059+0000 7f5c0dc09640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5c0815c2f0 0x7f5c080b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:19.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.059+0000 7f5c0dc09640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5c0815c2f0 0x7f5c080b18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37972/0 (socket says 172.21.15.89:37972) 2023-10-18T10:47:19.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.059+0000 7f5c0dc09640 1 -- 172.21.15.89:0/436192020 learned_addr learned my addr 172.21.15.89:0/436192020 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:19.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.059+0000 7f5c0d408640 1 --2- 172.21.15.89:0/436192020 >> v2:172.21.15.154:3300/0 conn(0x7f5c080d0270 0x7f5c080d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:19.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.059+0000 7f5c0e40a640 1 --2- 172.21.15.89:0/436192020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c0815c6d0 0x7f5c0815cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:19.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.059+0000 7f5c0dc09640 1 -- 172.21.15.89:0/436192020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c0815c6d0 msgr2=0x7f5c0815cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:19.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.059+0000 7f5c0dc09640 1 --2- 172.21.15.89:0/436192020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c0815c6d0 0x7f5c0815cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:19.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.059+0000 7f5c0dc09640 1 -- 172.21.15.89:0/436192020 >> v2:172.21.15.154:3300/0 conn(0x7f5c080d0270 msgr2=0x7f5c080d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:19.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.059+0000 7f5c0dc09640 1 --2- 172.21.15.89:0/436192020 >> v2:172.21.15.154:3300/0 conn(0x7f5c080d0270 0x7f5c080d0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:19.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.059+0000 7f5c0dc09640 1 -- 172.21.15.89:0/436192020 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0815da50 con 0x7f5c0815c2f0 2023-10-18T10:47:19.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.063+0000 7f5c0dc09640 1 --2- 172.21.15.89:0/436192020 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c0815c2f0 0x7f5c080b18e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5c080c96b0 tx=0x7f5c0401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23059751fc474f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:19.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.063+0000 7f5c0cc07640 1 -- 172.21.15.89:0/436192020 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c040aa610 con 0x7f5c0815c2f0 2023-10-18T10:47:19.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.063+0000 7f5c0cc07640 1 -- 172.21.15.89:0/436192020 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c040a6040 con 0x7f5c0815c2f0 2023-10-18T10:47:19.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.063+0000 7f5c0cc07640 1 -- 172.21.15.89:0/436192020 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c040aa990 con 0x7f5c0815c2f0 2023-10-18T10:47:19.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.063+0000 7f5c0fe6d640 1 -- 172.21.15.89:0/436192020 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c0815c2f0 msgr2=0x7f5c080b18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:19.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.063+0000 7f5c0fe6d640 1 --2- 172.21.15.89:0/436192020 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c0815c2f0 0x7f5c080b18e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5c080c96b0 tx=0x7f5c0401c4d0 comp rx=0 tx=0).stop 2023-10-18T10:47:19.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.063+0000 7f5c0fe6d640 1 -- 172.21.15.89:0/436192020 shutdown_connections 2023-10-18T10:47:19.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.063+0000 7f5c0fe6d640 1 --2- 172.21.15.89:0/436192020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c0815c6d0 0x7f5c0815cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:19.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.063+0000 7f5c0fe6d640 1 --2- 172.21.15.89:0/436192020 >> v2:172.21.15.154:3300/0 conn(0x7f5c080d0270 0x7f5c080d0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:19.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.063+0000 7f5c0fe6d640 1 --2- 172.21.15.89:0/436192020 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c0815c2f0 0x7f5c080b18e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:19.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.063+0000 7f5c0fe6d640 1 -- 172.21.15.89:0/436192020 >> 172.21.15.89:0/436192020 conn(0x7f5c08069ad0 msgr2=0x7f5c08069eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:19.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.063+0000 7f5c0fe6d640 1 -- 172.21.15.89:0/436192020 shutdown_connections 2023-10-18T10:47:19.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.063+0000 7f5c0fe6d640 1 -- 172.21.15.89:0/436192020 wait complete. 2023-10-18T10:47:19.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.067+0000 7f5c0fe6d640 1 Processor -- start 2023-10-18T10:47:19.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.067+0000 7f5c0fe6d640 1 -- start start 2023-10-18T10:47:19.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.067+0000 7f5c0fe6d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c080d0270 0x7f5c080cc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:19.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.067+0000 7f5c0fe6d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c0815c2f0 0x7f5c08156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:19.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.067+0000 7f5c0fe6d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c0815c6d0 0x7f5c0815ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:19.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.067+0000 7f5c0fe6d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c08157220 con 0x7f5c080d0270 2023-10-18T10:47:19.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.067+0000 7f5c0fe6d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c081fa660 con 0x7f5c0815c2f0 2023-10-18T10:47:19.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.067+0000 7f5c0fe6d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c081fabe0 con 0x7f5c0815c6d0 2023-10-18T10:47:19.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.067+0000 7f5c0dc09640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c080d0270 0x7f5c080cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:19.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.067+0000 7f5c0dc09640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c080d0270 0x7f5c080cc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37982/0 (socket says 172.21.15.89:37982) 2023-10-18T10:47:19.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.067+0000 7f5c0dc09640 1 -- 172.21.15.89:0/118964272 learned_addr learned my addr 172.21.15.89:0/118964272 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:19.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.067+0000 7f5c0d408640 1 --2- 172.21.15.89:0/118964272 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c0815c2f0 0x7f5c08156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:19.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.067+0000 7f5c0e40a640 1 --2- 172.21.15.89:0/118964272 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c0815c6d0 0x7f5c0815ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:19.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.067+0000 7f5c0dc09640 1 -- 172.21.15.89:0/118964272 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c0815c6d0 msgr2=0x7f5c0815ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:19.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.067+0000 7f5c0dc09640 1 --2- 172.21.15.89:0/118964272 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c0815c6d0 0x7f5c0815ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:19.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.067+0000 7f5c0dc09640 1 -- 172.21.15.89:0/118964272 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c0815c2f0 msgr2=0x7f5c08156240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:19.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.067+0000 7f5c0dc09640 1 --2- 172.21.15.89:0/118964272 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c0815c2f0 0x7f5c08156240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:19.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.067+0000 7f5c0dc09640 1 -- 172.21.15.89:0/118964272 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c040094d0 con 0x7f5c080d0270 2023-10-18T10:47:19.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.067+0000 7f5c0dc09640 1 --2- 172.21.15.89:0/118964272 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c080d0270 0x7f5c080cc300 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5c0401ab80 tx=0x7f5c0401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:19.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.067+0000 7f5bfe7fc640 1 -- 172.21.15.89:0/118964272 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c040bd5f0 con 0x7f5c080d0270 2023-10-18T10:47:19.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.067+0000 7f5bfe7fc640 1 -- 172.21.15.89:0/118964272 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c040bd750 con 0x7f5c080d0270 2023-10-18T10:47:19.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.067+0000 7f5c0fe6d640 1 -- 172.21.15.89:0/118964272 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c08156780 con 0x7f5c080d0270 2023-10-18T10:47:19.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.071+0000 7f5bfe7fc640 1 -- 172.21.15.89:0/118964272 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c040bdb00 con 0x7f5c080d0270 2023-10-18T10:47:19.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.071+0000 7f5c0fe6d640 1 -- 172.21.15.89:0/118964272 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c081fa870 con 0x7f5c080d0270 2023-10-18T10:47:19.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.071+0000 7f5c0fe6d640 1 -- 172.21.15.89:0/118964272 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c08066ca0 con 0x7f5c080d0270 2023-10-18T10:47:19.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.071+0000 7f5bfe7fc640 1 -- 172.21.15.89:0/118964272 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f5c040b33f0 con 0x7f5c080d0270 2023-10-18T10:47:19.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.071+0000 7f5bfe7fc640 1 --2- 172.21.15.89:0/118964272 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5bec065680 0x7f5bec067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:19.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.071+0000 7f5bfe7fc640 1 -- 172.21.15.89:0/118964272 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f5c0411dbb0 con 0x7f5c080d0270 2023-10-18T10:47:19.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.071+0000 7f5c0d408640 1 --2- 172.21.15.89:0/118964272 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5bec065680 0x7f5bec067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:19.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.075+0000 7f5c0d408640 1 --2- 172.21.15.89:0/118964272 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5bec065680 0x7f5bec067b40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5c080cc840 tx=0x7f5bf8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:19.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.075+0000 7f5bfe7fc640 1 -- 172.21.15.89:0/118964272 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f5c040eb540 con 0x7f5c080d0270 2023-10-18T10:47:19.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.259+0000 7f5c0fe6d640 1 -- 172.21.15.89:0/118964272 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c080c94c0 con 0x7f5c080d0270 2023-10-18T10:47:19.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.259+0000 7f5bfe7fc640 1 -- 172.21.15.89:0/118964272 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4875 (secure 0 0 0) 0x7f5c040c3090 con 0x7f5c080d0270 2023-10-18T10:47:19.263 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:47:19.263 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/654729641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":654729641},{"type":"v1","addr":"172.21.15.154:6836","nonce":654729641}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/964566480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":964566480},{"type":"v1","addr":"172.21.15.154:6837","nonce":964566480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2261878538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2261878538},{"type":"v1","addr":"172.21.15.89:6833","nonce":2261878538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:47:15.199575+0000","modified":"2023-10-18T10:47:18.846656+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5517":{"gid":5517,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6835/2092274731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2092274731},{"type":"v1","addr":"172.21.15.89:6835","nonce":2092274731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T10:47:19.263 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 18 2023-10-18T10:47:19.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.263+0000 7f5c0fe6d640 1 -- 172.21.15.89:0/118964272 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5bec065680 msgr2=0x7f5bec067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:19.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.263+0000 7f5c0fe6d640 1 --2- 172.21.15.89:0/118964272 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5bec065680 0x7f5bec067b40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5c080cc840 tx=0x7f5bf8008040 comp rx=0 tx=0).stop 2023-10-18T10:47:19.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.263+0000 7f5c0fe6d640 1 -- 172.21.15.89:0/118964272 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c080d0270 msgr2=0x7f5c080cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:19.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.263+0000 7f5c0fe6d640 1 --2- 172.21.15.89:0/118964272 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c080d0270 0x7f5c080cc300 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5c0401ab80 tx=0x7f5c0401ad10 comp rx=0 tx=0).stop 2023-10-18T10:47:19.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.263+0000 7f5c0fe6d640 1 -- 172.21.15.89:0/118964272 shutdown_connections 2023-10-18T10:47:19.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.263+0000 7f5c0fe6d640 1 --2- 172.21.15.89:0/118964272 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5bec065680 0x7f5bec067b40 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:19.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.263+0000 7f5c0fe6d640 1 --2- 172.21.15.89:0/118964272 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c0815c6d0 0x7f5c0815ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:19.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.263+0000 7f5c0fe6d640 1 --2- 172.21.15.89:0/118964272 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c0815c2f0 0x7f5c08156240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:19.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.263+0000 7f5c0fe6d640 1 --2- 172.21.15.89:0/118964272 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c080d0270 0x7f5c080cc300 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:19.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.263+0000 7f5c0fe6d640 1 -- 172.21.15.89:0/118964272 >> 172.21.15.89:0/118964272 conn(0x7f5c08069ad0 msgr2=0x7f5c080b4570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:19.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.263+0000 7f5c0fe6d640 1 -- 172.21.15.89:0/118964272 shutdown_connections 2023-10-18T10:47:19.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.263+0000 7f5c0fe6d640 1 -- 172.21.15.89:0/118964272 wait complete. 2023-10-18T10:47:19.284 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:47:19.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.419+0000 7f5935aca640 1 Processor -- start 2023-10-18T10:47:19.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.419+0000 7f5935aca640 1 -- start start 2023-10-18T10:47:19.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.419+0000 7f5935aca640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f593015e600 0x7f59300c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:19.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.419+0000 7f5935aca640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59301614e0 con 0x7f59300afd90 2023-10-18T10:47:19.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.419+0000 7f5935aca640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f593006c640 con 0x7f593006c1b0 2023-10-18T10:47:19.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.419+0000 7f5935aca640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5930160f80 con 0x7f593015e600 2023-10-18T10:47:19.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.419+0000 7f592ffff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f59300afd90 0x7f59300b01a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40662/0 (socket says 172.21.15.89:40662) 2023-10-18T10:47:19.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.419+0000 7f592ffff640 1 -- 172.21.15.89:0/983785221 learned_addr learned my addr 172.21.15.89:0/983785221 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:47:19.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.419+0000 7f592f7fe640 1 --2- 172.21.15.89:0/983785221 >> v2:172.21.15.154:3300/0 conn(0x7f593015e600 0x7f59300c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:19.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.419+0000 7f592effd640 1 -- 172.21.15.89:0/983785221 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2443474526 0 0) 0x7f59301614e0 con 0x7f59300afd90 2023-10-18T10:47:19.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.419+0000 7f592effd640 1 -- 172.21.15.89:0/983785221 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f591c003530 con 0x7f59300afd90 2023-10-18T10:47:19.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.423+0000 7f592f7fe640 1 -- 172.21.15.89:0/983785221 >> v1:172.21.15.154:6789/0 conn(0x7f593006c1b0 legacy=0x7f5930161970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:19.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.423+0000 7f592f7fe640 1 -- 172.21.15.89:0/983785221 >> v1:172.21.15.89:6789/0 conn(0x7f59300afd90 legacy=0x7f59300b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:19.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.423+0000 7f592f7fe640 1 -- 172.21.15.89:0/983785221 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f593015e9e0 con 0x7f593015e600 2023-10-18T10:47:19.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.423+0000 7f592f7fe640 1 --2- 172.21.15.89:0/983785221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f593015e600 0x7f59300c8960 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f59200097c0 tx=0x7f592001c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43150f232ad69486 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:19.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.423+0000 7f592effd640 1 -- 172.21.15.89:0/983785221 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59200ab650 con 0x7f593015e600 2023-10-18T10:47:19.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.423+0000 7f592effd640 1 -- 172.21.15.89:0/983785221 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59200ab7b0 con 0x7f593015e600 2023-10-18T10:47:19.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.423+0000 7f592effd640 1 -- 172.21.15.89:0/983785221 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59200abb40 con 0x7f593015e600 2023-10-18T10:47:19.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.427+0000 7f5935aca640 1 -- 172.21.15.89:0/983785221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f593015e600 msgr2=0x7f59300c8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:19.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.427+0000 7f5935aca640 1 --2- 172.21.15.89:0/983785221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f593015e600 0x7f59300c8960 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f59200097c0 tx=0x7f592001c450 comp rx=0 tx=0).stop 2023-10-18T10:47:19.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.427+0000 7f5935aca640 1 -- 172.21.15.89:0/983785221 shutdown_connections 2023-10-18T10:47:19.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.427+0000 7f5935aca640 1 --2- 172.21.15.89:0/983785221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f593015e600 0x7f59300c8960 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:19.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.427+0000 7f5935aca640 1 -- 172.21.15.89:0/983785221 >> 172.21.15.89:0/983785221 conn(0x7f5930069f00 msgr2=0x7f59300cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:19.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.427+0000 7f5935aca640 1 -- 172.21.15.89:0/983785221 shutdown_connections 2023-10-18T10:47:19.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.427+0000 7f5935aca640 1 -- 172.21.15.89:0/983785221 wait complete. 2023-10-18T10:47:19.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.427+0000 7f5935aca640 1 Processor -- start 2023-10-18T10:47:19.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.427+0000 7f5935aca640 1 -- start start 2023-10-18T10:47:19.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.427+0000 7f5935aca640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f593006c1b0 0x7f59300b3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:19.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.427+0000 7f5935aca640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59300afd90 0x7f5930155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:19.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.427+0000 7f5935aca640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f593015e600 0x7f59300b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:19.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.427+0000 7f5935aca640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5930161dd0 con 0x7f593006c1b0 2023-10-18T10:47:19.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.427+0000 7f5935aca640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59300b2110 con 0x7f593015e600 2023-10-18T10:47:19.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.427+0000 7f5935aca640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59300c6f70 con 0x7f59300afd90 2023-10-18T10:47:19.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.427+0000 7f592f7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f593006c1b0 0x7f59300b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:19.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.427+0000 7f592f7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f593006c1b0 0x7f59300b3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37986/0 (socket says 172.21.15.89:37986) 2023-10-18T10:47:19.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.427+0000 7f592f7fe640 1 -- 172.21.15.89:0/1719680538 learned_addr learned my addr 172.21.15.89:0/1719680538 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:19.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.427+0000 7f5927fff640 1 --2- 172.21.15.89:0/1719680538 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59300afd90 0x7f5930155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:19.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.427+0000 7f592f7fe640 1 -- 172.21.15.89:0/1719680538 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59300afd90 msgr2=0x7f5930155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:19.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.427+0000 7f592ffff640 1 --2- 172.21.15.89:0/1719680538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f593015e600 0x7f59300b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:19.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.427+0000 7f592f7fe640 1 --2- 172.21.15.89:0/1719680538 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59300afd90 0x7f5930155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:19.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.427+0000 7f592f7fe640 1 -- 172.21.15.89:0/1719680538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f593015e600 msgr2=0x7f59300b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:19.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.427+0000 7f592f7fe640 1 --2- 172.21.15.89:0/1719680538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f593015e600 0x7f59300b0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:19.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.427+0000 7f592f7fe640 1 -- 172.21.15.89:0/1719680538 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5920009420 con 0x7f593006c1b0 2023-10-18T10:47:19.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.427+0000 7f5927fff640 1 --2- 172.21.15.89:0/1719680538 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59300afd90 0x7f5930155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:47:19.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.427+0000 7f592ffff640 1 --2- 172.21.15.89:0/1719680538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f593015e600 0x7f59300b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:47:19.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.431+0000 7f592f7fe640 1 --2- 172.21.15.89:0/1719680538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f593006c1b0 0x7f59300b3930 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f59200ad040 tx=0x7f592001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:19.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.431+0000 7f592cff9640 1 -- 172.21.15.89:0/1719680538 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59200bf610 con 0x7f593006c1b0 2023-10-18T10:47:19.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.431+0000 7f592cff9640 1 -- 172.21.15.89:0/1719680538 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59200bf770 con 0x7f593006c1b0 2023-10-18T10:47:19.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.431+0000 7f592cff9640 1 -- 172.21.15.89:0/1719680538 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59200bfb20 con 0x7f593006c1b0 2023-10-18T10:47:19.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.431+0000 7f5935aca640 1 -- 172.21.15.89:0/1719680538 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59300c7250 con 0x7f593006c1b0 2023-10-18T10:47:19.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.431+0000 7f5935aca640 1 -- 172.21.15.89:0/1719680538 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59300c7810 con 0x7f593006c1b0 2023-10-18T10:47:19.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.431+0000 7f5935aca640 1 -- 172.21.15.89:0/1719680538 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f593004dbe0 con 0x7f593006c1b0 2023-10-18T10:47:19.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.431+0000 7f592cff9640 1 -- 172.21.15.89:0/1719680538 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f59200b5400 con 0x7f593006c1b0 2023-10-18T10:47:19.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.431+0000 7f592cff9640 1 --2- 172.21.15.89:0/1719680538 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5910065680 0x7f5910067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:19.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.431+0000 7f592cff9640 1 -- 172.21.15.89:0/1719680538 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f592011dd20 con 0x7f593006c1b0 2023-10-18T10:47:19.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.431+0000 7f5927fff640 1 --2- 172.21.15.89:0/1719680538 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5910065680 0x7f5910067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:19.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.435+0000 7f5927fff640 1 --2- 172.21.15.89:0/1719680538 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5910065680 0x7f5910067b40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f59300adb10 tx=0x7f5918009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:19.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.435+0000 7f592cff9640 1 -- 172.21.15.89:0/1719680538 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f59200eb690 con 0x7f593006c1b0 2023-10-18T10:47:19.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.583+0000 7f5935aca640 1 -- 172.21.15.89:0/1719680538 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59300b4290 con 0x7f593006c1b0 2023-10-18T10:47:19.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.587+0000 7f592cff9640 1 -- 172.21.15.89:0/1719680538 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+18476 (secure 0 0 0) 0x7f59200eecb0 con 0x7f593006c1b0 2023-10-18T10:47:19.588 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:47:19.589 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":41,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:47:16.832871+0000","last_up_change":"2023-10-18T10:45:42.164688+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:47:13.188701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-18T10:47:14.198224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6824","nonce":773078364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6827","nonce":773078364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6831","nonce":773078364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":773078364},{"type":"v1","addr":"172.21.15.154:6829","nonce":773078364}]},"public_addr":"172.21.15.154:6824/773078364","cluster_addr":"172.21.15.154:6827/773078364","heartbeat_back_addr":"172.21.15.154:6831/773078364","heartbeat_front_addr":"172.21.15.154:6829/773078364","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:47:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.587+0000 7f5935aca640 1 -- 172.21.15.89:0/1719680538 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5910065680 msgr2=0x7f5910067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.587+0000 7f5935aca640 1 --2- 172.21.15.89:0/1719680538 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5910065680 0x7f5910067b40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f59300adb10 tx=0x7f5918009040 comp rx=0 tx=0).stop 2023-10-18T10:47:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.587+0000 7f5935aca640 1 -- 172.21.15.89:0/1719680538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f593006c1b0 msgr2=0x7f59300b3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:19.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.587+0000 7f5935aca640 1 --2- 172.21.15.89:0/1719680538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f593006c1b0 0x7f59300b3930 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f59200ad040 tx=0x7f592001ac90 comp rx=0 tx=0).stop 2023-10-18T10:47:19.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.587+0000 7f5935aca640 1 -- 172.21.15.89:0/1719680538 shutdown_connections 2023-10-18T10:47:19.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.587+0000 7f5935aca640 1 --2- 172.21.15.89:0/1719680538 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5910065680 0x7f5910067b40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:19.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.587+0000 7f5935aca640 1 --2- 172.21.15.89:0/1719680538 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59300afd90 0x7f5930155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:19.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.587+0000 7f5935aca640 1 --2- 172.21.15.89:0/1719680538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f593015e600 0x7f59300b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.587+0000 7f5935aca640 1 --2- 172.21.15.89:0/1719680538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f593006c1b0 0x7f59300b3930 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.587+0000 7f5935aca640 1 -- 172.21.15.89:0/1719680538 >> 172.21.15.89:0/1719680538 conn(0x7f5930069f00 msgr2=0x7f59300af220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.587+0000 7f5935aca640 1 -- 172.21.15.89:0/1719680538 shutdown_connections 2023-10-18T10:47:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.587+0000 7f5935aca640 1 -- 172.21.15.89:0/1719680538 wait complete. 2023-10-18T10:47:19.610 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-18T10:47:19.610 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:47:19.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.747+0000 7fc45189e640 1 Processor -- start 2023-10-18T10:47:19.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.747+0000 7fc45189e640 1 -- start start 2023-10-18T10:47:19.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.747+0000 7fc45189e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc44c0d1a40 0x7fc44c0cee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:19.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.747+0000 7fc45189e640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc44c0cf7e0 con 0x7fc44c15c550 2023-10-18T10:47:19.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.747+0000 7fc45189e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc44c0d0e40 con 0x7fc44c0cf370 2023-10-18T10:47:19.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.747+0000 7fc45189e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c0d0c50 con 0x7fc44c0d1a40 2023-10-18T10:47:19.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.747+0000 7fc45089c640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc44c15c550 0x7fc44c0d0540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40664/0 (socket says 172.21.15.89:40664) 2023-10-18T10:47:19.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.747+0000 7fc45089c640 1 -- 172.21.15.89:0/418745672 learned_addr learned my addr 172.21.15.89:0/418745672 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:47:19.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.747+0000 7fc44bfff640 1 --2- 172.21.15.89:0/418745672 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc44c0d1a40 0x7fc44c0cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:19.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.747+0000 7fc44b7fe640 1 -- 172.21.15.89:0/418745672 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 311149313 0 0) 0x7fc44c0cf7e0 con 0x7fc44c15c550 2023-10-18T10:47:19.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.747+0000 7fc44b7fe640 1 -- 172.21.15.89:0/418745672 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc434003530 con 0x7fc44c15c550 2023-10-18T10:47:19.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.747+0000 7fc44b7fe640 1 -- 172.21.15.89:0/418745672 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4100338691 0 0) 0x7fc44c0d0e40 con 0x7fc44c0cf370 2023-10-18T10:47:19.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.747+0000 7fc44b7fe640 1 -- 172.21.15.89:0/418745672 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc44c0cf7e0 con 0x7fc44c0cf370 2023-10-18T10:47:19.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.747+0000 7fc44bfff640 1 -- 172.21.15.89:0/418745672 >> v1:172.21.15.154:6789/0 conn(0x7fc44c0cf370 legacy=0x7fc44c0b5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:19.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.747+0000 7fc44bfff640 1 -- 172.21.15.89:0/418745672 >> v1:172.21.15.89:6789/0 conn(0x7fc44c15c550 legacy=0x7fc44c0d0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:19.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.747+0000 7fc44bfff640 1 -- 172.21.15.89:0/418745672 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc44c0d1120 con 0x7fc44c0d1a40 2023-10-18T10:47:19.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.751+0000 7fc44bfff640 1 --2- 172.21.15.89:0/418745672 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc44c0d1a40 0x7fc44c0cee30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc4400097f0 tx=0x7fc44001c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37d1ac456dc41925 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:19.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.751+0000 7fc44b7fe640 1 -- 172.21.15.89:0/418745672 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4400aa5f0 con 0x7fc44c0d1a40 2023-10-18T10:47:19.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.751+0000 7fc44b7fe640 1 -- 172.21.15.89:0/418745672 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4400aa750 con 0x7fc44c0d1a40 2023-10-18T10:47:19.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.751+0000 7fc44b7fe640 1 -- 172.21.15.89:0/418745672 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4400aaae0 con 0x7fc44c0d1a40 2023-10-18T10:47:19.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.751+0000 7fc45189e640 1 -- 172.21.15.89:0/418745672 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc44c0d1a40 msgr2=0x7fc44c0cee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:19.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.751+0000 7fc45189e640 1 --2- 172.21.15.89:0/418745672 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc44c0d1a40 0x7fc44c0cee30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc4400097f0 tx=0x7fc44001c490 comp rx=0 tx=0).stop 2023-10-18T10:47:19.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.751+0000 7fc45189e640 1 -- 172.21.15.89:0/418745672 shutdown_connections 2023-10-18T10:47:19.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.751+0000 7fc45189e640 1 --2- 172.21.15.89:0/418745672 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc44c0d1a40 0x7fc44c0cee30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:19.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.751+0000 7fc45189e640 1 -- 172.21.15.89:0/418745672 >> 172.21.15.89:0/418745672 conn(0x7fc44c070320 msgr2=0x7fc44c070730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:19.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.751+0000 7fc45189e640 1 -- 172.21.15.89:0/418745672 shutdown_connections 2023-10-18T10:47:19.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.751+0000 7fc45189e640 1 -- 172.21.15.89:0/418745672 wait complete. 2023-10-18T10:47:19.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.755+0000 7fc45189e640 1 Processor -- start 2023-10-18T10:47:19.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.755+0000 7fc45189e640 1 -- start start 2023-10-18T10:47:19.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.755+0000 7fc45189e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc44c0cf370 0x7fc44c14be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:19.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.755+0000 7fc45189e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc44c0d1a40 0x7fc44c0c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:19.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.755+0000 7fc45189e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc44c15c550 0x7fc44c0c8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:19.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.755+0000 7fc45189e640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c14c390 con 0x7fc44c0cf370 2023-10-18T10:47:19.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.755+0000 7fc45189e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c0c9240 con 0x7fc44c0d1a40 2023-10-18T10:47:19.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.755+0000 7fc45189e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c0c96e0 con 0x7fc44c15c550 2023-10-18T10:47:19.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.755+0000 7fc45089c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc44c0cf370 0x7fc44c14be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:19.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.755+0000 7fc45089c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc44c0cf370 0x7fc44c14be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37994/0 (socket says 172.21.15.89:37994) 2023-10-18T10:47:19.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.755+0000 7fc45089c640 1 -- 172.21.15.89:0/2987151948 learned_addr learned my addr 172.21.15.89:0/2987151948 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:19.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.755+0000 7fc44bfff640 1 --2- 172.21.15.89:0/2987151948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc44c0d1a40 0x7fc44c0c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:19.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.755+0000 7fc45109d640 1 --2- 172.21.15.89:0/2987151948 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc44c15c550 0x7fc44c0c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:19.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.755+0000 7fc45089c640 1 -- 172.21.15.89:0/2987151948 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc44c15c550 msgr2=0x7fc44c0c8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:19.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.755+0000 7fc45089c640 1 --2- 172.21.15.89:0/2987151948 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc44c15c550 0x7fc44c0c8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:19.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.755+0000 7fc45089c640 1 -- 172.21.15.89:0/2987151948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc44c0d1a40 msgr2=0x7fc44c0c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:19.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.755+0000 7fc45089c640 1 --2- 172.21.15.89:0/2987151948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc44c0d1a40 0x7fc44c0c80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:19.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.755+0000 7fc45089c640 1 -- 172.21.15.89:0/2987151948 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc440009420 con 0x7fc44c0cf370 2023-10-18T10:47:19.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.755+0000 7fc45089c640 1 --2- 172.21.15.89:0/2987151948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc44c0cf370 0x7fc44c14be50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc43c00cbf0 tx=0x7fc43c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:19.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.755+0000 7fc4497fa640 1 -- 172.21.15.89:0/2987151948 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c00ad50 con 0x7fc44c0cf370 2023-10-18T10:47:19.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.755+0000 7fc4497fa640 1 -- 172.21.15.89:0/2987151948 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43c012400 con 0x7fc44c0cf370 2023-10-18T10:47:19.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.755+0000 7fc45189e640 1 -- 172.21.15.89:0/2987151948 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc44c15cab0 con 0x7fc44c0cf370 2023-10-18T10:47:19.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.755+0000 7fc4497fa640 1 -- 172.21.15.89:0/2987151948 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c012790 con 0x7fc44c0cf370 2023-10-18T10:47:19.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.755+0000 7fc45189e640 1 -- 172.21.15.89:0/2987151948 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc44c0c8c80 con 0x7fc44c0cf370 2023-10-18T10:47:19.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.755+0000 7fc45189e640 1 -- 172.21.15.89:0/2987151948 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc44c0c9520 con 0x7fc44c0cf370 2023-10-18T10:47:19.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.759+0000 7fc4497fa640 1 -- 172.21.15.89:0/2987151948 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fc43c007070 con 0x7fc44c0cf370 2023-10-18T10:47:19.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.759+0000 7fc4497fa640 1 --2- 172.21.15.89:0/2987151948 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc4380656b0 0x7fc438067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:19.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.759+0000 7fc4497fa640 1 -- 172.21.15.89:0/2987151948 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fc43c082f40 con 0x7fc44c0cf370 2023-10-18T10:47:19.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.759+0000 7fc44bfff640 1 --2- 172.21.15.89:0/2987151948 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc4380656b0 0x7fc438067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:19.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.759+0000 7fc44bfff640 1 --2- 172.21.15.89:0/2987151948 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc4380656b0 0x7fc438067b70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc4400ae000 tx=0x7fc4400a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:19.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.763+0000 7fc4497fa640 1 -- 172.21.15.89:0/2987151948 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fc43c050960 con 0x7fc44c0cf370 2023-10-18T10:47:19.796 INFO:tasks.ceph.mds.c.smithi089.stderr:2023-10-18T10:47:19.795+0000 7f712e82d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T10:47:19.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.915+0000 7fc45189e640 1 -- 172.21.15.89:0/2987151948 --> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc44c15c930 con 0x7fc4380656b0 2023-10-18T10:47:19.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.923+0000 7fc4497fa640 1 -- 172.21.15.89:0/2987151948 <== mgr.4099 v2:172.21.15.154:6832/1086608968 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311045 (secure 0 0 0) 0x7fc4400aa5d0 con 0x7fc4380656b0 2023-10-18T10:47:19.930 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:47:19.932 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-18T10:47:19.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.927+0000 7fc45189e640 1 -- 172.21.15.89:0/2987151948 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc4380656b0 msgr2=0x7fc438067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:19.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.927+0000 7fc45189e640 1 --2- 172.21.15.89:0/2987151948 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc4380656b0 0x7fc438067b70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc4400ae000 tx=0x7fc4400a7000 comp rx=0 tx=0).stop 2023-10-18T10:47:19.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.927+0000 7fc45189e640 1 -- 172.21.15.89:0/2987151948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc44c0cf370 msgr2=0x7fc44c14be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:19.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.927+0000 7fc45189e640 1 --2- 172.21.15.89:0/2987151948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc44c0cf370 0x7fc44c14be50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc43c00cbf0 tx=0x7fc43c00a430 comp rx=0 tx=0).stop 2023-10-18T10:47:19.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.927+0000 7fc45189e640 1 -- 172.21.15.89:0/2987151948 shutdown_connections 2023-10-18T10:47:19.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.927+0000 7fc45189e640 1 --2- 172.21.15.89:0/2987151948 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc4380656b0 0x7fc438067b70 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:19.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.927+0000 7fc45189e640 1 --2- 172.21.15.89:0/2987151948 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc44c15c550 0x7fc44c0c8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:19.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.927+0000 7fc45189e640 1 --2- 172.21.15.89:0/2987151948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc44c0d1a40 0x7fc44c0c80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:19.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.927+0000 7fc45189e640 1 --2- 172.21.15.89:0/2987151948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc44c0cf370 0x7fc44c14be50 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:19.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.927+0000 7fc45189e640 1 -- 172.21.15.89:0/2987151948 >> 172.21.15.89:0/2987151948 conn(0x7fc44c070320 msgr2=0x7fc44c070700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:19.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.931+0000 7fc45189e640 1 -- 172.21.15.89:0/2987151948 shutdown_connections 2023-10-18T10:47:19.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:19.931+0000 7fc45189e640 1 -- 172.21.15.89:0/2987151948 wait complete. 2023-10-18T10:47:19.951 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2023-10-18T10:47:19.074024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":142,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221060,"kb_used_data":4876,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754753660,"statfs":{"total":773094113280,"available":772867747840,"internally_reserved":0,"allocated":4993024,"data_stored":2852995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.258723"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489199+0000","last_change":"2023-10-18T10:47:15.837486+0000","last_active":"2023-10-18T10:47:17.489199+0000","last_peered":"2023-10-18T10:47:17.489199+0000","last_clean":"2023-10-18T10:47:17.489199+0000","last_became_active":"2023-10-18T10:47:15.837138+0000","last_became_peered":"2023-10-18T10:47:15.837138+0000","last_unstale":"2023-10-18T10:47:17.489199+0000","last_undegraded":"2023-10-18T10:47:17.489199+0000","last_fullsized":"2023-10-18T10:47:17.489199+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:20:23.286839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488805+0000","last_change":"2023-10-18T10:47:15.857501+0000","last_active":"2023-10-18T10:47:17.488805+0000","last_peered":"2023-10-18T10:47:17.488805+0000","last_clean":"2023-10-18T10:47:17.488805+0000","last_became_active":"2023-10-18T10:47:15.857173+0000","last_became_peered":"2023-10-18T10:47:15.857173+0000","last_unstale":"2023-10-18T10:47:17.488805+0000","last_undegraded":"2023-10-18T10:47:17.488805+0000","last_fullsized":"2023-10-18T10:47:17.488805+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:36:02.352590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.844070+0000","last_change":"2023-10-18T10:47:15.851720+0000","last_active":"2023-10-18T10:47:16.844070+0000","last_peered":"2023-10-18T10:47:16.844070+0000","last_clean":"2023-10-18T10:47:16.844070+0000","last_became_active":"2023-10-18T10:47:15.851361+0000","last_became_peered":"2023-10-18T10:47:15.851361+0000","last_unstale":"2023-10-18T10:47:16.844070+0000","last_undegraded":"2023-10-18T10:47:16.844070+0000","last_fullsized":"2023-10-18T10:47:16.844070+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:29:48.874952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488913+0000","last_change":"2023-10-18T10:47:15.861433+0000","last_active":"2023-10-18T10:47:17.488913+0000","last_peered":"2023-10-18T10:47:17.488913+0000","last_clean":"2023-10-18T10:47:17.488913+0000","last_became_active":"2023-10-18T10:47:15.861154+0000","last_became_peered":"2023-10-18T10:47:15.861154+0000","last_unstale":"2023-10-18T10:47:17.488913+0000","last_undegraded":"2023-10-18T10:47:17.488913+0000","last_fullsized":"2023-10-18T10:47:17.488913+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:03:06.298352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488573+0000","last_change":"2023-10-18T10:47:15.858755+0000","last_active":"2023-10-18T10:47:17.488573+0000","last_peered":"2023-10-18T10:47:17.488573+0000","last_clean":"2023-10-18T10:47:17.488573+0000","last_became_active":"2023-10-18T10:47:15.858528+0000","last_became_peered":"2023-10-18T10:47:15.858528+0000","last_unstale":"2023-10-18T10:47:17.488573+0000","last_undegraded":"2023-10-18T10:47:17.488573+0000","last_fullsized":"2023-10-18T10:47:17.488573+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:26:16.315364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.841388+0000","last_change":"2023-10-18T10:47:15.840342+0000","last_active":"2023-10-18T10:47:16.841388+0000","last_peered":"2023-10-18T10:47:16.841388+0000","last_clean":"2023-10-18T10:47:16.841388+0000","last_became_active":"2023-10-18T10:47:15.840027+0000","last_became_peered":"2023-10-18T10:47:15.840027+0000","last_unstale":"2023-10-18T10:47:16.841388+0000","last_undegraded":"2023-10-18T10:47:16.841388+0000","last_fullsized":"2023-10-18T10:47:16.841388+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:43:50.664999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489035+0000","last_change":"2023-10-18T10:47:15.840188+0000","last_active":"2023-10-18T10:47:17.489035+0000","last_peered":"2023-10-18T10:47:17.489035+0000","last_clean":"2023-10-18T10:47:17.489035+0000","last_became_active":"2023-10-18T10:47:15.839890+0000","last_became_peered":"2023-10-18T10:47:15.839890+0000","last_unstale":"2023-10-18T10:47:17.489035+0000","last_undegraded":"2023-10-18T10:47:17.489035+0000","last_fullsized":"2023-10-18T10:47:17.489035+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:43:26.476206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489134+0000","last_change":"2023-10-18T10:47:15.862982+0000","last_active":"2023-10-18T10:47:17.489134+0000","last_peered":"2023-10-18T10:47:17.489134+0000","last_clean":"2023-10-18T10:47:17.489134+0000","last_became_active":"2023-10-18T10:47:15.862607+0000","last_became_peered":"2023-10-18T10:47:15.862607+0000","last_unstale":"2023-10-18T10:47:17.489134+0000","last_undegraded":"2023-10-18T10:47:17.489134+0000","last_fullsized":"2023-10-18T10:47:17.489134+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:43:50.664999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489990+0000","last_change":"2023-10-18T10:47:15.842867+0000","last_active":"2023-10-18T10:47:17.489990+0000","last_peered":"2023-10-18T10:47:17.489990+0000","last_clean":"2023-10-18T10:47:17.489990+0000","last_became_active":"2023-10-18T10:47:15.842521+0000","last_became_peered":"2023-10-18T10:47:15.842521+0000","last_unstale":"2023-10-18T10:47:17.489990+0000","last_undegraded":"2023-10-18T10:47:17.489990+0000","last_fullsized":"2023-10-18T10:47:17.489990+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:54:49.219935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489902+0000","last_change":"2023-10-18T10:47:15.861062+0000","last_active":"2023-10-18T10:47:17.489902+0000","last_peered":"2023-10-18T10:47:17.489902+0000","last_clean":"2023-10-18T10:47:17.489902+0000","last_became_active":"2023-10-18T10:47:15.860790+0000","last_became_peered":"2023-10-18T10:47:15.860790+0000","last_unstale":"2023-10-18T10:47:17.489902+0000","last_undegraded":"2023-10-18T10:47:17.489902+0000","last_fullsized":"2023-10-18T10:47:17.489902+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:36:02.352590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488976+0000","last_change":"2023-10-18T10:47:15.857520+0000","last_active":"2023-10-18T10:47:17.488976+0000","last_peered":"2023-10-18T10:47:17.488976+0000","last_clean":"2023-10-18T10:47:17.488976+0000","last_became_active":"2023-10-18T10:47:15.856115+0000","last_became_peered":"2023-10-18T10:47:15.856115+0000","last_unstale":"2023-10-18T10:47:17.488976+0000","last_undegraded":"2023-10-18T10:47:17.488976+0000","last_fullsized":"2023-10-18T10:47:17.488976+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:35:08.646421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488552+0000","last_change":"2023-10-18T10:47:15.852071+0000","last_active":"2023-10-18T10:47:17.488552+0000","last_peered":"2023-10-18T10:47:17.488552+0000","last_clean":"2023-10-18T10:47:17.488552+0000","last_became_active":"2023-10-18T10:47:15.851481+0000","last_became_peered":"2023-10-18T10:47:15.851481+0000","last_unstale":"2023-10-18T10:47:17.488552+0000","last_undegraded":"2023-10-18T10:47:17.488552+0000","last_fullsized":"2023-10-18T10:47:17.488552+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:03:46.146073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489462+0000","last_change":"2023-10-18T10:47:15.844524+0000","last_active":"2023-10-18T10:47:17.489462+0000","last_peered":"2023-10-18T10:47:17.489462+0000","last_clean":"2023-10-18T10:47:17.489462+0000","last_became_active":"2023-10-18T10:47:15.844138+0000","last_became_peered":"2023-10-18T10:47:15.844138+0000","last_unstale":"2023-10-18T10:47:17.489462+0000","last_undegraded":"2023-10-18T10:47:17.489462+0000","last_fullsized":"2023-10-18T10:47:17.489462+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:35:22.551306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489625+0000","last_change":"2023-10-18T10:47:15.844727+0000","last_active":"2023-10-18T10:47:17.489625+0000","last_peered":"2023-10-18T10:47:17.489625+0000","last_clean":"2023-10-18T10:47:17.489625+0000","last_became_active":"2023-10-18T10:47:15.842617+0000","last_became_peered":"2023-10-18T10:47:15.842617+0000","last_unstale":"2023-10-18T10:47:17.489625+0000","last_undegraded":"2023-10-18T10:47:17.489625+0000","last_fullsized":"2023-10-18T10:47:17.489625+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:18:37.376778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.841556+0000","last_change":"2023-10-18T10:47:15.835368+0000","last_active":"2023-10-18T10:47:16.841556+0000","last_peered":"2023-10-18T10:47:16.841556+0000","last_clean":"2023-10-18T10:47:16.841556+0000","last_became_active":"2023-10-18T10:47:15.835074+0000","last_became_peered":"2023-10-18T10:47:15.835074+0000","last_unstale":"2023-10-18T10:47:16.841556+0000","last_undegraded":"2023-10-18T10:47:16.841556+0000","last_fullsized":"2023-10-18T10:47:16.841556+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:43:53.526897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.843642+0000","last_change":"2023-10-18T10:47:15.861341+0000","last_active":"2023-10-18T10:47:16.843642+0000","last_peered":"2023-10-18T10:47:16.843642+0000","last_clean":"2023-10-18T10:47:16.843642+0000","last_became_active":"2023-10-18T10:47:15.861041+0000","last_became_peered":"2023-10-18T10:47:15.861041+0000","last_unstale":"2023-10-18T10:47:16.843642+0000","last_undegraded":"2023-10-18T10:47:16.843642+0000","last_fullsized":"2023-10-18T10:47:16.843642+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:02:26.920118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.844380+0000","last_change":"2023-10-18T10:47:15.841201+0000","last_active":"2023-10-18T10:47:16.844380+0000","last_peered":"2023-10-18T10:47:16.844380+0000","last_clean":"2023-10-18T10:47:16.844380+0000","last_became_active":"2023-10-18T10:47:15.840880+0000","last_became_peered":"2023-10-18T10:47:15.840880+0000","last_unstale":"2023-10-18T10:47:16.844380+0000","last_undegraded":"2023-10-18T10:47:16.844380+0000","last_fullsized":"2023-10-18T10:47:16.844380+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:21:38.088695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.843651+0000","last_change":"2023-10-18T10:47:15.858086+0000","last_active":"2023-10-18T10:47:16.843651+0000","last_peered":"2023-10-18T10:47:16.843651+0000","last_clean":"2023-10-18T10:47:16.843651+0000","last_became_active":"2023-10-18T10:47:15.857812+0000","last_became_peered":"2023-10-18T10:47:15.857812+0000","last_unstale":"2023-10-18T10:47:16.843651+0000","last_undegraded":"2023-10-18T10:47:16.843651+0000","last_fullsized":"2023-10-18T10:47:16.843651+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:40:36.982411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.844661+0000","last_change":"2023-10-18T10:47:15.858895+0000","last_active":"2023-10-18T10:47:16.844661+0000","last_peered":"2023-10-18T10:47:16.844661+0000","last_clean":"2023-10-18T10:47:16.844661+0000","last_became_active":"2023-10-18T10:47:15.858483+0000","last_became_peered":"2023-10-18T10:47:15.858483+0000","last_unstale":"2023-10-18T10:47:16.844661+0000","last_undegraded":"2023-10-18T10:47:16.844661+0000","last_fullsized":"2023-10-18T10:47:16.844661+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:14:26.143046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489648+0000","last_change":"2023-10-18T10:47:15.850040+0000","last_active":"2023-10-18T10:47:17.489648+0000","last_peered":"2023-10-18T10:47:17.489648+0000","last_clean":"2023-10-18T10:47:17.489648+0000","last_became_active":"2023-10-18T10:47:15.849723+0000","last_became_peered":"2023-10-18T10:47:15.849723+0000","last_unstale":"2023-10-18T10:47:17.489648+0000","last_undegraded":"2023-10-18T10:47:17.489648+0000","last_fullsized":"2023-10-18T10:47:17.489648+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:50.509688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.841544+0000","last_change":"2023-10-18T10:47:15.851831+0000","last_active":"2023-10-18T10:47:16.841544+0000","last_peered":"2023-10-18T10:47:16.841544+0000","last_clean":"2023-10-18T10:47:16.841544+0000","last_became_active":"2023-10-18T10:47:15.851385+0000","last_became_peered":"2023-10-18T10:47:15.851385+0000","last_unstale":"2023-10-18T10:47:16.841544+0000","last_undegraded":"2023-10-18T10:47:16.841544+0000","last_fullsized":"2023-10-18T10:47:16.841544+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:36:02.352590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488924+0000","last_change":"2023-10-18T10:47:15.861409+0000","last_active":"2023-10-18T10:47:17.488924+0000","last_peered":"2023-10-18T10:47:17.488924+0000","last_clean":"2023-10-18T10:47:17.488924+0000","last_became_active":"2023-10-18T10:47:15.861085+0000","last_became_peered":"2023-10-18T10:47:15.861085+0000","last_unstale":"2023-10-18T10:47:17.488924+0000","last_undegraded":"2023-10-18T10:47:17.488924+0000","last_fullsized":"2023-10-18T10:47:17.488924+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:13:35.452997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.844076+0000","last_change":"2023-10-18T10:47:15.848186+0000","last_active":"2023-10-18T10:47:16.844076+0000","last_peered":"2023-10-18T10:47:16.844076+0000","last_clean":"2023-10-18T10:47:16.844076+0000","last_became_active":"2023-10-18T10:47:15.847909+0000","last_became_peered":"2023-10-18T10:47:15.847909+0000","last_unstale":"2023-10-18T10:47:16.844076+0000","last_undegraded":"2023-10-18T10:47:16.844076+0000","last_fullsized":"2023-10-18T10:47:16.844076+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:45:39.396012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.843785+0000","last_change":"2023-10-18T10:47:15.853844+0000","last_active":"2023-10-18T10:47:16.843785+0000","last_peered":"2023-10-18T10:47:16.843785+0000","last_clean":"2023-10-18T10:47:16.843785+0000","last_became_active":"2023-10-18T10:47:15.853505+0000","last_became_peered":"2023-10-18T10:47:15.853505+0000","last_unstale":"2023-10-18T10:47:16.843785+0000","last_undegraded":"2023-10-18T10:47:16.843785+0000","last_fullsized":"2023-10-18T10:47:16.843785+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:38:22.856976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.844805+0000","last_change":"2023-10-18T10:47:15.845600+0000","last_active":"2023-10-18T10:47:16.844805+0000","last_peered":"2023-10-18T10:47:16.844805+0000","last_clean":"2023-10-18T10:47:16.844805+0000","last_became_active":"2023-10-18T10:47:15.845399+0000","last_became_peered":"2023-10-18T10:47:15.845399+0000","last_unstale":"2023-10-18T10:47:16.844805+0000","last_undegraded":"2023-10-18T10:47:16.844805+0000","last_fullsized":"2023-10-18T10:47:16.844805+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:59:39.963888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489770+0000","last_change":"2023-10-18T10:47:15.848901+0000","last_active":"2023-10-18T10:47:17.489770+0000","last_peered":"2023-10-18T10:47:17.489770+0000","last_clean":"2023-10-18T10:47:17.489770+0000","last_became_active":"2023-10-18T10:47:15.848221+0000","last_became_peered":"2023-10-18T10:47:15.848221+0000","last_unstale":"2023-10-18T10:47:17.489770+0000","last_undegraded":"2023-10-18T10:47:17.489770+0000","last_fullsized":"2023-10-18T10:47:17.489770+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:53:50.604882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.844207+0000","last_change":"2023-10-18T10:47:15.844213+0000","last_active":"2023-10-18T10:47:16.844207+0000","last_peered":"2023-10-18T10:47:16.844207+0000","last_clean":"2023-10-18T10:47:16.844207+0000","last_became_active":"2023-10-18T10:47:15.843321+0000","last_became_peered":"2023-10-18T10:47:15.843321+0000","last_unstale":"2023-10-18T10:47:16.844207+0000","last_undegraded":"2023-10-18T10:47:16.844207+0000","last_fullsized":"2023-10-18T10:47:16.844207+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:26.573258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489419+0000","last_change":"2023-10-18T10:47:15.851877+0000","last_active":"2023-10-18T10:47:17.489419+0000","last_peered":"2023-10-18T10:47:17.489419+0000","last_clean":"2023-10-18T10:47:17.489419+0000","last_became_active":"2023-10-18T10:47:15.851046+0000","last_became_peered":"2023-10-18T10:47:15.851046+0000","last_unstale":"2023-10-18T10:47:17.489419+0000","last_undegraded":"2023-10-18T10:47:17.489419+0000","last_fullsized":"2023-10-18T10:47:17.489419+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:54:49.219935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.843903+0000","last_change":"2023-10-18T10:47:15.851823+0000","last_active":"2023-10-18T10:47:16.843903+0000","last_peered":"2023-10-18T10:47:16.843903+0000","last_clean":"2023-10-18T10:47:16.843903+0000","last_became_active":"2023-10-18T10:47:15.851513+0000","last_became_peered":"2023-10-18T10:47:15.851513+0000","last_unstale":"2023-10-18T10:47:16.843903+0000","last_undegraded":"2023-10-18T10:47:16.843903+0000","last_fullsized":"2023-10-18T10:47:16.843903+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:11:02.781422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.843495+0000","last_change":"2023-10-18T10:47:15.847560+0000","last_active":"2023-10-18T10:47:16.843495+0000","last_peered":"2023-10-18T10:47:16.843495+0000","last_clean":"2023-10-18T10:47:16.843495+0000","last_became_active":"2023-10-18T10:47:15.847241+0000","last_became_peered":"2023-10-18T10:47:15.847241+0000","last_unstale":"2023-10-18T10:47:16.843495+0000","last_undegraded":"2023-10-18T10:47:16.843495+0000","last_fullsized":"2023-10-18T10:47:16.843495+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:01:02.374958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.841407+0000","last_change":"2023-10-18T10:47:15.844563+0000","last_active":"2023-10-18T10:47:16.841407+0000","last_peered":"2023-10-18T10:47:16.841407+0000","last_clean":"2023-10-18T10:47:16.841407+0000","last_became_active":"2023-10-18T10:47:15.844300+0000","last_became_peered":"2023-10-18T10:47:15.844300+0000","last_unstale":"2023-10-18T10:47:16.841407+0000","last_undegraded":"2023-10-18T10:47:16.841407+0000","last_fullsized":"2023-10-18T10:47:16.841407+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:03:46.146073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488164+0000","last_change":"2023-10-18T10:47:15.841484+0000","last_active":"2023-10-18T10:47:17.488164+0000","last_peered":"2023-10-18T10:47:17.488164+0000","last_clean":"2023-10-18T10:47:17.488164+0000","last_became_active":"2023-10-18T10:47:15.841110+0000","last_became_peered":"2023-10-18T10:47:15.841110+0000","last_unstale":"2023-10-18T10:47:17.488164+0000","last_undegraded":"2023-10-18T10:47:17.488164+0000","last_fullsized":"2023-10-18T10:47:17.488164+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:58:49.080873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489208+0000","last_change":"2023-10-18T10:47:15.859551+0000","last_active":"2023-10-18T10:47:17.489208+0000","last_peered":"2023-10-18T10:47:17.489208+0000","last_clean":"2023-10-18T10:47:17.489208+0000","last_became_active":"2023-10-18T10:47:15.859273+0000","last_became_peered":"2023-10-18T10:47:15.859273+0000","last_unstale":"2023-10-18T10:47:17.489208+0000","last_undegraded":"2023-10-18T10:47:17.489208+0000","last_fullsized":"2023-10-18T10:47:17.489208+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:57:18.382770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"40'4","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.842253+0000","last_change":"2023-10-18T10:47:14.832823+0000","last_active":"2023-10-18T10:47:16.842253+0000","last_peered":"2023-10-18T10:47:16.842253+0000","last_clean":"2023-10-18T10:47:16.842253+0000","last_became_active":"2023-10-18T10:47:14.832542+0000","last_became_peered":"2023-10-18T10:47:14.832542+0000","last_unstale":"2023-10-18T10:47:16.842253+0000","last_undegraded":"2023-10-18T10:47:16.842253+0000","last_fullsized":"2023-10-18T10:47:16.842253+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:21:09.552522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.844960+0000","last_change":"2023-10-18T10:47:15.846403+0000","last_active":"2023-10-18T10:47:16.844960+0000","last_peered":"2023-10-18T10:47:16.844960+0000","last_clean":"2023-10-18T10:47:16.844960+0000","last_became_active":"2023-10-18T10:47:15.845794+0000","last_became_peered":"2023-10-18T10:47:15.845794+0000","last_unstale":"2023-10-18T10:47:16.844960+0000","last_undegraded":"2023-10-18T10:47:16.844960+0000","last_fullsized":"2023-10-18T10:47:16.844960+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:49:15.746889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"40'1","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.843931+0000","last_change":"2023-10-18T10:47:14.839287+0000","last_active":"2023-10-18T10:47:16.843931+0000","last_peered":"2023-10-18T10:47:16.843931+0000","last_clean":"2023-10-18T10:47:16.843931+0000","last_became_active":"2023-10-18T10:47:14.838994+0000","last_became_peered":"2023-10-18T10:47:14.838994+0000","last_unstale":"2023-10-18T10:47:16.843931+0000","last_undegraded":"2023-10-18T10:47:16.843931+0000","last_fullsized":"2023-10-18T10:47:16.843931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:19:27.257823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488696+0000","last_change":"2023-10-18T10:47:15.857607+0000","last_active":"2023-10-18T10:47:17.488696+0000","last_peered":"2023-10-18T10:47:17.488696+0000","last_clean":"2023-10-18T10:47:17.488696+0000","last_became_active":"2023-10-18T10:47:15.854475+0000","last_became_peered":"2023-10-18T10:47:15.854475+0000","last_unstale":"2023-10-18T10:47:17.488696+0000","last_undegraded":"2023-10-18T10:47:17.488696+0000","last_fullsized":"2023-10-18T10:47:17.488696+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:33:49.953156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"40'2","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489206+0000","last_change":"2023-10-18T10:47:14.847383+0000","last_active":"2023-10-18T10:47:17.489206+0000","last_peered":"2023-10-18T10:47:17.489206+0000","last_clean":"2023-10-18T10:47:17.489206+0000","last_became_active":"2023-10-18T10:47:14.847052+0000","last_became_peered":"2023-10-18T10:47:14.847052+0000","last_unstale":"2023-10-18T10:47:17.489206+0000","last_undegraded":"2023-10-18T10:47:17.489206+0000","last_fullsized":"2023-10-18T10:47:17.489206+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:23:25.645520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489056+0000","last_change":"2023-10-18T10:47:15.851519+0000","last_active":"2023-10-18T10:47:17.489056+0000","last_peered":"2023-10-18T10:47:17.489056+0000","last_clean":"2023-10-18T10:47:17.489056+0000","last_became_active":"2023-10-18T10:47:15.850963+0000","last_became_peered":"2023-10-18T10:47:15.850963+0000","last_unstale":"2023-10-18T10:47:17.489056+0000","last_undegraded":"2023-10-18T10:47:17.489056+0000","last_fullsized":"2023-10-18T10:47:17.489056+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:02:06.039754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"40'2","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489149+0000","last_change":"2023-10-18T10:47:14.850426+0000","last_active":"2023-10-18T10:47:17.489149+0000","last_peered":"2023-10-18T10:47:17.489149+0000","last_clean":"2023-10-18T10:47:17.489149+0000","last_became_active":"2023-10-18T10:47:14.849654+0000","last_became_peered":"2023-10-18T10:47:14.849654+0000","last_unstale":"2023-10-18T10:47:17.489149+0000","last_undegraded":"2023-10-18T10:47:17.489149+0000","last_fullsized":"2023-10-18T10:47:17.489149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:19:09.958118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.843655+0000","last_change":"2023-10-18T10:47:15.852762+0000","last_active":"2023-10-18T10:47:16.843655+0000","last_peered":"2023-10-18T10:47:16.843655+0000","last_clean":"2023-10-18T10:47:16.843655+0000","last_became_active":"2023-10-18T10:47:15.852460+0000","last_became_peered":"2023-10-18T10:47:15.852460+0000","last_unstale":"2023-10-18T10:47:16.843655+0000","last_undegraded":"2023-10-18T10:47:16.843655+0000","last_fullsized":"2023-10-18T10:47:16.843655+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:10:43.821057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"40'1","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.841656+0000","last_change":"2023-10-18T10:47:14.834346+0000","last_active":"2023-10-18T10:47:16.841656+0000","last_peered":"2023-10-18T10:47:16.841656+0000","last_clean":"2023-10-18T10:47:16.841656+0000","last_became_active":"2023-10-18T10:47:14.834098+0000","last_became_peered":"2023-10-18T10:47:14.834098+0000","last_unstale":"2023-10-18T10:47:16.841656+0000","last_undegraded":"2023-10-18T10:47:16.841656+0000","last_fullsized":"2023-10-18T10:47:16.841656+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:54:48.213290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488328+0000","last_change":"2023-10-18T10:47:15.852962+0000","last_active":"2023-10-18T10:47:17.488328+0000","last_peered":"2023-10-18T10:47:17.488328+0000","last_clean":"2023-10-18T10:47:17.488328+0000","last_became_active":"2023-10-18T10:47:15.852632+0000","last_became_peered":"2023-10-18T10:47:15.852632+0000","last_unstale":"2023-10-18T10:47:17.488328+0000","last_undegraded":"2023-10-18T10:47:17.488328+0000","last_fullsized":"2023-10-18T10:47:17.488328+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:00:02.907322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"40'1","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.843789+0000","last_change":"2023-10-18T10:47:14.846897+0000","last_active":"2023-10-18T10:47:16.843789+0000","last_peered":"2023-10-18T10:47:16.843789+0000","last_clean":"2023-10-18T10:47:16.843789+0000","last_became_active":"2023-10-18T10:47:14.846628+0000","last_became_peered":"2023-10-18T10:47:14.846628+0000","last_unstale":"2023-10-18T10:47:16.843789+0000","last_undegraded":"2023-10-18T10:47:16.843789+0000","last_fullsized":"2023-10-18T10:47:16.843789+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:26:38.444860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489121+0000","last_change":"2023-10-18T10:47:15.854526+0000","last_active":"2023-10-18T10:47:17.489121+0000","last_peered":"2023-10-18T10:47:17.489121+0000","last_clean":"2023-10-18T10:47:17.489121+0000","last_became_active":"2023-10-18T10:47:15.854257+0000","last_became_peered":"2023-10-18T10:47:15.854257+0000","last_unstale":"2023-10-18T10:47:17.489121+0000","last_undegraded":"2023-10-18T10:47:17.489121+0000","last_fullsized":"2023-10-18T10:47:17.489121+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:09:50.729532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488046+0000","last_change":"2023-10-18T10:47:14.848143+0000","last_active":"2023-10-18T10:47:17.488046+0000","last_peered":"2023-10-18T10:47:17.488046+0000","last_clean":"2023-10-18T10:47:17.488046+0000","last_became_active":"2023-10-18T10:47:14.847834+0000","last_became_peered":"2023-10-18T10:47:14.847834+0000","last_unstale":"2023-10-18T10:47:17.488046+0000","last_undegraded":"2023-10-18T10:47:17.488046+0000","last_fullsized":"2023-10-18T10:47:17.488046+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:35:07.639776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.843880+0000","last_change":"2023-10-18T10:47:15.848544+0000","last_active":"2023-10-18T10:47:16.843880+0000","last_peered":"2023-10-18T10:47:16.843880+0000","last_clean":"2023-10-18T10:47:16.843880+0000","last_became_active":"2023-10-18T10:47:15.848162+0000","last_became_peered":"2023-10-18T10:47:15.848162+0000","last_unstale":"2023-10-18T10:47:16.843880+0000","last_undegraded":"2023-10-18T10:47:16.843880+0000","last_fullsized":"2023-10-18T10:47:16.843880+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:32:48.986074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489241+0000","last_change":"2023-10-18T10:47:14.842407+0000","last_active":"2023-10-18T10:47:17.489241+0000","last_peered":"2023-10-18T10:47:17.489241+0000","last_clean":"2023-10-18T10:47:17.489241+0000","last_became_active":"2023-10-18T10:47:14.842152+0000","last_became_peered":"2023-10-18T10:47:14.842152+0000","last_unstale":"2023-10-18T10:47:17.489241+0000","last_undegraded":"2023-10-18T10:47:17.489241+0000","last_fullsized":"2023-10-18T10:47:17.489241+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:03:45.139428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.843793+0000","last_change":"2023-10-18T10:47:15.857610+0000","last_active":"2023-10-18T10:47:16.843793+0000","last_peered":"2023-10-18T10:47:16.843793+0000","last_clean":"2023-10-18T10:47:16.843793+0000","last_became_active":"2023-10-18T10:47:15.857308+0000","last_became_peered":"2023-10-18T10:47:15.857308+0000","last_unstale":"2023-10-18T10:47:16.843793+0000","last_undegraded":"2023-10-18T10:47:16.843793+0000","last_fullsized":"2023-10-18T10:47:16.843793+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:28:18.012270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.844519+0000","last_change":"2023-10-18T10:47:14.839653+0000","last_active":"2023-10-18T10:47:16.844519+0000","last_peered":"2023-10-18T10:47:16.844519+0000","last_clean":"2023-10-18T10:47:16.844519+0000","last_became_active":"2023-10-18T10:47:14.838492+0000","last_became_peered":"2023-10-18T10:47:14.838492+0000","last_unstale":"2023-10-18T10:47:16.844519+0000","last_undegraded":"2023-10-18T10:47:16.844519+0000","last_fullsized":"2023-10-18T10:47:16.844519+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:45:04.943284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489235+0000","last_change":"2023-10-18T10:47:15.843928+0000","last_active":"2023-10-18T10:47:17.489235+0000","last_peered":"2023-10-18T10:47:17.489235+0000","last_clean":"2023-10-18T10:47:17.489235+0000","last_became_active":"2023-10-18T10:47:15.842875+0000","last_became_peered":"2023-10-18T10:47:15.842875+0000","last_unstale":"2023-10-18T10:47:17.489235+0000","last_undegraded":"2023-10-18T10:47:17.489235+0000","last_fullsized":"2023-10-18T10:47:17.489235+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:03.621992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.490020+0000","last_change":"2023-10-18T10:47:14.844999+0000","last_active":"2023-10-18T10:47:17.490020+0000","last_peered":"2023-10-18T10:47:17.490020+0000","last_clean":"2023-10-18T10:47:17.490020+0000","last_became_active":"2023-10-18T10:47:14.844746+0000","last_became_peered":"2023-10-18T10:47:14.844746+0000","last_unstale":"2023-10-18T10:47:17.490020+0000","last_undegraded":"2023-10-18T10:47:17.490020+0000","last_fullsized":"2023-10-18T10:47:17.490020+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:33:17.479207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.841938+0000","last_change":"2023-10-18T10:47:15.859400+0000","last_active":"2023-10-18T10:47:16.841938+0000","last_peered":"2023-10-18T10:47:16.841938+0000","last_clean":"2023-10-18T10:47:16.841938+0000","last_became_active":"2023-10-18T10:47:15.859044+0000","last_became_peered":"2023-10-18T10:47:15.859044+0000","last_unstale":"2023-10-18T10:47:16.841938+0000","last_undegraded":"2023-10-18T10:47:16.841938+0000","last_fullsized":"2023-10-18T10:47:16.841938+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:00:28.228199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"40'3","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489251+0000","last_change":"2023-10-18T10:47:14.845856+0000","last_active":"2023-10-18T10:47:17.489251+0000","last_peered":"2023-10-18T10:47:17.489251+0000","last_clean":"2023-10-18T10:47:17.489251+0000","last_became_active":"2023-10-18T10:47:14.845575+0000","last_became_peered":"2023-10-18T10:47:14.845575+0000","last_unstale":"2023-10-18T10:47:17.489251+0000","last_undegraded":"2023-10-18T10:47:17.489251+0000","last_fullsized":"2023-10-18T10:47:17.489251+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:33:10.378488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488845+0000","last_change":"2023-10-18T10:47:15.843972+0000","last_active":"2023-10-18T10:47:17.488845+0000","last_peered":"2023-10-18T10:47:17.488845+0000","last_clean":"2023-10-18T10:47:17.488845+0000","last_became_active":"2023-10-18T10:47:15.843649+0000","last_became_peered":"2023-10-18T10:47:15.843649+0000","last_unstale":"2023-10-18T10:47:17.488845+0000","last_undegraded":"2023-10-18T10:47:17.488845+0000","last_fullsized":"2023-10-18T10:47:17.488845+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:32:29.921799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"40'6","reported_seq":19,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489745+0000","last_change":"2023-10-18T10:47:14.840158+0000","last_active":"2023-10-18T10:47:17.489745+0000","last_peered":"2023-10-18T10:47:17.489745+0000","last_clean":"2023-10-18T10:47:17.489745+0000","last_became_active":"2023-10-18T10:47:14.839881+0000","last_became_peered":"2023-10-18T10:47:14.839881+0000","last_unstale":"2023-10-18T10:47:17.489745+0000","last_undegraded":"2023-10-18T10:47:17.489745+0000","last_fullsized":"2023-10-18T10:47:17.489745+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:38:22.391824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.844871+0000","last_change":"2023-10-18T10:47:15.861260+0000","last_active":"2023-10-18T10:47:16.844871+0000","last_peered":"2023-10-18T10:47:16.844871+0000","last_clean":"2023-10-18T10:47:16.844871+0000","last_became_active":"2023-10-18T10:47:15.860958+0000","last_became_peered":"2023-10-18T10:47:15.860958+0000","last_unstale":"2023-10-18T10:47:16.844871+0000","last_undegraded":"2023-10-18T10:47:16.844871+0000","last_fullsized":"2023-10-18T10:47:16.844871+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:38:24.521066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"40'2","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.841721+0000","last_change":"2023-10-18T10:47:14.840478+0000","last_active":"2023-10-18T10:47:16.841721+0000","last_peered":"2023-10-18T10:47:16.841721+0000","last_clean":"2023-10-18T10:47:16.841721+0000","last_became_active":"2023-10-18T10:47:14.839906+0000","last_became_peered":"2023-10-18T10:47:14.839906+0000","last_unstale":"2023-10-18T10:47:16.841721+0000","last_undegraded":"2023-10-18T10:47:16.841721+0000","last_fullsized":"2023-10-18T10:47:16.841721+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:23:49.688625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488388+0000","last_change":"2023-10-18T10:47:15.850243+0000","last_active":"2023-10-18T10:47:17.488388+0000","last_peered":"2023-10-18T10:47:17.488388+0000","last_clean":"2023-10-18T10:47:17.488388+0000","last_became_active":"2023-10-18T10:47:15.849925+0000","last_became_peered":"2023-10-18T10:47:15.849925+0000","last_unstale":"2023-10-18T10:47:17.488388+0000","last_undegraded":"2023-10-18T10:47:17.488388+0000","last_fullsized":"2023-10-18T10:47:17.488388+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:28:54.785138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489170+0000","last_change":"2023-10-18T10:47:14.841945+0000","last_active":"2023-10-18T10:47:17.489170+0000","last_peered":"2023-10-18T10:47:17.489170+0000","last_clean":"2023-10-18T10:47:17.489170+0000","last_became_active":"2023-10-18T10:47:14.841614+0000","last_became_peered":"2023-10-18T10:47:14.841614+0000","last_unstale":"2023-10-18T10:47:17.489170+0000","last_undegraded":"2023-10-18T10:47:17.489170+0000","last_fullsized":"2023-10-18T10:47:17.489170+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:34:54.832263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.841320+0000","last_change":"2023-10-18T10:47:15.852810+0000","last_active":"2023-10-18T10:47:16.841320+0000","last_peered":"2023-10-18T10:47:16.841320+0000","last_clean":"2023-10-18T10:47:16.841320+0000","last_became_active":"2023-10-18T10:47:15.852509+0000","last_became_peered":"2023-10-18T10:47:15.852509+0000","last_unstale":"2023-10-18T10:47:16.841320+0000","last_undegraded":"2023-10-18T10:47:16.841320+0000","last_fullsized":"2023-10-18T10:47:16.841320+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:09:50.729532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.841518+0000","last_change":"2023-10-18T10:47:14.845377+0000","last_active":"2023-10-18T10:47:16.841518+0000","last_peered":"2023-10-18T10:47:16.841518+0000","last_clean":"2023-10-18T10:47:16.841518+0000","last_became_active":"2023-10-18T10:47:14.845102+0000","last_became_peered":"2023-10-18T10:47:14.845102+0000","last_unstale":"2023-10-18T10:47:16.841518+0000","last_undegraded":"2023-10-18T10:47:16.841518+0000","last_fullsized":"2023-10-18T10:47:16.841518+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:18:36.370134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489451+0000","last_change":"2023-10-18T10:47:15.849181+0000","last_active":"2023-10-18T10:47:17.489451+0000","last_peered":"2023-10-18T10:47:17.489451+0000","last_clean":"2023-10-18T10:47:17.489451+0000","last_became_active":"2023-10-18T10:47:15.848694+0000","last_became_peered":"2023-10-18T10:47:15.848694+0000","last_unstale":"2023-10-18T10:47:17.489451+0000","last_undegraded":"2023-10-18T10:47:17.489451+0000","last_fullsized":"2023-10-18T10:47:17.489451+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:04:44.835305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"40'8","reported_seq":21,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489254+0000","last_change":"2023-10-18T10:47:14.850050+0000","last_active":"2023-10-18T10:47:17.489254+0000","last_peered":"2023-10-18T10:47:17.489254+0000","last_clean":"2023-10-18T10:47:17.489254+0000","last_became_active":"2023-10-18T10:47:14.849205+0000","last_became_peered":"2023-10-18T10:47:14.849205+0000","last_unstale":"2023-10-18T10:47:17.489254+0000","last_undegraded":"2023-10-18T10:47:17.489254+0000","last_fullsized":"2023-10-18T10:47:17.489254+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:26:15.308720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489838+0000","last_change":"2023-10-18T10:47:15.849087+0000","last_active":"2023-10-18T10:47:17.489838+0000","last_peered":"2023-10-18T10:47:17.489838+0000","last_clean":"2023-10-18T10:47:17.489838+0000","last_became_active":"2023-10-18T10:47:15.848606+0000","last_became_peered":"2023-10-18T10:47:15.848606+0000","last_unstale":"2023-10-18T10:47:17.489838+0000","last_undegraded":"2023-10-18T10:47:17.489838+0000","last_fullsized":"2023-10-18T10:47:17.489838+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:20:24.951086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"40'1","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489591+0000","last_change":"2023-10-18T10:47:14.842314+0000","last_active":"2023-10-18T10:47:17.489591+0000","last_peered":"2023-10-18T10:47:17.489591+0000","last_clean":"2023-10-18T10:47:17.489591+0000","last_became_active":"2023-10-18T10:47:14.841832+0000","last_became_peered":"2023-10-18T10:47:14.841832+0000","last_unstale":"2023-10-18T10:47:17.489591+0000","last_undegraded":"2023-10-18T10:47:17.489591+0000","last_fullsized":"2023-10-18T10:47:17.489591+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:28:31.383054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.490156+0000","last_change":"2023-10-18T10:47:15.866629+0000","last_active":"2023-10-18T10:47:17.490156+0000","last_peered":"2023-10-18T10:47:17.490156+0000","last_clean":"2023-10-18T10:47:17.490156+0000","last_became_active":"2023-10-18T10:47:15.866353+0000","last_became_peered":"2023-10-18T10:47:15.866353+0000","last_unstale":"2023-10-18T10:47:17.490156+0000","last_undegraded":"2023-10-18T10:47:17.490156+0000","last_fullsized":"2023-10-18T10:47:17.490156+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:23:50.695270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"40'2","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.844067+0000","last_change":"2023-10-18T10:47:14.843544+0000","last_active":"2023-10-18T10:47:16.844067+0000","last_peered":"2023-10-18T10:47:16.844067+0000","last_clean":"2023-10-18T10:47:16.844067+0000","last_became_active":"2023-10-18T10:47:14.843095+0000","last_became_peered":"2023-10-18T10:47:14.843095+0000","last_unstale":"2023-10-18T10:47:16.844067+0000","last_undegraded":"2023-10-18T10:47:16.844067+0000","last_fullsized":"2023-10-18T10:47:16.844067+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:04:07.055793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.844032+0000","last_change":"2023-10-18T10:47:15.853760+0000","last_active":"2023-10-18T10:47:16.844032+0000","last_peered":"2023-10-18T10:47:16.844032+0000","last_clean":"2023-10-18T10:47:16.844032+0000","last_became_active":"2023-10-18T10:47:15.853466+0000","last_became_peered":"2023-10-18T10:47:15.853466+0000","last_unstale":"2023-10-18T10:47:16.844032+0000","last_undegraded":"2023-10-18T10:47:16.844032+0000","last_fullsized":"2023-10-18T10:47:16.844032+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:37:35.177266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"40'1","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489207+0000","last_change":"2023-10-18T10:47:14.847139+0000","last_active":"2023-10-18T10:47:17.489207+0000","last_peered":"2023-10-18T10:47:17.489207+0000","last_clean":"2023-10-18T10:47:17.489207+0000","last_became_active":"2023-10-18T10:47:14.846845+0000","last_became_peered":"2023-10-18T10:47:14.846845+0000","last_unstale":"2023-10-18T10:47:17.489207+0000","last_undegraded":"2023-10-18T10:47:17.489207+0000","last_fullsized":"2023-10-18T10:47:17.489207+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:43:52.520253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.844454+0000","last_change":"2023-10-18T10:47:15.845736+0000","last_active":"2023-10-18T10:47:16.844454+0000","last_peered":"2023-10-18T10:47:16.844454+0000","last_clean":"2023-10-18T10:47:16.844454+0000","last_became_active":"2023-10-18T10:47:15.844036+0000","last_became_peered":"2023-10-18T10:47:15.844036+0000","last_unstale":"2023-10-18T10:47:16.844454+0000","last_undegraded":"2023-10-18T10:47:16.844454+0000","last_fullsized":"2023-10-18T10:47:16.844454+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:44:06.107284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.841959+0000","last_change":"2023-10-18T10:47:14.833953+0000","last_active":"2023-10-18T10:47:16.841959+0000","last_peered":"2023-10-18T10:47:16.841959+0000","last_clean":"2023-10-18T10:47:16.841959+0000","last_became_active":"2023-10-18T10:47:14.833512+0000","last_became_peered":"2023-10-18T10:47:14.833512+0000","last_unstale":"2023-10-18T10:47:16.841959+0000","last_undegraded":"2023-10-18T10:47:16.841959+0000","last_fullsized":"2023-10-18T10:47:16.841959+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:24:47.792031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.841466+0000","last_change":"2023-10-18T10:47:15.858560+0000","last_active":"2023-10-18T10:47:16.841466+0000","last_peered":"2023-10-18T10:47:16.841466+0000","last_clean":"2023-10-18T10:47:16.841466+0000","last_became_active":"2023-10-18T10:47:15.858325+0000","last_became_peered":"2023-10-18T10:47:15.858325+0000","last_unstale":"2023-10-18T10:47:16.841466+0000","last_undegraded":"2023-10-18T10:47:16.841466+0000","last_fullsized":"2023-10-18T10:47:16.841466+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:26:16.315364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"40'2","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.841818+0000","last_change":"2023-10-18T10:47:14.834264+0000","last_active":"2023-10-18T10:47:16.841818+0000","last_peered":"2023-10-18T10:47:16.841818+0000","last_clean":"2023-10-18T10:47:16.841818+0000","last_became_active":"2023-10-18T10:47:14.833996+0000","last_became_peered":"2023-10-18T10:47:14.833996+0000","last_unstale":"2023-10-18T10:47:16.841818+0000","last_undegraded":"2023-10-18T10:47:16.841818+0000","last_fullsized":"2023-10-18T10:47:16.841818+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:15:55.232241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489347+0000","last_change":"2023-10-18T10:47:15.845011+0000","last_active":"2023-10-18T10:47:17.489347+0000","last_peered":"2023-10-18T10:47:17.489347+0000","last_clean":"2023-10-18T10:47:17.489347+0000","last_became_active":"2023-10-18T10:47:15.844475+0000","last_became_peered":"2023-10-18T10:47:15.844475+0000","last_unstale":"2023-10-18T10:47:17.489347+0000","last_undegraded":"2023-10-18T10:47:17.489347+0000","last_fullsized":"2023-10-18T10:47:17.489347+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:03:40.394860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488360+0000","last_change":"2023-10-18T10:47:14.848894+0000","last_active":"2023-10-18T10:47:17.488360+0000","last_peered":"2023-10-18T10:47:17.488360+0000","last_clean":"2023-10-18T10:47:17.488360+0000","last_became_active":"2023-10-18T10:47:14.848556+0000","last_became_peered":"2023-10-18T10:47:14.848556+0000","last_unstale":"2023-10-18T10:47:17.488360+0000","last_undegraded":"2023-10-18T10:47:17.488360+0000","last_fullsized":"2023-10-18T10:47:17.488360+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:00:46.711890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488321+0000","last_change":"2023-10-18T10:47:15.848304+0000","last_active":"2023-10-18T10:47:17.488321+0000","last_peered":"2023-10-18T10:47:17.488321+0000","last_clean":"2023-10-18T10:47:17.488321+0000","last_became_active":"2023-10-18T10:47:15.847980+0000","last_became_peered":"2023-10-18T10:47:15.847980+0000","last_unstale":"2023-10-18T10:47:17.488321+0000","last_undegraded":"2023-10-18T10:47:17.488321+0000","last_fullsized":"2023-10-18T10:47:17.488321+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:07:39.705667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488534+0000","last_change":"2023-10-18T10:47:14.838905+0000","last_active":"2023-10-18T10:47:17.488534+0000","last_peered":"2023-10-18T10:47:17.488534+0000","last_clean":"2023-10-18T10:47:17.488534+0000","last_became_active":"2023-10-18T10:47:14.838537+0000","last_became_peered":"2023-10-18T10:47:14.838537+0000","last_unstale":"2023-10-18T10:47:17.488534+0000","last_undegraded":"2023-10-18T10:47:17.488534+0000","last_fullsized":"2023-10-18T10:47:17.488534+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:16:34.301296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488332+0000","last_change":"2023-10-18T10:47:15.853190+0000","last_active":"2023-10-18T10:47:17.488332+0000","last_peered":"2023-10-18T10:47:17.488332+0000","last_clean":"2023-10-18T10:47:17.488332+0000","last_became_active":"2023-10-18T10:47:15.852895+0000","last_became_peered":"2023-10-18T10:47:15.852895+0000","last_unstale":"2023-10-18T10:47:17.488332+0000","last_undegraded":"2023-10-18T10:47:17.488332+0000","last_fullsized":"2023-10-18T10:47:17.488332+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:29:24.304875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488188+0000","last_change":"2023-10-18T10:47:14.841944+0000","last_active":"2023-10-18T10:47:17.488188+0000","last_peered":"2023-10-18T10:47:17.488188+0000","last_clean":"2023-10-18T10:47:17.488188+0000","last_became_active":"2023-10-18T10:47:14.841623+0000","last_became_peered":"2023-10-18T10:47:14.841623+0000","last_unstale":"2023-10-18T10:47:17.488188+0000","last_undegraded":"2023-10-18T10:47:17.488188+0000","last_fullsized":"2023-10-18T10:47:17.488188+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:00:27.221555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.841493+0000","last_change":"2023-10-18T10:47:15.841976+0000","last_active":"2023-10-18T10:47:16.841493+0000","last_peered":"2023-10-18T10:47:16.841493+0000","last_clean":"2023-10-18T10:47:16.841493+0000","last_became_active":"2023-10-18T10:47:15.841713+0000","last_became_peered":"2023-10-18T10:47:15.841713+0000","last_unstale":"2023-10-18T10:47:16.841493+0000","last_undegraded":"2023-10-18T10:47:16.841493+0000","last_fullsized":"2023-10-18T10:47:16.841493+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:33:49.953156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488920+0000","last_change":"2023-10-18T10:47:14.847120+0000","last_active":"2023-10-18T10:47:17.488920+0000","last_peered":"2023-10-18T10:47:17.488920+0000","last_clean":"2023-10-18T10:47:17.488920+0000","last_became_active":"2023-10-18T10:47:14.846008+0000","last_became_peered":"2023-10-18T10:47:14.846008+0000","last_unstale":"2023-10-18T10:47:17.488920+0000","last_undegraded":"2023-10-18T10:47:17.488920+0000","last_fullsized":"2023-10-18T10:47:17.488920+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:21:09.552522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.845037+0000","last_change":"2023-10-18T10:45:50.252021+0000","last_active":"2023-10-18T10:47:16.845037+0000","last_peered":"2023-10-18T10:47:16.845037+0000","last_clean":"2023-10-18T10:47:16.845037+0000","last_became_active":"2023-10-18T10:45:50.251427+0000","last_became_peered":"2023-10-18T10:45:50.251427+0000","last_unstale":"2023-10-18T10:47:16.845037+0000","last_undegraded":"2023-10-18T10:47:16.845037+0000","last_fullsized":"2023-10-18T10:47:16.845037+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:57:23.835848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.841677+0000","last_change":"2023-10-18T10:47:15.859453+0000","last_active":"2023-10-18T10:47:16.841677+0000","last_peered":"2023-10-18T10:47:16.841677+0000","last_clean":"2023-10-18T10:47:16.841677+0000","last_became_active":"2023-10-18T10:47:15.858668+0000","last_became_peered":"2023-10-18T10:47:15.858668+0000","last_unstale":"2023-10-18T10:47:16.841677+0000","last_undegraded":"2023-10-18T10:47:16.841677+0000","last_fullsized":"2023-10-18T10:47:16.841677+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:33:18.485852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.843909+0000","last_change":"2023-10-18T10:47:14.841091+0000","last_active":"2023-10-18T10:47:16.843909+0000","last_peered":"2023-10-18T10:47:16.843909+0000","last_clean":"2023-10-18T10:47:16.843909+0000","last_became_active":"2023-10-18T10:47:14.840807+0000","last_became_peered":"2023-10-18T10:47:14.840807+0000","last_unstale":"2023-10-18T10:47:16.843909+0000","last_undegraded":"2023-10-18T10:47:16.843909+0000","last_fullsized":"2023-10-18T10:47:16.843909+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:43:51.292223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":47,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489287+0000","last_change":"2023-10-18T10:45:50.252876+0000","last_active":"2023-10-18T10:47:17.489287+0000","last_peered":"2023-10-18T10:47:17.489287+0000","last_clean":"2023-10-18T10:47:17.489287+0000","last_became_active":"2023-10-18T10:45:50.252559+0000","last_became_peered":"2023-10-18T10:45:50.252559+0000","last_unstale":"2023-10-18T10:47:17.489287+0000","last_undegraded":"2023-10-18T10:47:17.489287+0000","last_fullsized":"2023-10-18T10:47:17.489287+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:43:05.360433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488017+0000","last_change":"2023-10-18T10:47:15.859101+0000","last_active":"2023-10-18T10:47:17.488017+0000","last_peered":"2023-10-18T10:47:17.488017+0000","last_clean":"2023-10-18T10:47:17.488017+0000","last_became_active":"2023-10-18T10:47:15.857002+0000","last_became_peered":"2023-10-18T10:47:15.857002+0000","last_unstale":"2023-10-18T10:47:17.488017+0000","last_undegraded":"2023-10-18T10:47:17.488017+0000","last_fullsized":"2023-10-18T10:47:17.488017+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:13:35.452997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488307+0000","last_change":"2023-10-18T10:47:14.842188+0000","last_active":"2023-10-18T10:47:17.488307+0000","last_peered":"2023-10-18T10:47:17.488307+0000","last_clean":"2023-10-18T10:47:17.488307+0000","last_became_active":"2023-10-18T10:47:14.841932+0000","last_became_peered":"2023-10-18T10:47:14.841932+0000","last_unstale":"2023-10-18T10:47:17.488307+0000","last_undegraded":"2023-10-18T10:47:17.488307+0000","last_fullsized":"2023-10-18T10:47:17.488307+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:50:12.229901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489285+0000","last_change":"2023-10-18T10:45:50.585224+0000","last_active":"2023-10-18T10:47:17.489285+0000","last_peered":"2023-10-18T10:47:17.489285+0000","last_clean":"2023-10-18T10:47:17.489285+0000","last_became_active":"2023-10-18T10:45:50.584872+0000","last_became_peered":"2023-10-18T10:45:50.584872+0000","last_unstale":"2023-10-18T10:47:17.489285+0000","last_undegraded":"2023-10-18T10:47:17.489285+0000","last_fullsized":"2023-10-18T10:47:17.489285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:20:47.535961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.843582+0000","last_change":"2023-10-18T10:47:15.842480+0000","last_active":"2023-10-18T10:47:16.843582+0000","last_peered":"2023-10-18T10:47:16.843582+0000","last_clean":"2023-10-18T10:47:16.843582+0000","last_became_active":"2023-10-18T10:47:15.842161+0000","last_became_peered":"2023-10-18T10:47:15.842161+0000","last_unstale":"2023-10-18T10:47:16.843582+0000","last_undegraded":"2023-10-18T10:47:16.843582+0000","last_fullsized":"2023-10-18T10:47:16.843582+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:54:43.295790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489501+0000","last_change":"2023-10-18T10:47:14.841126+0000","last_active":"2023-10-18T10:47:17.489501+0000","last_peered":"2023-10-18T10:47:17.489501+0000","last_clean":"2023-10-18T10:47:17.489501+0000","last_became_active":"2023-10-18T10:47:14.840452+0000","last_became_peered":"2023-10-18T10:47:14.840452+0000","last_unstale":"2023-10-18T10:47:17.489501+0000","last_undegraded":"2023-10-18T10:47:17.489501+0000","last_fullsized":"2023-10-18T10:47:17.489501+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:33:48.946511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488874+0000","last_change":"2023-10-18T10:45:50.584946+0000","last_active":"2023-10-18T10:47:17.488874+0000","last_peered":"2023-10-18T10:47:17.488874+0000","last_clean":"2023-10-18T10:47:17.488874+0000","last_became_active":"2023-10-18T10:45:50.584638+0000","last_became_peered":"2023-10-18T10:45:50.584638+0000","last_unstale":"2023-10-18T10:47:17.488874+0000","last_undegraded":"2023-10-18T10:47:17.488874+0000","last_fullsized":"2023-10-18T10:47:17.488874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:35:48.515328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.844976+0000","last_change":"2023-10-18T10:47:15.855056+0000","last_active":"2023-10-18T10:47:16.844976+0000","last_peered":"2023-10-18T10:47:16.844976+0000","last_clean":"2023-10-18T10:47:16.844976+0000","last_became_active":"2023-10-18T10:47:15.854157+0000","last_became_peered":"2023-10-18T10:47:15.854157+0000","last_unstale":"2023-10-18T10:47:16.844976+0000","last_undegraded":"2023-10-18T10:47:16.844976+0000","last_fullsized":"2023-10-18T10:47:16.844976+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:11:14.235733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.842031+0000","last_change":"2023-10-18T10:47:14.838883+0000","last_active":"2023-10-18T10:47:16.842031+0000","last_peered":"2023-10-18T10:47:16.842031+0000","last_clean":"2023-10-18T10:47:16.842031+0000","last_became_active":"2023-10-18T10:47:14.838464+0000","last_became_peered":"2023-10-18T10:47:14.838464+0000","last_unstale":"2023-10-18T10:47:16.842031+0000","last_undegraded":"2023-10-18T10:47:16.842031+0000","last_fullsized":"2023-10-18T10:47:16.842031+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:32:28.915154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488442+0000","last_change":"2023-10-18T10:45:50.251505+0000","last_active":"2023-10-18T10:47:17.488442+0000","last_peered":"2023-10-18T10:47:17.488442+0000","last_clean":"2023-10-18T10:47:17.488442+0000","last_became_active":"2023-10-18T10:45:50.251111+0000","last_became_peered":"2023-10-18T10:45:50.251111+0000","last_unstale":"2023-10-18T10:47:17.488442+0000","last_undegraded":"2023-10-18T10:47:17.488442+0000","last_fullsized":"2023-10-18T10:47:17.488442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:08:25.147676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489060+0000","last_change":"2023-10-18T10:47:15.844637+0000","last_active":"2023-10-18T10:47:17.489060+0000","last_peered":"2023-10-18T10:47:17.489060+0000","last_clean":"2023-10-18T10:47:17.489060+0000","last_became_active":"2023-10-18T10:47:15.844372+0000","last_became_peered":"2023-10-18T10:47:15.844372+0000","last_unstale":"2023-10-18T10:47:17.489060+0000","last_undegraded":"2023-10-18T10:47:17.489060+0000","last_fullsized":"2023-10-18T10:47:17.489060+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:00:47.718535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489221+0000","last_change":"2023-10-18T10:47:14.850005+0000","last_active":"2023-10-18T10:47:17.489221+0000","last_peered":"2023-10-18T10:47:17.489221+0000","last_clean":"2023-10-18T10:47:17.489221+0000","last_became_active":"2023-10-18T10:47:14.849708+0000","last_became_peered":"2023-10-18T10:47:14.849708+0000","last_unstale":"2023-10-18T10:47:17.489221+0000","last_undegraded":"2023-10-18T10:47:17.489221+0000","last_fullsized":"2023-10-18T10:47:17.489221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:23:03.852551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488004+0000","last_change":"2023-10-18T10:45:50.259019+0000","last_active":"2023-10-18T10:47:17.488004+0000","last_peered":"2023-10-18T10:47:17.488004+0000","last_clean":"2023-10-18T10:47:17.488004+0000","last_became_active":"2023-10-18T10:45:50.258607+0000","last_became_peered":"2023-10-18T10:45:50.258607+0000","last_unstale":"2023-10-18T10:47:17.488004+0000","last_undegraded":"2023-10-18T10:47:17.488004+0000","last_fullsized":"2023-10-18T10:47:17.488004+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:20:38.830994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489088+0000","last_change":"2023-10-18T10:47:15.846658+0000","last_active":"2023-10-18T10:47:17.489088+0000","last_peered":"2023-10-18T10:47:17.489088+0000","last_clean":"2023-10-18T10:47:17.489088+0000","last_became_active":"2023-10-18T10:47:15.846173+0000","last_became_peered":"2023-10-18T10:47:15.846173+0000","last_unstale":"2023-10-18T10:47:17.489088+0000","last_undegraded":"2023-10-18T10:47:17.489088+0000","last_fullsized":"2023-10-18T10:47:17.489088+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:56:56.255147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489060+0000","last_change":"2023-10-18T10:47:14.849782+0000","last_active":"2023-10-18T10:47:17.489060+0000","last_peered":"2023-10-18T10:47:17.489060+0000","last_clean":"2023-10-18T10:47:17.489060+0000","last_became_active":"2023-10-18T10:47:14.849493+0000","last_became_peered":"2023-10-18T10:47:14.849493+0000","last_unstale":"2023-10-18T10:47:17.489060+0000","last_undegraded":"2023-10-18T10:47:17.489060+0000","last_fullsized":"2023-10-18T10:47:17.489060+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:56:59.839610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488131+0000","last_change":"2023-10-18T10:45:50.251375+0000","last_active":"2023-10-18T10:47:17.488131+0000","last_peered":"2023-10-18T10:47:17.488131+0000","last_clean":"2023-10-18T10:47:17.488131+0000","last_became_active":"2023-10-18T10:45:50.251055+0000","last_became_peered":"2023-10-18T10:45:50.251055+0000","last_unstale":"2023-10-18T10:47:17.488131+0000","last_undegraded":"2023-10-18T10:47:17.488131+0000","last_fullsized":"2023-10-18T10:47:17.488131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:14:30.657029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489985+0000","last_change":"2023-10-18T10:47:15.855868+0000","last_active":"2023-10-18T10:47:17.489985+0000","last_peered":"2023-10-18T10:47:17.489985+0000","last_clean":"2023-10-18T10:47:17.489985+0000","last_became_active":"2023-10-18T10:47:15.852068+0000","last_became_peered":"2023-10-18T10:47:15.852068+0000","last_unstale":"2023-10-18T10:47:17.489985+0000","last_undegraded":"2023-10-18T10:47:17.489985+0000","last_fullsized":"2023-10-18T10:47:17.489985+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:43:53.526897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"40'2","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488784+0000","last_change":"2023-10-18T10:47:14.849861+0000","last_active":"2023-10-18T10:47:17.488784+0000","last_peered":"2023-10-18T10:47:17.488784+0000","last_clean":"2023-10-18T10:47:17.488784+0000","last_became_active":"2023-10-18T10:47:14.849100+0000","last_became_peered":"2023-10-18T10:47:14.849100+0000","last_unstale":"2023-10-18T10:47:17.488784+0000","last_undegraded":"2023-10-18T10:47:17.488784+0000","last_fullsized":"2023-10-18T10:47:17.488784+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:18:36.370134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488491+0000","last_change":"2023-10-18T10:45:50.251310+0000","last_active":"2023-10-18T10:47:17.488491+0000","last_peered":"2023-10-18T10:47:17.488491+0000","last_clean":"2023-10-18T10:47:17.488491+0000","last_became_active":"2023-10-18T10:45:50.250823+0000","last_became_peered":"2023-10-18T10:45:50.250823+0000","last_unstale":"2023-10-18T10:47:17.488491+0000","last_undegraded":"2023-10-18T10:47:17.488491+0000","last_fullsized":"2023-10-18T10:47:17.488491+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:42:25.083142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":205,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488272+0000","last_change":"2023-10-18T10:45:44.563531+0000","last_active":"2023-10-18T10:47:17.488272+0000","last_peered":"2023-10-18T10:47:17.488272+0000","last_clean":"2023-10-18T10:47:17.488272+0000","last_became_active":"2023-10-18T10:45:44.562504+0000","last_became_peered":"2023-10-18T10:45:44.562504+0000","last_unstale":"2023-10-18T10:47:17.488272+0000","last_undegraded":"2023-10-18T10:47:17.488272+0000","last_fullsized":"2023-10-18T10:47:17.488272+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:43.181461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:43.181461+0000","last_clean_scrub_stamp":"2023-10-18T10:45:43.181461+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:23:17.171109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343780,"statfs":{"total":96636764160,"available":96608030720,"internally_reserved":0,"allocated":1060864,"data_stored":796273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:46:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.72599999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:46:43 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.76800000000000002}]},{"osd":2,"last update":"Wed Oct 18 10:46:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.69299999999999995}]},{"osd":3,"last update":"Wed Oct 18 10:46:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.749}]},{"osd":4,"last update":"Wed Oct 18 10:46:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.501}]},{"osd":5,"last update":"Wed Oct 18 10:46:43 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.63300000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:46:43 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.56999999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":483328,"data_stored":213460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.89700000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.754}]},{"osd":2,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.1469999999999998,"5min":2.1469999999999998,"15min":2.1469999999999998},"last":0.91900000000000004}]},{"osd":3,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.57899999999999996}]},{"osd":4,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.72999999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.84299999999999997}]},{"osd":7,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":1.026},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.95899999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":466944,"data_stored":205885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.71799999999999997}]},{"osd":1,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.63900000000000001}]},{"osd":2,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.76200000000000001}]},{"osd":3,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":5.0970000000000004,"5min":5.0970000000000004,"15min":5.0970000000000004},"last":0.79200000000000004}]},{"osd":4,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.505}]},{"osd":6,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.441}]},{"osd":7,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.67600000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":487424,"data_stored":214400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:46:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.61899999999999999}]},{"osd":1,"last update":"Wed Oct 18 10:46:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.853,"5min":1.853,"15min":1.853},"last":0.65600000000000003}]},{"osd":2,"last update":"Wed Oct 18 10:46:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":0.748}]},{"osd":3,"last update":"Wed Oct 18 10:46:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.879}]},{"osd":5,"last update":"Wed Oct 18 10:46:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.77500000000000002}]},{"osd":6,"last update":"Wed Oct 18 10:46:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":0.90400000000000003}]},{"osd":7,"last update":"Wed Oct 18 10:46:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.95999999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":487424,"data_stored":213426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.69699999999999995}]},{"osd":1,"last update":"Wed Oct 18 10:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.52900000000000003}]},{"osd":2,"last update":"Wed Oct 18 10:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.57499999999999996}]},{"osd":4,"last update":"Wed Oct 18 10:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":1.0069999999999999}]},{"osd":5,"last update":"Wed Oct 18 10:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.91500000000000004}]},{"osd":6,"last update":"Wed Oct 18 10:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":1.046},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":1.0269999999999999}]},{"osd":7,"last update":"Wed Oct 18 10:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.95899999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":479232,"data_stored":207449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.53400000000000003}]},{"osd":1,"last update":"Wed Oct 18 10:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.40000000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.45300000000000001}]},{"osd":4,"last update":"Wed Oct 18 10:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.72999999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.73999999999999999}]},{"osd":6,"last update":"Wed Oct 18 10:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.84699999999999998}]},{"osd":7,"last update":"Wed Oct 18 10:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.875},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.754}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":466944,"data_stored":205885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.375},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.47899999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.41899999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.503},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.45700000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.71799999999999997}]},{"osd":5,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.9850000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":0.67200000000000004}]},{"osd":6,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.65900000000000003}]},{"osd":7,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.57399999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343780,"statfs":{"total":96636764160,"available":96608030720,"internally_reserved":0,"allocated":1060864,"data_stored":796217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:46:45 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":9.0410000000000004,"5min":9.0410000000000004,"15min":9.0410000000000004},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":9.1349999999999998,"5min":9.1349999999999998,"15min":9.1349999999999998},"last":0.40600000000000003}]},{"osd":2,"last update":"Wed Oct 18 10:46:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":5.3150000000000004,"5min":5.3150000000000004,"15min":5.3150000000000004},"last":0.48999999999999999},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":9.0980000000000008,"5min":9.0980000000000008,"15min":9.0980000000000008},"last":0.47199999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:46:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":8.9700000000000006,"5min":8.9700000000000006,"15min":8.9700000000000006},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":5.7210000000000001,"5min":5.7210000000000001,"15min":5.7210000000000001},"last":0.58699999999999997}]},{"osd":4,"last update":"Wed Oct 18 10:46:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":5.625,"5min":5.625,"15min":5.625},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":5.8040000000000003,"5min":5.8040000000000003,"15min":5.8040000000000003},"last":0.84299999999999997}]},{"osd":5,"last update":"Wed Oct 18 10:46:45 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":5.6790000000000003,"5min":5.6790000000000003,"15min":5.6790000000000003},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.149,"5min":1.149,"15min":1.149},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":5.5739999999999998,"5min":5.5739999999999998,"15min":5.5739999999999998},"last":0.91400000000000003}]},{"osd":6,"last update":"Wed Oct 18 10:46:45 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":5.6559999999999997,"5min":5.6559999999999997,"15min":5.6559999999999997},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":5.7610000000000001,"5min":5.7610000000000001,"15min":5.7610000000000001},"last":0.82799999999999996}]},{"osd":7,"last update":"Wed Oct 18 10:46:45 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":5.782,"5min":5.782,"15min":5.782},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":5.6109999999999998,"5min":5.6109999999999998,"15min":5.6109999999999998},"last":0.95599999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:47:19.959 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:47:20.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.131+0000 7f052efe1640 1 Processor -- start 2023-10-18T10:47:20.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.135+0000 7f052efe1640 1 -- start start 2023-10-18T10:47:20.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.135+0000 7f052efe1640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f05280cd230 0x7f05280ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:20.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.135+0000 7f052efe1640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f052815d4c0 con 0x7f05280709b0 2023-10-18T10:47:20.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.135+0000 7f052efe1640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f052806b210 con 0x7f05281594b0 2023-10-18T10:47:20.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.135+0000 7f052efe1640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f052815e060 con 0x7f05280cd230 2023-10-18T10:47:20.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.135+0000 7f0527fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f05280cd230 0x7f05280ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:20.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.135+0000 7f052d57e640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f05280709b0 0x7f05280ca650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40680/0 (socket says 172.21.15.89:40680) 2023-10-18T10:47:20.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.135+0000 7f052d57e640 1 -- 172.21.15.89:0/3193114488 learned_addr learned my addr 172.21.15.89:0/3193114488 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:47:20.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.135+0000 7f0527fff640 1 -- 172.21.15.89:0/3193114488 >> v1:172.21.15.154:6789/0 conn(0x7f05281594b0 legacy=0x7f052815d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:20.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.135+0000 7f0527fff640 1 -- 172.21.15.89:0/3193114488 >> v1:172.21.15.89:6789/0 conn(0x7f05280709b0 legacy=0x7f05280ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:20.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.135+0000 7f0527fff640 1 -- 172.21.15.89:0/3193114488 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052806ac50 con 0x7f05280cd230 2023-10-18T10:47:20.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.135+0000 7f05277fe640 1 -- 172.21.15.89:0/3193114488 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2775136601 0 0) 0x7f052815d4c0 con 0x7f05280709b0 2023-10-18T10:47:20.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.139+0000 7f0527fff640 1 --2- 172.21.15.89:0/3193114488 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05280cd230 0x7f05280ce6e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f05180097c0 tx=0x7f051801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4053f8c390c09f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:20.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.139+0000 7f05277fe640 1 -- 172.21.15.89:0/3193114488 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05180ac620 con 0x7f05280cd230 2023-10-18T10:47:20.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.139+0000 7f05277fe640 1 -- 172.21.15.89:0/3193114488 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05180ac780 con 0x7f05280cd230 2023-10-18T10:47:20.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.139+0000 7f05277fe640 1 -- 172.21.15.89:0/3193114488 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05180acb10 con 0x7f05280cd230 2023-10-18T10:47:20.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.139+0000 7f052efe1640 1 -- 172.21.15.89:0/3193114488 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05280cd230 msgr2=0x7f05280ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:20.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.139+0000 7f052efe1640 1 --2- 172.21.15.89:0/3193114488 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05280cd230 0x7f05280ce6e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f05180097c0 tx=0x7f051801c3c0 comp rx=0 tx=0).stop 2023-10-18T10:47:20.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.139+0000 7f052efe1640 1 -- 172.21.15.89:0/3193114488 shutdown_connections 2023-10-18T10:47:20.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.139+0000 7f052efe1640 1 --2- 172.21.15.89:0/3193114488 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05280cd230 0x7f05280ce6e0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:20.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.139+0000 7f052efe1640 1 -- 172.21.15.89:0/3193114488 >> 172.21.15.89:0/3193114488 conn(0x7f05280cbdc0 msgr2=0x7f05280cc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:20.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.139+0000 7f052efe1640 1 -- 172.21.15.89:0/3193114488 shutdown_connections 2023-10-18T10:47:20.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.139+0000 7f052efe1640 1 -- 172.21.15.89:0/3193114488 wait complete. 2023-10-18T10:47:20.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.139+0000 7f052efe1640 1 Processor -- start 2023-10-18T10:47:20.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.139+0000 7f052efe1640 1 -- start start 2023-10-18T10:47:20.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.143+0000 7f052efe1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05280709b0 0x7f0528156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:20.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.143+0000 7f052efe1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05280cd230 0x7f05280c2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:20.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.143+0000 7f052efe1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05281594b0 0x7f05280c3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:20.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.143+0000 7f052efe1640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052815c380 con 0x7f05281594b0 2023-10-18T10:47:20.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.143+0000 7f052efe1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05280c5990 con 0x7f05280709b0 2023-10-18T10:47:20.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.143+0000 7f052efe1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f052815b5c0 con 0x7f05280cd230 2023-10-18T10:47:20.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.143+0000 7f052d57e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05281594b0 0x7f05280c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:20.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.143+0000 7f0527fff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05280cd230 0x7f05280c2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:20.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.143+0000 7f0527fff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05280cd230 0x7f05280c2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:50588/0 (socket says 172.21.15.89:50588) 2023-10-18T10:47:20.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.143+0000 7f052cd7d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05280709b0 0x7f0528156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:20.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.143+0000 7f0527fff640 1 -- 172.21.15.89:0/2575283901 learned_addr learned my addr 172.21.15.89:0/2575283901 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:20.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.143+0000 7f052d57e640 1 -- 172.21.15.89:0/2575283901 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05280cd230 msgr2=0x7f05280c2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:20.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.143+0000 7f052d57e640 1 --2- 172.21.15.89:0/2575283901 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05280cd230 0x7f05280c2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:20.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.143+0000 7f052d57e640 1 -- 172.21.15.89:0/2575283901 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05280709b0 msgr2=0x7f0528156710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:20.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.143+0000 7f052d57e640 1 --2- 172.21.15.89:0/2575283901 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05280709b0 0x7f0528156710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:20.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.143+0000 7f052d57e640 1 -- 172.21.15.89:0/2575283901 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0518009420 con 0x7f05281594b0 2023-10-18T10:47:20.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.143+0000 7f052d57e640 1 --2- 172.21.15.89:0/2575283901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05281594b0 0x7f05280c3190 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f052000c750 tx=0x7f052000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:20.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.143+0000 7f05257fa640 1 -- 172.21.15.89:0/2575283901 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0520004d80 con 0x7f05281594b0 2023-10-18T10:47:20.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.143+0000 7f05257fa640 1 -- 172.21.15.89:0/2575283901 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052000a640 con 0x7f05281594b0 2023-10-18T10:47:20.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.143+0000 7f05257fa640 1 -- 172.21.15.89:0/2575283901 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052000aa70 con 0x7f05281594b0 2023-10-18T10:47:20.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.143+0000 7f052efe1640 1 -- 172.21.15.89:0/2575283901 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f052815b8a0 con 0x7f05281594b0 2023-10-18T10:47:20.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.143+0000 7f052efe1640 1 -- 172.21.15.89:0/2575283901 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052815be60 con 0x7f05281594b0 2023-10-18T10:47:20.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.143+0000 7f052efe1640 1 -- 172.21.15.89:0/2575283901 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f052815c050 con 0x7f05281594b0 2023-10-18T10:47:20.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.143+0000 7f05257fa640 1 -- 172.21.15.89:0/2575283901 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f052000abd0 con 0x7f05281594b0 2023-10-18T10:47:20.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.147+0000 7f05257fa640 1 --2- 172.21.15.89:0/2575283901 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0508065680 0x7f0508067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:20.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.147+0000 7f05257fa640 1 -- 172.21.15.89:0/2575283901 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f05200822f0 con 0x7f05281594b0 2023-10-18T10:47:20.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.147+0000 7f052cd7d640 1 --2- 172.21.15.89:0/2575283901 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0508065680 0x7f0508067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:20.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.147+0000 7f052cd7d640 1 --2- 172.21.15.89:0/2575283901 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0508065680 0x7f0508067b40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0510007fd0 tx=0x7f0510009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:20.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.151+0000 7f05257fa640 1 -- 172.21.15.89:0/2575283901 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f052004fdb0 con 0x7f05281594b0 2023-10-18T10:47:20.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.295+0000 7f052efe1640 1 -- 172.21.15.89:0/2575283901 --> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f052815b730 con 0x7f0508065680 2023-10-18T10:47:20.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.303+0000 7f05257fa640 1 -- 172.21.15.89:0/2575283901 <== mgr.4099 v2:172.21.15.154:6832/1086608968 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311045 (secure 0 0 0) 0x7f052815b730 con 0x7f0508065680 2023-10-18T10:47:20.310 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:47:20.312 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-18T10:47:20.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.307+0000 7f052efe1640 1 -- 172.21.15.89:0/2575283901 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0508065680 msgr2=0x7f0508067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:20.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.307+0000 7f052efe1640 1 --2- 172.21.15.89:0/2575283901 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0508065680 0x7f0508067b40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0510007fd0 tx=0x7f0510009040 comp rx=0 tx=0).stop 2023-10-18T10:47:20.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.307+0000 7f052efe1640 1 -- 172.21.15.89:0/2575283901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05281594b0 msgr2=0x7f05280c3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:20.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.307+0000 7f052efe1640 1 --2- 172.21.15.89:0/2575283901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05281594b0 0x7f05280c3190 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f052000c750 tx=0x7f052000cc10 comp rx=0 tx=0).stop 2023-10-18T10:47:20.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.307+0000 7f052efe1640 1 -- 172.21.15.89:0/2575283901 shutdown_connections 2023-10-18T10:47:20.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.307+0000 7f052efe1640 1 --2- 172.21.15.89:0/2575283901 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0508065680 0x7f0508067b40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:20.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.307+0000 7f052efe1640 1 --2- 172.21.15.89:0/2575283901 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05280cd230 0x7f05280c2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:20.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.307+0000 7f052efe1640 1 --2- 172.21.15.89:0/2575283901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05281594b0 0x7f05280c3190 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:20.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.307+0000 7f052efe1640 1 --2- 172.21.15.89:0/2575283901 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05280709b0 0x7f0528156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:20.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.307+0000 7f052efe1640 1 -- 172.21.15.89:0/2575283901 >> 172.21.15.89:0/2575283901 conn(0x7f05280cbdc0 msgr2=0x7f052815d950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:20.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.307+0000 7f052efe1640 1 -- 172.21.15.89:0/2575283901 shutdown_connections 2023-10-18T10:47:20.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.307+0000 7f052efe1640 1 -- 172.21.15.89:0/2575283901 wait complete. 2023-10-18T10:47:20.330 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2023-10-18T10:47:19.074024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":142,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221060,"kb_used_data":4876,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754753660,"statfs":{"total":773094113280,"available":772867747840,"internally_reserved":0,"allocated":4993024,"data_stored":2852995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.258723"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489199+0000","last_change":"2023-10-18T10:47:15.837486+0000","last_active":"2023-10-18T10:47:17.489199+0000","last_peered":"2023-10-18T10:47:17.489199+0000","last_clean":"2023-10-18T10:47:17.489199+0000","last_became_active":"2023-10-18T10:47:15.837138+0000","last_became_peered":"2023-10-18T10:47:15.837138+0000","last_unstale":"2023-10-18T10:47:17.489199+0000","last_undegraded":"2023-10-18T10:47:17.489199+0000","last_fullsized":"2023-10-18T10:47:17.489199+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:20:23.286839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488805+0000","last_change":"2023-10-18T10:47:15.857501+0000","last_active":"2023-10-18T10:47:17.488805+0000","last_peered":"2023-10-18T10:47:17.488805+0000","last_clean":"2023-10-18T10:47:17.488805+0000","last_became_active":"2023-10-18T10:47:15.857173+0000","last_became_peered":"2023-10-18T10:47:15.857173+0000","last_unstale":"2023-10-18T10:47:17.488805+0000","last_undegraded":"2023-10-18T10:47:17.488805+0000","last_fullsized":"2023-10-18T10:47:17.488805+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:36:02.352590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.844070+0000","last_change":"2023-10-18T10:47:15.851720+0000","last_active":"2023-10-18T10:47:16.844070+0000","last_peered":"2023-10-18T10:47:16.844070+0000","last_clean":"2023-10-18T10:47:16.844070+0000","last_became_active":"2023-10-18T10:47:15.851361+0000","last_became_peered":"2023-10-18T10:47:15.851361+0000","last_unstale":"2023-10-18T10:47:16.844070+0000","last_undegraded":"2023-10-18T10:47:16.844070+0000","last_fullsized":"2023-10-18T10:47:16.844070+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:29:48.874952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488913+0000","last_change":"2023-10-18T10:47:15.861433+0000","last_active":"2023-10-18T10:47:17.488913+0000","last_peered":"2023-10-18T10:47:17.488913+0000","last_clean":"2023-10-18T10:47:17.488913+0000","last_became_active":"2023-10-18T10:47:15.861154+0000","last_became_peered":"2023-10-18T10:47:15.861154+0000","last_unstale":"2023-10-18T10:47:17.488913+0000","last_undegraded":"2023-10-18T10:47:17.488913+0000","last_fullsized":"2023-10-18T10:47:17.488913+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:03:06.298352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488573+0000","last_change":"2023-10-18T10:47:15.858755+0000","last_active":"2023-10-18T10:47:17.488573+0000","last_peered":"2023-10-18T10:47:17.488573+0000","last_clean":"2023-10-18T10:47:17.488573+0000","last_became_active":"2023-10-18T10:47:15.858528+0000","last_became_peered":"2023-10-18T10:47:15.858528+0000","last_unstale":"2023-10-18T10:47:17.488573+0000","last_undegraded":"2023-10-18T10:47:17.488573+0000","last_fullsized":"2023-10-18T10:47:17.488573+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:26:16.315364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.841388+0000","last_change":"2023-10-18T10:47:15.840342+0000","last_active":"2023-10-18T10:47:16.841388+0000","last_peered":"2023-10-18T10:47:16.841388+0000","last_clean":"2023-10-18T10:47:16.841388+0000","last_became_active":"2023-10-18T10:47:15.840027+0000","last_became_peered":"2023-10-18T10:47:15.840027+0000","last_unstale":"2023-10-18T10:47:16.841388+0000","last_undegraded":"2023-10-18T10:47:16.841388+0000","last_fullsized":"2023-10-18T10:47:16.841388+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:43:50.664999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489035+0000","last_change":"2023-10-18T10:47:15.840188+0000","last_active":"2023-10-18T10:47:17.489035+0000","last_peered":"2023-10-18T10:47:17.489035+0000","last_clean":"2023-10-18T10:47:17.489035+0000","last_became_active":"2023-10-18T10:47:15.839890+0000","last_became_peered":"2023-10-18T10:47:15.839890+0000","last_unstale":"2023-10-18T10:47:17.489035+0000","last_undegraded":"2023-10-18T10:47:17.489035+0000","last_fullsized":"2023-10-18T10:47:17.489035+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:43:26.476206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489134+0000","last_change":"2023-10-18T10:47:15.862982+0000","last_active":"2023-10-18T10:47:17.489134+0000","last_peered":"2023-10-18T10:47:17.489134+0000","last_clean":"2023-10-18T10:47:17.489134+0000","last_became_active":"2023-10-18T10:47:15.862607+0000","last_became_peered":"2023-10-18T10:47:15.862607+0000","last_unstale":"2023-10-18T10:47:17.489134+0000","last_undegraded":"2023-10-18T10:47:17.489134+0000","last_fullsized":"2023-10-18T10:47:17.489134+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:43:50.664999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489990+0000","last_change":"2023-10-18T10:47:15.842867+0000","last_active":"2023-10-18T10:47:17.489990+0000","last_peered":"2023-10-18T10:47:17.489990+0000","last_clean":"2023-10-18T10:47:17.489990+0000","last_became_active":"2023-10-18T10:47:15.842521+0000","last_became_peered":"2023-10-18T10:47:15.842521+0000","last_unstale":"2023-10-18T10:47:17.489990+0000","last_undegraded":"2023-10-18T10:47:17.489990+0000","last_fullsized":"2023-10-18T10:47:17.489990+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:54:49.219935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489902+0000","last_change":"2023-10-18T10:47:15.861062+0000","last_active":"2023-10-18T10:47:17.489902+0000","last_peered":"2023-10-18T10:47:17.489902+0000","last_clean":"2023-10-18T10:47:17.489902+0000","last_became_active":"2023-10-18T10:47:15.860790+0000","last_became_peered":"2023-10-18T10:47:15.860790+0000","last_unstale":"2023-10-18T10:47:17.489902+0000","last_undegraded":"2023-10-18T10:47:17.489902+0000","last_fullsized":"2023-10-18T10:47:17.489902+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:36:02.352590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488976+0000","last_change":"2023-10-18T10:47:15.857520+0000","last_active":"2023-10-18T10:47:17.488976+0000","last_peered":"2023-10-18T10:47:17.488976+0000","last_clean":"2023-10-18T10:47:17.488976+0000","last_became_active":"2023-10-18T10:47:15.856115+0000","last_became_peered":"2023-10-18T10:47:15.856115+0000","last_unstale":"2023-10-18T10:47:17.488976+0000","last_undegraded":"2023-10-18T10:47:17.488976+0000","last_fullsized":"2023-10-18T10:47:17.488976+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:35:08.646421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488552+0000","last_change":"2023-10-18T10:47:15.852071+0000","last_active":"2023-10-18T10:47:17.488552+0000","last_peered":"2023-10-18T10:47:17.488552+0000","last_clean":"2023-10-18T10:47:17.488552+0000","last_became_active":"2023-10-18T10:47:15.851481+0000","last_became_peered":"2023-10-18T10:47:15.851481+0000","last_unstale":"2023-10-18T10:47:17.488552+0000","last_undegraded":"2023-10-18T10:47:17.488552+0000","last_fullsized":"2023-10-18T10:47:17.488552+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:03:46.146073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489462+0000","last_change":"2023-10-18T10:47:15.844524+0000","last_active":"2023-10-18T10:47:17.489462+0000","last_peered":"2023-10-18T10:47:17.489462+0000","last_clean":"2023-10-18T10:47:17.489462+0000","last_became_active":"2023-10-18T10:47:15.844138+0000","last_became_peered":"2023-10-18T10:47:15.844138+0000","last_unstale":"2023-10-18T10:47:17.489462+0000","last_undegraded":"2023-10-18T10:47:17.489462+0000","last_fullsized":"2023-10-18T10:47:17.489462+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:35:22.551306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489625+0000","last_change":"2023-10-18T10:47:15.844727+0000","last_active":"2023-10-18T10:47:17.489625+0000","last_peered":"2023-10-18T10:47:17.489625+0000","last_clean":"2023-10-18T10:47:17.489625+0000","last_became_active":"2023-10-18T10:47:15.842617+0000","last_became_peered":"2023-10-18T10:47:15.842617+0000","last_unstale":"2023-10-18T10:47:17.489625+0000","last_undegraded":"2023-10-18T10:47:17.489625+0000","last_fullsized":"2023-10-18T10:47:17.489625+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:18:37.376778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.841556+0000","last_change":"2023-10-18T10:47:15.835368+0000","last_active":"2023-10-18T10:47:16.841556+0000","last_peered":"2023-10-18T10:47:16.841556+0000","last_clean":"2023-10-18T10:47:16.841556+0000","last_became_active":"2023-10-18T10:47:15.835074+0000","last_became_peered":"2023-10-18T10:47:15.835074+0000","last_unstale":"2023-10-18T10:47:16.841556+0000","last_undegraded":"2023-10-18T10:47:16.841556+0000","last_fullsized":"2023-10-18T10:47:16.841556+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:43:53.526897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.843642+0000","last_change":"2023-10-18T10:47:15.861341+0000","last_active":"2023-10-18T10:47:16.843642+0000","last_peered":"2023-10-18T10:47:16.843642+0000","last_clean":"2023-10-18T10:47:16.843642+0000","last_became_active":"2023-10-18T10:47:15.861041+0000","last_became_peered":"2023-10-18T10:47:15.861041+0000","last_unstale":"2023-10-18T10:47:16.843642+0000","last_undegraded":"2023-10-18T10:47:16.843642+0000","last_fullsized":"2023-10-18T10:47:16.843642+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:02:26.920118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.844380+0000","last_change":"2023-10-18T10:47:15.841201+0000","last_active":"2023-10-18T10:47:16.844380+0000","last_peered":"2023-10-18T10:47:16.844380+0000","last_clean":"2023-10-18T10:47:16.844380+0000","last_became_active":"2023-10-18T10:47:15.840880+0000","last_became_peered":"2023-10-18T10:47:15.840880+0000","last_unstale":"2023-10-18T10:47:16.844380+0000","last_undegraded":"2023-10-18T10:47:16.844380+0000","last_fullsized":"2023-10-18T10:47:16.844380+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:21:38.088695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.843651+0000","last_change":"2023-10-18T10:47:15.858086+0000","last_active":"2023-10-18T10:47:16.843651+0000","last_peered":"2023-10-18T10:47:16.843651+0000","last_clean":"2023-10-18T10:47:16.843651+0000","last_became_active":"2023-10-18T10:47:15.857812+0000","last_became_peered":"2023-10-18T10:47:15.857812+0000","last_unstale":"2023-10-18T10:47:16.843651+0000","last_undegraded":"2023-10-18T10:47:16.843651+0000","last_fullsized":"2023-10-18T10:47:16.843651+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:40:36.982411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.844661+0000","last_change":"2023-10-18T10:47:15.858895+0000","last_active":"2023-10-18T10:47:16.844661+0000","last_peered":"2023-10-18T10:47:16.844661+0000","last_clean":"2023-10-18T10:47:16.844661+0000","last_became_active":"2023-10-18T10:47:15.858483+0000","last_became_peered":"2023-10-18T10:47:15.858483+0000","last_unstale":"2023-10-18T10:47:16.844661+0000","last_undegraded":"2023-10-18T10:47:16.844661+0000","last_fullsized":"2023-10-18T10:47:16.844661+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:14:26.143046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489648+0000","last_change":"2023-10-18T10:47:15.850040+0000","last_active":"2023-10-18T10:47:17.489648+0000","last_peered":"2023-10-18T10:47:17.489648+0000","last_clean":"2023-10-18T10:47:17.489648+0000","last_became_active":"2023-10-18T10:47:15.849723+0000","last_became_peered":"2023-10-18T10:47:15.849723+0000","last_unstale":"2023-10-18T10:47:17.489648+0000","last_undegraded":"2023-10-18T10:47:17.489648+0000","last_fullsized":"2023-10-18T10:47:17.489648+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:50.509688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.841544+0000","last_change":"2023-10-18T10:47:15.851831+0000","last_active":"2023-10-18T10:47:16.841544+0000","last_peered":"2023-10-18T10:47:16.841544+0000","last_clean":"2023-10-18T10:47:16.841544+0000","last_became_active":"2023-10-18T10:47:15.851385+0000","last_became_peered":"2023-10-18T10:47:15.851385+0000","last_unstale":"2023-10-18T10:47:16.841544+0000","last_undegraded":"2023-10-18T10:47:16.841544+0000","last_fullsized":"2023-10-18T10:47:16.841544+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:36:02.352590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488924+0000","last_change":"2023-10-18T10:47:15.861409+0000","last_active":"2023-10-18T10:47:17.488924+0000","last_peered":"2023-10-18T10:47:17.488924+0000","last_clean":"2023-10-18T10:47:17.488924+0000","last_became_active":"2023-10-18T10:47:15.861085+0000","last_became_peered":"2023-10-18T10:47:15.861085+0000","last_unstale":"2023-10-18T10:47:17.488924+0000","last_undegraded":"2023-10-18T10:47:17.488924+0000","last_fullsized":"2023-10-18T10:47:17.488924+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:13:35.452997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.844076+0000","last_change":"2023-10-18T10:47:15.848186+0000","last_active":"2023-10-18T10:47:16.844076+0000","last_peered":"2023-10-18T10:47:16.844076+0000","last_clean":"2023-10-18T10:47:16.844076+0000","last_became_active":"2023-10-18T10:47:15.847909+0000","last_became_peered":"2023-10-18T10:47:15.847909+0000","last_unstale":"2023-10-18T10:47:16.844076+0000","last_undegraded":"2023-10-18T10:47:16.844076+0000","last_fullsized":"2023-10-18T10:47:16.844076+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:45:39.396012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.843785+0000","last_change":"2023-10-18T10:47:15.853844+0000","last_active":"2023-10-18T10:47:16.843785+0000","last_peered":"2023-10-18T10:47:16.843785+0000","last_clean":"2023-10-18T10:47:16.843785+0000","last_became_active":"2023-10-18T10:47:15.853505+0000","last_became_peered":"2023-10-18T10:47:15.853505+0000","last_unstale":"2023-10-18T10:47:16.843785+0000","last_undegraded":"2023-10-18T10:47:16.843785+0000","last_fullsized":"2023-10-18T10:47:16.843785+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:38:22.856976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.844805+0000","last_change":"2023-10-18T10:47:15.845600+0000","last_active":"2023-10-18T10:47:16.844805+0000","last_peered":"2023-10-18T10:47:16.844805+0000","last_clean":"2023-10-18T10:47:16.844805+0000","last_became_active":"2023-10-18T10:47:15.845399+0000","last_became_peered":"2023-10-18T10:47:15.845399+0000","last_unstale":"2023-10-18T10:47:16.844805+0000","last_undegraded":"2023-10-18T10:47:16.844805+0000","last_fullsized":"2023-10-18T10:47:16.844805+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:59:39.963888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489770+0000","last_change":"2023-10-18T10:47:15.848901+0000","last_active":"2023-10-18T10:47:17.489770+0000","last_peered":"2023-10-18T10:47:17.489770+0000","last_clean":"2023-10-18T10:47:17.489770+0000","last_became_active":"2023-10-18T10:47:15.848221+0000","last_became_peered":"2023-10-18T10:47:15.848221+0000","last_unstale":"2023-10-18T10:47:17.489770+0000","last_undegraded":"2023-10-18T10:47:17.489770+0000","last_fullsized":"2023-10-18T10:47:17.489770+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:53:50.604882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.844207+0000","last_change":"2023-10-18T10:47:15.844213+0000","last_active":"2023-10-18T10:47:16.844207+0000","last_peered":"2023-10-18T10:47:16.844207+0000","last_clean":"2023-10-18T10:47:16.844207+0000","last_became_active":"2023-10-18T10:47:15.843321+0000","last_became_peered":"2023-10-18T10:47:15.843321+0000","last_unstale":"2023-10-18T10:47:16.844207+0000","last_undegraded":"2023-10-18T10:47:16.844207+0000","last_fullsized":"2023-10-18T10:47:16.844207+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:26.573258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489419+0000","last_change":"2023-10-18T10:47:15.851877+0000","last_active":"2023-10-18T10:47:17.489419+0000","last_peered":"2023-10-18T10:47:17.489419+0000","last_clean":"2023-10-18T10:47:17.489419+0000","last_became_active":"2023-10-18T10:47:15.851046+0000","last_became_peered":"2023-10-18T10:47:15.851046+0000","last_unstale":"2023-10-18T10:47:17.489419+0000","last_undegraded":"2023-10-18T10:47:17.489419+0000","last_fullsized":"2023-10-18T10:47:17.489419+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:54:49.219935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.843903+0000","last_change":"2023-10-18T10:47:15.851823+0000","last_active":"2023-10-18T10:47:16.843903+0000","last_peered":"2023-10-18T10:47:16.843903+0000","last_clean":"2023-10-18T10:47:16.843903+0000","last_became_active":"2023-10-18T10:47:15.851513+0000","last_became_peered":"2023-10-18T10:47:15.851513+0000","last_unstale":"2023-10-18T10:47:16.843903+0000","last_undegraded":"2023-10-18T10:47:16.843903+0000","last_fullsized":"2023-10-18T10:47:16.843903+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:11:02.781422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.843495+0000","last_change":"2023-10-18T10:47:15.847560+0000","last_active":"2023-10-18T10:47:16.843495+0000","last_peered":"2023-10-18T10:47:16.843495+0000","last_clean":"2023-10-18T10:47:16.843495+0000","last_became_active":"2023-10-18T10:47:15.847241+0000","last_became_peered":"2023-10-18T10:47:15.847241+0000","last_unstale":"2023-10-18T10:47:16.843495+0000","last_undegraded":"2023-10-18T10:47:16.843495+0000","last_fullsized":"2023-10-18T10:47:16.843495+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:01:02.374958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.841407+0000","last_change":"2023-10-18T10:47:15.844563+0000","last_active":"2023-10-18T10:47:16.841407+0000","last_peered":"2023-10-18T10:47:16.841407+0000","last_clean":"2023-10-18T10:47:16.841407+0000","last_became_active":"2023-10-18T10:47:15.844300+0000","last_became_peered":"2023-10-18T10:47:15.844300+0000","last_unstale":"2023-10-18T10:47:16.841407+0000","last_undegraded":"2023-10-18T10:47:16.841407+0000","last_fullsized":"2023-10-18T10:47:16.841407+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:03:46.146073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488164+0000","last_change":"2023-10-18T10:47:15.841484+0000","last_active":"2023-10-18T10:47:17.488164+0000","last_peered":"2023-10-18T10:47:17.488164+0000","last_clean":"2023-10-18T10:47:17.488164+0000","last_became_active":"2023-10-18T10:47:15.841110+0000","last_became_peered":"2023-10-18T10:47:15.841110+0000","last_unstale":"2023-10-18T10:47:17.488164+0000","last_undegraded":"2023-10-18T10:47:17.488164+0000","last_fullsized":"2023-10-18T10:47:17.488164+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:58:49.080873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489208+0000","last_change":"2023-10-18T10:47:15.859551+0000","last_active":"2023-10-18T10:47:17.489208+0000","last_peered":"2023-10-18T10:47:17.489208+0000","last_clean":"2023-10-18T10:47:17.489208+0000","last_became_active":"2023-10-18T10:47:15.859273+0000","last_became_peered":"2023-10-18T10:47:15.859273+0000","last_unstale":"2023-10-18T10:47:17.489208+0000","last_undegraded":"2023-10-18T10:47:17.489208+0000","last_fullsized":"2023-10-18T10:47:17.489208+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:57:18.382770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"40'4","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.842253+0000","last_change":"2023-10-18T10:47:14.832823+0000","last_active":"2023-10-18T10:47:16.842253+0000","last_peered":"2023-10-18T10:47:16.842253+0000","last_clean":"2023-10-18T10:47:16.842253+0000","last_became_active":"2023-10-18T10:47:14.832542+0000","last_became_peered":"2023-10-18T10:47:14.832542+0000","last_unstale":"2023-10-18T10:47:16.842253+0000","last_undegraded":"2023-10-18T10:47:16.842253+0000","last_fullsized":"2023-10-18T10:47:16.842253+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:21:09.552522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.844960+0000","last_change":"2023-10-18T10:47:15.846403+0000","last_active":"2023-10-18T10:47:16.844960+0000","last_peered":"2023-10-18T10:47:16.844960+0000","last_clean":"2023-10-18T10:47:16.844960+0000","last_became_active":"2023-10-18T10:47:15.845794+0000","last_became_peered":"2023-10-18T10:47:15.845794+0000","last_unstale":"2023-10-18T10:47:16.844960+0000","last_undegraded":"2023-10-18T10:47:16.844960+0000","last_fullsized":"2023-10-18T10:47:16.844960+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:49:15.746889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"40'1","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.843931+0000","last_change":"2023-10-18T10:47:14.839287+0000","last_active":"2023-10-18T10:47:16.843931+0000","last_peered":"2023-10-18T10:47:16.843931+0000","last_clean":"2023-10-18T10:47:16.843931+0000","last_became_active":"2023-10-18T10:47:14.838994+0000","last_became_peered":"2023-10-18T10:47:14.838994+0000","last_unstale":"2023-10-18T10:47:16.843931+0000","last_undegraded":"2023-10-18T10:47:16.843931+0000","last_fullsized":"2023-10-18T10:47:16.843931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:19:27.257823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488696+0000","last_change":"2023-10-18T10:47:15.857607+0000","last_active":"2023-10-18T10:47:17.488696+0000","last_peered":"2023-10-18T10:47:17.488696+0000","last_clean":"2023-10-18T10:47:17.488696+0000","last_became_active":"2023-10-18T10:47:15.854475+0000","last_became_peered":"2023-10-18T10:47:15.854475+0000","last_unstale":"2023-10-18T10:47:17.488696+0000","last_undegraded":"2023-10-18T10:47:17.488696+0000","last_fullsized":"2023-10-18T10:47:17.488696+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:33:49.953156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"40'2","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489206+0000","last_change":"2023-10-18T10:47:14.847383+0000","last_active":"2023-10-18T10:47:17.489206+0000","last_peered":"2023-10-18T10:47:17.489206+0000","last_clean":"2023-10-18T10:47:17.489206+0000","last_became_active":"2023-10-18T10:47:14.847052+0000","last_became_peered":"2023-10-18T10:47:14.847052+0000","last_unstale":"2023-10-18T10:47:17.489206+0000","last_undegraded":"2023-10-18T10:47:17.489206+0000","last_fullsized":"2023-10-18T10:47:17.489206+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:23:25.645520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489056+0000","last_change":"2023-10-18T10:47:15.851519+0000","last_active":"2023-10-18T10:47:17.489056+0000","last_peered":"2023-10-18T10:47:17.489056+0000","last_clean":"2023-10-18T10:47:17.489056+0000","last_became_active":"2023-10-18T10:47:15.850963+0000","last_became_peered":"2023-10-18T10:47:15.850963+0000","last_unstale":"2023-10-18T10:47:17.489056+0000","last_undegraded":"2023-10-18T10:47:17.489056+0000","last_fullsized":"2023-10-18T10:47:17.489056+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:02:06.039754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"40'2","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489149+0000","last_change":"2023-10-18T10:47:14.850426+0000","last_active":"2023-10-18T10:47:17.489149+0000","last_peered":"2023-10-18T10:47:17.489149+0000","last_clean":"2023-10-18T10:47:17.489149+0000","last_became_active":"2023-10-18T10:47:14.849654+0000","last_became_peered":"2023-10-18T10:47:14.849654+0000","last_unstale":"2023-10-18T10:47:17.489149+0000","last_undegraded":"2023-10-18T10:47:17.489149+0000","last_fullsized":"2023-10-18T10:47:17.489149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:19:09.958118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.843655+0000","last_change":"2023-10-18T10:47:15.852762+0000","last_active":"2023-10-18T10:47:16.843655+0000","last_peered":"2023-10-18T10:47:16.843655+0000","last_clean":"2023-10-18T10:47:16.843655+0000","last_became_active":"2023-10-18T10:47:15.852460+0000","last_became_peered":"2023-10-18T10:47:15.852460+0000","last_unstale":"2023-10-18T10:47:16.843655+0000","last_undegraded":"2023-10-18T10:47:16.843655+0000","last_fullsized":"2023-10-18T10:47:16.843655+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:10:43.821057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"40'1","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.841656+0000","last_change":"2023-10-18T10:47:14.834346+0000","last_active":"2023-10-18T10:47:16.841656+0000","last_peered":"2023-10-18T10:47:16.841656+0000","last_clean":"2023-10-18T10:47:16.841656+0000","last_became_active":"2023-10-18T10:47:14.834098+0000","last_became_peered":"2023-10-18T10:47:14.834098+0000","last_unstale":"2023-10-18T10:47:16.841656+0000","last_undegraded":"2023-10-18T10:47:16.841656+0000","last_fullsized":"2023-10-18T10:47:16.841656+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:54:48.213290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488328+0000","last_change":"2023-10-18T10:47:15.852962+0000","last_active":"2023-10-18T10:47:17.488328+0000","last_peered":"2023-10-18T10:47:17.488328+0000","last_clean":"2023-10-18T10:47:17.488328+0000","last_became_active":"2023-10-18T10:47:15.852632+0000","last_became_peered":"2023-10-18T10:47:15.852632+0000","last_unstale":"2023-10-18T10:47:17.488328+0000","last_undegraded":"2023-10-18T10:47:17.488328+0000","last_fullsized":"2023-10-18T10:47:17.488328+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:00:02.907322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"40'1","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.843789+0000","last_change":"2023-10-18T10:47:14.846897+0000","last_active":"2023-10-18T10:47:16.843789+0000","last_peered":"2023-10-18T10:47:16.843789+0000","last_clean":"2023-10-18T10:47:16.843789+0000","last_became_active":"2023-10-18T10:47:14.846628+0000","last_became_peered":"2023-10-18T10:47:14.846628+0000","last_unstale":"2023-10-18T10:47:16.843789+0000","last_undegraded":"2023-10-18T10:47:16.843789+0000","last_fullsized":"2023-10-18T10:47:16.843789+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:26:38.444860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489121+0000","last_change":"2023-10-18T10:47:15.854526+0000","last_active":"2023-10-18T10:47:17.489121+0000","last_peered":"2023-10-18T10:47:17.489121+0000","last_clean":"2023-10-18T10:47:17.489121+0000","last_became_active":"2023-10-18T10:47:15.854257+0000","last_became_peered":"2023-10-18T10:47:15.854257+0000","last_unstale":"2023-10-18T10:47:17.489121+0000","last_undegraded":"2023-10-18T10:47:17.489121+0000","last_fullsized":"2023-10-18T10:47:17.489121+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:09:50.729532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488046+0000","last_change":"2023-10-18T10:47:14.848143+0000","last_active":"2023-10-18T10:47:17.488046+0000","last_peered":"2023-10-18T10:47:17.488046+0000","last_clean":"2023-10-18T10:47:17.488046+0000","last_became_active":"2023-10-18T10:47:14.847834+0000","last_became_peered":"2023-10-18T10:47:14.847834+0000","last_unstale":"2023-10-18T10:47:17.488046+0000","last_undegraded":"2023-10-18T10:47:17.488046+0000","last_fullsized":"2023-10-18T10:47:17.488046+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:35:07.639776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.843880+0000","last_change":"2023-10-18T10:47:15.848544+0000","last_active":"2023-10-18T10:47:16.843880+0000","last_peered":"2023-10-18T10:47:16.843880+0000","last_clean":"2023-10-18T10:47:16.843880+0000","last_became_active":"2023-10-18T10:47:15.848162+0000","last_became_peered":"2023-10-18T10:47:15.848162+0000","last_unstale":"2023-10-18T10:47:16.843880+0000","last_undegraded":"2023-10-18T10:47:16.843880+0000","last_fullsized":"2023-10-18T10:47:16.843880+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:32:48.986074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489241+0000","last_change":"2023-10-18T10:47:14.842407+0000","last_active":"2023-10-18T10:47:17.489241+0000","last_peered":"2023-10-18T10:47:17.489241+0000","last_clean":"2023-10-18T10:47:17.489241+0000","last_became_active":"2023-10-18T10:47:14.842152+0000","last_became_peered":"2023-10-18T10:47:14.842152+0000","last_unstale":"2023-10-18T10:47:17.489241+0000","last_undegraded":"2023-10-18T10:47:17.489241+0000","last_fullsized":"2023-10-18T10:47:17.489241+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:03:45.139428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.843793+0000","last_change":"2023-10-18T10:47:15.857610+0000","last_active":"2023-10-18T10:47:16.843793+0000","last_peered":"2023-10-18T10:47:16.843793+0000","last_clean":"2023-10-18T10:47:16.843793+0000","last_became_active":"2023-10-18T10:47:15.857308+0000","last_became_peered":"2023-10-18T10:47:15.857308+0000","last_unstale":"2023-10-18T10:47:16.843793+0000","last_undegraded":"2023-10-18T10:47:16.843793+0000","last_fullsized":"2023-10-18T10:47:16.843793+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:28:18.012270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.844519+0000","last_change":"2023-10-18T10:47:14.839653+0000","last_active":"2023-10-18T10:47:16.844519+0000","last_peered":"2023-10-18T10:47:16.844519+0000","last_clean":"2023-10-18T10:47:16.844519+0000","last_became_active":"2023-10-18T10:47:14.838492+0000","last_became_peered":"2023-10-18T10:47:14.838492+0000","last_unstale":"2023-10-18T10:47:16.844519+0000","last_undegraded":"2023-10-18T10:47:16.844519+0000","last_fullsized":"2023-10-18T10:47:16.844519+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:45:04.943284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489235+0000","last_change":"2023-10-18T10:47:15.843928+0000","last_active":"2023-10-18T10:47:17.489235+0000","last_peered":"2023-10-18T10:47:17.489235+0000","last_clean":"2023-10-18T10:47:17.489235+0000","last_became_active":"2023-10-18T10:47:15.842875+0000","last_became_peered":"2023-10-18T10:47:15.842875+0000","last_unstale":"2023-10-18T10:47:17.489235+0000","last_undegraded":"2023-10-18T10:47:17.489235+0000","last_fullsized":"2023-10-18T10:47:17.489235+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:03.621992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.490020+0000","last_change":"2023-10-18T10:47:14.844999+0000","last_active":"2023-10-18T10:47:17.490020+0000","last_peered":"2023-10-18T10:47:17.490020+0000","last_clean":"2023-10-18T10:47:17.490020+0000","last_became_active":"2023-10-18T10:47:14.844746+0000","last_became_peered":"2023-10-18T10:47:14.844746+0000","last_unstale":"2023-10-18T10:47:17.490020+0000","last_undegraded":"2023-10-18T10:47:17.490020+0000","last_fullsized":"2023-10-18T10:47:17.490020+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:33:17.479207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.841938+0000","last_change":"2023-10-18T10:47:15.859400+0000","last_active":"2023-10-18T10:47:16.841938+0000","last_peered":"2023-10-18T10:47:16.841938+0000","last_clean":"2023-10-18T10:47:16.841938+0000","last_became_active":"2023-10-18T10:47:15.859044+0000","last_became_peered":"2023-10-18T10:47:15.859044+0000","last_unstale":"2023-10-18T10:47:16.841938+0000","last_undegraded":"2023-10-18T10:47:16.841938+0000","last_fullsized":"2023-10-18T10:47:16.841938+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:00:28.228199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"40'3","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489251+0000","last_change":"2023-10-18T10:47:14.845856+0000","last_active":"2023-10-18T10:47:17.489251+0000","last_peered":"2023-10-18T10:47:17.489251+0000","last_clean":"2023-10-18T10:47:17.489251+0000","last_became_active":"2023-10-18T10:47:14.845575+0000","last_became_peered":"2023-10-18T10:47:14.845575+0000","last_unstale":"2023-10-18T10:47:17.489251+0000","last_undegraded":"2023-10-18T10:47:17.489251+0000","last_fullsized":"2023-10-18T10:47:17.489251+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:33:10.378488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488845+0000","last_change":"2023-10-18T10:47:15.843972+0000","last_active":"2023-10-18T10:47:17.488845+0000","last_peered":"2023-10-18T10:47:17.488845+0000","last_clean":"2023-10-18T10:47:17.488845+0000","last_became_active":"2023-10-18T10:47:15.843649+0000","last_became_peered":"2023-10-18T10:47:15.843649+0000","last_unstale":"2023-10-18T10:47:17.488845+0000","last_undegraded":"2023-10-18T10:47:17.488845+0000","last_fullsized":"2023-10-18T10:47:17.488845+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:32:29.921799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"40'6","reported_seq":19,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489745+0000","last_change":"2023-10-18T10:47:14.840158+0000","last_active":"2023-10-18T10:47:17.489745+0000","last_peered":"2023-10-18T10:47:17.489745+0000","last_clean":"2023-10-18T10:47:17.489745+0000","last_became_active":"2023-10-18T10:47:14.839881+0000","last_became_peered":"2023-10-18T10:47:14.839881+0000","last_unstale":"2023-10-18T10:47:17.489745+0000","last_undegraded":"2023-10-18T10:47:17.489745+0000","last_fullsized":"2023-10-18T10:47:17.489745+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:38:22.391824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.844871+0000","last_change":"2023-10-18T10:47:15.861260+0000","last_active":"2023-10-18T10:47:16.844871+0000","last_peered":"2023-10-18T10:47:16.844871+0000","last_clean":"2023-10-18T10:47:16.844871+0000","last_became_active":"2023-10-18T10:47:15.860958+0000","last_became_peered":"2023-10-18T10:47:15.860958+0000","last_unstale":"2023-10-18T10:47:16.844871+0000","last_undegraded":"2023-10-18T10:47:16.844871+0000","last_fullsized":"2023-10-18T10:47:16.844871+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:38:24.521066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"40'2","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.841721+0000","last_change":"2023-10-18T10:47:14.840478+0000","last_active":"2023-10-18T10:47:16.841721+0000","last_peered":"2023-10-18T10:47:16.841721+0000","last_clean":"2023-10-18T10:47:16.841721+0000","last_became_active":"2023-10-18T10:47:14.839906+0000","last_became_peered":"2023-10-18T10:47:14.839906+0000","last_unstale":"2023-10-18T10:47:16.841721+0000","last_undegraded":"2023-10-18T10:47:16.841721+0000","last_fullsized":"2023-10-18T10:47:16.841721+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:23:49.688625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488388+0000","last_change":"2023-10-18T10:47:15.850243+0000","last_active":"2023-10-18T10:47:17.488388+0000","last_peered":"2023-10-18T10:47:17.488388+0000","last_clean":"2023-10-18T10:47:17.488388+0000","last_became_active":"2023-10-18T10:47:15.849925+0000","last_became_peered":"2023-10-18T10:47:15.849925+0000","last_unstale":"2023-10-18T10:47:17.488388+0000","last_undegraded":"2023-10-18T10:47:17.488388+0000","last_fullsized":"2023-10-18T10:47:17.488388+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:28:54.785138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489170+0000","last_change":"2023-10-18T10:47:14.841945+0000","last_active":"2023-10-18T10:47:17.489170+0000","last_peered":"2023-10-18T10:47:17.489170+0000","last_clean":"2023-10-18T10:47:17.489170+0000","last_became_active":"2023-10-18T10:47:14.841614+0000","last_became_peered":"2023-10-18T10:47:14.841614+0000","last_unstale":"2023-10-18T10:47:17.489170+0000","last_undegraded":"2023-10-18T10:47:17.489170+0000","last_fullsized":"2023-10-18T10:47:17.489170+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:34:54.832263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.841320+0000","last_change":"2023-10-18T10:47:15.852810+0000","last_active":"2023-10-18T10:47:16.841320+0000","last_peered":"2023-10-18T10:47:16.841320+0000","last_clean":"2023-10-18T10:47:16.841320+0000","last_became_active":"2023-10-18T10:47:15.852509+0000","last_became_peered":"2023-10-18T10:47:15.852509+0000","last_unstale":"2023-10-18T10:47:16.841320+0000","last_undegraded":"2023-10-18T10:47:16.841320+0000","last_fullsized":"2023-10-18T10:47:16.841320+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:09:50.729532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.841518+0000","last_change":"2023-10-18T10:47:14.845377+0000","last_active":"2023-10-18T10:47:16.841518+0000","last_peered":"2023-10-18T10:47:16.841518+0000","last_clean":"2023-10-18T10:47:16.841518+0000","last_became_active":"2023-10-18T10:47:14.845102+0000","last_became_peered":"2023-10-18T10:47:14.845102+0000","last_unstale":"2023-10-18T10:47:16.841518+0000","last_undegraded":"2023-10-18T10:47:16.841518+0000","last_fullsized":"2023-10-18T10:47:16.841518+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:18:36.370134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489451+0000","last_change":"2023-10-18T10:47:15.849181+0000","last_active":"2023-10-18T10:47:17.489451+0000","last_peered":"2023-10-18T10:47:17.489451+0000","last_clean":"2023-10-18T10:47:17.489451+0000","last_became_active":"2023-10-18T10:47:15.848694+0000","last_became_peered":"2023-10-18T10:47:15.848694+0000","last_unstale":"2023-10-18T10:47:17.489451+0000","last_undegraded":"2023-10-18T10:47:17.489451+0000","last_fullsized":"2023-10-18T10:47:17.489451+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:04:44.835305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"40'8","reported_seq":21,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489254+0000","last_change":"2023-10-18T10:47:14.850050+0000","last_active":"2023-10-18T10:47:17.489254+0000","last_peered":"2023-10-18T10:47:17.489254+0000","last_clean":"2023-10-18T10:47:17.489254+0000","last_became_active":"2023-10-18T10:47:14.849205+0000","last_became_peered":"2023-10-18T10:47:14.849205+0000","last_unstale":"2023-10-18T10:47:17.489254+0000","last_undegraded":"2023-10-18T10:47:17.489254+0000","last_fullsized":"2023-10-18T10:47:17.489254+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:26:15.308720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489838+0000","last_change":"2023-10-18T10:47:15.849087+0000","last_active":"2023-10-18T10:47:17.489838+0000","last_peered":"2023-10-18T10:47:17.489838+0000","last_clean":"2023-10-18T10:47:17.489838+0000","last_became_active":"2023-10-18T10:47:15.848606+0000","last_became_peered":"2023-10-18T10:47:15.848606+0000","last_unstale":"2023-10-18T10:47:17.489838+0000","last_undegraded":"2023-10-18T10:47:17.489838+0000","last_fullsized":"2023-10-18T10:47:17.489838+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:20:24.951086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"40'1","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489591+0000","last_change":"2023-10-18T10:47:14.842314+0000","last_active":"2023-10-18T10:47:17.489591+0000","last_peered":"2023-10-18T10:47:17.489591+0000","last_clean":"2023-10-18T10:47:17.489591+0000","last_became_active":"2023-10-18T10:47:14.841832+0000","last_became_peered":"2023-10-18T10:47:14.841832+0000","last_unstale":"2023-10-18T10:47:17.489591+0000","last_undegraded":"2023-10-18T10:47:17.489591+0000","last_fullsized":"2023-10-18T10:47:17.489591+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:28:31.383054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.490156+0000","last_change":"2023-10-18T10:47:15.866629+0000","last_active":"2023-10-18T10:47:17.490156+0000","last_peered":"2023-10-18T10:47:17.490156+0000","last_clean":"2023-10-18T10:47:17.490156+0000","last_became_active":"2023-10-18T10:47:15.866353+0000","last_became_peered":"2023-10-18T10:47:15.866353+0000","last_unstale":"2023-10-18T10:47:17.490156+0000","last_undegraded":"2023-10-18T10:47:17.490156+0000","last_fullsized":"2023-10-18T10:47:17.490156+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:23:50.695270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"40'2","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.844067+0000","last_change":"2023-10-18T10:47:14.843544+0000","last_active":"2023-10-18T10:47:16.844067+0000","last_peered":"2023-10-18T10:47:16.844067+0000","last_clean":"2023-10-18T10:47:16.844067+0000","last_became_active":"2023-10-18T10:47:14.843095+0000","last_became_peered":"2023-10-18T10:47:14.843095+0000","last_unstale":"2023-10-18T10:47:16.844067+0000","last_undegraded":"2023-10-18T10:47:16.844067+0000","last_fullsized":"2023-10-18T10:47:16.844067+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:04:07.055793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.844032+0000","last_change":"2023-10-18T10:47:15.853760+0000","last_active":"2023-10-18T10:47:16.844032+0000","last_peered":"2023-10-18T10:47:16.844032+0000","last_clean":"2023-10-18T10:47:16.844032+0000","last_became_active":"2023-10-18T10:47:15.853466+0000","last_became_peered":"2023-10-18T10:47:15.853466+0000","last_unstale":"2023-10-18T10:47:16.844032+0000","last_undegraded":"2023-10-18T10:47:16.844032+0000","last_fullsized":"2023-10-18T10:47:16.844032+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:37:35.177266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"40'1","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489207+0000","last_change":"2023-10-18T10:47:14.847139+0000","last_active":"2023-10-18T10:47:17.489207+0000","last_peered":"2023-10-18T10:47:17.489207+0000","last_clean":"2023-10-18T10:47:17.489207+0000","last_became_active":"2023-10-18T10:47:14.846845+0000","last_became_peered":"2023-10-18T10:47:14.846845+0000","last_unstale":"2023-10-18T10:47:17.489207+0000","last_undegraded":"2023-10-18T10:47:17.489207+0000","last_fullsized":"2023-10-18T10:47:17.489207+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:43:52.520253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.844454+0000","last_change":"2023-10-18T10:47:15.845736+0000","last_active":"2023-10-18T10:47:16.844454+0000","last_peered":"2023-10-18T10:47:16.844454+0000","last_clean":"2023-10-18T10:47:16.844454+0000","last_became_active":"2023-10-18T10:47:15.844036+0000","last_became_peered":"2023-10-18T10:47:15.844036+0000","last_unstale":"2023-10-18T10:47:16.844454+0000","last_undegraded":"2023-10-18T10:47:16.844454+0000","last_fullsized":"2023-10-18T10:47:16.844454+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:44:06.107284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.841959+0000","last_change":"2023-10-18T10:47:14.833953+0000","last_active":"2023-10-18T10:47:16.841959+0000","last_peered":"2023-10-18T10:47:16.841959+0000","last_clean":"2023-10-18T10:47:16.841959+0000","last_became_active":"2023-10-18T10:47:14.833512+0000","last_became_peered":"2023-10-18T10:47:14.833512+0000","last_unstale":"2023-10-18T10:47:16.841959+0000","last_undegraded":"2023-10-18T10:47:16.841959+0000","last_fullsized":"2023-10-18T10:47:16.841959+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:24:47.792031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.841466+0000","last_change":"2023-10-18T10:47:15.858560+0000","last_active":"2023-10-18T10:47:16.841466+0000","last_peered":"2023-10-18T10:47:16.841466+0000","last_clean":"2023-10-18T10:47:16.841466+0000","last_became_active":"2023-10-18T10:47:15.858325+0000","last_became_peered":"2023-10-18T10:47:15.858325+0000","last_unstale":"2023-10-18T10:47:16.841466+0000","last_undegraded":"2023-10-18T10:47:16.841466+0000","last_fullsized":"2023-10-18T10:47:16.841466+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:26:16.315364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"40'2","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.841818+0000","last_change":"2023-10-18T10:47:14.834264+0000","last_active":"2023-10-18T10:47:16.841818+0000","last_peered":"2023-10-18T10:47:16.841818+0000","last_clean":"2023-10-18T10:47:16.841818+0000","last_became_active":"2023-10-18T10:47:14.833996+0000","last_became_peered":"2023-10-18T10:47:14.833996+0000","last_unstale":"2023-10-18T10:47:16.841818+0000","last_undegraded":"2023-10-18T10:47:16.841818+0000","last_fullsized":"2023-10-18T10:47:16.841818+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:15:55.232241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489347+0000","last_change":"2023-10-18T10:47:15.845011+0000","last_active":"2023-10-18T10:47:17.489347+0000","last_peered":"2023-10-18T10:47:17.489347+0000","last_clean":"2023-10-18T10:47:17.489347+0000","last_became_active":"2023-10-18T10:47:15.844475+0000","last_became_peered":"2023-10-18T10:47:15.844475+0000","last_unstale":"2023-10-18T10:47:17.489347+0000","last_undegraded":"2023-10-18T10:47:17.489347+0000","last_fullsized":"2023-10-18T10:47:17.489347+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:03:40.394860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488360+0000","last_change":"2023-10-18T10:47:14.848894+0000","last_active":"2023-10-18T10:47:17.488360+0000","last_peered":"2023-10-18T10:47:17.488360+0000","last_clean":"2023-10-18T10:47:17.488360+0000","last_became_active":"2023-10-18T10:47:14.848556+0000","last_became_peered":"2023-10-18T10:47:14.848556+0000","last_unstale":"2023-10-18T10:47:17.488360+0000","last_undegraded":"2023-10-18T10:47:17.488360+0000","last_fullsized":"2023-10-18T10:47:17.488360+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:00:46.711890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488321+0000","last_change":"2023-10-18T10:47:15.848304+0000","last_active":"2023-10-18T10:47:17.488321+0000","last_peered":"2023-10-18T10:47:17.488321+0000","last_clean":"2023-10-18T10:47:17.488321+0000","last_became_active":"2023-10-18T10:47:15.847980+0000","last_became_peered":"2023-10-18T10:47:15.847980+0000","last_unstale":"2023-10-18T10:47:17.488321+0000","last_undegraded":"2023-10-18T10:47:17.488321+0000","last_fullsized":"2023-10-18T10:47:17.488321+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:07:39.705667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488534+0000","last_change":"2023-10-18T10:47:14.838905+0000","last_active":"2023-10-18T10:47:17.488534+0000","last_peered":"2023-10-18T10:47:17.488534+0000","last_clean":"2023-10-18T10:47:17.488534+0000","last_became_active":"2023-10-18T10:47:14.838537+0000","last_became_peered":"2023-10-18T10:47:14.838537+0000","last_unstale":"2023-10-18T10:47:17.488534+0000","last_undegraded":"2023-10-18T10:47:17.488534+0000","last_fullsized":"2023-10-18T10:47:17.488534+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:16:34.301296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488332+0000","last_change":"2023-10-18T10:47:15.853190+0000","last_active":"2023-10-18T10:47:17.488332+0000","last_peered":"2023-10-18T10:47:17.488332+0000","last_clean":"2023-10-18T10:47:17.488332+0000","last_became_active":"2023-10-18T10:47:15.852895+0000","last_became_peered":"2023-10-18T10:47:15.852895+0000","last_unstale":"2023-10-18T10:47:17.488332+0000","last_undegraded":"2023-10-18T10:47:17.488332+0000","last_fullsized":"2023-10-18T10:47:17.488332+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:29:24.304875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488188+0000","last_change":"2023-10-18T10:47:14.841944+0000","last_active":"2023-10-18T10:47:17.488188+0000","last_peered":"2023-10-18T10:47:17.488188+0000","last_clean":"2023-10-18T10:47:17.488188+0000","last_became_active":"2023-10-18T10:47:14.841623+0000","last_became_peered":"2023-10-18T10:47:14.841623+0000","last_unstale":"2023-10-18T10:47:17.488188+0000","last_undegraded":"2023-10-18T10:47:17.488188+0000","last_fullsized":"2023-10-18T10:47:17.488188+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:00:27.221555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.841493+0000","last_change":"2023-10-18T10:47:15.841976+0000","last_active":"2023-10-18T10:47:16.841493+0000","last_peered":"2023-10-18T10:47:16.841493+0000","last_clean":"2023-10-18T10:47:16.841493+0000","last_became_active":"2023-10-18T10:47:15.841713+0000","last_became_peered":"2023-10-18T10:47:15.841713+0000","last_unstale":"2023-10-18T10:47:16.841493+0000","last_undegraded":"2023-10-18T10:47:16.841493+0000","last_fullsized":"2023-10-18T10:47:16.841493+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:33:49.953156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488920+0000","last_change":"2023-10-18T10:47:14.847120+0000","last_active":"2023-10-18T10:47:17.488920+0000","last_peered":"2023-10-18T10:47:17.488920+0000","last_clean":"2023-10-18T10:47:17.488920+0000","last_became_active":"2023-10-18T10:47:14.846008+0000","last_became_peered":"2023-10-18T10:47:14.846008+0000","last_unstale":"2023-10-18T10:47:17.488920+0000","last_undegraded":"2023-10-18T10:47:17.488920+0000","last_fullsized":"2023-10-18T10:47:17.488920+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:21:09.552522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.845037+0000","last_change":"2023-10-18T10:45:50.252021+0000","last_active":"2023-10-18T10:47:16.845037+0000","last_peered":"2023-10-18T10:47:16.845037+0000","last_clean":"2023-10-18T10:47:16.845037+0000","last_became_active":"2023-10-18T10:45:50.251427+0000","last_became_peered":"2023-10-18T10:45:50.251427+0000","last_unstale":"2023-10-18T10:47:16.845037+0000","last_undegraded":"2023-10-18T10:47:16.845037+0000","last_fullsized":"2023-10-18T10:47:16.845037+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:57:23.835848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.841677+0000","last_change":"2023-10-18T10:47:15.859453+0000","last_active":"2023-10-18T10:47:16.841677+0000","last_peered":"2023-10-18T10:47:16.841677+0000","last_clean":"2023-10-18T10:47:16.841677+0000","last_became_active":"2023-10-18T10:47:15.858668+0000","last_became_peered":"2023-10-18T10:47:15.858668+0000","last_unstale":"2023-10-18T10:47:16.841677+0000","last_undegraded":"2023-10-18T10:47:16.841677+0000","last_fullsized":"2023-10-18T10:47:16.841677+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:33:18.485852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.843909+0000","last_change":"2023-10-18T10:47:14.841091+0000","last_active":"2023-10-18T10:47:16.843909+0000","last_peered":"2023-10-18T10:47:16.843909+0000","last_clean":"2023-10-18T10:47:16.843909+0000","last_became_active":"2023-10-18T10:47:14.840807+0000","last_became_peered":"2023-10-18T10:47:14.840807+0000","last_unstale":"2023-10-18T10:47:16.843909+0000","last_undegraded":"2023-10-18T10:47:16.843909+0000","last_fullsized":"2023-10-18T10:47:16.843909+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:43:51.292223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":47,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489287+0000","last_change":"2023-10-18T10:45:50.252876+0000","last_active":"2023-10-18T10:47:17.489287+0000","last_peered":"2023-10-18T10:47:17.489287+0000","last_clean":"2023-10-18T10:47:17.489287+0000","last_became_active":"2023-10-18T10:45:50.252559+0000","last_became_peered":"2023-10-18T10:45:50.252559+0000","last_unstale":"2023-10-18T10:47:17.489287+0000","last_undegraded":"2023-10-18T10:47:17.489287+0000","last_fullsized":"2023-10-18T10:47:17.489287+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:43:05.360433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488017+0000","last_change":"2023-10-18T10:47:15.859101+0000","last_active":"2023-10-18T10:47:17.488017+0000","last_peered":"2023-10-18T10:47:17.488017+0000","last_clean":"2023-10-18T10:47:17.488017+0000","last_became_active":"2023-10-18T10:47:15.857002+0000","last_became_peered":"2023-10-18T10:47:15.857002+0000","last_unstale":"2023-10-18T10:47:17.488017+0000","last_undegraded":"2023-10-18T10:47:17.488017+0000","last_fullsized":"2023-10-18T10:47:17.488017+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:13:35.452997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488307+0000","last_change":"2023-10-18T10:47:14.842188+0000","last_active":"2023-10-18T10:47:17.488307+0000","last_peered":"2023-10-18T10:47:17.488307+0000","last_clean":"2023-10-18T10:47:17.488307+0000","last_became_active":"2023-10-18T10:47:14.841932+0000","last_became_peered":"2023-10-18T10:47:14.841932+0000","last_unstale":"2023-10-18T10:47:17.488307+0000","last_undegraded":"2023-10-18T10:47:17.488307+0000","last_fullsized":"2023-10-18T10:47:17.488307+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:50:12.229901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489285+0000","last_change":"2023-10-18T10:45:50.585224+0000","last_active":"2023-10-18T10:47:17.489285+0000","last_peered":"2023-10-18T10:47:17.489285+0000","last_clean":"2023-10-18T10:47:17.489285+0000","last_became_active":"2023-10-18T10:45:50.584872+0000","last_became_peered":"2023-10-18T10:45:50.584872+0000","last_unstale":"2023-10-18T10:47:17.489285+0000","last_undegraded":"2023-10-18T10:47:17.489285+0000","last_fullsized":"2023-10-18T10:47:17.489285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:20:47.535961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.843582+0000","last_change":"2023-10-18T10:47:15.842480+0000","last_active":"2023-10-18T10:47:16.843582+0000","last_peered":"2023-10-18T10:47:16.843582+0000","last_clean":"2023-10-18T10:47:16.843582+0000","last_became_active":"2023-10-18T10:47:15.842161+0000","last_became_peered":"2023-10-18T10:47:15.842161+0000","last_unstale":"2023-10-18T10:47:16.843582+0000","last_undegraded":"2023-10-18T10:47:16.843582+0000","last_fullsized":"2023-10-18T10:47:16.843582+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:54:43.295790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489501+0000","last_change":"2023-10-18T10:47:14.841126+0000","last_active":"2023-10-18T10:47:17.489501+0000","last_peered":"2023-10-18T10:47:17.489501+0000","last_clean":"2023-10-18T10:47:17.489501+0000","last_became_active":"2023-10-18T10:47:14.840452+0000","last_became_peered":"2023-10-18T10:47:14.840452+0000","last_unstale":"2023-10-18T10:47:17.489501+0000","last_undegraded":"2023-10-18T10:47:17.489501+0000","last_fullsized":"2023-10-18T10:47:17.489501+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:33:48.946511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488874+0000","last_change":"2023-10-18T10:45:50.584946+0000","last_active":"2023-10-18T10:47:17.488874+0000","last_peered":"2023-10-18T10:47:17.488874+0000","last_clean":"2023-10-18T10:47:17.488874+0000","last_became_active":"2023-10-18T10:45:50.584638+0000","last_became_peered":"2023-10-18T10:45:50.584638+0000","last_unstale":"2023-10-18T10:47:17.488874+0000","last_undegraded":"2023-10-18T10:47:17.488874+0000","last_fullsized":"2023-10-18T10:47:17.488874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:35:48.515328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.844976+0000","last_change":"2023-10-18T10:47:15.855056+0000","last_active":"2023-10-18T10:47:16.844976+0000","last_peered":"2023-10-18T10:47:16.844976+0000","last_clean":"2023-10-18T10:47:16.844976+0000","last_became_active":"2023-10-18T10:47:15.854157+0000","last_became_peered":"2023-10-18T10:47:15.854157+0000","last_unstale":"2023-10-18T10:47:16.844976+0000","last_undegraded":"2023-10-18T10:47:16.844976+0000","last_fullsized":"2023-10-18T10:47:16.844976+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:11:14.235733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:16.842031+0000","last_change":"2023-10-18T10:47:14.838883+0000","last_active":"2023-10-18T10:47:16.842031+0000","last_peered":"2023-10-18T10:47:16.842031+0000","last_clean":"2023-10-18T10:47:16.842031+0000","last_became_active":"2023-10-18T10:47:14.838464+0000","last_became_peered":"2023-10-18T10:47:14.838464+0000","last_unstale":"2023-10-18T10:47:16.842031+0000","last_undegraded":"2023-10-18T10:47:16.842031+0000","last_fullsized":"2023-10-18T10:47:16.842031+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:32:28.915154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488442+0000","last_change":"2023-10-18T10:45:50.251505+0000","last_active":"2023-10-18T10:47:17.488442+0000","last_peered":"2023-10-18T10:47:17.488442+0000","last_clean":"2023-10-18T10:47:17.488442+0000","last_became_active":"2023-10-18T10:45:50.251111+0000","last_became_peered":"2023-10-18T10:45:50.251111+0000","last_unstale":"2023-10-18T10:47:17.488442+0000","last_undegraded":"2023-10-18T10:47:17.488442+0000","last_fullsized":"2023-10-18T10:47:17.488442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:08:25.147676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489060+0000","last_change":"2023-10-18T10:47:15.844637+0000","last_active":"2023-10-18T10:47:17.489060+0000","last_peered":"2023-10-18T10:47:17.489060+0000","last_clean":"2023-10-18T10:47:17.489060+0000","last_became_active":"2023-10-18T10:47:15.844372+0000","last_became_peered":"2023-10-18T10:47:15.844372+0000","last_unstale":"2023-10-18T10:47:17.489060+0000","last_undegraded":"2023-10-18T10:47:17.489060+0000","last_fullsized":"2023-10-18T10:47:17.489060+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:00:47.718535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489221+0000","last_change":"2023-10-18T10:47:14.850005+0000","last_active":"2023-10-18T10:47:17.489221+0000","last_peered":"2023-10-18T10:47:17.489221+0000","last_clean":"2023-10-18T10:47:17.489221+0000","last_became_active":"2023-10-18T10:47:14.849708+0000","last_became_peered":"2023-10-18T10:47:14.849708+0000","last_unstale":"2023-10-18T10:47:17.489221+0000","last_undegraded":"2023-10-18T10:47:17.489221+0000","last_fullsized":"2023-10-18T10:47:17.489221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:23:03.852551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488004+0000","last_change":"2023-10-18T10:45:50.259019+0000","last_active":"2023-10-18T10:47:17.488004+0000","last_peered":"2023-10-18T10:47:17.488004+0000","last_clean":"2023-10-18T10:47:17.488004+0000","last_became_active":"2023-10-18T10:45:50.258607+0000","last_became_peered":"2023-10-18T10:45:50.258607+0000","last_unstale":"2023-10-18T10:47:17.488004+0000","last_undegraded":"2023-10-18T10:47:17.488004+0000","last_fullsized":"2023-10-18T10:47:17.488004+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:20:38.830994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489088+0000","last_change":"2023-10-18T10:47:15.846658+0000","last_active":"2023-10-18T10:47:17.489088+0000","last_peered":"2023-10-18T10:47:17.489088+0000","last_clean":"2023-10-18T10:47:17.489088+0000","last_became_active":"2023-10-18T10:47:15.846173+0000","last_became_peered":"2023-10-18T10:47:15.846173+0000","last_unstale":"2023-10-18T10:47:17.489088+0000","last_undegraded":"2023-10-18T10:47:17.489088+0000","last_fullsized":"2023-10-18T10:47:17.489088+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:56:56.255147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489060+0000","last_change":"2023-10-18T10:47:14.849782+0000","last_active":"2023-10-18T10:47:17.489060+0000","last_peered":"2023-10-18T10:47:17.489060+0000","last_clean":"2023-10-18T10:47:17.489060+0000","last_became_active":"2023-10-18T10:47:14.849493+0000","last_became_peered":"2023-10-18T10:47:14.849493+0000","last_unstale":"2023-10-18T10:47:17.489060+0000","last_undegraded":"2023-10-18T10:47:17.489060+0000","last_fullsized":"2023-10-18T10:47:17.489060+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:56:59.839610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488131+0000","last_change":"2023-10-18T10:45:50.251375+0000","last_active":"2023-10-18T10:47:17.488131+0000","last_peered":"2023-10-18T10:47:17.488131+0000","last_clean":"2023-10-18T10:47:17.488131+0000","last_became_active":"2023-10-18T10:45:50.251055+0000","last_became_peered":"2023-10-18T10:45:50.251055+0000","last_unstale":"2023-10-18T10:47:17.488131+0000","last_undegraded":"2023-10-18T10:47:17.488131+0000","last_fullsized":"2023-10-18T10:47:17.488131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:14:30.657029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.489985+0000","last_change":"2023-10-18T10:47:15.855868+0000","last_active":"2023-10-18T10:47:17.489985+0000","last_peered":"2023-10-18T10:47:17.489985+0000","last_clean":"2023-10-18T10:47:17.489985+0000","last_became_active":"2023-10-18T10:47:15.852068+0000","last_became_peered":"2023-10-18T10:47:15.852068+0000","last_unstale":"2023-10-18T10:47:17.489985+0000","last_undegraded":"2023-10-18T10:47:17.489985+0000","last_fullsized":"2023-10-18T10:47:17.489985+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:14.809028+0000","last_clean_scrub_stamp":"2023-10-18T10:47:14.809028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:43:53.526897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"40'2","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488784+0000","last_change":"2023-10-18T10:47:14.849861+0000","last_active":"2023-10-18T10:47:17.488784+0000","last_peered":"2023-10-18T10:47:17.488784+0000","last_clean":"2023-10-18T10:47:17.488784+0000","last_became_active":"2023-10-18T10:47:14.849100+0000","last_became_peered":"2023-10-18T10:47:14.849100+0000","last_unstale":"2023-10-18T10:47:17.488784+0000","last_undegraded":"2023-10-18T10:47:17.488784+0000","last_fullsized":"2023-10-18T10:47:17.488784+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:47:13.802384+0000","last_clean_scrub_stamp":"2023-10-18T10:47:13.802384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:18:36.370134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488491+0000","last_change":"2023-10-18T10:45:50.251310+0000","last_active":"2023-10-18T10:47:17.488491+0000","last_peered":"2023-10-18T10:47:17.488491+0000","last_clean":"2023-10-18T10:47:17.488491+0000","last_became_active":"2023-10-18T10:45:50.250823+0000","last_became_peered":"2023-10-18T10:45:50.250823+0000","last_unstale":"2023-10-18T10:47:17.488491+0000","last_undegraded":"2023-10-18T10:47:17.488491+0000","last_fullsized":"2023-10-18T10:47:17.488491+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:42:25.083142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":205,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-18T10:47:17.488272+0000","last_change":"2023-10-18T10:45:44.563531+0000","last_active":"2023-10-18T10:47:17.488272+0000","last_peered":"2023-10-18T10:47:17.488272+0000","last_clean":"2023-10-18T10:47:17.488272+0000","last_became_active":"2023-10-18T10:45:44.562504+0000","last_became_peered":"2023-10-18T10:45:44.562504+0000","last_unstale":"2023-10-18T10:47:17.488272+0000","last_undegraded":"2023-10-18T10:47:17.488272+0000","last_fullsized":"2023-10-18T10:47:17.488272+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:43.181461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:43.181461+0000","last_clean_scrub_stamp":"2023-10-18T10:45:43.181461+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:23:17.171109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343780,"statfs":{"total":96636764160,"available":96608030720,"internally_reserved":0,"allocated":1060864,"data_stored":796273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:46:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.72599999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:46:43 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.76800000000000002}]},{"osd":2,"last update":"Wed Oct 18 10:46:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.69299999999999995}]},{"osd":3,"last update":"Wed Oct 18 10:46:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.749}]},{"osd":4,"last update":"Wed Oct 18 10:46:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.501}]},{"osd":5,"last update":"Wed Oct 18 10:46:43 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.63300000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:46:43 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.56999999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":483328,"data_stored":213460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.89700000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.754}]},{"osd":2,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.1469999999999998,"5min":2.1469999999999998,"15min":2.1469999999999998},"last":0.91900000000000004}]},{"osd":3,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.57899999999999996}]},{"osd":4,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.72999999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.84299999999999997}]},{"osd":7,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":1.026},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.95899999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":466944,"data_stored":205885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.71799999999999997}]},{"osd":1,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.63900000000000001}]},{"osd":2,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.76200000000000001}]},{"osd":3,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":5.0970000000000004,"5min":5.0970000000000004,"15min":5.0970000000000004},"last":0.79200000000000004}]},{"osd":4,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.505}]},{"osd":6,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.441}]},{"osd":7,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.67600000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":487424,"data_stored":214400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:46:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.61899999999999999}]},{"osd":1,"last update":"Wed Oct 18 10:46:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.853,"5min":1.853,"15min":1.853},"last":0.65600000000000003}]},{"osd":2,"last update":"Wed Oct 18 10:46:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":0.748}]},{"osd":3,"last update":"Wed Oct 18 10:46:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.879}]},{"osd":5,"last update":"Wed Oct 18 10:46:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.77500000000000002}]},{"osd":6,"last update":"Wed Oct 18 10:46:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":0.90400000000000003}]},{"osd":7,"last update":"Wed Oct 18 10:46:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.95999999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":487424,"data_stored":213426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.69699999999999995}]},{"osd":1,"last update":"Wed Oct 18 10:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.52900000000000003}]},{"osd":2,"last update":"Wed Oct 18 10:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.57499999999999996}]},{"osd":4,"last update":"Wed Oct 18 10:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":1.0069999999999999}]},{"osd":5,"last update":"Wed Oct 18 10:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.91500000000000004}]},{"osd":6,"last update":"Wed Oct 18 10:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":1.046},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":1.0269999999999999}]},{"osd":7,"last update":"Wed Oct 18 10:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.95899999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":479232,"data_stored":207449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.53400000000000003}]},{"osd":1,"last update":"Wed Oct 18 10:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.40000000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.45300000000000001}]},{"osd":4,"last update":"Wed Oct 18 10:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.72999999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.73999999999999999}]},{"osd":6,"last update":"Wed Oct 18 10:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.84699999999999998}]},{"osd":7,"last update":"Wed Oct 18 10:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.875},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.754}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":466944,"data_stored":205885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.375},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.47899999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.41899999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.503},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.45700000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.71799999999999997}]},{"osd":5,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.9850000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":0.67200000000000004}]},{"osd":6,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.65900000000000003}]},{"osd":7,"last update":"Wed Oct 18 10:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.57399999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343780,"statfs":{"total":96636764160,"available":96608030720,"internally_reserved":0,"allocated":1060864,"data_stored":796217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:46:45 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":9.0410000000000004,"5min":9.0410000000000004,"15min":9.0410000000000004},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":9.1349999999999998,"5min":9.1349999999999998,"15min":9.1349999999999998},"last":0.40600000000000003}]},{"osd":2,"last update":"Wed Oct 18 10:46:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":5.3150000000000004,"5min":5.3150000000000004,"15min":5.3150000000000004},"last":0.48999999999999999},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":9.0980000000000008,"5min":9.0980000000000008,"15min":9.0980000000000008},"last":0.47199999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:46:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":8.9700000000000006,"5min":8.9700000000000006,"15min":8.9700000000000006},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":5.7210000000000001,"5min":5.7210000000000001,"15min":5.7210000000000001},"last":0.58699999999999997}]},{"osd":4,"last update":"Wed Oct 18 10:46:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":5.625,"5min":5.625,"15min":5.625},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":5.8040000000000003,"5min":5.8040000000000003,"15min":5.8040000000000003},"last":0.84299999999999997}]},{"osd":5,"last update":"Wed Oct 18 10:46:45 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":5.6790000000000003,"5min":5.6790000000000003,"15min":5.6790000000000003},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.149,"5min":1.149,"15min":1.149},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":5.5739999999999998,"5min":5.5739999999999998,"15min":5.5739999999999998},"last":0.91400000000000003}]},{"osd":6,"last update":"Wed Oct 18 10:46:45 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":5.6559999999999997,"5min":5.6559999999999997,"15min":5.6559999999999997},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":5.7610000000000001,"5min":5.7610000000000001,"15min":5.7610000000000001},"last":0.82799999999999996}]},{"osd":7,"last update":"Wed Oct 18 10:46:45 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":5.782,"5min":5.782,"15min":5.782},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":5.6109999999999998,"5min":5.6109999999999998,"15min":5.6109999999999998},"last":0.95599999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:47:20.337 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-18T10:47:20.338 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-18T10:47:20.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.515+0000 7fe6212ca640 1 Processor -- start 2023-10-18T10:47:20.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.515+0000 7fe6212ca640 1 -- start start 2023-10-18T10:47:20.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.515+0000 7fe6212ca640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe61c0d0760 0x7fe61c0d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:20.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.515+0000 7fe6212ca640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe61c0cd160 0x7fe61c0cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:20.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.515+0000 7fe6212ca640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe61c0d1100 con 0x7fe61c15c940 2023-10-18T10:47:20.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.515+0000 7fe6212ca640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe61c15da50 con 0x7fe61c0d0760 2023-10-18T10:47:20.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.515+0000 7fe6212ca640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c0d3110 con 0x7fe61c0cd160 2023-10-18T10:47:20.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.515+0000 7fe6127fc640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe61c0d0760 0x7fe61c0d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:20.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.515+0000 7fe61b7fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe61c0cd160 0x7fe61c0cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:20.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.515+0000 7fe6127fc640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe61c0d0760 0x7fe61c0d0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:45620/0 (socket says 172.21.15.89:45620) 2023-10-18T10:47:20.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.515+0000 7fe6127fc640 1 -- 172.21.15.89:0/2506271095 learned_addr learned my addr 172.21.15.89:0/2506271095 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:20.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.515+0000 7fe6127fc640 1 -- 172.21.15.89:0/2506271095 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe61c0cd160 msgr2=0x7fe61c0cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:20.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.515+0000 7fe6127fc640 1 --2- 172.21.15.89:0/2506271095 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe61c0cd160 0x7fe61c0cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:20.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.515+0000 7fe6127fc640 1 -- 172.21.15.89:0/2506271095 >> v1:172.21.15.154:6789/0 conn(0x7fe61c15c940 legacy=0x7fe61c0d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:20.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.515+0000 7fe6127fc640 1 -- 172.21.15.89:0/2506271095 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61c15ce40 con 0x7fe61c0d0760 2023-10-18T10:47:20.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.515+0000 7fe61a7fc640 1 -- 172.21.15.89:0/2506271095 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2303946615 0 0) 0x7fe61c0d1100 con 0x7fe61c15c940 2023-10-18T10:47:20.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.515+0000 7fe61b7fe640 1 --2- 172.21.15.89:0/2506271095 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe61c0cd160 0x7fe61c0cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:47:20.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.519+0000 7fe6127fc640 1 --2- 172.21.15.89:0/2506271095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe61c0d0760 0x7fe61c0d0bc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe604009870 tx=0x7fe60401c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27e3a85786f41c02 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:20.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.519+0000 7fe61a7fc640 1 -- 172.21.15.89:0/2506271095 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6040aa670 con 0x7fe61c0d0760 2023-10-18T10:47:20.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.519+0000 7fe61a7fc640 1 -- 172.21.15.89:0/2506271095 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6040aa7d0 con 0x7fe61c0d0760 2023-10-18T10:47:20.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.519+0000 7fe61a7fc640 1 -- 172.21.15.89:0/2506271095 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6040aab60 con 0x7fe61c0d0760 2023-10-18T10:47:20.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.519+0000 7fe6212ca640 1 -- 172.21.15.89:0/2506271095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe61c0d0760 msgr2=0x7fe61c0d0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:20.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.519+0000 7fe6212ca640 1 --2- 172.21.15.89:0/2506271095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe61c0d0760 0x7fe61c0d0bc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe604009870 tx=0x7fe60401c510 comp rx=0 tx=0).stop 2023-10-18T10:47:20.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.519+0000 7fe6212ca640 1 -- 172.21.15.89:0/2506271095 shutdown_connections 2023-10-18T10:47:20.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.519+0000 7fe6212ca640 1 --2- 172.21.15.89:0/2506271095 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe61c0cd160 0x7fe61c0cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:20.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.519+0000 7fe6212ca640 1 --2- 172.21.15.89:0/2506271095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe61c0d0760 0x7fe61c0d0bc0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:20.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.519+0000 7fe6212ca640 1 -- 172.21.15.89:0/2506271095 >> 172.21.15.89:0/2506271095 conn(0x7fe61c089fd0 msgr2=0x7fe61c09b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:20.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.519+0000 7fe6212ca640 1 -- 172.21.15.89:0/2506271095 shutdown_connections 2023-10-18T10:47:20.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.519+0000 7fe6212ca640 1 -- 172.21.15.89:0/2506271095 wait complete. 2023-10-18T10:47:20.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.519+0000 7fe6212ca640 1 Processor -- start 2023-10-18T10:47:20.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.523+0000 7fe6212ca640 1 -- start start 2023-10-18T10:47:20.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.523+0000 7fe6212ca640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe61c0cd160 0x7fe61c0c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:20.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.523+0000 7fe6212ca640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe61c0d0760 0x7fe61c1578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:20.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.523+0000 7fe6212ca640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe61c1f4cb0 0x7fe61c157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:20.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.523+0000 7fe6212ca640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c1f5260 con 0x7fe61c0d0760 2023-10-18T10:47:20.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.523+0000 7fe6212ca640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c1f5540 con 0x7fe61c1f4cb0 2023-10-18T10:47:20.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.523+0000 7fe6212ca640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c1f5820 con 0x7fe61c0cd160 2023-10-18T10:47:20.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.523+0000 7fe6127fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe61c0d0760 0x7fe61c1578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:20.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.523+0000 7fe6127fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe61c0d0760 0x7fe61c1578f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38038/0 (socket says 172.21.15.89:38038) 2023-10-18T10:47:20.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.523+0000 7fe6127fc640 1 -- 172.21.15.89:0/3779037909 learned_addr learned my addr 172.21.15.89:0/3779037909 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:20.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.523+0000 7fe61affd640 1 --2- 172.21.15.89:0/3779037909 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe61c0cd160 0x7fe61c0c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:20.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.523+0000 7fe61b7fe640 1 --2- 172.21.15.89:0/3779037909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe61c1f4cb0 0x7fe61c157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:20.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.523+0000 7fe6127fc640 1 -- 172.21.15.89:0/3779037909 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe61c0cd160 msgr2=0x7fe61c0c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:20.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.523+0000 7fe6127fc640 1 --2- 172.21.15.89:0/3779037909 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe61c0cd160 0x7fe61c0c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:20.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.523+0000 7fe6127fc640 1 -- 172.21.15.89:0/3779037909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe61c1f4cb0 msgr2=0x7fe61c157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:20.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.523+0000 7fe6127fc640 1 --2- 172.21.15.89:0/3779037909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe61c1f4cb0 0x7fe61c157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:20.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.523+0000 7fe6127fc640 1 -- 172.21.15.89:0/3779037909 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe614009a70 con 0x7fe61c0d0760 2023-10-18T10:47:20.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.523+0000 7fe6127fc640 1 --2- 172.21.15.89:0/3779037909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe61c0d0760 0x7fe61c1578f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe6040ac040 tx=0x7fe60401ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:20.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.523+0000 7fe613fff640 1 -- 172.21.15.89:0/3779037909 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6040aad50 con 0x7fe61c0d0760 2023-10-18T10:47:20.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.523+0000 7fe613fff640 1 -- 172.21.15.89:0/3779037909 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6040b7070 con 0x7fe61c0d0760 2023-10-18T10:47:20.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.523+0000 7fe6212ca640 1 -- 172.21.15.89:0/3779037909 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6040094d0 con 0x7fe61c0d0760 2023-10-18T10:47:20.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.523+0000 7fe613fff640 1 -- 172.21.15.89:0/3779037909 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe604004220 con 0x7fe61c0d0760 2023-10-18T10:47:20.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.523+0000 7fe6212ca640 1 -- 172.21.15.89:0/3779037909 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe61c1f5de0 con 0x7fe61c0d0760 2023-10-18T10:47:20.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.523+0000 7fe6212ca640 1 -- 172.21.15.89:0/3779037909 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe61c1f5960 con 0x7fe61c0d0760 2023-10-18T10:47:20.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.527+0000 7fe613fff640 1 -- 172.21.15.89:0/3779037909 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fe604004380 con 0x7fe61c0d0760 2023-10-18T10:47:20.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.527+0000 7fe613fff640 1 --2- 172.21.15.89:0/3779037909 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe5fc0656b0 0x7fe5fc067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:20.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.527+0000 7fe613fff640 1 -- 172.21.15.89:0/3779037909 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe60411cfe0 con 0x7fe61c0d0760 2023-10-18T10:47:20.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.527+0000 7fe61affd640 1 --2- 172.21.15.89:0/3779037909 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe5fc0656b0 0x7fe5fc067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:20.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.527+0000 7fe61affd640 1 --2- 172.21.15.89:0/3779037909 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe5fc0656b0 0x7fe5fc067b70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe60c007fd0 tx=0x7fe60c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:20.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.531+0000 7fe613fff640 1 -- 172.21.15.89:0/3779037909 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fe61c1f5960 con 0x7fe61c0d0760 2023-10-18T10:47:20.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.731+0000 7fe6212ca640 1 -- 172.21.15.89:0/3779037909 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe61c1f5960 con 0x7fe61c0d0760 2023-10-18T10:47:20.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.735+0000 7fe613fff640 1 -- 172.21.15.89:0/3779037909 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7fe61c1f5960 con 0x7fe61c0d0760 2023-10-18T10:47:20.738 INFO:teuthology.orchestra.run.smithi089.stdout:[client.0] 2023-10-18T10:47:20.738 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQAity9lApuiJhAAUNpuF1S7ZLrsIuRG0mrVNw== 2023-10-18T10:47:20.739 INFO:teuthology.orchestra.run.smithi089.stdout: caps mds = "allow" 2023-10-18T10:47:20.739 INFO:teuthology.orchestra.run.smithi089.stdout: caps mon = "allow r" 2023-10-18T10:47:20.739 INFO:teuthology.orchestra.run.smithi089.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-18T10:47:20.741 INFO:teuthology.orchestra.run.smithi089.stderr:updated caps for client.0 2023-10-18T10:47:20.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.735+0000 7fe6212ca640 1 -- 172.21.15.89:0/3779037909 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe5fc0656b0 msgr2=0x7fe5fc067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:20.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.735+0000 7fe6212ca640 1 --2- 172.21.15.89:0/3779037909 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe5fc0656b0 0x7fe5fc067b70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe60c007fd0 tx=0x7fe60c009040 comp rx=0 tx=0).stop 2023-10-18T10:47:20.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.735+0000 7fe6212ca640 1 -- 172.21.15.89:0/3779037909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe61c0d0760 msgr2=0x7fe61c1578f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:20.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.735+0000 7fe6212ca640 1 --2- 172.21.15.89:0/3779037909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe61c0d0760 0x7fe61c1578f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe6040ac040 tx=0x7fe60401ad50 comp rx=0 tx=0).stop 2023-10-18T10:47:20.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.735+0000 7fe6212ca640 1 -- 172.21.15.89:0/3779037909 shutdown_connections 2023-10-18T10:47:20.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.735+0000 7fe6212ca640 1 --2- 172.21.15.89:0/3779037909 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe5fc0656b0 0x7fe5fc067b70 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:20.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.735+0000 7fe6212ca640 1 --2- 172.21.15.89:0/3779037909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe61c1f4cb0 0x7fe61c157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:20.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.735+0000 7fe6212ca640 1 --2- 172.21.15.89:0/3779037909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe61c0d0760 0x7fe61c1578f0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:20.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.735+0000 7fe6212ca640 1 --2- 172.21.15.89:0/3779037909 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe61c0cd160 0x7fe61c0c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:20.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.735+0000 7fe6212ca640 1 -- 172.21.15.89:0/3779037909 >> 172.21.15.89:0/3779037909 conn(0x7fe61c089fd0 msgr2=0x7fe61c0b4850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:20.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.739+0000 7fe6212ca640 1 -- 172.21.15.89:0/3779037909 shutdown_connections 2023-10-18T10:47:20.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.739+0000 7fe6212ca640 1 -- 172.21.15.89:0/3779037909 wait complete. 2023-10-18T10:47:20.759 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:47:20.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.895+0000 7f6ab1fba640 1 Processor -- start 2023-10-18T10:47:20.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.895+0000 7f6ab1fba640 1 -- start start 2023-10-18T10:47:20.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.895+0000 7f6ab1fba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6aac15c940 0x7f6aac0d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:20.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.895+0000 7f6ab1fba640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6aac0d2f40 0x7f6aac0d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:20.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.895+0000 7f6ab1fba640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aac15da50 con 0x7f6aac0d0ca0 2023-10-18T10:47:20.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.895+0000 7f6ab1fba640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aac0d1130 con 0x7f6aac0d2f40 2023-10-18T10:47:20.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.895+0000 7f6ab1fba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac15ce40 con 0x7f6aac15c940 2023-10-18T10:47:20.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.895+0000 7f6aabfff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6aac0d0ca0 0x7f6aac0cf060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40692/0 (socket says 172.21.15.89:40692) 2023-10-18T10:47:20.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.895+0000 7f6aaaffd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6aac0d2f40 0x7f6aac0d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:20.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.895+0000 7f6aabfff640 1 -- 172.21.15.89:0/3338535474 learned_addr learned my addr 172.21.15.89:0/3338535474 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:47:20.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.895+0000 7f6aab7fe640 1 --2- 172.21.15.89:0/3338535474 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6aac15c940 0x7f6aac0d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:20.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.895+0000 7f6aaaffd640 1 -- 172.21.15.89:0/3338535474 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6aac15c940 msgr2=0x7f6aac0d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:20.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.895+0000 7f6aaaffd640 1 --2- 172.21.15.89:0/3338535474 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6aac15c940 0x7f6aac0d2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:20.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.895+0000 7f6aaaffd640 1 -- 172.21.15.89:0/3338535474 >> v1:172.21.15.89:6789/0 conn(0x7f6aac0d0ca0 legacy=0x7f6aac0cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:20.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.895+0000 7f6aaaffd640 1 -- 172.21.15.89:0/3338535474 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aac15d120 con 0x7f6aac0d2f40 2023-10-18T10:47:20.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.899+0000 7f6aaaffd640 1 --2- 172.21.15.89:0/3338535474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6aac0d2f40 0x7f6aac0d0760 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6aa00097c0 tx=0x7f6aa001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d95651dc00ace2 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:20.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.899+0000 7f6aaa7fc640 1 -- 172.21.15.89:0/3338535474 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa00ad660 con 0x7f6aac0d2f40 2023-10-18T10:47:20.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.899+0000 7f6aaa7fc640 1 -- 172.21.15.89:0/3338535474 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aa00ad7c0 con 0x7f6aac0d2f40 2023-10-18T10:47:20.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.899+0000 7f6aaa7fc640 1 -- 172.21.15.89:0/3338535474 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa00adb50 con 0x7f6aac0d2f40 2023-10-18T10:47:20.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.899+0000 7f6ab1fba640 1 -- 172.21.15.89:0/3338535474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6aac0d2f40 msgr2=0x7f6aac0d0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:20.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.899+0000 7f6ab1fba640 1 --2- 172.21.15.89:0/3338535474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6aac0d2f40 0x7f6aac0d0760 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6aa00097c0 tx=0x7f6aa001c450 comp rx=0 tx=0).stop 2023-10-18T10:47:20.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.899+0000 7f6ab1fba640 1 -- 172.21.15.89:0/3338535474 shutdown_connections 2023-10-18T10:47:20.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.899+0000 7f6ab1fba640 1 --2- 172.21.15.89:0/3338535474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6aac0d2f40 0x7f6aac0d0760 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:20.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.899+0000 7f6ab1fba640 1 --2- 172.21.15.89:0/3338535474 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6aac15c940 0x7f6aac0d2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:20.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.899+0000 7f6ab1fba640 1 -- 172.21.15.89:0/3338535474 >> 172.21.15.89:0/3338535474 conn(0x7f6aac089fd0 msgr2=0x7f6aac09b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:20.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.899+0000 7f6ab1fba640 1 -- 172.21.15.89:0/3338535474 shutdown_connections 2023-10-18T10:47:20.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.899+0000 7f6ab1fba640 1 -- 172.21.15.89:0/3338535474 wait complete. 2023-10-18T10:47:20.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.903+0000 7f6ab1fba640 1 Processor -- start 2023-10-18T10:47:20.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.903+0000 7f6ab1fba640 1 -- start start 2023-10-18T10:47:20.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.903+0000 7f6ab1fba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6aac0d0ca0 0x7f6aac152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:20.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.903+0000 7f6ab1fba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6aac152860 0x7f6aac152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:20.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.903+0000 7f6ab1fba640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6aac15b130 0x7f6aac15b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:20.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.903+0000 7f6ab1fba640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac15baf0 con 0x7f6aac15b130 2023-10-18T10:47:20.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.903+0000 7f6ab1fba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac0b5240 con 0x7f6aac0d0ca0 2023-10-18T10:47:20.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.903+0000 7f6ab1fba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac0cb6c0 con 0x7f6aac152860 2023-10-18T10:47:20.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.903+0000 7f6aabfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6aac15b130 0x7f6aac15b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:20.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.903+0000 7f6aabfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6aac15b130 0x7f6aac15b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38062/0 (socket says 172.21.15.89:38062) 2023-10-18T10:47:20.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.903+0000 7f6aabfff640 1 -- 172.21.15.89:0/3483930081 learned_addr learned my addr 172.21.15.89:0/3483930081 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:20.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.903+0000 7f6aaaffd640 1 --2- 172.21.15.89:0/3483930081 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6aac152860 0x7f6aac152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:20.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.903+0000 7f6aab7fe640 1 --2- 172.21.15.89:0/3483930081 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6aac0d0ca0 0x7f6aac152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:20.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.903+0000 7f6aabfff640 1 -- 172.21.15.89:0/3483930081 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6aac152860 msgr2=0x7f6aac152ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:20.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.903+0000 7f6aabfff640 1 --2- 172.21.15.89:0/3483930081 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6aac152860 0x7f6aac152ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:20.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.903+0000 7f6aabfff640 1 -- 172.21.15.89:0/3483930081 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6aac0d0ca0 msgr2=0x7f6aac152000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:20.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.903+0000 7f6aabfff640 1 --2- 172.21.15.89:0/3483930081 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6aac0d0ca0 0x7f6aac152000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:20.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.903+0000 7f6aabfff640 1 -- 172.21.15.89:0/3483930081 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa0009420 con 0x7f6aac15b130 2023-10-18T10:47:20.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.903+0000 7f6aab7fe640 1 --2- 172.21.15.89:0/3483930081 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6aac0d0ca0 0x7f6aac152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:47:20.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.903+0000 7f6aabfff640 1 --2- 172.21.15.89:0/3483930081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6aac15b130 0x7f6aac15b5b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6aa400cbc0 tx=0x7f6aa400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:20.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.903+0000 7f6a93fff640 1 -- 172.21.15.89:0/3483930081 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa400ad40 con 0x7f6aac15b130 2023-10-18T10:47:20.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.903+0000 7f6a93fff640 1 -- 172.21.15.89:0/3483930081 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aa4012400 con 0x7f6aac15b130 2023-10-18T10:47:20.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.903+0000 7f6a93fff640 1 -- 172.21.15.89:0/3483930081 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa400aea0 con 0x7f6aac15b130 2023-10-18T10:47:20.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.903+0000 7f6ab1fba640 1 -- 172.21.15.89:0/3483930081 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aac0cb9a0 con 0x7f6aac15b130 2023-10-18T10:47:20.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.903+0000 7f6ab1fba640 1 -- 172.21.15.89:0/3483930081 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aac0cbf60 con 0x7f6aac15b130 2023-10-18T10:47:20.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.907+0000 7f6ab1fba640 1 -- 172.21.15.89:0/3483930081 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aac0cee70 con 0x7f6aac15b130 2023-10-18T10:47:20.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.907+0000 7f6a93fff640 1 -- 172.21.15.89:0/3483930081 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f6aa4007070 con 0x7f6aac15b130 2023-10-18T10:47:20.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.907+0000 7f6a93fff640 1 --2- 172.21.15.89:0/3483930081 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6a980656b0 0x7f6a98067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:20.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.907+0000 7f6a93fff640 1 -- 172.21.15.89:0/3483930081 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f6aa4082d70 con 0x7f6aac15b130 2023-10-18T10:47:20.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.907+0000 7f6aab7fe640 1 --2- 172.21.15.89:0/3483930081 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6a980656b0 0x7f6a98067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:20.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.907+0000 7f6aab7fe640 1 --2- 172.21.15.89:0/3483930081 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6a980656b0 0x7f6a98067b70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00b9f0 tx=0x7f6a9c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:20.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:20.911+0000 7f6a93fff640 1 -- 172.21.15.89:0/3483930081 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f6aa4050790 con 0x7f6aac15b130 2023-10-18T10:47:21.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.091+0000 7f6ab1fba640 1 -- 172.21.15.89:0/3483930081 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6aac0b53b0 con 0x7f6aac15b130 2023-10-18T10:47:21.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.091+0000 7f6a93fff640 1 -- 172.21.15.89:0/3483930081 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4875 (secure 0 0 0) 0x7f6aa4017020 con 0x7f6aac15b130 2023-10-18T10:47:21.095 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:47:21.095 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/654729641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":654729641},{"type":"v1","addr":"172.21.15.154:6836","nonce":654729641}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/964566480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":964566480},{"type":"v1","addr":"172.21.15.154:6837","nonce":964566480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2261878538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2261878538},{"type":"v1","addr":"172.21.15.89:6833","nonce":2261878538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:47:15.199575+0000","modified":"2023-10-18T10:47:18.846656+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5517":{"gid":5517,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6835/2092274731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2092274731},{"type":"v1","addr":"172.21.15.89:6835","nonce":2092274731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T10:47:21.096 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 18 2023-10-18T10:47:21.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.091+0000 7f6ab1fba640 1 -- 172.21.15.89:0/3483930081 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6a980656b0 msgr2=0x7f6a98067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:21.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.091+0000 7f6ab1fba640 1 --2- 172.21.15.89:0/3483930081 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6a980656b0 0x7f6a98067b70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00b9f0 tx=0x7f6a9c008040 comp rx=0 tx=0).stop 2023-10-18T10:47:21.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.091+0000 7f6ab1fba640 1 -- 172.21.15.89:0/3483930081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6aac15b130 msgr2=0x7f6aac15b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:21.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.091+0000 7f6ab1fba640 1 --2- 172.21.15.89:0/3483930081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6aac15b130 0x7f6aac15b5b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6aa400cbc0 tx=0x7f6aa400a430 comp rx=0 tx=0).stop 2023-10-18T10:47:21.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.095+0000 7f6ab1fba640 1 -- 172.21.15.89:0/3483930081 shutdown_connections 2023-10-18T10:47:21.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.095+0000 7f6ab1fba640 1 --2- 172.21.15.89:0/3483930081 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f6a980656b0 0x7f6a98067b70 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.095+0000 7f6ab1fba640 1 --2- 172.21.15.89:0/3483930081 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6aac152860 0x7f6aac152ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.095+0000 7f6ab1fba640 1 --2- 172.21.15.89:0/3483930081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6aac15b130 0x7f6aac15b5b0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.095+0000 7f6ab1fba640 1 --2- 172.21.15.89:0/3483930081 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6aac0d0ca0 0x7f6aac152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.095+0000 7f6ab1fba640 1 -- 172.21.15.89:0/3483930081 >> 172.21.15.89:0/3483930081 conn(0x7f6aac089fd0 msgr2=0x7f6aac069900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:21.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.095+0000 7f6ab1fba640 1 -- 172.21.15.89:0/3483930081 shutdown_connections 2023-10-18T10:47:21.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.095+0000 7f6ab1fba640 1 -- 172.21.15.89:0/3483930081 wait complete. 2023-10-18T10:47:21.518 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-18T10:47:15.199575+0000', 'modified': '2023-10-18T10:47:18.846656+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5517}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5517': {'gid': 5517, 'name': 'c', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.89:6835/2092274731', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6834', 'nonce': 2092274731}, {'type': 'v1', 'addr': '172.21.15.89:6835', 'nonce': 2092274731}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-18T10:47:21.518 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T10:47:21.519 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-18T10:47:21.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.654+0000 7f534adba640 1 Processor -- start 2023-10-18T10:47:21.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.654+0000 7f534adba640 1 -- start start 2023-10-18T10:47:21.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.654+0000 7f534adba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5344160a10 0x7f5344060320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:21.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.654+0000 7f534adba640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53441602a0 con 0x7f5344160e30 2023-10-18T10:47:21.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.654+0000 7f534adba640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53440cd2b0 con 0x7f534415c670 2023-10-18T10:47:21.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.654+0000 7f534adba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53440cd590 con 0x7f5344160a10 2023-10-18T10:47:21.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.654+0000 7f5349357640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5344160e30 0x7f53440c9990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:40702/0 (socket says 172.21.15.89:40702) 2023-10-18T10:47:21.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.654+0000 7f5349357640 1 -- 172.21.15.89:0/1068171043 learned_addr learned my addr 172.21.15.89:0/1068171043 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:47:21.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.654+0000 7f533bfff640 1 --2- 172.21.15.89:0/1068171043 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5344160a10 0x7f5344060320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:21.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.658+0000 7f533b7fe640 1 -- 172.21.15.89:0/1068171043 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 841224288 0 0) 0x7f53441602a0 con 0x7f5344160e30 2023-10-18T10:47:21.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.658+0000 7f533b7fe640 1 -- 172.21.15.89:0/1068171043 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f532c003530 con 0x7f5344160e30 2023-10-18T10:47:21.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.658+0000 7f533b7fe640 1 -- 172.21.15.89:0/1068171043 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3140200138 0 0) 0x7f53440cd2b0 con 0x7f534415c670 2023-10-18T10:47:21.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.658+0000 7f533b7fe640 1 -- 172.21.15.89:0/1068171043 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53441602a0 con 0x7f534415c670 2023-10-18T10:47:21.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.658+0000 7f533bfff640 1 -- 172.21.15.89:0/1068171043 >> v1:172.21.15.154:6789/0 conn(0x7f534415c670 legacy=0x7f53440713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:21.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.658+0000 7f533bfff640 1 -- 172.21.15.89:0/1068171043 >> v1:172.21.15.89:6789/0 conn(0x7f5344160e30 legacy=0x7f53440c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:21.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.658+0000 7f533bfff640 1 -- 172.21.15.89:0/1068171043 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53440cd870 con 0x7f5344160a10 2023-10-18T10:47:21.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.658+0000 7f533bfff640 1 --2- 172.21.15.89:0/1068171043 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5344160a10 0x7f5344060320 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f53340097f0 tx=0x7f533401ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce1ace2b4b42fd6 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:21.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.658+0000 7f533b7fe640 1 -- 172.21.15.89:0/1068171043 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53340a89b0 con 0x7f5344160a10 2023-10-18T10:47:21.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.658+0000 7f533b7fe640 1 -- 172.21.15.89:0/1068171043 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53340a8b10 con 0x7f5344160a10 2023-10-18T10:47:21.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.658+0000 7f533b7fe640 1 -- 172.21.15.89:0/1068171043 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53340a8ea0 con 0x7f5344160a10 2023-10-18T10:47:21.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.662+0000 7f534adba640 1 -- 172.21.15.89:0/1068171043 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5344160a10 msgr2=0x7f5344060320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:21.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.662+0000 7f534adba640 1 --2- 172.21.15.89:0/1068171043 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5344160a10 0x7f5344060320 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f53340097f0 tx=0x7f533401ecb0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.662+0000 7f534adba640 1 -- 172.21.15.89:0/1068171043 shutdown_connections 2023-10-18T10:47:21.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.662+0000 7f534adba640 1 --2- 172.21.15.89:0/1068171043 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5344160a10 0x7f5344060320 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.662+0000 7f534adba640 1 -- 172.21.15.89:0/1068171043 >> 172.21.15.89:0/1068171043 conn(0x7f53440cbdc0 msgr2=0x7f53440cc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:21.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.662+0000 7f534adba640 1 -- 172.21.15.89:0/1068171043 shutdown_connections 2023-10-18T10:47:21.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.662+0000 7f534adba640 1 -- 172.21.15.89:0/1068171043 wait complete. 2023-10-18T10:47:21.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.662+0000 7f534adba640 1 Processor -- start 2023-10-18T10:47:21.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.662+0000 7f534adba640 1 -- start start 2023-10-18T10:47:21.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.662+0000 7f534adba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f534415c670 0x7f53441f5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:21.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.662+0000 7f534adba640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5344160a10 0x7f534415f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:21.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.662+0000 7f534adba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5344160e30 0x7f534414c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:21.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.662+0000 7f534adba640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53440c3ff0 con 0x7f5344160a10 2023-10-18T10:47:21.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.662+0000 7f534adba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53441f5710 con 0x7f534415c670 2023-10-18T10:47:21.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.662+0000 7f534adba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53441f59f0 con 0x7f5344160e30 2023-10-18T10:47:21.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.662+0000 7f533bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5344160a10 0x7f534415f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:21.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.662+0000 7f533bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5344160a10 0x7f534415f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38078/0 (socket says 172.21.15.89:38078) 2023-10-18T10:47:21.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.662+0000 7f533bfff640 1 -- 172.21.15.89:0/2004492934 learned_addr learned my addr 172.21.15.89:0/2004492934 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:21.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.662+0000 7f5348b56640 1 --2- 172.21.15.89:0/2004492934 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f534415c670 0x7f53441f5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:21.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.662+0000 7f5349357640 1 --2- 172.21.15.89:0/2004492934 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5344160e30 0x7f534414c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:21.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.666+0000 7f533bfff640 1 -- 172.21.15.89:0/2004492934 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5344160e30 msgr2=0x7f534414c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:21.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.666+0000 7f533bfff640 1 --2- 172.21.15.89:0/2004492934 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5344160e30 0x7f534414c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.666+0000 7f533bfff640 1 -- 172.21.15.89:0/2004492934 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f534415c670 msgr2=0x7f53441f5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:21.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.666+0000 7f533bfff640 1 --2- 172.21.15.89:0/2004492934 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f534415c670 0x7f53441f5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.666+0000 7f533bfff640 1 -- 172.21.15.89:0/2004492934 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5334009420 con 0x7f5344160a10 2023-10-18T10:47:21.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.666+0000 7f533bfff640 1 --2- 172.21.15.89:0/2004492934 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5344160a10 0x7f534415f640 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f533401b8c0 tx=0x7f533401ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:21.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.666+0000 7f53397fa640 1 -- 172.21.15.89:0/2004492934 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53340b0d60 con 0x7f5344160a10 2023-10-18T10:47:21.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.666+0000 7f53397fa640 1 -- 172.21.15.89:0/2004492934 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53340b9430 con 0x7f5344160a10 2023-10-18T10:47:21.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.666+0000 7f53397fa640 1 -- 172.21.15.89:0/2004492934 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53340b97e0 con 0x7f5344160a10 2023-10-18T10:47:21.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.666+0000 7f534adba640 1 -- 172.21.15.89:0/2004492934 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53441f5cd0 con 0x7f5344160a10 2023-10-18T10:47:21.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.666+0000 7f534adba640 1 -- 172.21.15.89:0/2004492934 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f534415dfe0 con 0x7f5344160a10 2023-10-18T10:47:21.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.666+0000 7f53397fa640 1 -- 172.21.15.89:0/2004492934 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f53340b7040 con 0x7f5344160a10 2023-10-18T10:47:21.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.666+0000 7f53397fa640 1 --2- 172.21.15.89:0/2004492934 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5330065680 0x7f5330067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:21.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.666+0000 7f53397fa640 1 -- 172.21.15.89:0/2004492934 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f53341208e0 con 0x7f5344160a10 2023-10-18T10:47:21.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.670+0000 7f5348b56640 1 --2- 172.21.15.89:0/2004492934 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5330065680 0x7f5330067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:21.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.670+0000 7f534e123000 1 Processor -- start 2023-10-18T10:47:21.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.670+0000 7f534e123000 1 -- start start 2023-10-18T10:47:21.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.670+0000 7f5348b56640 1 --2- 172.21.15.89:0/2004492934 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5330065680 0x7f5330067b40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f53400011f0 tx=0x7f5340009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:21.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.670+0000 7f534e123000 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55a1ae200a70 0x55a1ae200e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:21.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.670+0000 7f534e123000 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1ae201390 0x55a1ae1b6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:21.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.670+0000 7f534e123000 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1ae1b6e70 0x55a1ae1b7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:21.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.670+0000 7f534e123000 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1ae0205a0 con 0x55a1ae1b6e70 2023-10-18T10:47:21.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.670+0000 7f534e123000 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1ae00e320 con 0x55a1ae201390 2023-10-18T10:47:21.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.670+0000 7f534e123000 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1ae00b850 con 0x55a1ae200a70 2023-10-18T10:47:21.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.670+0000 7f5349357640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1ae1b6e70 0x55a1ae1b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:21.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.670+0000 7f5349357640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1ae1b6e70 0x55a1ae1b7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38086/0 (socket says 172.21.15.89:38086) 2023-10-18T10:47:21.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.670+0000 7f5349357640 1 -- 172.21.15.89:0/222366143 learned_addr learned my addr 172.21.15.89:0/222366143 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:21.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.670+0000 7f5348b56640 1 --2- 172.21.15.89:0/222366143 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55a1ae200a70 0x55a1ae200e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:21.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.670+0000 7f533bfff640 1 --2- 172.21.15.89:0/222366143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1ae201390 0x55a1ae1b6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:21.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.670+0000 7f5349357640 1 -- 172.21.15.89:0/222366143 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55a1ae200a70 msgr2=0x55a1ae200e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:21.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.670+0000 7f5349357640 1 --2- 172.21.15.89:0/222366143 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55a1ae200a70 0x55a1ae200e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.670+0000 7f5349357640 1 -- 172.21.15.89:0/222366143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1ae201390 msgr2=0x55a1ae1b6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:21.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.670+0000 7f5349357640 1 --2- 172.21.15.89:0/222366143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1ae201390 0x55a1ae1b6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.670+0000 7f5349357640 1 -- 172.21.15.89:0/222366143 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1add01890 con 0x55a1ae1b6e70 2023-10-18T10:47:21.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.670+0000 7f5349357640 1 --2- 172.21.15.89:0/222366143 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1ae1b6e70 0x55a1ae1b7250 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f533c002210 tx=0x7f533c0074d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c220eaf0f5e3748 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:21.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.670+0000 7f532a7fc640 1 -- 172.21.15.89:0/222366143 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533c011580 con 0x55a1ae1b6e70 2023-10-18T10:47:21.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.670+0000 7f532a7fc640 1 -- 172.21.15.89:0/222366143 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f533c0116e0 con 0x55a1ae1b6e70 2023-10-18T10:47:21.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.670+0000 7f532a7fc640 1 -- 172.21.15.89:0/222366143 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533c011a70 con 0x55a1ae1b6e70 2023-10-18T10:47:21.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.670+0000 7f534e123000 1 -- 172.21.15.89:0/222366143 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1ae1b6e70 msgr2=0x55a1ae1b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:21.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.670+0000 7f534e123000 1 --2- 172.21.15.89:0/222366143 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1ae1b6e70 0x55a1ae1b7250 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f533c002210 tx=0x7f533c0074d0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.670+0000 7f534e123000 1 -- 172.21.15.89:0/222366143 shutdown_connections 2023-10-18T10:47:21.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.670+0000 7f534e123000 1 --2- 172.21.15.89:0/222366143 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1ae1b6e70 0x55a1ae1b7250 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.670+0000 7f534e123000 1 --2- 172.21.15.89:0/222366143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1ae201390 0x55a1ae1b6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.670+0000 7f534e123000 1 --2- 172.21.15.89:0/222366143 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55a1ae200a70 0x55a1ae200e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.670+0000 7f534e123000 1 -- 172.21.15.89:0/222366143 >> 172.21.15.89:0/222366143 conn(0x55a1ae1ae1e0 msgr2=0x55a1ae1b5210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:21.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.670+0000 7f534e123000 1 -- 172.21.15.89:0/222366143 shutdown_connections 2023-10-18T10:47:21.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.670+0000 7f534e123000 1 -- 172.21.15.89:0/222366143 wait complete. 2023-10-18T10:47:21.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.670+0000 7f534e123000 1 -- start start 2023-10-18T10:47:21.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.670+0000 7f534e123000 1 Processor -- start 2023-10-18T10:47:21.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f534e123000 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1ae1b6e70 0x55a1ae1c2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:21.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f534e123000 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1ae200a70 0x55a1ae1c2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:21.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f534e123000 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55a1ae201390 0x55a1ae1c32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:21.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f534e123000 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1ae007140 con 0x55a1ae200a70 2023-10-18T10:47:21.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f534e123000 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1adfc1f60 con 0x55a1ae1b6e70 2023-10-18T10:47:21.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f534e123000 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1adfc1c00 con 0x55a1ae201390 2023-10-18T10:47:21.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f534e123000 10 client.0 _ll_register_callbacks cb 0x55a1ae1a2350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T10:47:21.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f533bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1ae200a70 0x55a1ae1c2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:21.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f533bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1ae200a70 0x55a1ae1c2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38096/0 (socket says 172.21.15.89:38096) 2023-10-18T10:47:21.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f533bfff640 1 -- 172.21.15.89:0/3324964660 learned_addr learned my addr 172.21.15.89:0/3324964660 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:21.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f5348b56640 1 --2- 172.21.15.89:0/3324964660 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1ae1b6e70 0x55a1ae1c2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:21.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f5349357640 1 --2- 172.21.15.89:0/3324964660 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55a1ae201390 0x55a1ae1c32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:21.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f533bfff640 1 -- 172.21.15.89:0/3324964660 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55a1ae201390 msgr2=0x55a1ae1c32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:21.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f533bfff640 1 --2- 172.21.15.89:0/3324964660 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55a1ae201390 0x55a1ae1c32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f533bfff640 1 -- 172.21.15.89:0/3324964660 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1ae1b6e70 msgr2=0x55a1ae1c2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:21.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f533bfff640 1 --2- 172.21.15.89:0/3324964660 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1ae1b6e70 0x55a1ae1c2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f533bfff640 1 -- 172.21.15.89:0/3324964660 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f532c003530 con 0x55a1ae200a70 2023-10-18T10:47:21.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f5349357640 1 --2- 172.21.15.89:0/3324964660 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55a1ae201390 0x55a1ae1c32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:47:21.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f533bfff640 1 --2- 172.21.15.89:0/3324964660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1ae200a70 0x55a1ae1c2db0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f53340202c0 tx=0x7f53340f1a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531fe20457e0feaa server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:21.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f53237fe640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-18T10:47:21.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f53237fe640 1 -- 172.21.15.89:0/3324964660 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53340f5ea0 con 0x55a1ae200a70 2023-10-18T10:47:21.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f53237fe640 1 -- 172.21.15.89:0/3324964660 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f53340f9b90 con 0x55a1ae200a70 2023-10-18T10:47:21.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f534e123000 1 -- 172.21.15.89:0/3324964660 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a1ae00d120 con 0x55a1ae200a70 2023-10-18T10:47:21.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f53237fe640 1 -- 172.21.15.89:0/3324964660 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53340f9900 con 0x55a1ae200a70 2023-10-18T10:47:21.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f534e123000 10 client.5691 fetch_fsmap learned FSMap version 18 2023-10-18T10:47:21.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f534e123000 1 -- 172.21.15.89:0/3324964660 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55a1adfca190 con 0x55a1ae200a70 2023-10-18T10:47:21.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f53237fe640 1 -- 172.21.15.89:0/3324964660 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a1ae00d120 con 0x55a1ae200a70 2023-10-18T10:47:21.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f53237fe640 1 -- 172.21.15.89:0/3324964660 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f533410b020 con 0x55a1ae200a70 2023-10-18T10:47:21.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f534e123000 10 client.5691 fetch_fsmap finished waiting for FSMap version 18 2023-10-18T10:47:21.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f534e123000 10 client.5691 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-18T10:47:21.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f534e123000 20 client.5691 populate_metadata read hostname 'smithi089' 2023-10-18T10:47:21.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f534e123000 1 --2- 172.21.15.89:0/3324964660 >> [v2:172.21.15.89:6834/2092274731,v1:172.21.15.89:6835/2092274731] conn(0x55a1ae1c3d00 0x55a1ae212b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:21.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f534e123000 4 client.5691 mds_command: new command op to 5517 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-18T10:47:21.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f534e123000 1 -- 172.21.15.89:0/3324964660 --> [v2:172.21.15.89:6834/2092274731,v1:172.21.15.89:6835/2092274731] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a1ae005a90 con 0x55a1ae1c3d00 2023-10-18T10:47:21.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f5348b56640 1 --2- 172.21.15.89:0/3324964660 >> [v2:172.21.15.89:6834/2092274731,v1:172.21.15.89:6835/2092274731] conn(0x55a1ae1c3d00 0x55a1ae212b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:21.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f5348b56640 1 --2- 172.21.15.89:0/3324964660 >> [v2:172.21.15.89:6834/2092274731,v1:172.21.15.89:6835/2092274731] conn(0x55a1ae1c3d00 0x55a1ae212b20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98898a95f9af395e server_cookie=b22ef1990735e530 in_seq=0 out_seq=0 2023-10-18T10:47:21.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f53237fe640 10 client.5691 ms_handle_connect on v2:172.21.15.89:6834/2092274731 2023-10-18T10:47:21.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f53237fe640 1 -- 172.21.15.89:0/3324964660 <== mds.0 v2:172.21.15.89:6834/2092274731 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a1ae005a90 con 0x55a1ae1c3d00 2023-10-18T10:47:21.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f53237fe640 10 client.5691 handle_command_reply: tid=0 2023-10-18T10:47:21.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.674+0000 7f534e123000 1 client.5691 shutdown 2023-10-18T10:47:21.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.678+0000 7f534e123000 1 -- 172.21.15.89:0/3324964660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1ae200a70 msgr2=0x55a1ae1c2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:21.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.678+0000 7f534e123000 1 --2- 172.21.15.89:0/3324964660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1ae200a70 0x55a1ae1c2db0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f53340202c0 tx=0x7f53340f1a70 comp rx=0 tx=0).stop 2023-10-18T10:47:21.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.678+0000 7f534e123000 1 -- 172.21.15.89:0/3324964660 shutdown_connections 2023-10-18T10:47:21.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.678+0000 7f534e123000 1 --2- 172.21.15.89:0/3324964660 >> [v2:172.21.15.89:6834/2092274731,v1:172.21.15.89:6835/2092274731] conn(0x55a1ae1c3d00 0x55a1ae212b20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.678+0000 7f534e123000 1 --2- 172.21.15.89:0/3324964660 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55a1ae201390 0x55a1ae1c32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.678+0000 7f534e123000 1 --2- 172.21.15.89:0/3324964660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1ae200a70 0x55a1ae1c2db0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.678+0000 7f53237fe640 0 client.5691 ms_handle_reset on v2:172.21.15.89:6834/2092274731 2023-10-18T10:47:21.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.678+0000 7f534e123000 1 --2- 172.21.15.89:0/3324964660 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1ae1b6e70 0x55a1ae1c2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.678+0000 7f534e123000 1 -- 172.21.15.89:0/3324964660 >> 172.21.15.89:0/3324964660 conn(0x55a1ae1ae1e0 msgr2=0x55a1ae1b5910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:21.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.678+0000 7f534e123000 1 -- 172.21.15.89:0/3324964660 shutdown_connections 2023-10-18T10:47:21.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.678+0000 7f534e123000 1 -- 172.21.15.89:0/3324964660 wait complete. 2023-10-18T10:47:21.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.678+0000 7f534e123000 20 client.5691 trim_cache size 0 max 16384 2023-10-18T10:47:21.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.690+0000 7f534e123000 1 Processor -- start 2023-10-18T10:47:21.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.690+0000 7f534e123000 1 -- start start 2023-10-18T10:47:21.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.694+0000 7f534e123000 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55a1ae1b6e70 0x55a1ae1ff2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:21.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.694+0000 7f534e123000 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1ae200a70 0x55a1ae1ff800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:21.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.694+0000 7f534e123000 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1ae201390 0x55a1ae210b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:21.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.694+0000 7f534e123000 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1adcc2940 con 0x55a1ae201390 2023-10-18T10:47:21.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.694+0000 7f534e123000 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1ae00c9f0 con 0x55a1ae200a70 2023-10-18T10:47:21.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.694+0000 7f534e123000 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1ae1afc90 con 0x55a1ae1b6e70 2023-10-18T10:47:21.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.694+0000 7f5349357640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1ae201390 0x55a1ae210b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:21.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.694+0000 7f5349357640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1ae201390 0x55a1ae210b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38106/0 (socket says 172.21.15.89:38106) 2023-10-18T10:47:21.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.694+0000 7f5349357640 1 -- 172.21.15.89:0/3592240161 learned_addr learned my addr 172.21.15.89:0/3592240161 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:21.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.694+0000 7f5348b56640 1 --2- 172.21.15.89:0/3592240161 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55a1ae1b6e70 0x55a1ae1ff2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:21.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.694+0000 7f533bfff640 1 --2- 172.21.15.89:0/3592240161 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1ae200a70 0x55a1ae1ff800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:21.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.694+0000 7f5349357640 1 -- 172.21.15.89:0/3592240161 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55a1ae1b6e70 msgr2=0x55a1ae1ff2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:21.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.694+0000 7f5349357640 1 --2- 172.21.15.89:0/3592240161 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55a1ae1b6e70 0x55a1ae1ff2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.694+0000 7f5349357640 1 -- 172.21.15.89:0/3592240161 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1ae200a70 msgr2=0x55a1ae1ff800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:21.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.694+0000 7f5349357640 1 --2- 172.21.15.89:0/3592240161 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1ae200a70 0x55a1ae1ff800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.694+0000 7f5349357640 1 -- 172.21.15.89:0/3592240161 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1adfb0420 con 0x55a1ae201390 2023-10-18T10:47:21.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.694+0000 7f5349357640 1 --2- 172.21.15.89:0/3592240161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1ae201390 0x55a1ae210b10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f533c007510 tx=0x7f533c00c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2f99e955850ca6e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:21.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.694+0000 7f532affd640 1 -- 172.21.15.89:0/3592240161 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533c001ec0 con 0x55a1ae201390 2023-10-18T10:47:21.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.694+0000 7f532affd640 1 -- 172.21.15.89:0/3592240161 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f533c00b030 con 0x55a1ae201390 2023-10-18T10:47:21.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.694+0000 7f532affd640 1 -- 172.21.15.89:0/3592240161 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533c001350 con 0x55a1ae201390 2023-10-18T10:47:21.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.694+0000 7f534e123000 1 -- 172.21.15.89:0/3592240161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1ae201390 msgr2=0x55a1ae210b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:21.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.694+0000 7f534e123000 1 --2- 172.21.15.89:0/3592240161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1ae201390 0x55a1ae210b10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f533c007510 tx=0x7f533c00c000 comp rx=0 tx=0).stop 2023-10-18T10:47:21.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.694+0000 7f534e123000 1 -- 172.21.15.89:0/3592240161 shutdown_connections 2023-10-18T10:47:21.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.694+0000 7f534e123000 1 --2- 172.21.15.89:0/3592240161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1ae201390 0x55a1ae210b10 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.694+0000 7f534e123000 1 --2- 172.21.15.89:0/3592240161 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1ae200a70 0x55a1ae1ff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.694+0000 7f534e123000 1 --2- 172.21.15.89:0/3592240161 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55a1ae1b6e70 0x55a1ae1ff2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.694+0000 7f534e123000 1 -- 172.21.15.89:0/3592240161 >> 172.21.15.89:0/3592240161 conn(0x55a1ae1ae1e0 msgr2=0x55a1ae1c2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:21.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.694+0000 7f534e123000 1 -- 172.21.15.89:0/3592240161 shutdown_connections 2023-10-18T10:47:21.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.694+0000 7f534e123000 1 -- 172.21.15.89:0/3592240161 wait complete. 2023-10-18T10:47:21.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.694+0000 7f534e123000 1 -- start start 2023-10-18T10:47:21.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.694+0000 7f534e123000 1 Processor -- start 2023-10-18T10:47:21.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f534e123000 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1ae1b6e70 0x55a1ae213b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:21.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f534e123000 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55a1ae200a70 0x55a1ae2246f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:21.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f534e123000 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1ae201390 0x55a1ae224c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:21.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f534e123000 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1ae009ca0 con 0x55a1ae1b6e70 2023-10-18T10:47:21.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f534e123000 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1ae00e930 con 0x55a1ae201390 2023-10-18T10:47:21.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f534e123000 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1adfe2a00 con 0x55a1ae200a70 2023-10-18T10:47:21.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f534e123000 10 client.0 _ll_register_callbacks cb 0x7f533c001e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T10:47:21.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f5348b56640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1ae1b6e70 0x55a1ae213b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:21.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f5348b56640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1ae1b6e70 0x55a1ae213b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38112/0 (socket says 172.21.15.89:38112) 2023-10-18T10:47:21.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f5348b56640 1 -- 172.21.15.89:0/2597382453 learned_addr learned my addr 172.21.15.89:0/2597382453 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:21.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f533bfff640 1 --2- 172.21.15.89:0/2597382453 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55a1ae200a70 0x55a1ae2246f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:21.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f5349357640 1 --2- 172.21.15.89:0/2597382453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1ae201390 0x55a1ae224c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:21.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f5348b56640 1 -- 172.21.15.89:0/2597382453 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55a1ae200a70 msgr2=0x55a1ae2246f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:21.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f5348b56640 1 --2- 172.21.15.89:0/2597382453 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55a1ae200a70 0x55a1ae2246f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f5348b56640 1 -- 172.21.15.89:0/2597382453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1ae201390 msgr2=0x55a1ae224c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:21.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f5348b56640 1 --2- 172.21.15.89:0/2597382453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1ae201390 0x55a1ae224c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f5348b56640 1 -- 172.21.15.89:0/2597382453 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a1add01890 con 0x55a1ae1b6e70 2023-10-18T10:47:21.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f5348b56640 1 --2- 172.21.15.89:0/2597382453 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1ae1b6e70 0x55a1ae213b40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f534000fdf0 tx=0x7f5340001160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67a40614e4c4fa5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:21.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f53237fe640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-18T10:47:21.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f53237fe640 1 -- 172.21.15.89:0/2597382453 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5340029480 con 0x55a1ae1b6e70 2023-10-18T10:47:21.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f53237fe640 1 -- 172.21.15.89:0/2597382453 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f534001c040 con 0x55a1ae1b6e70 2023-10-18T10:47:21.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f534e123000 1 -- 172.21.15.89:0/2597382453 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a1ae00c030 con 0x55a1ae1b6e70 2023-10-18T10:47:21.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f53237fe640 1 -- 172.21.15.89:0/2597382453 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5340014450 con 0x55a1ae1b6e70 2023-10-18T10:47:21.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f534e123000 10 client.5697 fetch_fsmap learned FSMap version 18 2023-10-18T10:47:21.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f534e123000 1 -- 172.21.15.89:0/2597382453 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55a1add01cc0 con 0x55a1ae1b6e70 2023-10-18T10:47:21.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f53237fe640 1 -- 172.21.15.89:0/2597382453 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a1ae00c030 con 0x55a1ae1b6e70 2023-10-18T10:47:21.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f53237fe640 1 -- 172.21.15.89:0/2597382453 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5340012070 con 0x55a1ae1b6e70 2023-10-18T10:47:21.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f534e123000 10 client.5697 fetch_fsmap finished waiting for FSMap version 18 2023-10-18T10:47:21.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f534e123000 10 client.5697 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-18T10:47:21.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f534e123000 20 client.5697 populate_metadata read hostname 'smithi089' 2023-10-18T10:47:21.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f534e123000 1 --2- 172.21.15.89:0/2597382453 >> [v2:172.21.15.89:6834/2092274731,v1:172.21.15.89:6835/2092274731] conn(0x55a1ae26db50 0x55a1ae26ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:21.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f534e123000 4 client.5697 mds_command: new command op to 5517 tid=0[{"prefix": "status"}] 2023-10-18T10:47:21.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f534e123000 1 -- 172.21.15.89:0/2597382453 --> [v2:172.21.15.89:6834/2092274731,v1:172.21.15.89:6835/2092274731] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a1ae000120 con 0x55a1ae26db50 2023-10-18T10:47:21.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f533bfff640 1 --2- 172.21.15.89:0/2597382453 >> [v2:172.21.15.89:6834/2092274731,v1:172.21.15.89:6835/2092274731] conn(0x55a1ae26db50 0x55a1ae26ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:21.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f53237fe640 10 client.5697 ms_handle_connect on v2:172.21.15.89:6834/2092274731 2023-10-18T10:47:21.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f533bfff640 1 --2- 172.21.15.89:0/2597382453 >> [v2:172.21.15.89:6834/2092274731,v1:172.21.15.89:6835/2092274731] conn(0x55a1ae26db50 0x55a1ae26ff40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a357a6d791b93588 server_cookie=31fe2f050589dd06 in_seq=0 out_seq=0 2023-10-18T10:47:21.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f53237fe640 1 -- 172.21.15.89:0/2597382453 <== mds.0 v2:172.21.15.89:6834/2092274731 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55a1ae000120 con 0x55a1ae26db50 2023-10-18T10:47:21.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f53237fe640 10 client.5697 handle_command_reply: tid=0 2023-10-18T10:47:21.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f534e123000 1 client.5697 shutdown 2023-10-18T10:47:21.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f534e123000 1 -- 172.21.15.89:0/2597382453 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1ae1b6e70 msgr2=0x55a1ae213b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:21.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f534e123000 1 --2- 172.21.15.89:0/2597382453 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1ae1b6e70 0x55a1ae213b40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f534000fdf0 tx=0x7f5340001160 comp rx=0 tx=0).stop 2023-10-18T10:47:21.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f534e123000 1 -- 172.21.15.89:0/2597382453 shutdown_connections 2023-10-18T10:47:21.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f534e123000 1 --2- 172.21.15.89:0/2597382453 >> [v2:172.21.15.89:6834/2092274731,v1:172.21.15.89:6835/2092274731] conn(0x55a1ae26db50 0x55a1ae26ff40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f534e123000 1 --2- 172.21.15.89:0/2597382453 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55a1ae200a70 0x55a1ae2246f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f534e123000 1 --2- 172.21.15.89:0/2597382453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a1ae201390 0x55a1ae224c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f534e123000 1 --2- 172.21.15.89:0/2597382453 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1ae1b6e70 0x55a1ae213b40 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f53237fe640 0 client.5697 ms_handle_reset on v2:172.21.15.89:6834/2092274731 2023-10-18T10:47:21.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f534e123000 1 -- 172.21.15.89:0/2597382453 >> 172.21.15.89:0/2597382453 conn(0x55a1ae1ae1e0 msgr2=0x55a1ae1fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:21.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f534e123000 1 -- 172.21.15.89:0/2597382453 shutdown_connections 2023-10-18T10:47:21.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f534e123000 1 -- 172.21.15.89:0/2597382453 wait complete. 2023-10-18T10:47:21.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f534e123000 20 client.5697 trim_cache size 0 max 16384 2023-10-18T10:47:21.719 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-18T10:47:21.719 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "1352f62a-2330-4e96-afcd-b8e87cd82667", 2023-10-18T10:47:21.719 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 0, 2023-10-18T10:47:21.720 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 5517, 2023-10-18T10:47:21.720 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2023-10-18T10:47:21.720 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2023-10-18T10:47:21.720 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2023-10-18T10:47:21.720 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 5.8648407489999999, 2023-10-18T10:47:21.721 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 18, 2023-10-18T10:47:21.721 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 41, 2023-10-18T10:47:21.721 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 41, 2023-10-18T10:47:21.721 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 13.737249952000001 2023-10-18T10:47:21.721 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-18T10:47:21.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f534adba640 1 -- 172.21.15.89:0/2004492934 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5330065680 msgr2=0x7f5330067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:21.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f534adba640 1 --2- 172.21.15.89:0/2004492934 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5330065680 0x7f5330067b40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f53400011f0 tx=0x7f5340009040 comp rx=0 tx=0).stop 2023-10-18T10:47:21.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f534adba640 1 -- 172.21.15.89:0/2004492934 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5344160a10 msgr2=0x7f534415f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:21.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f534adba640 1 --2- 172.21.15.89:0/2004492934 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5344160a10 0x7f534415f640 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f533401b8c0 tx=0x7f533401ba50 comp rx=0 tx=0).stop 2023-10-18T10:47:21.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f534adba640 1 -- 172.21.15.89:0/2004492934 shutdown_connections 2023-10-18T10:47:21.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.698+0000 7f534adba640 1 --2- 172.21.15.89:0/2004492934 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5330065680 0x7f5330067b40 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.702+0000 7f534adba640 1 --2- 172.21.15.89:0/2004492934 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5344160e30 0x7f534414c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.702+0000 7f534adba640 1 --2- 172.21.15.89:0/2004492934 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5344160a10 0x7f534415f640 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.702+0000 7f534adba640 1 --2- 172.21.15.89:0/2004492934 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f534415c670 0x7f53441f5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.702+0000 7f534adba640 1 -- 172.21.15.89:0/2004492934 >> 172.21.15.89:0/2004492934 conn(0x7f53440cbdc0 msgr2=0x7f53440b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:21.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.702+0000 7f534adba640 1 -- 172.21.15.89:0/2004492934 shutdown_connections 2023-10-18T10:47:21.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.702+0000 7f534adba640 1 -- 172.21.15.89:0/2004492934 wait complete. 2023-10-18T10:47:21.725 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T10:47:21.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.858+0000 7fefaf340640 1 Processor -- start 2023-10-18T10:47:21.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.862+0000 7fefaf340640 1 -- start start 2023-10-18T10:47:21.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.862+0000 7fefaf340640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fefa8069900 0x7fefa8069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:21.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.862+0000 7fefaf340640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fefa815c8f0 0x7fefa806c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:21.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.862+0000 7fefaf340640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefa806a220 con 0x7fefa806cd60 2023-10-18T10:47:21.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.862+0000 7fefaf340640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefa806d210 con 0x7fefa8069900 2023-10-18T10:47:21.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.862+0000 7fefaf340640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefa815cdf0 con 0x7fefa815c8f0 2023-10-18T10:47:21.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.862+0000 7fefad0dc640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fefa8069900 0x7fefa8069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:21.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.862+0000 7fefad0dc640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fefa8069900 0x7fefa8069ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38118/0 (socket says 172.21.15.89:38118) 2023-10-18T10:47:21.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.862+0000 7fefad0dc640 1 -- 172.21.15.89:0/266936097 learned_addr learned my addr 172.21.15.89:0/266936097 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:21.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.862+0000 7fefac8db640 1 --2- 172.21.15.89:0/266936097 >> v2:172.21.15.154:3300/0 conn(0x7fefa815c8f0 0x7fefa806c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:21.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.862+0000 7fefad0dc640 1 -- 172.21.15.89:0/266936097 >> v2:172.21.15.154:3300/0 conn(0x7fefa815c8f0 msgr2=0x7fefa806c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:21.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.862+0000 7fefad0dc640 1 --2- 172.21.15.89:0/266936097 >> v2:172.21.15.154:3300/0 conn(0x7fefa815c8f0 0x7fefa806c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.862+0000 7fefad0dc640 1 -- 172.21.15.89:0/266936097 >> v1:172.21.15.154:6789/0 conn(0x7fefa806cd60 legacy=0x7fefa806da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:21.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.862+0000 7fefad0dc640 1 -- 172.21.15.89:0/266936097 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa815d0d0 con 0x7fefa8069900 2023-10-18T10:47:21.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.862+0000 7fefac8db640 1 --2- 172.21.15.89:0/266936097 >> v2:172.21.15.154:3300/0 conn(0x7fefa815c8f0 0x7fefa806c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:47:21.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.866+0000 7fefad0dc640 1 --2- 172.21.15.89:0/266936097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefa8069900 0x7fefa8069ce0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fefa80cc4e0 tx=0x7fefa401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9089d1584df92ce3 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:21.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.866+0000 7fef9ffff640 1 -- 172.21.15.89:0/266936097 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa40aa430 con 0x7fefa8069900 2023-10-18T10:47:21.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.866+0000 7fef9ffff640 1 -- 172.21.15.89:0/266936097 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa40aa590 con 0x7fefa8069900 2023-10-18T10:47:21.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.866+0000 7fef9ffff640 1 -- 172.21.15.89:0/266936097 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa40aa920 con 0x7fefa8069900 2023-10-18T10:47:21.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.866+0000 7fefaf340640 1 -- 172.21.15.89:0/266936097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefa8069900 msgr2=0x7fefa8069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:21.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.866+0000 7fefaf340640 1 --2- 172.21.15.89:0/266936097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefa8069900 0x7fefa8069ce0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fefa80cc4e0 tx=0x7fefa401c4d0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.866+0000 7fefaf340640 1 -- 172.21.15.89:0/266936097 shutdown_connections 2023-10-18T10:47:21.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.866+0000 7fefaf340640 1 --2- 172.21.15.89:0/266936097 >> v2:172.21.15.154:3300/0 conn(0x7fefa815c8f0 0x7fefa806c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.866+0000 7fefaf340640 1 --2- 172.21.15.89:0/266936097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefa8069900 0x7fefa8069ce0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.866+0000 7fefaf340640 1 -- 172.21.15.89:0/266936097 >> 172.21.15.89:0/266936097 conn(0x7fefa80cb5a0 msgr2=0x7fefa80cb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:21.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.866+0000 7fefaf340640 1 -- 172.21.15.89:0/266936097 shutdown_connections 2023-10-18T10:47:21.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.866+0000 7fefaf340640 1 -- 172.21.15.89:0/266936097 wait complete. 2023-10-18T10:47:21.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.870+0000 7fefaf340640 1 Processor -- start 2023-10-18T10:47:21.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.870+0000 7fefaf340640 1 -- start start 2023-10-18T10:47:21.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.870+0000 7fefaf340640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefa8069900 0x7fefa8159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:21.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.870+0000 7fefaf340640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefa806cd60 0x7fefa80c1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:21.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.870+0000 7fefaf340640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefa815c8f0 0x7fefa80c1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:21.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.870+0000 7fefaf340640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa814cb00 con 0x7fefa815c8f0 2023-10-18T10:47:21.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.870+0000 7fefaf340640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa815b080 con 0x7fefa806cd60 2023-10-18T10:47:21.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.870+0000 7fefaf340640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa815b360 con 0x7fefa8069900 2023-10-18T10:47:21.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.870+0000 7fefad8dd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefa815c8f0 0x7fefa80c1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:21.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.870+0000 7fefad0dc640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefa8069900 0x7fefa8159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:21.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.870+0000 7fefad8dd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefa815c8f0 0x7fefa80c1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38132/0 (socket says 172.21.15.89:38132) 2023-10-18T10:47:21.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.870+0000 7fefad0dc640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefa8069900 0x7fefa8159480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:50676/0 (socket says 172.21.15.89:50676) 2023-10-18T10:47:21.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.870+0000 7fefac8db640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefa806cd60 0x7fefa80c1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:21.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.870+0000 7fefad8dd640 1 -- 172.21.15.89:0/2148976441 learned_addr learned my addr 172.21.15.89:0/2148976441 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:21.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.870+0000 7fefad8dd640 1 -- 172.21.15.89:0/2148976441 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefa8069900 msgr2=0x7fefa8159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:21.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.870+0000 7fefad8dd640 1 --2- 172.21.15.89:0/2148976441 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefa8069900 0x7fefa8159480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.870+0000 7fefad8dd640 1 -- 172.21.15.89:0/2148976441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefa806cd60 msgr2=0x7fefa80c1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:21.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.870+0000 7fefad8dd640 1 --2- 172.21.15.89:0/2148976441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefa806cd60 0x7fefa80c1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:21.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.870+0000 7fefad8dd640 1 -- 172.21.15.89:0/2148976441 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa40094d0 con 0x7fefa815c8f0 2023-10-18T10:47:21.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.870+0000 7fefad8dd640 1 --2- 172.21.15.89:0/2148976441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefa815c8f0 0x7fefa80c1ed0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fefa000c770 tx=0x7fefa000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:21.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.870+0000 7fef9dffb640 1 -- 172.21.15.89:0/2148976441 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa0003b80 con 0x7fefa815c8f0 2023-10-18T10:47:21.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.870+0000 7fef9dffb640 1 -- 172.21.15.89:0/2148976441 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa0003ce0 con 0x7fefa815c8f0 2023-10-18T10:47:21.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.870+0000 7fef9dffb640 1 -- 172.21.15.89:0/2148976441 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa000aa90 con 0x7fefa815c8f0 2023-10-18T10:47:21.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.870+0000 7fefaf340640 1 -- 172.21.15.89:0/2148976441 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefa815b640 con 0x7fefa815c8f0 2023-10-18T10:47:21.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.870+0000 7fefaf340640 1 -- 172.21.15.89:0/2148976441 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa815bc00 con 0x7fefa815c8f0 2023-10-18T10:47:21.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.870+0000 7fefaf340640 1 -- 172.21.15.89:0/2148976441 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefa804dbe0 con 0x7fefa815c8f0 2023-10-18T10:47:21.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.874+0000 7fef9dffb640 1 -- 172.21.15.89:0/2148976441 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fefa000abf0 con 0x7fefa815c8f0 2023-10-18T10:47:21.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.874+0000 7fef9dffb640 1 --2- 172.21.15.89:0/2148976441 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fef8c0656b0 0x7fef8c067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:21.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.874+0000 7fef9dffb640 1 -- 172.21.15.89:0/2148976441 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fefa0082c00 con 0x7fefa815c8f0 2023-10-18T10:47:21.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.874+0000 7fefad0dc640 1 --2- 172.21.15.89:0/2148976441 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fef8c0656b0 0x7fef8c067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:21.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.874+0000 7fefad0dc640 1 --2- 172.21.15.89:0/2148976441 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fef8c0656b0 0x7fef8c067b70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fefa40ae000 tx=0x7fefa40b5040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:21.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:21.878+0000 7fef9dffb640 1 -- 172.21.15.89:0/2148976441 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fefa0089080 con 0x7fefa815c8f0 2023-10-18T10:47:22.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.058+0000 7fefaf340640 1 -- 172.21.15.89:0/2148976441 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fefa806de10 con 0x7fefa815c8f0 2023-10-18T10:47:22.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.058+0000 7fef9dffb640 1 -- 172.21.15.89:0/2148976441 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+73 (secure 0 0 0) 0x7fefa0050630 con 0x7fefa815c8f0 2023-10-18T10:47:22.063 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T10:47:22.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.062+0000 7fefaf340640 1 -- 172.21.15.89:0/2148976441 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fef8c0656b0 msgr2=0x7fef8c067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:22.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.062+0000 7fefaf340640 1 --2- 172.21.15.89:0/2148976441 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fef8c0656b0 0x7fef8c067b70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fefa40ae000 tx=0x7fefa40b5040 comp rx=0 tx=0).stop 2023-10-18T10:47:22.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.062+0000 7fefaf340640 1 -- 172.21.15.89:0/2148976441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefa815c8f0 msgr2=0x7fefa80c1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:22.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.062+0000 7fefaf340640 1 --2- 172.21.15.89:0/2148976441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefa815c8f0 0x7fefa80c1ed0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fefa000c770 tx=0x7fefa000cc30 comp rx=0 tx=0).stop 2023-10-18T10:47:22.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.062+0000 7fefaf340640 1 -- 172.21.15.89:0/2148976441 shutdown_connections 2023-10-18T10:47:22.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.062+0000 7fefaf340640 1 --2- 172.21.15.89:0/2148976441 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fef8c0656b0 0x7fef8c067b70 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:22.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.062+0000 7fefaf340640 1 --2- 172.21.15.89:0/2148976441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefa815c8f0 0x7fefa80c1ed0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:22.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.062+0000 7fefaf340640 1 --2- 172.21.15.89:0/2148976441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefa806cd60 0x7fefa80c1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:22.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.062+0000 7fefaf340640 1 --2- 172.21.15.89:0/2148976441 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fefa8069900 0x7fefa8159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:22.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.062+0000 7fefaf340640 1 -- 172.21.15.89:0/2148976441 >> 172.21.15.89:0/2148976441 conn(0x7fefa80cb5a0 msgr2=0x7fefa80cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:22.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.062+0000 7fefaf340640 1 -- 172.21.15.89:0/2148976441 shutdown_connections 2023-10-18T10:47:22.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.062+0000 7fefaf340640 1 -- 172.21.15.89:0/2148976441 wait complete. 2023-10-18T10:47:22.083 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-18T10:47:22.084 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T10:47:22.084 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi089.front.sepia.ceph.com 2023-10-18T10:47:22.084 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-18T10:47:22.085 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-18T10:47:22.085 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-18T10:47:22.085 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-18T10:47:22.086 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T10:47:22.086 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T10:47:22.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:47:22.087 DEBUG:teuthology.orchestra.run.smithi089:> ip addr 2023-10-18T10:47:22.097 INFO:teuthology.orchestra.run.smithi089.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T10:47:22.097 INFO:teuthology.orchestra.run.smithi089.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T10:47:22.097 INFO:teuthology.orchestra.run.smithi089.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T10:47:22.098 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:47:22.098 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 ::1/128 scope host 2023-10-18T10:47:22.098 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:47:22.098 INFO:teuthology.orchestra.run.smithi089.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-18T10:47:22.099 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c6 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:47:22.099 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f0 2023-10-18T10:47:22.099 INFO:teuthology.orchestra.run.smithi089.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-18T10:47:22.099 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c7 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:47:22.099 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f1 2023-10-18T10:47:22.100 INFO:teuthology.orchestra.run.smithi089.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-18T10:47:22.100 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:00 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:47:22.100 INFO:teuthology.orchestra.run.smithi089.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T10:47:22.100 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:01 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:47:22.100 INFO:teuthology.orchestra.run.smithi089.stdout: inet 172.21.15.89/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-18T10:47:22.101 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft 42307sec preferred_lft 42307sec 2023-10-18T10:47:22.101 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec4:7aff:fe8f:d501/64 scope link 2023-10-18T10:47:22.101 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:47:22.101 INFO:teuthology.orchestra.run.smithi089.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-18T10:47:22.101 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 02:42:3f:fd:35:8d brd ff:ff:ff:ff:ff:ff 2023-10-18T10:47:22.102 INFO:teuthology.orchestra.run.smithi089.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-18T10:47:22.102 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:47:22.102 INFO:teuthology.orchestra.run.smithi089.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T10:47:22.102 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-18T10:47:22.102 INFO:teuthology.orchestra.run.smithi089.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T10:47:22.103 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:47:22.103 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-18T10:47:22.103 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:47:22.103 INFO:teuthology.orchestra.run.smithi089.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T10:47:22.104 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:47:22.104 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-18T10:47:22.104 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:47:22.104 INFO:teuthology.orchestra.run.smithi089.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T10:47:22.104 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:47:22.105 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-10-18T10:47:22.105 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:47:22.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:47:22.106 DEBUG:teuthology.orchestra.run.smithi089:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T10:47:22.162 INFO:teuthology.orchestra.run.smithi089.stdout:1 2023-10-18T10:47:22.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:47:22.165 DEBUG:teuthology.orchestra.run.smithi089:> route 2023-10-18T10:47:22.218 INFO:teuthology.orchestra.run.smithi089.stdout:Kernel IP routing table 2023-10-18T10:47:22.218 INFO:teuthology.orchestra.run.smithi089.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T10:47:22.218 INFO:teuthology.orchestra.run.smithi089.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-18T10:47:22.219 INFO:teuthology.orchestra.run.smithi089.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T10:47:22.219 INFO:teuthology.orchestra.run.smithi089.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-18T10:47:22.219 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T10:47:22.219 INFO:teuthology.orchestra.run.smithi089.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-18T10:47:22.219 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-18T10:47:22.220 INFO:teuthology.orchestra.run.smithi089.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-18T10:47:22.220 INFO:teuthology.orchestra.run.smithi089.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T10:47:22.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:47:22.221 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-18T10:47:22.221 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-18T10:47:22.221 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T10:47:22.222 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T10:47:22.222 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T10:47:22.222 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-18T10:47:22.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:47:22.310 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-10-18T10:47:22.319 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-18T10:47:22.319 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-18T10:47:22.320 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T10:47:22.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:47:22.321 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set brx.0 up 2023-10-18T10:47:22.384 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:47:22.385 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-18T10:47:22.435 INFO:teuthology.orchestra.run.smithi089.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-18T10:47:22.437 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:47:22.437 DEBUG:teuthology.orchestra.run.smithi089:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-18T10:47:22.491 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-18T10:47:22.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:47:22.492 DEBUG:teuthology.orchestra.run.smithi089:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-18T10:47:22.580 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-18T10:47:22.580 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-18T10:47:22.581 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-18T10:47:22.595 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-18T10:47:22.595 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-18T10:47:22.595 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-18T10:47:22.596 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.154:3300,172.21.15.154:3301" 2023-10-18T10:47:22.596 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-18T10:47:22.605 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-18T10:47:22.605 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-18T10:47:22.605 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-18T10:47:22.605 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.154:3300,172.21.15.154:3301" 2023-10-18T10:47:22.606 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-18T10:47:22.606 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.154:6789,172.21.15.154:6790" 2023-10-18T10:47:22.606 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.154:6789,172.21.15.154:6790:/ 2023-10-18T10:47:22.606 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1352f62a-2330-4e96-afcd-b8e87cd82667" will pass to kernel 2023-10-18T10:47:22.609 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-18T10:47:22.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:47:22.610 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-18T10:47:22.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:47:22.632 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T10:47:22.696 INFO:teuthology.orchestra.run.smithi089.stdout:client5461 2023-10-18T10:47:22.696 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:47:22.697 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:47:22.748 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T10:47:22.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.930+0000 7fd1d685f640 1 Processor -- start 2023-10-18T10:47:22.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.930+0000 7fd1d685f640 1 -- start start 2023-10-18T10:47:22.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.930+0000 7fd1d685f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd1d0145b70 0x7fd1d0143470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:22.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.930+0000 7fd1d685f640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1d00cd8d0 con 0x7fd1d00cd4f0 2023-10-18T10:47:22.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.930+0000 7fd1d685f640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1d015cbf0 con 0x7fd1d00ce070 2023-10-18T10:47:22.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.930+0000 7fd1d685f640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1d0069900 con 0x7fd1d0145b70 2023-10-18T10:47:22.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.930+0000 7fd1cffff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd1d00cd4f0 0x7fd1d0160cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48932/0 (socket says 172.21.15.89:48932) 2023-10-18T10:47:22.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.930+0000 7fd1cffff640 1 -- 172.21.15.89:0/1717751232 learned_addr learned my addr 172.21.15.89:0/1717751232 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:47:22.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.934+0000 7fd1cf7fe640 1 --2- 172.21.15.89:0/1717751232 >> v2:172.21.15.154:3300/0 conn(0x7fd1d0145b70 0x7fd1d0143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:22.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.934+0000 7fd1ceffd640 1 -- 172.21.15.89:0/1717751232 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1399551058 0 0) 0x7fd1d00cd8d0 con 0x7fd1d00cd4f0 2023-10-18T10:47:22.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.934+0000 7fd1ceffd640 1 -- 172.21.15.89:0/1717751232 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1b8003530 con 0x7fd1d00cd4f0 2023-10-18T10:47:22.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.934+0000 7fd1ceffd640 1 -- 172.21.15.89:0/1717751232 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1257487244 0 0) 0x7fd1b8003530 con 0x7fd1d00cd4f0 2023-10-18T10:47:22.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.934+0000 7fd1ceffd640 1 -- 172.21.15.89:0/1717751232 >> v2:172.21.15.154:3300/0 conn(0x7fd1d0145b70 msgr2=0x7fd1d0143470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:22.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.934+0000 7fd1ceffd640 1 --2- 172.21.15.89:0/1717751232 >> v2:172.21.15.154:3300/0 conn(0x7fd1d0145b70 0x7fd1d0143470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:22.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.934+0000 7fd1ceffd640 1 -- 172.21.15.89:0/1717751232 >> v1:172.21.15.154:6789/0 conn(0x7fd1d00ce070 legacy=0x7fd1d006abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:22.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.934+0000 7fd1ceffd640 1 -- 172.21.15.89:0/1717751232 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d0069b10 con 0x7fd1d00cd4f0 2023-10-18T10:47:22.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.934+0000 7fd1ceffd640 1 -- 172.21.15.89:0/1717751232 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7fd1c00016a0 con 0x7fd1d00cd4f0 2023-10-18T10:47:22.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.934+0000 7fd1ceffd640 1 -- 172.21.15.89:0/1717751232 >> v1:172.21.15.89:6789/0 conn(0x7fd1d00cd4f0 legacy=0x7fd1d0160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:22.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.934+0000 7fd1ceffd640 1 --2- 172.21.15.89:0/1717751232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1b8003980 0x7fd1b8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:22.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.934+0000 7fd1ceffd640 1 --2- 172.21.15.89:0/1717751232 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1b80047d0 0x7fd1b8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:22.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.934+0000 7fd1ceffd640 1 --2- 172.21.15.89:0/1717751232 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1b8005190 0x7fd1b800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:22.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.934+0000 7fd1ceffd640 1 -- 172.21.15.89:0/1717751232 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d0069900 con 0x7fd1b8003980 2023-10-18T10:47:22.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.934+0000 7fd1ceffd640 1 -- 172.21.15.89:0/1717751232 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b800c030 con 0x7fd1b8005190 2023-10-18T10:47:22.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.934+0000 7fd1ceffd640 1 -- 172.21.15.89:0/1717751232 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b800c1f0 con 0x7fd1b80047d0 2023-10-18T10:47:22.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.934+0000 7fd1cffff640 1 --2- 172.21.15.89:0/1717751232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1b8003980 0x7fd1b8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:22.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.934+0000 7fd1cf7fe640 1 --2- 172.21.15.89:0/1717751232 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1b80047d0 0x7fd1b8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:22.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.934+0000 7fd1d4dfc640 1 --2- 172.21.15.89:0/1717751232 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1b8005190 0x7fd1b800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:22.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.934+0000 7fd1cffff640 1 -- 172.21.15.89:0/1717751232 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1b80047d0 msgr2=0x7fd1b8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:22.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.934+0000 7fd1cffff640 1 --2- 172.21.15.89:0/1717751232 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1b80047d0 0x7fd1b8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:22.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.934+0000 7fd1cffff640 1 -- 172.21.15.89:0/1717751232 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1b8005190 msgr2=0x7fd1b800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:22.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.934+0000 7fd1cffff640 1 --2- 172.21.15.89:0/1717751232 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1b8005190 0x7fd1b800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:22.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.934+0000 7fd1cffff640 1 -- 172.21.15.89:0/1717751232 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b8003530 con 0x7fd1b8003980 2023-10-18T10:47:22.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.934+0000 7fd1d4dfc640 1 --2- 172.21.15.89:0/1717751232 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1b8005190 0x7fd1b800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:47:22.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.934+0000 7fd1d4dfc640 1 -- 172.21.15.89:0/1717751232 reap_dead start 2023-10-18T10:47:22.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.938+0000 7fd1cffff640 1 --2- 172.21.15.89:0/1717751232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1b8003980 0x7fd1b8003da0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd1d00cda60 tx=0x7fd1c001cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2a98cb44831d94 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:22.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.938+0000 7fd1ceffd640 1 -- 172.21.15.89:0/1717751232 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0005ae0 con 0x7fd1b8003980 2023-10-18T10:47:22.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.938+0000 7fd1ceffd640 1 -- 172.21.15.89:0/1717751232 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c0005c40 con 0x7fd1b8003980 2023-10-18T10:47:22.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.938+0000 7fd1ceffd640 1 -- 172.21.15.89:0/1717751232 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c00a59b0 con 0x7fd1b8003980 2023-10-18T10:47:22.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.938+0000 7fd1d685f640 1 -- 172.21.15.89:0/1717751232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1b8003980 msgr2=0x7fd1b8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:22.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.938+0000 7fd1d685f640 1 --2- 172.21.15.89:0/1717751232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1b8003980 0x7fd1b8003da0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd1d00cda60 tx=0x7fd1c001cbd0 comp rx=0 tx=0).stop 2023-10-18T10:47:22.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.938+0000 7fd1d685f640 1 -- 172.21.15.89:0/1717751232 shutdown_connections 2023-10-18T10:47:22.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.938+0000 7fd1d685f640 1 --2- 172.21.15.89:0/1717751232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1b8003980 0x7fd1b8003da0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:22.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.938+0000 7fd1d685f640 1 -- 172.21.15.89:0/1717751232 >> 172.21.15.89:0/1717751232 conn(0x7fd1d006dad0 msgr2=0x7fd1d006dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:22.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.938+0000 7fd1d685f640 1 -- 172.21.15.89:0/1717751232 shutdown_connections 2023-10-18T10:47:22.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.938+0000 7fd1d685f640 1 -- 172.21.15.89:0/1717751232 wait complete. 2023-10-18T10:47:22.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.938+0000 7fd1d685f640 1 Processor -- start 2023-10-18T10:47:22.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.938+0000 7fd1d685f640 1 -- start start 2023-10-18T10:47:22.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.942+0000 7fd1d685f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1b8003980 0x7fd1d01f60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:22.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.942+0000 7fd1d685f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1d00b53c0 0x7fd1d01579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:22.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.942+0000 7fd1d685f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1d0157f00 0x7fd1d015de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:22.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.942+0000 7fd1d685f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d01f2f00 con 0x7fd1b8003980 2023-10-18T10:47:22.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.942+0000 7fd1d685f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d0158680 con 0x7fd1d00b53c0 2023-10-18T10:47:22.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.942+0000 7fd1d685f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d01583a0 con 0x7fd1d0157f00 2023-10-18T10:47:22.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.942+0000 7fd1cffff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1b8003980 0x7fd1d01f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:22.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.942+0000 7fd1cffff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1b8003980 0x7fd1d01f60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50388/0 (socket says 172.21.15.89:50388) 2023-10-18T10:47:22.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.942+0000 7fd1cffff640 1 -- 172.21.15.89:0/3770404647 learned_addr learned my addr 172.21.15.89:0/3770404647 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:22.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.942+0000 7fd1cf7fe640 1 --2- 172.21.15.89:0/3770404647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1d00b53c0 0x7fd1d01579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:22.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.942+0000 7fd1d4dfc640 1 --2- 172.21.15.89:0/3770404647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1d0157f00 0x7fd1d015de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:22.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.942+0000 7fd1cffff640 1 -- 172.21.15.89:0/3770404647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1d0157f00 msgr2=0x7fd1d015de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:22.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.942+0000 7fd1cffff640 1 --2- 172.21.15.89:0/3770404647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1d0157f00 0x7fd1d015de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:22.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.942+0000 7fd1cffff640 1 -- 172.21.15.89:0/3770404647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1d00b53c0 msgr2=0x7fd1d01579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:22.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.942+0000 7fd1cffff640 1 --2- 172.21.15.89:0/3770404647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1d00b53c0 0x7fd1d01579c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:22.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.942+0000 7fd1cffff640 1 -- 172.21.15.89:0/3770404647 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d01f55e0 con 0x7fd1b8003980 2023-10-18T10:47:22.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.942+0000 7fd1cffff640 1 --2- 172.21.15.89:0/3770404647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1b8003980 0x7fd1d01f60c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd1c001c640 tx=0x7fd1b8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:22.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.942+0000 7fd1ccff9640 1 -- 172.21.15.89:0/3770404647 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c00b4070 con 0x7fd1b8003980 2023-10-18T10:47:22.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.942+0000 7fd1ccff9640 1 -- 172.21.15.89:0/3770404647 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c0001560 con 0x7fd1b8003980 2023-10-18T10:47:22.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.942+0000 7fd1d685f640 1 -- 172.21.15.89:0/3770404647 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1d01f58c0 con 0x7fd1b8003980 2023-10-18T10:47:22.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.942+0000 7fd1ccff9640 1 -- 172.21.15.89:0/3770404647 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c00a59d0 con 0x7fd1b8003980 2023-10-18T10:47:22.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.942+0000 7fd1d685f640 1 -- 172.21.15.89:0/3770404647 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1d015e350 con 0x7fd1b8003980 2023-10-18T10:47:22.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.942+0000 7fd1d685f640 1 -- 172.21.15.89:0/3770404647 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1d004dbe0 con 0x7fd1b8003980 2023-10-18T10:47:22.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.942+0000 7fd1ccff9640 1 -- 172.21.15.89:0/3770404647 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fd1c00a5b30 con 0x7fd1b8003980 2023-10-18T10:47:22.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.946+0000 7fd1ccff9640 1 --2- 172.21.15.89:0/3770404647 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd1bc0656b0 0x7fd1bc067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:22.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.946+0000 7fd1ccff9640 1 -- 172.21.15.89:0/3770404647 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd1c011c750 con 0x7fd1b8003980 2023-10-18T10:47:22.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.946+0000 7fd1cf7fe640 1 --2- 172.21.15.89:0/3770404647 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd1bc0656b0 0x7fd1bc067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:22.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.946+0000 7fd1cf7fe640 1 --2- 172.21.15.89:0/3770404647 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd1bc0656b0 0x7fd1bc067b70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0161b00 tx=0x7fd1c4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:22.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:22.950+0000 7fd1ccff9640 1 -- 172.21.15.89:0/3770404647 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fd1c00ea0d0 con 0x7fd1b8003980 2023-10-18T10:47:23.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.130+0000 7fd1d685f640 1 -- 172.21.15.89:0/3770404647 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd1d006a380 con 0x7fd1b8003980 2023-10-18T10:47:23.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.130+0000 7fd1ccff9640 1 -- 172.21.15.89:0/3770404647 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd1c00ed6f0 con 0x7fd1b8003980 2023-10-18T10:47:23.134 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T10:47:23.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.130+0000 7fd1d685f640 1 -- 172.21.15.89:0/3770404647 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd1bc0656b0 msgr2=0x7fd1bc067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:23.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.130+0000 7fd1d685f640 1 --2- 172.21.15.89:0/3770404647 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd1bc0656b0 0x7fd1bc067b70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0161b00 tx=0x7fd1c4008040 comp rx=0 tx=0).stop 2023-10-18T10:47:23.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.130+0000 7fd1d685f640 1 -- 172.21.15.89:0/3770404647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1b8003980 msgr2=0x7fd1d01f60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:23.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.130+0000 7fd1d685f640 1 --2- 172.21.15.89:0/3770404647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1b8003980 0x7fd1d01f60c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd1c001c640 tx=0x7fd1b8005800 comp rx=0 tx=0).stop 2023-10-18T10:47:23.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.130+0000 7fd1d685f640 1 -- 172.21.15.89:0/3770404647 shutdown_connections 2023-10-18T10:47:23.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.130+0000 7fd1d685f640 1 --2- 172.21.15.89:0/3770404647 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd1bc0656b0 0x7fd1bc067b70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:23.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.130+0000 7fd1d685f640 1 --2- 172.21.15.89:0/3770404647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1d0157f00 0x7fd1d015de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:23.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.130+0000 7fd1d685f640 1 --2- 172.21.15.89:0/3770404647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1d00b53c0 0x7fd1d01579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:23.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.130+0000 7fd1d685f640 1 --2- 172.21.15.89:0/3770404647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1b8003980 0x7fd1d01f60c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:23.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.130+0000 7fd1d685f640 1 -- 172.21.15.89:0/3770404647 >> 172.21.15.89:0/3770404647 conn(0x7fd1d006dad0 msgr2=0x7fd1d0160470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:23.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.130+0000 7fd1d685f640 1 -- 172.21.15.89:0/3770404647 shutdown_connections 2023-10-18T10:47:23.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.130+0000 7fd1d685f640 1 -- 172.21.15.89:0/3770404647 wait complete. 2023-10-18T10:47:23.154 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-18T10:47:23.155 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T10:47:23.155 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2023-10-18T10:47:23.156 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-18T10:47:23.156 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-18T10:47:23.156 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-18T10:47:23.157 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-18T10:47:23.157 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T10:47:23.157 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-18T10:47:23.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:47:23.158 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2023-10-18T10:47:23.168 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T10:47:23.169 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T10:47:23.169 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T10:47:23.169 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:47:23.169 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2023-10-18T10:47:23.170 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:47:23.170 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-18T10:47:23.170 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:47:23.170 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f0 2023-10-18T10:47:23.170 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-18T10:47:23.170 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:47:23.171 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f1 2023-10-18T10:47:23.171 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-18T10:47:23.171 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:47:23.171 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T10:47:23.171 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:47:23.172 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-18T10:47:23.172 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 42336sec preferred_lft 42336sec 2023-10-18T10:47:23.172 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::ec4:7aff:fe88:7761/64 scope link 2023-10-18T10:47:23.172 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:47:23.172 INFO:teuthology.orchestra.run.smithi154.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-18T10:47:23.173 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 02:42:60:ea:f7:c5 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:47:23.173 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-18T10:47:23.173 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:47:23.173 INFO:teuthology.orchestra.run.smithi154.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T10:47:23.173 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-18T10:47:23.174 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T10:47:23.174 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:47:23.174 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-18T10:47:23.174 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:47:23.174 INFO:teuthology.orchestra.run.smithi154.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T10:47:23.175 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-18T10:47:23.175 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-18T10:47:23.175 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:47:23.175 INFO:teuthology.orchestra.run.smithi154.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T10:47:23.175 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-18T10:47:23.175 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-10-18T10:47:23.176 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:47:23.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:47:23.177 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T10:47:23.237 INFO:teuthology.orchestra.run.smithi154.stdout:1 2023-10-18T10:47:23.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:47:23.239 DEBUG:teuthology.orchestra.run.smithi154:> route 2023-10-18T10:47:23.297 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2023-10-18T10:47:23.297 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T10:47:23.297 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-18T10:47:23.297 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T10:47:23.298 INFO:teuthology.orchestra.run.smithi154.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-18T10:47:23.298 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T10:47:23.298 INFO:teuthology.orchestra.run.smithi154.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-18T10:47:23.298 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-18T10:47:23.299 INFO:teuthology.orchestra.run.smithi154.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-18T10:47:23.299 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T10:47:23.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:47:23.300 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-10-18T10:47:23.300 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-10-18T10:47:23.300 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T10:47:23.301 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T10:47:23.301 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T10:47:23.301 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-10-18T10:47:23.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:47:23.395 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2023-10-18T10:47:23.404 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-18T10:47:23.404 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-18T10:47:23.405 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-18T10:47:23.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:47:23.406 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.0 up 2023-10-18T10:47:23.474 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:47:23.474 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-18T10:47:23.525 INFO:teuthology.orchestra.run.smithi154.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-18T10:47:23.525 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:47:23.526 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-18T10:47:23.581 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-18T10:47:23.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:47:23.582 DEBUG:teuthology.orchestra.run.smithi154:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-18T10:47:23.675 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-18T10:47:23.676 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-18T10:47:23.676 INFO:teuthology.orchestra.run.smithi154.stdout:invalid new device string format 2023-10-18T10:47:23.691 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-18T10:47:23.691 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-18T10:47:23.691 INFO:teuthology.orchestra.run.smithi154.stdout:invalid new device string format 2023-10-18T10:47:23.691 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.154:3300,172.21.15.154:3301" 2023-10-18T10:47:23.692 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-18T10:47:23.700 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-18T10:47:23.700 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-18T10:47:23.701 INFO:teuthology.orchestra.run.smithi154.stdout:invalid new device string format 2023-10-18T10:47:23.701 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.154:3300,172.21.15.154:3301" 2023-10-18T10:47:23.701 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-18T10:47:23.701 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.154:6789,172.21.15.154:6790" 2023-10-18T10:47:23.702 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.154:6789,172.21.15.154:6790:/ 2023-10-18T10:47:23.702 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=1352f62a-2330-4e96-afcd-b8e87cd82667" will pass to kernel 2023-10-18T10:47:23.705 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-18T10:47:23.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:47:23.706 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-18T10:47:23.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:47:23.727 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T10:47:23.780 INFO:teuthology.orchestra.run.smithi154.stdout:client5183 2023-10-18T10:47:23.781 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T10:47:23.781 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:47:23.834 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:47:23.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.974+0000 7fc57ce34640 1 Processor -- start 2023-10-18T10:47:23.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.974+0000 7fc57ce34640 1 -- start start 2023-10-18T10:47:23.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.978+0000 7fc57ce34640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc57806ff00 0x7fc57815e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:23.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.978+0000 7fc57ce34640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5780ca340 0x7fc57815b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:23.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.978+0000 7fc57ce34640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc57815b680 0x7fc5780c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:23.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.978+0000 7fc57ce34640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc57815ef30 con 0x7fc57815b680 2023-10-18T10:47:23.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.978+0000 7fc57ce34640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc57815aa10 con 0x7fc57806ff00 2023-10-18T10:47:23.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.978+0000 7fc57ce34640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc57815acb0 con 0x7fc5780ca340 2023-10-18T10:47:23.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.978+0000 7fc576d9d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc57815b680 0x7fc5780c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:23.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.978+0000 7fc56ffff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5780ca340 0x7fc57815b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:23.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.978+0000 7fc56ffff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5780ca340 0x7fc57815b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:44876/0 (socket says 172.21.15.89:44876) 2023-10-18T10:47:23.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.978+0000 7fc56ffff640 1 -- 172.21.15.89:0/3938830905 learned_addr learned my addr 172.21.15.89:0/3938830905 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:23.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.978+0000 7fc57659c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc57806ff00 0x7fc57815e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:23.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.978+0000 7fc576d9d640 1 -- 172.21.15.89:0/3938830905 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5780ca340 msgr2=0x7fc57815b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:23.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.978+0000 7fc576d9d640 1 --2- 172.21.15.89:0/3938830905 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5780ca340 0x7fc57815b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:23.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.978+0000 7fc576d9d640 1 -- 172.21.15.89:0/3938830905 >> v2:172.21.15.154:3300/0 conn(0x7fc57806ff00 msgr2=0x7fc57815e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:23.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.978+0000 7fc576d9d640 1 --2- 172.21.15.89:0/3938830905 >> v2:172.21.15.154:3300/0 conn(0x7fc57806ff00 0x7fc57815e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:23.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.978+0000 7fc576d9d640 1 -- 172.21.15.89:0/3938830905 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5780702e0 con 0x7fc57815b680 2023-10-18T10:47:23.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.982+0000 7fc576d9d640 1 --2- 172.21.15.89:0/3938830905 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc57815b680 0x7fc5780c8960 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc57806a830 tx=0x7fc57001c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e7201ddc3e99577 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:23.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.982+0000 7fc575d9b640 1 -- 172.21.15.89:0/3938830905 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5700a57e0 con 0x7fc57815b680 2023-10-18T10:47:23.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.982+0000 7fc575d9b640 1 -- 172.21.15.89:0/3938830905 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5700a5940 con 0x7fc57815b680 2023-10-18T10:47:23.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.982+0000 7fc575d9b640 1 -- 172.21.15.89:0/3938830905 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5700a5cd0 con 0x7fc57815b680 2023-10-18T10:47:23.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.982+0000 7fc57ce34640 1 -- 172.21.15.89:0/3938830905 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc57815b680 msgr2=0x7fc5780c8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:23.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.982+0000 7fc57ce34640 1 --2- 172.21.15.89:0/3938830905 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc57815b680 0x7fc5780c8960 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc57806a830 tx=0x7fc57001c850 comp rx=0 tx=0).stop 2023-10-18T10:47:23.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.982+0000 7fc57ce34640 1 -- 172.21.15.89:0/3938830905 shutdown_connections 2023-10-18T10:47:23.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.982+0000 7fc57ce34640 1 --2- 172.21.15.89:0/3938830905 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc57815b680 0x7fc5780c8960 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:23.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.982+0000 7fc57ce34640 1 --2- 172.21.15.89:0/3938830905 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5780ca340 0x7fc57815b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:23.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.982+0000 7fc57ce34640 1 --2- 172.21.15.89:0/3938830905 >> v2:172.21.15.154:3300/0 conn(0x7fc57806ff00 0x7fc57815e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:23.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.982+0000 7fc57ce34640 1 -- 172.21.15.89:0/3938830905 >> 172.21.15.89:0/3938830905 conn(0x7fc578069f00 msgr2=0x7fc5780cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:23.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.982+0000 7fc57ce34640 1 -- 172.21.15.89:0/3938830905 shutdown_connections 2023-10-18T10:47:23.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.982+0000 7fc57ce34640 1 -- 172.21.15.89:0/3938830905 wait complete. 2023-10-18T10:47:23.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.982+0000 7fc57ce34640 1 Processor -- start 2023-10-18T10:47:23.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.982+0000 7fc57ce34640 1 -- start start 2023-10-18T10:47:23.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.986+0000 7fc57ce34640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc57806ff00 0x7fc57814b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:23.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.986+0000 7fc57ce34640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5780ca340 0x7fc5780c2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:23.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.986+0000 7fc57ce34640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc578152910 0x7fc578153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:23.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.986+0000 7fc57ce34640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5780ccf80 con 0x7fc5780ca340 2023-10-18T10:47:23.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.986+0000 7fc57ce34640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5780cc0b0 con 0x7fc57806ff00 2023-10-18T10:47:23.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.986+0000 7fc57ce34640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc57814b630 con 0x7fc578152910 2023-10-18T10:47:23.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.986+0000 7fc56ffff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5780ca340 0x7fc5780c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:23.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.986+0000 7fc56ffff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5780ca340 0x7fc5780c2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50402/0 (socket says 172.21.15.89:50402) 2023-10-18T10:47:23.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.986+0000 7fc56ffff640 1 -- 172.21.15.89:0/3778401713 learned_addr learned my addr 172.21.15.89:0/3778401713 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:23.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.986+0000 7fc57659c640 1 --2- 172.21.15.89:0/3778401713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc57806ff00 0x7fc57814b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:23.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.986+0000 7fc576d9d640 1 --2- 172.21.15.89:0/3778401713 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc578152910 0x7fc578153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:23.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.986+0000 7fc56ffff640 1 -- 172.21.15.89:0/3778401713 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc578152910 msgr2=0x7fc578153320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:23.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.986+0000 7fc56ffff640 1 --2- 172.21.15.89:0/3778401713 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc578152910 0x7fc578153320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:23.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.986+0000 7fc56ffff640 1 -- 172.21.15.89:0/3778401713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc57806ff00 msgr2=0x7fc57814b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:23.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.986+0000 7fc56ffff640 1 --2- 172.21.15.89:0/3778401713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc57806ff00 0x7fc57814b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:23.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.986+0000 7fc56ffff640 1 -- 172.21.15.89:0/3778401713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc570009a70 con 0x7fc5780ca340 2023-10-18T10:47:23.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.986+0000 7fc56ffff640 1 --2- 172.21.15.89:0/3778401713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5780ca340 0x7fc5780c2110 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc56000e8c0 tx=0x7fc56000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:23.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.986+0000 7fc56effd640 1 -- 172.21.15.89:0/3778401713 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc560009620 con 0x7fc5780ca340 2023-10-18T10:47:23.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.986+0000 7fc56effd640 1 -- 172.21.15.89:0/3778401713 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc56000c940 con 0x7fc5780ca340 2023-10-18T10:47:23.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.986+0000 7fc57ce34640 1 -- 172.21.15.89:0/3778401713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc57814bb30 con 0x7fc5780ca340 2023-10-18T10:47:23.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.986+0000 7fc56effd640 1 -- 172.21.15.89:0/3778401713 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56000ccf0 con 0x7fc5780ca340 2023-10-18T10:47:23.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.986+0000 7fc57ce34640 1 -- 172.21.15.89:0/3778401713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5780cc5e0 con 0x7fc5780ca340 2023-10-18T10:47:23.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.986+0000 7fc57ce34640 1 -- 172.21.15.89:0/3778401713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5780cd0c0 con 0x7fc5780ca340 2023-10-18T10:47:23.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.990+0000 7fc56effd640 1 -- 172.21.15.89:0/3778401713 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fc560013470 con 0x7fc5780ca340 2023-10-18T10:47:23.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.990+0000 7fc56effd640 1 --2- 172.21.15.89:0/3778401713 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc558065680 0x7fc558067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:23.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.990+0000 7fc56effd640 1 -- 172.21.15.89:0/3778401713 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fc560083040 con 0x7fc5780ca340 2023-10-18T10:47:23.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.990+0000 7fc57659c640 1 --2- 172.21.15.89:0/3778401713 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc558065680 0x7fc558067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:23.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.990+0000 7fc57659c640 1 --2- 172.21.15.89:0/3778401713 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc558065680 0x7fc558067b40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc568009ae0 tx=0x7fc568009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:23.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:23.994+0000 7fc56effd640 1 -- 172.21.15.89:0/3778401713 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fc5780cd0c0 con 0x7fc5780ca340 2023-10-18T10:47:24.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.170+0000 7fc57ce34640 1 -- 172.21.15.89:0/3778401713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5780cd0c0 con 0x7fc5780ca340 2023-10-18T10:47:24.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.170+0000 7fc56effd640 1 -- 172.21.15.89:0/3778401713 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4875 (secure 0 0 0) 0x7fc5780cd0c0 con 0x7fc5780ca340 2023-10-18T10:47:24.176 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:47:24.176 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/654729641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":654729641},{"type":"v1","addr":"172.21.15.154:6836","nonce":654729641}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/964566480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":964566480},{"type":"v1","addr":"172.21.15.154:6837","nonce":964566480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2261878538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2261878538},{"type":"v1","addr":"172.21.15.89:6833","nonce":2261878538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:47:15.199575+0000","modified":"2023-10-18T10:47:18.846656+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5517":{"gid":5517,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6835/2092274731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2092274731},{"type":"v1","addr":"172.21.15.89:6835","nonce":2092274731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T10:47:24.177 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 18 2023-10-18T10:47:24.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.174+0000 7fc57ce34640 1 -- 172.21.15.89:0/3778401713 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc558065680 msgr2=0x7fc558067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:24.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.174+0000 7fc57ce34640 1 --2- 172.21.15.89:0/3778401713 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc558065680 0x7fc558067b40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc568009ae0 tx=0x7fc568009190 comp rx=0 tx=0).stop 2023-10-18T10:47:24.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.174+0000 7fc57ce34640 1 -- 172.21.15.89:0/3778401713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5780ca340 msgr2=0x7fc5780c2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:24.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.174+0000 7fc57ce34640 1 --2- 172.21.15.89:0/3778401713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5780ca340 0x7fc5780c2110 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc56000e8c0 tx=0x7fc56000ed80 comp rx=0 tx=0).stop 2023-10-18T10:47:24.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.174+0000 7fc57ce34640 1 -- 172.21.15.89:0/3778401713 shutdown_connections 2023-10-18T10:47:24.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.174+0000 7fc57ce34640 1 --2- 172.21.15.89:0/3778401713 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc558065680 0x7fc558067b40 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:24.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.174+0000 7fc57ce34640 1 --2- 172.21.15.89:0/3778401713 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc578152910 0x7fc578153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:24.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.174+0000 7fc57ce34640 1 --2- 172.21.15.89:0/3778401713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5780ca340 0x7fc5780c2110 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:24.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.174+0000 7fc57ce34640 1 --2- 172.21.15.89:0/3778401713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc57806ff00 0x7fc57814b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:24.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.174+0000 7fc57ce34640 1 -- 172.21.15.89:0/3778401713 >> 172.21.15.89:0/3778401713 conn(0x7fc578069f00 msgr2=0x7fc57815bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:24.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.174+0000 7fc57ce34640 1 -- 172.21.15.89:0/3778401713 shutdown_connections 2023-10-18T10:47:24.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.174+0000 7fc57ce34640 1 -- 172.21.15.89:0/3778401713 wait complete. 2023-10-18T10:47:24.197 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_defer_session_stale false 2023-10-18T10:47:24.351 INFO:teuthology.orchestra.run.smithi089.stdout:{"success":"mds_defer_session_stale = 'false' (not observed, change may require restart) "} 2023-10-18T10:47:24.352 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_defer_session_stale = 'false' (not observed, change may require restart) " } 2023-10-18T10:47:24.352 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds defer_client_eviction_on_laggy_osds true 2023-10-18T10:47:24.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.490+0000 7fe0217ba640 1 Processor -- start 2023-10-18T10:47:24.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.490+0000 7fe0217ba640 1 -- start start 2023-10-18T10:47:24.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.490+0000 7fe0217ba640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe01c15c940 0x7fe01c0d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:24.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.490+0000 7fe0217ba640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe01c0d2f40 0x7fe01c0d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:24.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.490+0000 7fe0217ba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe01c0d0ca0 0x7fe01c0cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:24.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.490+0000 7fe0217ba640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe01c0d1130 con 0x7fe01c0d2f40 2023-10-18T10:47:24.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.490+0000 7fe0217ba640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe01c15da50 con 0x7fe01c15c940 2023-10-18T10:47:24.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.490+0000 7fe0217ba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c15ce40 con 0x7fe01c0d0ca0 2023-10-18T10:47:24.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.490+0000 7fe01a7fc640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe01c0d2f40 0x7fe01c0d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:24.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.490+0000 7fe01a7fc640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe01c0d2f40 0x7fe01c0d0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50408/0 (socket says 172.21.15.89:50408) 2023-10-18T10:47:24.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.490+0000 7fe01a7fc640 1 -- 172.21.15.89:0/2987665039 learned_addr learned my addr 172.21.15.89:0/2987665039 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:24.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.490+0000 7fe01b7fe640 1 --2- 172.21.15.89:0/2987665039 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe01c0d0ca0 0x7fe01c0cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:24.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.490+0000 7fe01affd640 1 --2- 172.21.15.89:0/2987665039 >> v2:172.21.15.154:3300/0 conn(0x7fe01c15c940 0x7fe01c0d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:24.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.490+0000 7fe01a7fc640 1 -- 172.21.15.89:0/2987665039 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe01c0d0ca0 msgr2=0x7fe01c0cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:24.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.490+0000 7fe01a7fc640 1 --2- 172.21.15.89:0/2987665039 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe01c0d0ca0 0x7fe01c0cf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:24.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.490+0000 7fe01a7fc640 1 -- 172.21.15.89:0/2987665039 >> v2:172.21.15.154:3300/0 conn(0x7fe01c15c940 msgr2=0x7fe01c0d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:24.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.490+0000 7fe01a7fc640 1 --2- 172.21.15.89:0/2987665039 >> v2:172.21.15.154:3300/0 conn(0x7fe01c15c940 0x7fe01c0d2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:24.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.490+0000 7fe01a7fc640 1 -- 172.21.15.89:0/2987665039 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01c15d120 con 0x7fe01c0d2f40 2023-10-18T10:47:24.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.494+0000 7fe01a7fc640 1 --2- 172.21.15.89:0/2987665039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe01c0d2f40 0x7fe01c0d0760 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0725b0 tx=0x7fe01001c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ecaa54d194ced7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:24.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.494+0000 7fe019ffb640 1 -- 172.21.15.89:0/2987665039 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0100ab600 con 0x7fe01c0d2f40 2023-10-18T10:47:24.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.494+0000 7fe019ffb640 1 -- 172.21.15.89:0/2987665039 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0100a7050 con 0x7fe01c0d2f40 2023-10-18T10:47:24.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.494+0000 7fe019ffb640 1 -- 172.21.15.89:0/2987665039 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0100ab980 con 0x7fe01c0d2f40 2023-10-18T10:47:24.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.494+0000 7fe0217ba640 1 -- 172.21.15.89:0/2987665039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe01c0d2f40 msgr2=0x7fe01c0d0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:24.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.494+0000 7fe0217ba640 1 --2- 172.21.15.89:0/2987665039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe01c0d2f40 0x7fe01c0d0760 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0725b0 tx=0x7fe01001c210 comp rx=0 tx=0).stop 2023-10-18T10:47:24.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.494+0000 7fe0217ba640 1 -- 172.21.15.89:0/2987665039 shutdown_connections 2023-10-18T10:47:24.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.494+0000 7fe0217ba640 1 --2- 172.21.15.89:0/2987665039 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe01c0d0ca0 0x7fe01c0cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:24.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.494+0000 7fe0217ba640 1 --2- 172.21.15.89:0/2987665039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe01c0d2f40 0x7fe01c0d0760 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:24.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.494+0000 7fe0217ba640 1 --2- 172.21.15.89:0/2987665039 >> v2:172.21.15.154:3300/0 conn(0x7fe01c15c940 0x7fe01c0d2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:24.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.494+0000 7fe0217ba640 1 -- 172.21.15.89:0/2987665039 >> 172.21.15.89:0/2987665039 conn(0x7fe01c089fd0 msgr2=0x7fe01c09b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:24.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.494+0000 7fe0217ba640 1 -- 172.21.15.89:0/2987665039 shutdown_connections 2023-10-18T10:47:24.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.494+0000 7fe0217ba640 1 -- 172.21.15.89:0/2987665039 wait complete. 2023-10-18T10:47:24.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.498+0000 7fe0217ba640 1 Processor -- start 2023-10-18T10:47:24.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.498+0000 7fe0217ba640 1 -- start start 2023-10-18T10:47:24.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.498+0000 7fe0217ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe01c0d0ca0 0x7fe01c14e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:24.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.498+0000 7fe0217ba640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe01c15f6d0 0x7fe01c15fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:24.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.498+0000 7fe0217ba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe01c0c6fc0 0x7fe01c0c7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:24.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.498+0000 7fe0217ba640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c0c6180 con 0x7fe01c15f6d0 2023-10-18T10:47:24.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.498+0000 7fe0217ba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c160090 con 0x7fe01c0d0ca0 2023-10-18T10:47:24.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.498+0000 7fe0217ba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c1f8120 con 0x7fe01c0c6fc0 2023-10-18T10:47:24.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.498+0000 7fe01a7fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe01c15f6d0 0x7fe01c15fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:24.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.498+0000 7fe01a7fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe01c15f6d0 0x7fe01c15fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50412/0 (socket says 172.21.15.89:50412) 2023-10-18T10:47:24.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.498+0000 7fe01a7fc640 1 -- 172.21.15.89:0/1918177213 learned_addr learned my addr 172.21.15.89:0/1918177213 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:24.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.498+0000 7fe01b7fe640 1 --2- 172.21.15.89:0/1918177213 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe01c0c6fc0 0x7fe01c0c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:24.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.498+0000 7fe01affd640 1 --2- 172.21.15.89:0/1918177213 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe01c0d0ca0 0x7fe01c14e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:24.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.498+0000 7fe01a7fc640 1 -- 172.21.15.89:0/1918177213 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe01c0c6fc0 msgr2=0x7fe01c0c7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:24.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.498+0000 7fe01a7fc640 1 --2- 172.21.15.89:0/1918177213 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe01c0c6fc0 0x7fe01c0c7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:24.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.498+0000 7fe01a7fc640 1 -- 172.21.15.89:0/1918177213 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe01c0d0ca0 msgr2=0x7fe01c14e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:24.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.498+0000 7fe01a7fc640 1 --2- 172.21.15.89:0/1918177213 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe01c0d0ca0 0x7fe01c14e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:24.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.498+0000 7fe01a7fc640 1 -- 172.21.15.89:0/1918177213 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0100094d0 con 0x7fe01c15f6d0 2023-10-18T10:47:24.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.498+0000 7fe01a7fc640 1 --2- 172.21.15.89:0/1918177213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe01c15f6d0 0x7fe01c15fb50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe01001a8c0 tx=0x7fe010018fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:24.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.498+0000 7fe0037fe640 1 -- 172.21.15.89:0/1918177213 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0100b4420 con 0x7fe01c15f6d0 2023-10-18T10:47:24.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.498+0000 7fe0037fe640 1 -- 172.21.15.89:0/1918177213 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0100b4580 con 0x7fe01c15f6d0 2023-10-18T10:47:24.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.498+0000 7fe0217ba640 1 -- 172.21.15.89:0/1918177213 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe01c0c88c0 con 0x7fe01c15f6d0 2023-10-18T10:47:24.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.498+0000 7fe0037fe640 1 -- 172.21.15.89:0/1918177213 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0100b48f0 con 0x7fe01c15f6d0 2023-10-18T10:47:24.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.498+0000 7fe0217ba640 1 -- 172.21.15.89:0/1918177213 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe01c0c8020 con 0x7fe01c15f6d0 2023-10-18T10:47:24.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.498+0000 7fe0217ba640 1 -- 172.21.15.89:0/1918177213 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe01c04dbe0 con 0x7fe01c15f6d0 2023-10-18T10:47:24.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.502+0000 7fe0037fe640 1 -- 172.21.15.89:0/1918177213 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fe0100b2070 con 0x7fe01c15f6d0 2023-10-18T10:47:24.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.502+0000 7fe0037fe640 1 --2- 172.21.15.89:0/1918177213 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe0080656b0 0x7fe008067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:24.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.502+0000 7fe0037fe640 1 -- 172.21.15.89:0/1918177213 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe01011bf00 con 0x7fe01c15f6d0 2023-10-18T10:47:24.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.502+0000 7fe01affd640 1 --2- 172.21.15.89:0/1918177213 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe0080656b0 0x7fe008067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:24.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.502+0000 7fe01affd640 1 --2- 172.21.15.89:0/1918177213 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe0080656b0 0x7fe008067b70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe00c009a40 tx=0x7fe00c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:24.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.506+0000 7fe0037fe640 1 -- 172.21.15.89:0/1918177213 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fe0100e9800 con 0x7fe01c15f6d0 2023-10-18T10:47:24.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.654+0000 7fe0217ba640 1 -- 172.21.15.89:0/1918177213 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=defer_client_eviction_on_laggy_osds}] v 0) v1 -- 0x7fe01c06aab0 con 0x7fe01c15f6d0 2023-10-18T10:47:24.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.658+0000 7fe0037fe640 1 -- 172.21.15.89:0/1918177213 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=defer_client_eviction_on_laggy_osds}]=0 v11)=0 v11) v1 ==== 136+0+0 (secure 0 0 0) 0x7fe0100b00c0 con 0x7fe01c15f6d0 2023-10-18T10:47:24.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.662+0000 7fe0217ba640 1 -- 172.21.15.89:0/1918177213 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe0080656b0 msgr2=0x7fe008067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:24.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.662+0000 7fe0217ba640 1 --2- 172.21.15.89:0/1918177213 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe0080656b0 0x7fe008067b70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe00c009a40 tx=0x7fe00c009190 comp rx=0 tx=0).stop 2023-10-18T10:47:24.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.662+0000 7fe0217ba640 1 -- 172.21.15.89:0/1918177213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe01c15f6d0 msgr2=0x7fe01c15fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:24.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.662+0000 7fe0217ba640 1 --2- 172.21.15.89:0/1918177213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe01c15f6d0 0x7fe01c15fb50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe01001a8c0 tx=0x7fe010018fb0 comp rx=0 tx=0).stop 2023-10-18T10:47:24.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.662+0000 7fe0217ba640 1 -- 172.21.15.89:0/1918177213 shutdown_connections 2023-10-18T10:47:24.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.662+0000 7fe0217ba640 1 --2- 172.21.15.89:0/1918177213 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe0080656b0 0x7fe008067b70 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:24.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.662+0000 7fe0217ba640 1 --2- 172.21.15.89:0/1918177213 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe01c0c6fc0 0x7fe01c0c7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:24.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.662+0000 7fe0217ba640 1 --2- 172.21.15.89:0/1918177213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe01c15f6d0 0x7fe01c15fb50 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:24.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.662+0000 7fe0217ba640 1 --2- 172.21.15.89:0/1918177213 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe01c0d0ca0 0x7fe01c14e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:24.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.662+0000 7fe0217ba640 1 -- 172.21.15.89:0/1918177213 >> 172.21.15.89:0/1918177213 conn(0x7fe01c089fd0 msgr2=0x7fe01c0cd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:24.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.662+0000 7fe0217ba640 1 -- 172.21.15.89:0/1918177213 shutdown_connections 2023-10-18T10:47:24.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.662+0000 7fe0217ba640 1 -- 172.21.15.89:0/1918177213 wait complete. 2023-10-18T10:47:24.684 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds defer_client_eviction_on_laggy_osds 2023-10-18T10:47:24.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.818+0000 7fe91a4e5640 1 Processor -- start 2023-10-18T10:47:24.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.818+0000 7fe91a4e5640 1 -- start start 2023-10-18T10:47:24.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.822+0000 7fe91a4e5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe91415a480 0x7fe91415b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:24.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.822+0000 7fe91a4e5640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe91406c350 0x7fe91415df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:24.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.822+0000 7fe91a4e5640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe91415ee40 con 0x7fe9140c90e0 2023-10-18T10:47:24.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.822+0000 7fe91a4e5640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe91415a860 con 0x7fe91406c350 2023-10-18T10:47:24.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.822+0000 7fe91a4e5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe91415e880 con 0x7fe91415a480 2023-10-18T10:47:24.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.822+0000 7fe9137fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe91406c350 0x7fe91415df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:24.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.822+0000 7fe918a82640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe9140c90e0 0x7fe91406ffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48934/0 (socket says 172.21.15.89:48934) 2023-10-18T10:47:24.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.822+0000 7fe918a82640 1 -- 172.21.15.89:0/3172468959 learned_addr learned my addr 172.21.15.89:0/3172468959 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:47:24.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.822+0000 7fe913fff640 1 --2- 172.21.15.89:0/3172468959 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe91415a480 0x7fe91415b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:24.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.822+0000 7fe9137fe640 1 -- 172.21.15.89:0/3172468959 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe91415a480 msgr2=0x7fe91415b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:24.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.822+0000 7fe9137fe640 1 --2- 172.21.15.89:0/3172468959 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe91415a480 0x7fe91415b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:24.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.822+0000 7fe9137fe640 1 -- 172.21.15.89:0/3172468959 >> v1:172.21.15.89:6789/0 conn(0x7fe9140c90e0 legacy=0x7fe91406ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:24.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.822+0000 7fe9137fe640 1 -- 172.21.15.89:0/3172468959 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe91415eb60 con 0x7fe91406c350 2023-10-18T10:47:24.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.822+0000 7fe912ffd640 1 -- 172.21.15.89:0/3172468959 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 230655735 0 0) 0x7fe91415ee40 con 0x7fe9140c90e0 2023-10-18T10:47:24.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.826+0000 7fe9137fe640 1 --2- 172.21.15.89:0/3172468959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe91406c350 0x7fe91415df80 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe9080097c0 tx=0x7fe90801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e484391ab774cdb2 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:24.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.826+0000 7fe912ffd640 1 -- 172.21.15.89:0/3172468959 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9080ab650 con 0x7fe91406c350 2023-10-18T10:47:24.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.826+0000 7fe912ffd640 1 -- 172.21.15.89:0/3172468959 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9080ab7b0 con 0x7fe91406c350 2023-10-18T10:47:24.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.826+0000 7fe912ffd640 1 -- 172.21.15.89:0/3172468959 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9080abb40 con 0x7fe91406c350 2023-10-18T10:47:24.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.826+0000 7fe91a4e5640 1 -- 172.21.15.89:0/3172468959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe91406c350 msgr2=0x7fe91415df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:24.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.826+0000 7fe91a4e5640 1 --2- 172.21.15.89:0/3172468959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe91406c350 0x7fe91415df80 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe9080097c0 tx=0x7fe90801c450 comp rx=0 tx=0).stop 2023-10-18T10:47:24.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.826+0000 7fe91a4e5640 1 -- 172.21.15.89:0/3172468959 shutdown_connections 2023-10-18T10:47:24.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.826+0000 7fe91a4e5640 1 --2- 172.21.15.89:0/3172468959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe91406c350 0x7fe91415df80 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:24.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.826+0000 7fe91a4e5640 1 --2- 172.21.15.89:0/3172468959 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe91415a480 0x7fe91415b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:24.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.826+0000 7fe91a4e5640 1 -- 172.21.15.89:0/3172468959 >> 172.21.15.89:0/3172468959 conn(0x7fe9140cb5a0 msgr2=0x7fe9140cb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:24.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.826+0000 7fe91a4e5640 1 -- 172.21.15.89:0/3172468959 shutdown_connections 2023-10-18T10:47:24.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.826+0000 7fe91a4e5640 1 -- 172.21.15.89:0/3172468959 wait complete. 2023-10-18T10:47:24.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.826+0000 7fe91a4e5640 1 Processor -- start 2023-10-18T10:47:24.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.826+0000 7fe91a4e5640 1 -- start start 2023-10-18T10:47:24.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.830+0000 7fe91a4e5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe91406c350 0x7fe9140c62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:24.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.830+0000 7fe91a4e5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9140c90e0 0x7fe9140c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:24.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.830+0000 7fe91a4e5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe91415a480 0x7fe9140c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:24.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.830+0000 7fe91a4e5640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe914159070 con 0x7fe9140c90e0 2023-10-18T10:47:24.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.830+0000 7fe91a4e5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9140c88e0 con 0x7fe91415a480 2023-10-18T10:47:24.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.830+0000 7fe91a4e5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9140c5b60 con 0x7fe91406c350 2023-10-18T10:47:24.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.830+0000 7fe9137fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9140c90e0 0x7fe9140c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:24.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.830+0000 7fe9137fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9140c90e0 0x7fe9140c7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50432/0 (socket says 172.21.15.89:50432) 2023-10-18T10:47:24.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.830+0000 7fe9137fe640 1 -- 172.21.15.89:0/1400418529 learned_addr learned my addr 172.21.15.89:0/1400418529 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:24.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.830+0000 7fe913fff640 1 --2- 172.21.15.89:0/1400418529 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe91406c350 0x7fe9140c62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:24.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.830+0000 7fe918a82640 1 --2- 172.21.15.89:0/1400418529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe91415a480 0x7fe9140c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:24.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.830+0000 7fe9137fe640 1 -- 172.21.15.89:0/1400418529 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe91406c350 msgr2=0x7fe9140c62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:24.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.830+0000 7fe9137fe640 1 --2- 172.21.15.89:0/1400418529 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe91406c350 0x7fe9140c62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:24.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.830+0000 7fe9137fe640 1 -- 172.21.15.89:0/1400418529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe91415a480 msgr2=0x7fe9140c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:24.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.830+0000 7fe9137fe640 1 --2- 172.21.15.89:0/1400418529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe91415a480 0x7fe9140c83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:24.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.830+0000 7fe9137fe640 1 -- 172.21.15.89:0/1400418529 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe908009420 con 0x7fe9140c90e0 2023-10-18T10:47:24.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.830+0000 7fe9137fe640 1 --2- 172.21.15.89:0/1400418529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9140c90e0 0x7fe9140c7e60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe9080ad040 tx=0x7fe90801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:24.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.830+0000 7fe910ff9640 1 -- 172.21.15.89:0/1400418529 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9080b5810 con 0x7fe9140c90e0 2023-10-18T10:47:24.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.830+0000 7fe910ff9640 1 -- 172.21.15.89:0/1400418529 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9080b5990 con 0x7fe9140c90e0 2023-10-18T10:47:24.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.830+0000 7fe910ff9640 1 -- 172.21.15.89:0/1400418529 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9080b5d40 con 0x7fe9140c90e0 2023-10-18T10:47:24.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.830+0000 7fe91a4e5640 1 -- 172.21.15.89:0/1400418529 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9140c5e40 con 0x7fe9140c90e0 2023-10-18T10:47:24.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.830+0000 7fe91a4e5640 1 -- 172.21.15.89:0/1400418529 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe91415cb70 con 0x7fe9140c90e0 2023-10-18T10:47:24.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.830+0000 7fe91a4e5640 1 -- 172.21.15.89:0/1400418529 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9140c5fd0 con 0x7fe9140c90e0 2023-10-18T10:47:24.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.830+0000 7fe910ff9640 1 -- 172.21.15.89:0/1400418529 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fe9080bf410 con 0x7fe9140c90e0 2023-10-18T10:47:24.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.830+0000 7fe910ff9640 1 --2- 172.21.15.89:0/1400418529 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe900065680 0x7fe900067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:24.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.834+0000 7fe910ff9640 1 -- 172.21.15.89:0/1400418529 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe90811df60 con 0x7fe9140c90e0 2023-10-18T10:47:24.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.834+0000 7fe913fff640 1 --2- 172.21.15.89:0/1400418529 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe900065680 0x7fe900067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:24.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.834+0000 7fe913fff640 1 --2- 172.21.15.89:0/1400418529 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe900065680 0x7fe900067b40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe9040079c0 tx=0x7fe904008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:24.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.838+0000 7fe910ff9640 1 -- 172.21.15.89:0/1400418529 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fe9080ebaa0 con 0x7fe9140c90e0 2023-10-18T10:47:24.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.986+0000 7fe91a4e5640 1 -- 172.21.15.89:0/1400418529 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "defer_client_eviction_on_laggy_osds"} v 0) v1 -- 0x7fe91404dbe0 con 0x7fe9140c90e0 2023-10-18T10:47:24.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.986+0000 7fe910ff9640 1 -- 172.21.15.89:0/1400418529 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "defer_client_eviction_on_laggy_osds"}]=0 v11) v1 ==== 118+0+5 (secure 0 0 0) 0x7fe9080ef0c0 con 0x7fe9140c90e0 2023-10-18T10:47:24.991 INFO:teuthology.orchestra.run.smithi089.stdout:true 2023-10-18T10:47:24.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.990+0000 7fe91a4e5640 1 -- 172.21.15.89:0/1400418529 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe900065680 msgr2=0x7fe900067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:24.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.990+0000 7fe91a4e5640 1 --2- 172.21.15.89:0/1400418529 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe900065680 0x7fe900067b40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe9040079c0 tx=0x7fe904008040 comp rx=0 tx=0).stop 2023-10-18T10:47:24.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.990+0000 7fe91a4e5640 1 -- 172.21.15.89:0/1400418529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9140c90e0 msgr2=0x7fe9140c7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:24.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.990+0000 7fe91a4e5640 1 --2- 172.21.15.89:0/1400418529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9140c90e0 0x7fe9140c7e60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe9080ad040 tx=0x7fe90801ac90 comp rx=0 tx=0).stop 2023-10-18T10:47:24.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.990+0000 7fe91a4e5640 1 -- 172.21.15.89:0/1400418529 shutdown_connections 2023-10-18T10:47:24.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.990+0000 7fe91a4e5640 1 --2- 172.21.15.89:0/1400418529 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe900065680 0x7fe900067b40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:24.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.990+0000 7fe91a4e5640 1 --2- 172.21.15.89:0/1400418529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe91415a480 0x7fe9140c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:24.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.990+0000 7fe91a4e5640 1 --2- 172.21.15.89:0/1400418529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9140c90e0 0x7fe9140c7e60 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:24.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.990+0000 7fe91a4e5640 1 --2- 172.21.15.89:0/1400418529 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe91406c350 0x7fe9140c62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:24.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.990+0000 7fe91a4e5640 1 -- 172.21.15.89:0/1400418529 >> 172.21.15.89:0/1400418529 conn(0x7fe9140cb5a0 msgr2=0x7fe9141534f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:24.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.990+0000 7fe91a4e5640 1 -- 172.21.15.89:0/1400418529 shutdown_connections 2023-10-18T10:47:24.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:24.990+0000 7fe91a4e5640 1 -- 172.21.15.89:0/1400418529 wait complete. 2023-10-18T10:47:25.011 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2023-10-18T10:47:25.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.146+0000 7fb1924e5640 1 Processor -- start 2023-10-18T10:47:25.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.146+0000 7fb1924e5640 1 -- start start 2023-10-18T10:47:25.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.146+0000 7fb1924e5640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb18c159490 0x7fb18c15d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:25.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.146+0000 7fb1924e5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb18c070600 0x7fb18c06b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:25.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.146+0000 7fb1924e5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb18c06bd10 0x7fb18c0cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:25.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.146+0000 7fb1924e5640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb18c06c1a0 con 0x7fb18c159490 2023-10-18T10:47:25.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.146+0000 7fb1924e5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb18c15ddf0 con 0x7fb18c06bd10 2023-10-18T10:47:25.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.146+0000 7fb1924e5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c070b90 con 0x7fb18c070600 2023-10-18T10:47:25.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.146+0000 7fb18bfff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb18c159490 0x7fb18c15d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:25.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.146+0000 7fb18bfff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb18c159490 0x7fb18c15d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50434/0 (socket says 172.21.15.89:50434) 2023-10-18T10:47:25.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.146+0000 7fb18bfff640 1 -- 172.21.15.89:0/3217884533 learned_addr learned my addr 172.21.15.89:0/3217884533 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:25.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.146+0000 7fb18b7fe640 1 --2- 172.21.15.89:0/3217884533 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb18c070600 0x7fb18c06b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:25.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.146+0000 7fb190a82640 1 --2- 172.21.15.89:0/3217884533 >> v2:172.21.15.154:3300/0 conn(0x7fb18c06bd10 0x7fb18c0cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:25.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.146+0000 7fb18bfff640 1 -- 172.21.15.89:0/3217884533 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb18c070600 msgr2=0x7fb18c06b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:25.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.146+0000 7fb18bfff640 1 --2- 172.21.15.89:0/3217884533 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb18c070600 0x7fb18c06b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:25.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.146+0000 7fb18bfff640 1 -- 172.21.15.89:0/3217884533 >> v2:172.21.15.154:3300/0 conn(0x7fb18c06bd10 msgr2=0x7fb18c0cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:47:25.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.146+0000 7fb18bfff640 1 --2- 172.21.15.89:0/3217884533 >> v2:172.21.15.154:3300/0 conn(0x7fb18c06bd10 0x7fb18c0cbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:25.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.146+0000 7fb18bfff640 1 -- 172.21.15.89:0/3217884533 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18c070e70 con 0x7fb18c159490 2023-10-18T10:47:25.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.146+0000 7fb190a82640 1 --2- 172.21.15.89:0/3217884533 >> v2:172.21.15.154:3300/0 conn(0x7fb18c06bd10 0x7fb18c0cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:47:25.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.150+0000 7fb18bfff640 1 --2- 172.21.15.89:0/3217884533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb18c159490 0x7fb18c15d8b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb18c0add20 tx=0x7fb17c01c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c2a940e2e13ff4d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:25.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.150+0000 7fb18affd640 1 -- 172.21.15.89:0/3217884533 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c0ab600 con 0x7fb18c159490 2023-10-18T10:47:25.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.150+0000 7fb18affd640 1 -- 172.21.15.89:0/3217884533 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb17c0a7050 con 0x7fb18c159490 2023-10-18T10:47:25.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.150+0000 7fb18affd640 1 -- 172.21.15.89:0/3217884533 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c0ab980 con 0x7fb18c159490 2023-10-18T10:47:25.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.150+0000 7fb1924e5640 1 -- 172.21.15.89:0/3217884533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb18c159490 msgr2=0x7fb18c15d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:25.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.150+0000 7fb1924e5640 1 --2- 172.21.15.89:0/3217884533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb18c159490 0x7fb18c15d8b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb18c0add20 tx=0x7fb17c01c180 comp rx=0 tx=0).stop 2023-10-18T10:47:25.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.150+0000 7fb1924e5640 1 -- 172.21.15.89:0/3217884533 shutdown_connections 2023-10-18T10:47:25.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.150+0000 7fb1924e5640 1 --2- 172.21.15.89:0/3217884533 >> v2:172.21.15.154:3300/0 conn(0x7fb18c06bd10 0x7fb18c0cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:25.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.150+0000 7fb1924e5640 1 --2- 172.21.15.89:0/3217884533 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb18c070600 0x7fb18c06b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:25.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.150+0000 7fb1924e5640 1 --2- 172.21.15.89:0/3217884533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb18c159490 0x7fb18c15d8b0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:25.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.150+0000 7fb1924e5640 1 -- 172.21.15.89:0/3217884533 >> 172.21.15.89:0/3217884533 conn(0x7fb18c069f00 msgr2=0x7fb18c0cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:25.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.150+0000 7fb1924e5640 1 -- 172.21.15.89:0/3217884533 shutdown_connections 2023-10-18T10:47:25.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.150+0000 7fb1924e5640 1 -- 172.21.15.89:0/3217884533 wait complete. 2023-10-18T10:47:25.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.150+0000 7fb1924e5640 1 Processor -- start 2023-10-18T10:47:25.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.154+0000 7fb1924e5640 1 -- start start 2023-10-18T10:47:25.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.154+0000 7fb1924e5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb18c06bd10 0x7fb18c154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:25.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.154+0000 7fb1924e5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb18c070600 0x7fb18c0c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:25.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.154+0000 7fb1924e5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb18c159490 0x7fb18c1551d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:25.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.154+0000 7fb1924e5640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c0c4510 con 0x7fb18c06bd10 2023-10-18T10:47:25.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.154+0000 7fb1924e5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c1fd380 con 0x7fb18c070600 2023-10-18T10:47:25.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.154+0000 7fb1924e5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c0c3be0 con 0x7fb18c159490 2023-10-18T10:47:25.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.154+0000 7fb18bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb18c06bd10 0x7fb18c154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:25.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.154+0000 7fb18b7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb18c070600 0x7fb18c0c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:25.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.154+0000 7fb18b7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb18c070600 0x7fb18c0c49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:33632/0 (socket says 172.21.15.89:33632) 2023-10-18T10:47:25.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.154+0000 7fb18b7fe640 1 -- 172.21.15.89:0/3935625398 learned_addr learned my addr 172.21.15.89:0/3935625398 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:47:25.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.154+0000 7fb190a82640 1 --2- 172.21.15.89:0/3935625398 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb18c159490 0x7fb18c1551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:25.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.154+0000 7fb18bfff640 1 -- 172.21.15.89:0/3935625398 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb18c159490 msgr2=0x7fb18c1551d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:25.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.154+0000 7fb18bfff640 1 --2- 172.21.15.89:0/3935625398 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb18c159490 0x7fb18c1551d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:25.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.154+0000 7fb18bfff640 1 -- 172.21.15.89:0/3935625398 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb18c070600 msgr2=0x7fb18c0c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:25.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.154+0000 7fb18bfff640 1 --2- 172.21.15.89:0/3935625398 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb18c070600 0x7fb18c0c49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:25.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.154+0000 7fb18bfff640 1 -- 172.21.15.89:0/3935625398 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb17c0094d0 con 0x7fb18c06bd10 2023-10-18T10:47:25.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.154+0000 7fb18bfff640 1 --2- 172.21.15.89:0/3935625398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb18c06bd10 0x7fb18c154a20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb17c0ab000 tx=0x7fb17c01a9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:25.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.154+0000 7fb188ff9640 1 -- 172.21.15.89:0/3935625398 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c0b6620 con 0x7fb18c06bd10 2023-10-18T10:47:25.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.154+0000 7fb188ff9640 1 -- 172.21.15.89:0/3935625398 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb17c0b67a0 con 0x7fb18c06bd10 2023-10-18T10:47:25.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.154+0000 7fb188ff9640 1 -- 172.21.15.89:0/3935625398 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c0b6b50 con 0x7fb18c06bd10 2023-10-18T10:47:25.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.154+0000 7fb1924e5640 1 -- 172.21.15.89:0/3935625398 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb18c0c3ec0 con 0x7fb18c06bd10 2023-10-18T10:47:25.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.154+0000 7fb1924e5640 1 -- 172.21.15.89:0/3935625398 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb18c0c4ee0 con 0x7fb18c06bd10 2023-10-18T10:47:25.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.154+0000 7fb1924e5640 1 -- 172.21.15.89:0/3935625398 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb18c0c50d0 con 0x7fb18c06bd10 2023-10-18T10:47:25.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.158+0000 7fb188ff9640 1 -- 172.21.15.89:0/3935625398 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fb17c0b2040 con 0x7fb18c06bd10 2023-10-18T10:47:25.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.158+0000 7fb188ff9640 1 --2- 172.21.15.89:0/3935625398 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb178065680 0x7fb178067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:47:25.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.158+0000 7fb188ff9640 1 -- 172.21.15.89:0/3935625398 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb17c11d370 con 0x7fb18c06bd10 2023-10-18T10:47:25.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.158+0000 7fb18b7fe640 1 --2- 172.21.15.89:0/3935625398 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb178065680 0x7fb178067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:47:25.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.158+0000 7fb18b7fe640 1 --2- 172.21.15.89:0/3935625398 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb178065680 0x7fb178067b40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb18c158bc0 tx=0x7fb180008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:47:25.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.162+0000 7fb188ff9640 1 -- 172.21.15.89:0/3935625398 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fb17c0ead00 con 0x7fb18c06bd10 2023-10-18T10:47:25.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.310+0000 7fb1924e5640 1 -- 172.21.15.89:0/3935625398 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fb18c0adb80 con 0x7fb18c06bd10 2023-10-18T10:47:25.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.310+0000 7fb188ff9640 1 -- 172.21.15.89:0/3935625398 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v41) v1 ==== 56+0+3439 (secure 0 0 0) 0x7fb17c0ee320 con 0x7fb18c06bd10 2023-10-18T10:47:25.316 INFO:teuthology.orchestra.run.smithi089.stdout:epoch 41 2023-10-18T10:47:25.316 INFO:teuthology.orchestra.run.smithi089.stdout:fsid 1352f62a-2330-4e96-afcd-b8e87cd82667 2023-10-18T10:47:25.316 INFO:teuthology.orchestra.run.smithi089.stdout:created 2023-10-18T10:45:29.952211+0000 2023-10-18T10:47:25.316 INFO:teuthology.orchestra.run.smithi089.stdout:modified 2023-10-18T10:47:16.832871+0000 2023-10-18T10:47:25.316 INFO:teuthology.orchestra.run.smithi089.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-10-18T10:47:25.317 INFO:teuthology.orchestra.run.smithi089.stdout:crush_version 5 2023-10-18T10:47:25.317 INFO:teuthology.orchestra.run.smithi089.stdout:full_ratio 0.95 2023-10-18T10:47:25.317 INFO:teuthology.orchestra.run.smithi089.stdout:backfillfull_ratio 0.9 2023-10-18T10:47:25.317 INFO:teuthology.orchestra.run.smithi089.stdout:nearfull_ratio 0.85 2023-10-18T10:47:25.317 INFO:teuthology.orchestra.run.smithi089.stdout:require_min_compat_client luminous 2023-10-18T10:47:25.318 INFO:teuthology.orchestra.run.smithi089.stdout:min_compat_client jewel 2023-10-18T10:47:25.318 INFO:teuthology.orchestra.run.smithi089.stdout:require_osd_release reef 2023-10-18T10:47:25.318 INFO:teuthology.orchestra.run.smithi089.stdout:stretch_mode_enabled false 2023-10-18T10:47:25.318 INFO:teuthology.orchestra.run.smithi089.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 16 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2023-10-18T10:47:25.319 INFO:teuthology.orchestra.run.smithi089.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 19 flags hashpspool stripe_width 0 application rbd read_balance_score 1.99 2023-10-18T10:47:25.319 INFO:teuthology.orchestra.run.smithi089.stdout:pool 6 'cephfs_metadata' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 40 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.75 2023-10-18T10:47:25.319 INFO:teuthology.orchestra.run.smithi089.stdout:pool 7 'cephfs_data' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 41 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.75 2023-10-18T10:47:25.319 INFO:teuthology.orchestra.run.smithi089.stdout:max_osd 8 2023-10-18T10:47:25.319 INFO:teuthology.orchestra.run.smithi089.stdout:osd.0 up in weight 1 up_from 13 up_thru 39 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6800/308765932,v1:172.21.15.89:6801/308765932] [v2:172.21.15.89:6802/308765932,v1:172.21.15.89:6803/308765932] exists,up 842c9db2-b707-471b-a554-c3d6314ac38b 2023-10-18T10:47:25.320 INFO:teuthology.orchestra.run.smithi089.stdout:osd.1 up in weight 1 up_from 13 up_thru 39 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6816/2220030573,v1:172.21.15.89:6817/2220030573] [v2:172.21.15.89:6818/2220030573,v1:172.21.15.89:6819/2220030573] exists,up 0835cc88-e19b-4a4e-8857-d30c7e6804ef 2023-10-18T10:47:25.320 INFO:teuthology.orchestra.run.smithi089.stdout:osd.2 up in weight 1 up_from 13 up_thru 39 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6808/4229386532,v1:172.21.15.89:6809/4229386532] [v2:172.21.15.89:6810/4229386532,v1:172.21.15.89:6811/4229386532] exists,up 95eb0a15-8b08-4e57-8b37-34096114f81f 2023-10-18T10:47:25.320 INFO:teuthology.orchestra.run.smithi089.stdout:osd.3 up in weight 1 up_from 13 up_thru 39 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6822/2771730391,v1:172.21.15.89:6824/2771730391] [v2:172.21.15.89:6826/2771730391,v1:172.21.15.89:6827/2771730391] exists,up 40cd481e-d2af-4210-8d4a-add39171432d 2023-10-18T10:47:25.320 INFO:teuthology.orchestra.run.smithi089.stdout:osd.4 up in weight 1 up_from 13 up_thru 39 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6800/2025560293,v1:172.21.15.154:6801/2025560293] [v2:172.21.15.154:6802/2025560293,v1:172.21.15.154:6803/2025560293] exists,up 2fa04448-d520-4f8c-a9b0-684f526019a5 2023-10-18T10:47:25.320 INFO:teuthology.orchestra.run.smithi089.stdout:osd.5 up in weight 1 up_from 13 up_thru 39 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6808/357281174,v1:172.21.15.154:6809/357281174] [v2:172.21.15.154:6810/357281174,v1:172.21.15.154:6811/357281174] exists,up 93b639fd-01bb-4821-a426-7460cc1abb66 2023-10-18T10:47:25.321 INFO:teuthology.orchestra.run.smithi089.stdout:osd.6 up in weight 1 up_from 13 up_thru 39 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6822/773078364,v1:172.21.15.154:6824/773078364] [v2:172.21.15.154:6826/773078364,v1:172.21.15.154:6827/773078364] exists,up 81f3fc7f-239c-47de-a40b-34bf19289823 2023-10-18T10:47:25.321 INFO:teuthology.orchestra.run.smithi089.stdout:osd.7 up in weight 1 up_from 13 up_thru 39 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6815/4139186586,v1:172.21.15.154:6816/4139186586] [v2:172.21.15.154:6818/4139186586,v1:172.21.15.154:6819/4139186586] exists,up ceb2d886-a2ee-4974-9079-948986ac3778 2023-10-18T10:47:25.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.314+0000 7fb1924e5640 1 -- 172.21.15.89:0/3935625398 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb178065680 msgr2=0x7fb178067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:25.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.314+0000 7fb1924e5640 1 --2- 172.21.15.89:0/3935625398 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb178065680 0x7fb178067b40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb18c158bc0 tx=0x7fb180008040 comp rx=0 tx=0).stop 2023-10-18T10:47:25.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.314+0000 7fb1924e5640 1 -- 172.21.15.89:0/3935625398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb18c06bd10 msgr2=0x7fb18c154a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:47:25.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.314+0000 7fb1924e5640 1 --2- 172.21.15.89:0/3935625398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb18c06bd10 0x7fb18c154a20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb17c0ab000 tx=0x7fb17c01a9c0 comp rx=0 tx=0).stop 2023-10-18T10:47:25.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.314+0000 7fb1924e5640 1 -- 172.21.15.89:0/3935625398 shutdown_connections 2023-10-18T10:47:25.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.314+0000 7fb1924e5640 1 --2- 172.21.15.89:0/3935625398 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb178065680 0x7fb178067b40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:25.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.314+0000 7fb1924e5640 1 --2- 172.21.15.89:0/3935625398 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb18c159490 0x7fb18c1551d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:25.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.314+0000 7fb1924e5640 1 --2- 172.21.15.89:0/3935625398 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb18c070600 0x7fb18c0c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:25.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.314+0000 7fb1924e5640 1 --2- 172.21.15.89:0/3935625398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb18c06bd10 0x7fb18c154a20 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:47:25.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.314+0000 7fb1924e5640 1 -- 172.21.15.89:0/3935625398 >> 172.21.15.89:0/3935625398 conn(0x7fb18c069f00 msgr2=0x7fb18c0af570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:47:25.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.314+0000 7fb1924e5640 1 -- 172.21.15.89:0/3935625398 shutdown_connections 2023-10-18T10:47:25.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:47:25.314+0000 7fb1924e5640 1 -- 172.21.15.89:0/3935625398 wait complete. 2023-10-18T10:47:25.336 INFO:tasks.cephfs.filesystem.ceph_manager:['osd.0 up in weight 1 up_from 13 up_thru 39 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6800/308765932,v1:172.21.15.89:6801/308765932] [v2:172.21.15.89:6802/308765932,v1:172.21.15.89:6803/308765932] exists,up 842c9db2-b707-471b-a554-c3d6314ac38b', 'osd.1 up in weight 1 up_from 13 up_thru 39 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6816/2220030573,v1:172.21.15.89:6817/2220030573] [v2:172.21.15.89:6818/2220030573,v1:172.21.15.89:6819/2220030573] exists,up 0835cc88-e19b-4a4e-8857-d30c7e6804ef', 'osd.2 up in weight 1 up_from 13 up_thru 39 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6808/4229386532,v1:172.21.15.89:6809/4229386532] [v2:172.21.15.89:6810/4229386532,v1:172.21.15.89:6811/4229386532] exists,up 95eb0a15-8b08-4e57-8b37-34096114f81f', 'osd.3 up in weight 1 up_from 13 up_thru 39 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6822/2771730391,v1:172.21.15.89:6824/2771730391] [v2:172.21.15.89:6826/2771730391,v1:172.21.15.89:6827/2771730391] exists,up 40cd481e-d2af-4210-8d4a-add39171432d', 'osd.4 up in weight 1 up_from 13 up_thru 39 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6800/2025560293,v1:172.21.15.154:6801/2025560293] [v2:172.21.15.154:6802/2025560293,v1:172.21.15.154:6803/2025560293] exists,up 2fa04448-d520-4f8c-a9b0-684f526019a5', 'osd.5 up in weight 1 up_from 13 up_thru 39 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6808/357281174,v1:172.21.15.154:6809/357281174] [v2:172.21.15.154:6810/357281174,v1:172.21.15.154:6811/357281174] exists,up 93b639fd-01bb-4821-a426-7460cc1abb66', 'osd.6 up in weight 1 up_from 13 up_thru 39 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6822/773078364,v1:172.21.15.154:6824/773078364] [v2:172.21.15.154:6826/773078364,v1:172.21.15.154:6827/773078364] exists,up 81f3fc7f-239c-47de-a40b-34bf19289823', 'osd.7 up in weight 1 up_from 13 up_thru 39 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6815/4139186586,v1:172.21.15.154:6816/4139186586] [v2:172.21.15.154:6818/4139186586,v1:172.21.15.154:6819/4139186586] exists,up ceb2d886-a2ee-4974-9079-948986ac3778'] 2023-10-18T10:47:25.337 INFO:tasks.ceph.osd.6:Sent signal 19 2023-10-18T10:47:46.512 INFO:tasks.ceph.osd.7.smithi154.stderr:2023-10-18T10:47:46.506+0000 7fa1b1237640 -1 osd.7 43 heartbeat_check: no reply from 172.21.15.154:6828 osd.6 since back 2023-10-18T10:47:20.314765+0000 front 2023-10-18T10:47:20.314720+0000 (oldest deadline 2023-10-18T10:47:46.214585+0000) 2023-10-18T10:47:47.226 INFO:tasks.ceph.osd.2.smithi089.stderr:2023-10-18T10:47:47.222+0000 7f6873339640 -1 osd.2 43 heartbeat_check: no reply from 172.21.15.154:6828 osd.6 since back 2023-10-18T10:47:20.792334+0000 front 2023-10-18T10:47:20.792318+0000 (oldest deadline 2023-10-18T10:47:46.692079+0000) 2023-10-18T10:47:47.266 INFO:tasks.ceph.osd.4.smithi154.stderr:2023-10-18T10:47:47.262+0000 7f994a6e7640 -1 osd.4 43 heartbeat_check: no reply from 172.21.15.154:6828 osd.6 since back 2023-10-18T10:47:21.301162+0000 front 2023-10-18T10:47:21.301119+0000 (oldest deadline 2023-10-18T10:47:47.200866+0000) 2023-10-18T10:47:47.494 INFO:tasks.ceph.osd.7.smithi154.stderr:2023-10-18T10:47:47.490+0000 7fa1b1237640 -1 osd.7 43 heartbeat_check: no reply from 172.21.15.154:6828 osd.6 since back 2023-10-18T10:47:20.314765+0000 front 2023-10-18T10:47:20.314720+0000 (oldest deadline 2023-10-18T10:47:46.214585+0000) 2023-10-18T10:47:48.200 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:47:48.204 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:47:48.212 INFO:tasks.ceph.osd.2.smithi089.stderr:2023-10-18T10:47:48.210+0000 7f6873339640 -1 osd.2 43 heartbeat_check: no reply from 172.21.15.154:6828 osd.6 since back 2023-10-18T10:47:20.792334+0000 front 2023-10-18T10:47:20.792318+0000 (oldest deadline 2023-10-18T10:47:46.692079+0000) 2023-10-18T10:47:48.230 INFO:tasks.ceph.osd.4.smithi154.stderr:2023-10-18T10:47:48.226+0000 7f994a6e7640 -1 osd.4 43 heartbeat_check: no reply from 172.21.15.154:6828 osd.6 since back 2023-10-18T10:47:21.301162+0000 front 2023-10-18T10:47:21.301119+0000 (oldest deadline 2023-10-18T10:47:47.200866+0000) 2023-10-18T10:47:48.374 INFO:tasks.ceph.osd.0.smithi089.stderr:2023-10-18T10:47:48.370+0000 7f574628e640 -1 osd.0 43 heartbeat_check: no reply from 172.21.15.154:6828 osd.6 since back 2023-10-18T10:47:22.672572+0000 front 2023-10-18T10:47:22.672697+0000 (oldest deadline 2023-10-18T10:47:47.972320+0000) 2023-10-18T10:47:48.453 INFO:tasks.ceph.osd.7.smithi154.stderr:2023-10-18T10:47:48.446+0000 7fa1b1237640 -1 osd.7 43 heartbeat_check: no reply from 172.21.15.154:6828 osd.6 since back 2023-10-18T10:47:20.314765+0000 front 2023-10-18T10:47:20.314720+0000 (oldest deadline 2023-10-18T10:47:46.214585+0000) 2023-10-18T10:47:48.908 INFO:tasks.ceph.osd.3.smithi089.stderr:2023-10-18T10:47:48.906+0000 7fafd7cfb640 -1 osd.3 43 heartbeat_check: no reply from 172.21.15.154:6828 osd.6 since back 2023-10-18T10:47:22.910224+0000 front 2023-10-18T10:47:22.910125+0000 (oldest deadline 2023-10-18T10:47:48.209793+0000) 2023-10-18T10:47:49.181 INFO:tasks.ceph.osd.1.smithi089.stderr:2023-10-18T10:47:49.178+0000 7f4d0df64640 -1 osd.1 43 heartbeat_check: no reply from 172.21.15.154:6828 osd.6 since back 2023-10-18T10:47:23.198686+0000 front 2023-10-18T10:47:23.198794+0000 (oldest deadline 2023-10-18T10:47:49.098414+0000) 2023-10-18T10:47:49.631 INFO:tasks.ceph.osd.5.smithi154.stderr:2023-10-18T10:47:49.630+0000 7f5933009640 -1 osd.5 43 heartbeat_check: no reply from 172.21.15.154:6828 osd.6 since back 2023-10-18T10:47:23.980462+0000 front 2023-10-18T10:47:23.980375+0000 (oldest deadline 2023-10-18T10:47:49.280374+0000) 2023-10-18T10:48:18.274 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:48:18.280 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:48:48.630 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:48:48.635 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:49:18.811 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:49:18.816 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:49:25.338 INFO:tasks.ceph.osd.6:Sent signal 18 2023-10-18T10:49:25.339 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T10:49:25.339 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:49:25.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:49:25.352 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T10:49:25.407 INFO:teuthology.orchestra.run.smithi089.stdout:client5461 2023-10-18T10:49:25.408 INFO:tasks.cephfs.mount:Killing connection on ubuntu@smithi089.front.sepia.ceph.com... 2023-10-18T10:49:25.408 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T10:49:25.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:49:25.409 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set brx.0 down 2023-10-18T10:49:25.476 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:49:25.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.660+0000 7f3942b00640 1 Processor -- start 2023-10-18T10:49:25.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.660+0000 7f3942b00640 1 -- start start 2023-10-18T10:49:25.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.664+0000 7f3942b00640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f393c159490 0x7f393c15d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:49:25.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.664+0000 7f3942b00640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f393c06bee0 0x7f393c0c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:49:25.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.664+0000 7f3942b00640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f393c15ddf0 con 0x7f393c070600 2023-10-18T10:49:25.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.664+0000 7f3942b00640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f393c070b90 con 0x7f393c06bee0 2023-10-18T10:49:25.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.664+0000 7f3942b00640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f393c070e70 con 0x7f393c159490 2023-10-18T10:49:25.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.664+0000 7f394109d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f393c06bee0 0x7f393c0c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:49:25.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.664+0000 7f394109d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f393c06bee0 0x7f393c0c8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59662/0 (socket says 172.21.15.89:59662) 2023-10-18T10:49:25.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.664+0000 7f394109d640 1 -- 172.21.15.89:0/2641303162 learned_addr learned my addr 172.21.15.89:0/2641303162 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:49:25.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.664+0000 7f394089c640 1 --2- 172.21.15.89:0/2641303162 >> v2:172.21.15.154:3300/0 conn(0x7f393c159490 0x7f393c15d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:49:25.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.664+0000 7f394109d640 1 -- 172.21.15.89:0/2641303162 >> v2:172.21.15.154:3300/0 conn(0x7f393c159490 msgr2=0x7f393c15d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:49:25.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.664+0000 7f394109d640 1 --2- 172.21.15.89:0/2641303162 >> v2:172.21.15.154:3300/0 conn(0x7f393c159490 0x7f393c15d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:49:25.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.664+0000 7f394109d640 1 -- 172.21.15.89:0/2641303162 >> v1:172.21.15.154:6789/0 conn(0x7f393c070600 legacy=0x7f393c06b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:49:25.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.664+0000 7f394109d640 1 -- 172.21.15.89:0/2641303162 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f393c071150 con 0x7f393c06bee0 2023-10-18T10:49:25.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.664+0000 7f394089c640 1 --2- 172.21.15.89:0/2641303162 >> v2:172.21.15.154:3300/0 conn(0x7f393c159490 0x7f393c15d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:49:25.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.664+0000 7f394109d640 1 --2- 172.21.15.89:0/2641303162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f393c06bee0 0x7f393c0c8960 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f393c06c560 tx=0x7f393401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee93fb65b06f62b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:49:25.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.668+0000 7f393b7fe640 1 -- 172.21.15.89:0/2641303162 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39340ab600 con 0x7f393c06bee0 2023-10-18T10:49:25.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.668+0000 7f393b7fe640 1 -- 172.21.15.89:0/2641303162 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39340ab760 con 0x7f393c06bee0 2023-10-18T10:49:25.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.668+0000 7f393b7fe640 1 -- 172.21.15.89:0/2641303162 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39340abaf0 con 0x7f393c06bee0 2023-10-18T10:49:25.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.668+0000 7f3942b00640 1 -- 172.21.15.89:0/2641303162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f393c06bee0 msgr2=0x7f393c0c8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:49:25.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.668+0000 7f3942b00640 1 --2- 172.21.15.89:0/2641303162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f393c06bee0 0x7f393c0c8960 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f393c06c560 tx=0x7f393401c400 comp rx=0 tx=0).stop 2023-10-18T10:49:25.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.668+0000 7f3942b00640 1 -- 172.21.15.89:0/2641303162 shutdown_connections 2023-10-18T10:49:25.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.668+0000 7f3942b00640 1 --2- 172.21.15.89:0/2641303162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f393c06bee0 0x7f393c0c8960 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:49:25.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.668+0000 7f3942b00640 1 --2- 172.21.15.89:0/2641303162 >> v2:172.21.15.154:3300/0 conn(0x7f393c159490 0x7f393c15d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:49:25.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.668+0000 7f3942b00640 1 -- 172.21.15.89:0/2641303162 >> 172.21.15.89:0/2641303162 conn(0x7f393c069f00 msgr2=0x7f393c0cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:49:25.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.668+0000 7f3942b00640 1 -- 172.21.15.89:0/2641303162 shutdown_connections 2023-10-18T10:49:25.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.668+0000 7f3942b00640 1 -- 172.21.15.89:0/2641303162 wait complete. 2023-10-18T10:49:25.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.668+0000 7f3942b00640 1 Processor -- start 2023-10-18T10:49:25.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.668+0000 7f3942b00640 1 -- start start 2023-10-18T10:49:25.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.668+0000 7f3942b00640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f393c06bee0 0x7f393c0c4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:49:25.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.668+0000 7f3942b00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f393c070600 0x7f393c0af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:49:25.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.668+0000 7f3942b00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f393c159490 0x7f393c14faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:49:25.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.668+0000 7f3942b00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393c0c7f70 con 0x7f393c070600 2023-10-18T10:49:25.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.668+0000 7f3942b00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393c0c3d50 con 0x7f393c159490 2023-10-18T10:49:25.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.668+0000 7f3942b00640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f393c0c4030 con 0x7f393c06bee0 2023-10-18T10:49:25.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.668+0000 7f393bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f393c070600 0x7f393c0af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:49:25.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.672+0000 7f393bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f393c070600 0x7f393c0af2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59678/0 (socket says 172.21.15.89:59678) 2023-10-18T10:49:25.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.672+0000 7f393bfff640 1 -- 172.21.15.89:0/1989273487 learned_addr learned my addr 172.21.15.89:0/1989273487 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:49:25.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.672+0000 7f394089c640 1 --2- 172.21.15.89:0/1989273487 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f393c06bee0 0x7f393c0c4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:49:25.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.672+0000 7f394109d640 1 --2- 172.21.15.89:0/1989273487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f393c159490 0x7f393c14faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:49:25.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.672+0000 7f393bfff640 1 -- 172.21.15.89:0/1989273487 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f393c06bee0 msgr2=0x7f393c0c4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:49:25.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.672+0000 7f393bfff640 1 --2- 172.21.15.89:0/1989273487 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f393c06bee0 0x7f393c0c4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:49:25.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.672+0000 7f393bfff640 1 -- 172.21.15.89:0/1989273487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f393c159490 msgr2=0x7f393c14faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:49:25.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.672+0000 7f393bfff640 1 --2- 172.21.15.89:0/1989273487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f393c159490 0x7f393c14faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:49:25.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.672+0000 7f393bfff640 1 -- 172.21.15.89:0/1989273487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39340094d0 con 0x7f393c070600 2023-10-18T10:49:25.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.672+0000 7f393bfff640 1 --2- 172.21.15.89:0/1989273487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f393c070600 0x7f393c0af2f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f393000c450 tx=0x7f393000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:49:25.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.672+0000 7f39397fa640 1 -- 172.21.15.89:0/1989273487 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3930012830 con 0x7f393c070600 2023-10-18T10:49:25.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.672+0000 7f39397fa640 1 -- 172.21.15.89:0/1989273487 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39300129b0 con 0x7f393c070600 2023-10-18T10:49:25.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.672+0000 7f39397fa640 1 -- 172.21.15.89:0/1989273487 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3930012d60 con 0x7f393c070600 2023-10-18T10:49:25.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.672+0000 7f3942b00640 1 -- 172.21.15.89:0/1989273487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f393c0c4310 con 0x7f393c070600 2023-10-18T10:49:25.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.672+0000 7f3942b00640 1 -- 172.21.15.89:0/1989273487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f393c0af830 con 0x7f393c070600 2023-10-18T10:49:25.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.672+0000 7f3942b00640 1 -- 172.21.15.89:0/1989273487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f393c0afab0 con 0x7f393c070600 2023-10-18T10:49:25.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.672+0000 7f39397fa640 1 -- 172.21.15.89:0/1989273487 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f393001c460 con 0x7f393c070600 2023-10-18T10:49:25.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.676+0000 7f39397fa640 1 --2- 172.21.15.89:0/1989273487 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3928065680 0x7f3928067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:49:25.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.676+0000 7f39397fa640 1 -- 172.21.15.89:0/1989273487 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f3930082e50 con 0x7f393c070600 2023-10-18T10:49:25.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.676+0000 7f394089c640 1 --2- 172.21.15.89:0/1989273487 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3928065680 0x7f3928067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:49:25.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.676+0000 7f394089c640 1 --2- 172.21.15.89:0/1989273487 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3928065680 0x7f3928067b40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f392c007510 tx=0x7f392c0074a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:49:25.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.680+0000 7f39397fa640 1 -- 172.21.15.89:0/1989273487 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f3930050760 con 0x7f393c070600 2023-10-18T10:49:25.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.856+0000 7f3942b00640 1 -- 172.21.15.89:0/1989273487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f393c04dbe0 con 0x7f393c070600 2023-10-18T10:49:25.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.856+0000 7f39397fa640 1 -- 172.21.15.89:0/1989273487 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4875 (secure 0 0 0) 0x7f3930053d80 con 0x7f393c070600 2023-10-18T10:49:25.863 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:49:25.863 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/654729641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":654729641},{"type":"v1","addr":"172.21.15.154:6836","nonce":654729641}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/964566480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":964566480},{"type":"v1","addr":"172.21.15.154:6837","nonce":964566480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2261878538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2261878538},{"type":"v1","addr":"172.21.15.89:6833","nonce":2261878538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:47:15.199575+0000","modified":"2023-10-18T10:47:18.846656+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5517":{"gid":5517,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6835/2092274731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2092274731},{"type":"v1","addr":"172.21.15.89:6835","nonce":2092274731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T10:49:25.865 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 18 2023-10-18T10:49:25.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.860+0000 7f3942b00640 1 -- 172.21.15.89:0/1989273487 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3928065680 msgr2=0x7f3928067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:49:25.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.860+0000 7f3942b00640 1 --2- 172.21.15.89:0/1989273487 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3928065680 0x7f3928067b40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f392c007510 tx=0x7f392c0074a0 comp rx=0 tx=0).stop 2023-10-18T10:49:25.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.860+0000 7f3942b00640 1 -- 172.21.15.89:0/1989273487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f393c070600 msgr2=0x7f393c0af2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:49:25.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.860+0000 7f3942b00640 1 --2- 172.21.15.89:0/1989273487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f393c070600 0x7f393c0af2f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f393000c450 tx=0x7f393000c910 comp rx=0 tx=0).stop 2023-10-18T10:49:25.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.860+0000 7f3942b00640 1 -- 172.21.15.89:0/1989273487 shutdown_connections 2023-10-18T10:49:25.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.860+0000 7f3942b00640 1 --2- 172.21.15.89:0/1989273487 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3928065680 0x7f3928067b40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:49:25.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.860+0000 7f3942b00640 1 --2- 172.21.15.89:0/1989273487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f393c159490 0x7f393c14faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:49:25.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.860+0000 7f3942b00640 1 --2- 172.21.15.89:0/1989273487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f393c070600 0x7f393c0af2f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:49:25.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.860+0000 7f3942b00640 1 --2- 172.21.15.89:0/1989273487 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f393c06bee0 0x7f393c0c4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:49:25.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.860+0000 7f3942b00640 1 -- 172.21.15.89:0/1989273487 >> 172.21.15.89:0/1989273487 conn(0x7f393c069f00 msgr2=0x7f393c153810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:49:25.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.860+0000 7f3942b00640 1 -- 172.21.15.89:0/1989273487 shutdown_connections 2023-10-18T10:49:25.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:25.860+0000 7f3942b00640 1 -- 172.21.15.89:0/1989273487 wait complete. 2023-10-18T10:49:25.883 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-18T10:49:26.042 INFO:teuthology.orchestra.run.smithi089.stdout:[{"id":5183,"entity":{"name":{"type":"client","num":5183},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":4272238049}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":122.326000392,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.75379632438930622,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":3,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5183 v1:192.168.0.1:0/4272238049","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi154","kernel_version":"6.6.0-rc5-g47672119149b","root":"/"}},{"id":5461,"entity":{"name":{"type":"client","num":5461},"addr":{"type":"v1","addr":"172.21.15.89:0","nonce":330049992}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":8,"uptime":123.421970469,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.75189690328315972,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":25,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5461 v1:172.21.15.89:0/330049992","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi089","kernel_version":"6.6.0-rc5-g47672119149b","root":"/"}}] 2023-10-18T10:49:26.044 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "1", "hostname": "smithi154", "kernel_version": "6.6.0-rc5-g47672119149b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 4272238049, "type": "v1" }, "name": { "num": 5183, "type": "client" } }, "id": 5183, "inst": "client.5183 v1:192.168.0.1:0/4272238049", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 3, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.7537963243893062 }, "state": "open", "uptime": 122.326000392 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "0", "hostname": "smithi089", "kernel_version": "6.6.0-rc5-g47672119149b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.89:0", "nonce": 330049992, "type": "v1" }, "name": { "num": 5461, "type": "client" } }, "id": 5461, "inst": "client.5461 v1:172.21.15.89:0/330049992", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 25, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.7518969032831597 }, "state": "open", "uptime": 123.421970469 } ] 2023-10-18T10:49:26.045 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:49:26.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.176+0000 7f4264cca640 1 Processor -- start 2023-10-18T10:49:26.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.176+0000 7f4264cca640 1 -- start start 2023-10-18T10:49:26.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.176+0000 7f4264cca640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f42600b1860 0x7f42600cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:49:26.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.176+0000 7f4264cca640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42600b1cc0 0x7f42600cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:49:26.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.176+0000 7f4264cca640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42600d1e50 con 0x7f426015c550 2023-10-18T10:49:26.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.176+0000 7f4264cca640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42600d2130 con 0x7f42600b1860 2023-10-18T10:49:26.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.176+0000 7f4264cca640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42600d2410 con 0x7f42600b1cc0 2023-10-18T10:49:26.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.176+0000 7f425ed9d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42600b1cc0 0x7f42600cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:49:26.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.176+0000 7f425ed9d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42600b1cc0 0x7f42600cfb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:52814/0 (socket says 172.21.15.89:52814) 2023-10-18T10:49:26.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.176+0000 7f425ed9d640 1 -- 172.21.15.89:0/3998918644 learned_addr learned my addr 172.21.15.89:0/3998918644 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:49:26.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.176+0000 7f4255d9b640 1 --2- 172.21.15.89:0/3998918644 >> v2:172.21.15.154:3300/0 conn(0x7f42600b1860 0x7f42600cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:49:26.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.176+0000 7f425ed9d640 1 -- 172.21.15.89:0/3998918644 >> v2:172.21.15.154:3300/0 conn(0x7f42600b1860 msgr2=0x7f42600cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:49:26.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.176+0000 7f425ed9d640 1 --2- 172.21.15.89:0/3998918644 >> v2:172.21.15.154:3300/0 conn(0x7f42600b1860 0x7f42600cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:49:26.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.176+0000 7f425ed9d640 1 -- 172.21.15.89:0/3998918644 >> v1:172.21.15.154:6789/0 conn(0x7f426015c550 legacy=0x7f42600d2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:49:26.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.176+0000 7f425dd9b640 1 -- 172.21.15.89:0/3998918644 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 779538476 0 0) 0x7f42600d1e50 con 0x7f426015c550 2023-10-18T10:49:26.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.176+0000 7f425ed9d640 1 -- 172.21.15.89:0/3998918644 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42600d26f0 con 0x7f42600b1cc0 2023-10-18T10:49:26.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.180+0000 7f425ed9d640 1 --2- 172.21.15.89:0/3998918644 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42600b1cc0 0x7f42600cfb50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4258009e50 tx=0x7f425801c9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d1ef9d784ce40c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:49:26.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.180+0000 7f425dd9b640 1 -- 172.21.15.89:0/3998918644 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42580a58d0 con 0x7f42600b1cc0 2023-10-18T10:49:26.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.180+0000 7f425dd9b640 1 -- 172.21.15.89:0/3998918644 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42580a5a30 con 0x7f42600b1cc0 2023-10-18T10:49:26.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.180+0000 7f425dd9b640 1 -- 172.21.15.89:0/3998918644 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42580a5dc0 con 0x7f42600b1cc0 2023-10-18T10:49:26.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.180+0000 7f4264cca640 1 -- 172.21.15.89:0/3998918644 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42600b1cc0 msgr2=0x7f42600cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:49:26.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.180+0000 7f4264cca640 1 --2- 172.21.15.89:0/3998918644 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42600b1cc0 0x7f42600cfb50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4258009e50 tx=0x7f425801c9e0 comp rx=0 tx=0).stop 2023-10-18T10:49:26.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.180+0000 7f4264cca640 1 -- 172.21.15.89:0/3998918644 shutdown_connections 2023-10-18T10:49:26.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.180+0000 7f4264cca640 1 --2- 172.21.15.89:0/3998918644 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42600b1cc0 0x7f42600cfb50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:49:26.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.180+0000 7f4264cca640 1 --2- 172.21.15.89:0/3998918644 >> v2:172.21.15.154:3300/0 conn(0x7f42600b1860 0x7f42600cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:49:26.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.180+0000 7f4264cca640 1 -- 172.21.15.89:0/3998918644 >> 172.21.15.89:0/3998918644 conn(0x7f4260069d00 msgr2=0x7f426006a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:49:26.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.180+0000 7f4264cca640 1 -- 172.21.15.89:0/3998918644 shutdown_connections 2023-10-18T10:49:26.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.180+0000 7f4264cca640 1 -- 172.21.15.89:0/3998918644 wait complete. 2023-10-18T10:49:26.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.184+0000 7f4264cca640 1 Processor -- start 2023-10-18T10:49:26.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.184+0000 7f4264cca640 1 -- start start 2023-10-18T10:49:26.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.184+0000 7f4264cca640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42600b1860 0x7f42600b7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:49:26.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.184+0000 7f4264cca640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42600b1cc0 0x7f42601608e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:49:26.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.184+0000 7f4264cca640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f426015c550 0x7f4260166b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:49:26.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.184+0000 7f4264cca640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4260160e20 con 0x7f42600b1cc0 2023-10-18T10:49:26.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.184+0000 7f4264cca640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4260167060 con 0x7f426015c550 2023-10-18T10:49:26.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.184+0000 7f4264cca640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4260157a80 con 0x7f42600b1860 2023-10-18T10:49:26.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.184+0000 7f4255d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42600b1cc0 0x7f42601608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:49:26.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.184+0000 7f4255d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42600b1cc0 0x7f42601608e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59682/0 (socket says 172.21.15.89:59682) 2023-10-18T10:49:26.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.184+0000 7f4255d9b640 1 -- 172.21.15.89:0/4016926470 learned_addr learned my addr 172.21.15.89:0/4016926470 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:49:26.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.184+0000 7f425e59c640 1 --2- 172.21.15.89:0/4016926470 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42600b1860 0x7f42600b7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:49:26.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.184+0000 7f425ed9d640 1 --2- 172.21.15.89:0/4016926470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f426015c550 0x7f4260166b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:49:26.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.184+0000 7f4255d9b640 1 -- 172.21.15.89:0/4016926470 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42600b1860 msgr2=0x7f42600b7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:49:26.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.184+0000 7f4255d9b640 1 --2- 172.21.15.89:0/4016926470 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42600b1860 0x7f42600b7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:49:26.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.184+0000 7f4255d9b640 1 -- 172.21.15.89:0/4016926470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f426015c550 msgr2=0x7f4260166b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:49:26.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.184+0000 7f4255d9b640 1 --2- 172.21.15.89:0/4016926470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f426015c550 0x7f4260166b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:49:26.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.184+0000 7f4255d9b640 1 -- 172.21.15.89:0/4016926470 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4258009b30 con 0x7f42600b1cc0 2023-10-18T10:49:26.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.184+0000 7f425ed9d640 1 --2- 172.21.15.89:0/4016926470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f426015c550 0x7f4260166b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:49:26.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.184+0000 7f4255d9b640 1 --2- 172.21.15.89:0/4016926470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42600b1cc0 0x7f42601608e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f425000e9b0 tx=0x7f425000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:49:26.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.188+0000 7f42577fe640 1 -- 172.21.15.89:0/4016926470 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42500096b0 con 0x7f42600b1cc0 2023-10-18T10:49:26.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.188+0000 7f42577fe640 1 -- 172.21.15.89:0/4016926470 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f425000c9a0 con 0x7f42600b1cc0 2023-10-18T10:49:26.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.188+0000 7f4264cca640 1 -- 172.21.15.89:0/4016926470 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4260157d60 con 0x7f42600b1cc0 2023-10-18T10:49:26.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.188+0000 7f42577fe640 1 -- 172.21.15.89:0/4016926470 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425000cd30 con 0x7f42600b1cc0 2023-10-18T10:49:26.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.188+0000 7f4264cca640 1 -- 172.21.15.89:0/4016926470 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4260161010 con 0x7f42600b1cc0 2023-10-18T10:49:26.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.188+0000 7f4264cca640 1 -- 172.21.15.89:0/4016926470 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42600b2490 con 0x7f42600b1cc0 2023-10-18T10:49:26.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.188+0000 7f42577fe640 1 -- 172.21.15.89:0/4016926470 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f4250013470 con 0x7f42600b1cc0 2023-10-18T10:49:26.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.188+0000 7f42577fe640 1 --2- 172.21.15.89:0/4016926470 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4240065680 0x7f4240067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:49:26.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.188+0000 7f42577fe640 1 -- 172.21.15.89:0/4016926470 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f4250083230 con 0x7f42600b1cc0 2023-10-18T10:49:26.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.188+0000 7f425e59c640 1 --2- 172.21.15.89:0/4016926470 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4240065680 0x7f4240067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:49:26.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.192+0000 7f425e59c640 1 --2- 172.21.15.89:0/4016926470 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4240065680 0x7f4240067b40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4248007fd0 tx=0x7f4248009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:49:26.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.192+0000 7f42577fe640 1 -- 172.21.15.89:0/4016926470 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f4250050c00 con 0x7f42600b1cc0 2023-10-18T10:49:26.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.372+0000 7f4264cca640 1 -- 172.21.15.89:0/4016926470 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f426004dbe0 con 0x7f42600b1cc0 2023-10-18T10:49:26.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.372+0000 7f42577fe640 1 -- 172.21.15.89:0/4016926470 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4875 (secure 0 0 0) 0x7f4250054220 con 0x7f42600b1cc0 2023-10-18T10:49:26.377 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:49:26.377 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/654729641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":654729641},{"type":"v1","addr":"172.21.15.154:6836","nonce":654729641}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/964566480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":964566480},{"type":"v1","addr":"172.21.15.154:6837","nonce":964566480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2261878538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2261878538},{"type":"v1","addr":"172.21.15.89:6833","nonce":2261878538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:47:15.199575+0000","modified":"2023-10-18T10:47:18.846656+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5517":{"gid":5517,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6835/2092274731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2092274731},{"type":"v1","addr":"172.21.15.89:6835","nonce":2092274731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T10:49:26.377 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 18 2023-10-18T10:49:26.378 INFO:tasks.ceph.osd.6.smithi154.stderr:2023-10-18T10:49:26.372+0000 7f0c7d73e640 -1 osd.6 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T10:49:26.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.372+0000 7f4264cca640 1 -- 172.21.15.89:0/4016926470 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4240065680 msgr2=0x7f4240067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:49:26.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.372+0000 7f4264cca640 1 --2- 172.21.15.89:0/4016926470 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4240065680 0x7f4240067b40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4248007fd0 tx=0x7f4248009040 comp rx=0 tx=0).stop 2023-10-18T10:49:26.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.372+0000 7f4264cca640 1 -- 172.21.15.89:0/4016926470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42600b1cc0 msgr2=0x7f42601608e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:49:26.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.372+0000 7f4264cca640 1 --2- 172.21.15.89:0/4016926470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42600b1cc0 0x7f42601608e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f425000e9b0 tx=0x7f425000ee70 comp rx=0 tx=0).stop 2023-10-18T10:49:26.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.372+0000 7f4264cca640 1 -- 172.21.15.89:0/4016926470 shutdown_connections 2023-10-18T10:49:26.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.372+0000 7f4264cca640 1 --2- 172.21.15.89:0/4016926470 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4240065680 0x7f4240067b40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:49:26.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.372+0000 7f4264cca640 1 --2- 172.21.15.89:0/4016926470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f426015c550 0x7f4260166b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:49:26.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.372+0000 7f4264cca640 1 --2- 172.21.15.89:0/4016926470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42600b1cc0 0x7f42601608e0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:49:26.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.372+0000 7f4264cca640 1 --2- 172.21.15.89:0/4016926470 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42600b1860 0x7f42600b7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:49:26.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.372+0000 7f4264cca640 1 -- 172.21.15.89:0/4016926470 >> 172.21.15.89:0/4016926470 conn(0x7f4260069d00 msgr2=0x7f42600d0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:49:26.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.372+0000 7f4264cca640 1 -- 172.21.15.89:0/4016926470 shutdown_connections 2023-10-18T10:49:26.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:49:26.372+0000 7f4264cca640 1 -- 172.21.15.89:0/4016926470 wait complete. 2023-10-18T10:49:49.008 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:49:49.013 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:50:19.067 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:50:19.072 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:50:49.154 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:50:49.159 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:50:56.398 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:50:56.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.547+0000 7f5ea3dfa640 1 Processor -- start 2023-10-18T10:50:56.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.547+0000 7f5ea3dfa640 1 -- start start 2023-10-18T10:50:56.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.547+0000 7f5ea3dfa640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5e9c15c940 0x7f5e9c0d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:50:56.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.547+0000 7f5ea3dfa640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5e9c0d2f40 0x7f5e9c0d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:50:56.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.547+0000 7f5ea3dfa640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e9c0d10f0 con 0x7f5e9c0d0ca0 2023-10-18T10:50:56.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.547+0000 7f5ea3dfa640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e9c15da50 con 0x7f5e9c15c940 2023-10-18T10:50:56.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.547+0000 7f5ea3dfa640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e9c15ce40 con 0x7f5e9c0d2f40 2023-10-18T10:50:56.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.547+0000 7f5ea1b96640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5e9c15c940 0x7f5e9c0d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:50:56.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.547+0000 7f5ea2397640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5e9c0d0ca0 0x7f5e9c0cf060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:53788/0 (socket says 172.21.15.89:53788) 2023-10-18T10:50:56.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.547+0000 7f5ea2397640 1 -- 172.21.15.89:0/4011229527 learned_addr learned my addr 172.21.15.89:0/4011229527 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:50:56.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.547+0000 7f5ea1395640 1 --2- 172.21.15.89:0/4011229527 >> v2:172.21.15.154:3300/0 conn(0x7f5e9c0d2f40 0x7f5e9c0d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:50:56.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.547+0000 7f5ea1b96640 1 -- 172.21.15.89:0/4011229527 >> v2:172.21.15.154:3300/0 conn(0x7f5e9c0d2f40 msgr2=0x7f5e9c0d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:50:56.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.547+0000 7f5ea1b96640 1 --2- 172.21.15.89:0/4011229527 >> v2:172.21.15.154:3300/0 conn(0x7f5e9c0d2f40 0x7f5e9c0d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:50:56.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.547+0000 7f5ea1b96640 1 -- 172.21.15.89:0/4011229527 >> v1:172.21.15.89:6789/0 conn(0x7f5e9c0d0ca0 legacy=0x7f5e9c0cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:50:56.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.547+0000 7f5ea1b96640 1 -- 172.21.15.89:0/4011229527 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e9c15d120 con 0x7f5e9c15c940 2023-10-18T10:50:56.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.551+0000 7f5ea1b96640 1 --2- 172.21.15.89:0/4011229527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e9c15c940 0x7f5e9c0d2a00 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c0724f0 tx=0x7f5e9801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca9f5c261e442d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:50:56.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.551+0000 7f5ea0b94640 1 -- 172.21.15.89:0/4011229527 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e980a58b0 con 0x7f5e9c15c940 2023-10-18T10:50:56.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.551+0000 7f5ea0b94640 1 -- 172.21.15.89:0/4011229527 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e980a5a10 con 0x7f5e9c15c940 2023-10-18T10:50:56.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.551+0000 7f5ea0b94640 1 -- 172.21.15.89:0/4011229527 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e980a5da0 con 0x7f5e9c15c940 2023-10-18T10:50:56.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.551+0000 7f5ea3dfa640 1 -- 172.21.15.89:0/4011229527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e9c15c940 msgr2=0x7f5e9c0d2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:50:56.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.551+0000 7f5ea3dfa640 1 --2- 172.21.15.89:0/4011229527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e9c15c940 0x7f5e9c0d2a00 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c0724f0 tx=0x7f5e9801c8e0 comp rx=0 tx=0).stop 2023-10-18T10:50:56.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.551+0000 7f5ea3dfa640 1 -- 172.21.15.89:0/4011229527 shutdown_connections 2023-10-18T10:50:56.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.551+0000 7f5ea3dfa640 1 --2- 172.21.15.89:0/4011229527 >> v2:172.21.15.154:3300/0 conn(0x7f5e9c0d2f40 0x7f5e9c0d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:50:56.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.551+0000 7f5ea3dfa640 1 --2- 172.21.15.89:0/4011229527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e9c15c940 0x7f5e9c0d2a00 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:50:56.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.551+0000 7f5ea3dfa640 1 -- 172.21.15.89:0/4011229527 >> 172.21.15.89:0/4011229527 conn(0x7f5e9c089fd0 msgr2=0x7f5e9c09b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:50:56.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.551+0000 7f5ea3dfa640 1 -- 172.21.15.89:0/4011229527 shutdown_connections 2023-10-18T10:50:56.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.551+0000 7f5ea3dfa640 1 -- 172.21.15.89:0/4011229527 wait complete. 2023-10-18T10:50:56.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.555+0000 7f5ea3dfa640 1 Processor -- start 2023-10-18T10:50:56.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.555+0000 7f5ea3dfa640 1 -- start start 2023-10-18T10:50:56.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.555+0000 7f5ea3dfa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e9c0d0ca0 0x7f5e9c0c7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:50:56.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.555+0000 7f5ea3dfa640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e9c0c8d40 0x7f5e9c154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:50:56.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.555+0000 7f5ea3dfa640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e9c15e430 0x7f5e9c15e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:50:56.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.555+0000 7f5ea3dfa640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9c15ee20 con 0x7f5e9c15e430 2023-10-18T10:50:56.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.555+0000 7f5ea3dfa640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9c154ca0 con 0x7f5e9c0c8d40 2023-10-18T10:50:56.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.555+0000 7f5ea3dfa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9c154f80 con 0x7f5e9c0d0ca0 2023-10-18T10:50:56.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.555+0000 7f5ea2397640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e9c15e430 0x7f5e9c15e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:50:56.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.555+0000 7f5ea2397640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e9c15e430 0x7f5e9c15e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38270/0 (socket says 172.21.15.89:38270) 2023-10-18T10:50:56.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.555+0000 7f5ea2397640 1 -- 172.21.15.89:0/2270707516 learned_addr learned my addr 172.21.15.89:0/2270707516 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:50:56.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.555+0000 7f5ea1b96640 1 --2- 172.21.15.89:0/2270707516 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e9c0d0ca0 0x7f5e9c0c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:50:56.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.555+0000 7f5ea1395640 1 --2- 172.21.15.89:0/2270707516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e9c0c8d40 0x7f5e9c154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:50:56.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.555+0000 7f5ea2397640 1 -- 172.21.15.89:0/2270707516 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e9c0d0ca0 msgr2=0x7f5e9c0c7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:50:56.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.555+0000 7f5ea2397640 1 --2- 172.21.15.89:0/2270707516 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e9c0d0ca0 0x7f5e9c0c7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:50:56.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.555+0000 7f5ea2397640 1 -- 172.21.15.89:0/2270707516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e9c0c8d40 msgr2=0x7f5e9c154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:50:56.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.555+0000 7f5ea2397640 1 --2- 172.21.15.89:0/2270707516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e9c0c8d40 0x7f5e9c154330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:50:56.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.555+0000 7f5ea2397640 1 -- 172.21.15.89:0/2270707516 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e98009a70 con 0x7f5e9c15e430 2023-10-18T10:50:56.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.555+0000 7f5ea1b96640 1 --2- 172.21.15.89:0/2270707516 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e9c0d0ca0 0x7f5e9c0c7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:50:56.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.555+0000 7f5ea1395640 1 --2- 172.21.15.89:0/2270707516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e9c0c8d40 0x7f5e9c154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:50:56.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.555+0000 7f5ea2397640 1 --2- 172.21.15.89:0/2270707516 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e9c15e430 0x7f5e9c15e8e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f5e9400c400 tx=0x7f5e9400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:50:56.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.555+0000 7f5e927fc640 1 -- 172.21.15.89:0/2270707516 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e94005cf0 con 0x7f5e9c15e430 2023-10-18T10:50:56.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.555+0000 7f5e927fc640 1 -- 172.21.15.89:0/2270707516 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e94005e50 con 0x7f5e9c15e430 2023-10-18T10:50:56.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.555+0000 7f5e927fc640 1 -- 172.21.15.89:0/2270707516 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e940072d0 con 0x7f5e9c15e430 2023-10-18T10:50:56.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.555+0000 7f5ea3dfa640 1 -- 172.21.15.89:0/2270707516 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e9c155260 con 0x7f5e9c15e430 2023-10-18T10:50:56.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.555+0000 7f5ea3dfa640 1 -- 172.21.15.89:0/2270707516 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e9c155820 con 0x7f5e9c15e430 2023-10-18T10:50:56.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.559+0000 7f5ea3dfa640 1 -- 172.21.15.89:0/2270707516 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e9c155a10 con 0x7f5e9c15e430 2023-10-18T10:50:56.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.559+0000 7f5e927fc640 1 -- 172.21.15.89:0/2270707516 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f5e94007430 con 0x7f5e9c15e430 2023-10-18T10:50:56.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.559+0000 7f5e927fc640 1 --2- 172.21.15.89:0/2270707516 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5e800656b0 0x7f5e80067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:50:56.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.559+0000 7f5e927fc640 1 -- 172.21.15.89:0/2270707516 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f5e94082370 con 0x7f5e9c15e430 2023-10-18T10:50:56.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.559+0000 7f5ea1b96640 1 --2- 172.21.15.89:0/2270707516 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5e800656b0 0x7f5e80067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:50:56.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.559+0000 7f5ea1b96640 1 --2- 172.21.15.89:0/2270707516 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5e800656b0 0x7f5e80067b70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5e980af000 tx=0x7f5e980a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:50:56.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.563+0000 7f5e927fc640 1 -- 172.21.15.89:0/2270707516 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f5e9404fcc0 con 0x7f5e9c15e430 2023-10-18T10:50:56.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.743+0000 7f5ea3dfa640 1 -- 172.21.15.89:0/2270707516 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e9c0d10f0 con 0x7f5e9c15e430 2023-10-18T10:50:56.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.743+0000 7f5e927fc640 1 -- 172.21.15.89:0/2270707516 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4875 (secure 0 0 0) 0x7f5e9c0d10f0 con 0x7f5e9c15e430 2023-10-18T10:50:56.743 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:50:56.744 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/654729641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":654729641},{"type":"v1","addr":"172.21.15.154:6836","nonce":654729641}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/964566480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":964566480},{"type":"v1","addr":"172.21.15.154:6837","nonce":964566480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2261878538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2261878538},{"type":"v1","addr":"172.21.15.89:6833","nonce":2261878538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:47:15.199575+0000","modified":"2023-10-18T10:47:18.846656+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5517":{"gid":5517,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6835/2092274731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2092274731},{"type":"v1","addr":"172.21.15.89:6835","nonce":2092274731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T10:50:56.745 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 18 2023-10-18T10:50:56.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.743+0000 7f5ea3dfa640 1 -- 172.21.15.89:0/2270707516 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5e800656b0 msgr2=0x7f5e80067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:50:56.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.743+0000 7f5ea3dfa640 1 --2- 172.21.15.89:0/2270707516 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5e800656b0 0x7f5e80067b70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5e980af000 tx=0x7f5e980a7070 comp rx=0 tx=0).stop 2023-10-18T10:50:56.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.743+0000 7f5ea3dfa640 1 -- 172.21.15.89:0/2270707516 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e9c15e430 msgr2=0x7f5e9c15e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:50:56.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.743+0000 7f5ea3dfa640 1 --2- 172.21.15.89:0/2270707516 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e9c15e430 0x7f5e9c15e8e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f5e9400c400 tx=0x7f5e9400c8c0 comp rx=0 tx=0).stop 2023-10-18T10:50:56.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.743+0000 7f5ea3dfa640 1 -- 172.21.15.89:0/2270707516 shutdown_connections 2023-10-18T10:50:56.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.743+0000 7f5ea3dfa640 1 --2- 172.21.15.89:0/2270707516 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5e800656b0 0x7f5e80067b70 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:50:56.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.743+0000 7f5ea3dfa640 1 --2- 172.21.15.89:0/2270707516 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e9c15e430 0x7f5e9c15e8e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:50:56.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.743+0000 7f5ea3dfa640 1 --2- 172.21.15.89:0/2270707516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e9c0c8d40 0x7f5e9c154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:50:56.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.743+0000 7f5ea3dfa640 1 --2- 172.21.15.89:0/2270707516 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e9c0d0ca0 0x7f5e9c0c7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:50:56.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.743+0000 7f5ea3dfa640 1 -- 172.21.15.89:0/2270707516 >> 172.21.15.89:0/2270707516 conn(0x7f5e9c089fd0 msgr2=0x7f5e9c06af80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:50:56.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.743+0000 7f5ea3dfa640 1 -- 172.21.15.89:0/2270707516 shutdown_connections 2023-10-18T10:50:56.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:56.743+0000 7f5ea3dfa640 1 -- 172.21.15.89:0/2270707516 wait complete. 2023-10-18T10:50:56.763 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-18T10:50:56.921 INFO:teuthology.orchestra.run.smithi089.stdout:[{"id":5183,"entity":{"name":{"type":"client","num":5183},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":4272238049}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":213.21304664600001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.61101929528118393,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":3,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5183 v1:192.168.0.1:0/4272238049","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi154","kernel_version":"6.6.0-rc5-g47672119149b","root":"/"}},{"id":5461,"entity":{"name":{"type":"client","num":5461},"addr":{"type":"v1","addr":"172.21.15.89:0","nonce":330049992}},"state":"stale","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":214.30901672300001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.60947964470427218,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":25,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5461 v1:172.21.15.89:0/330049992","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi089","kernel_version":"6.6.0-rc5-g47672119149b","root":"/"}}] 2023-10-18T10:50:56.922 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "1", "hostname": "smithi154", "kernel_version": "6.6.0-rc5-g47672119149b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 4272238049, "type": "v1" }, "name": { "num": 5183, "type": "client" } }, "id": 5183, "inst": "client.5183 v1:192.168.0.1:0/4272238049", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 3, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.6110192952811839 }, "state": "open", "uptime": 213.213046646 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "0", "hostname": "smithi089", "kernel_version": "6.6.0-rc5-g47672119149b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.89:0", "nonce": 330049992, "type": "v1" }, "name": { "num": 5461, "type": "client" } }, "id": 5461, "inst": "client.5461 v1:172.21.15.89:0/330049992", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 25, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.6094796447042722 }, "state": "stale", "uptime": 214.309016723 } ] 2023-10-18T10:50:56.923 DEBUG:teuthology.orchestra.run.smithi089:> sudo daemon-helper kill ceph --cluster ceph -w 2023-10-18T10:50:56.927 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:50:57.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.067+0000 7fef04c19640 1 Processor -- start 2023-10-18T10:50:57.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.067+0000 7fef04c19640 1 -- start start 2023-10-18T10:50:57.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.067+0000 7fef04c19640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fef0015c940 0x7fef000d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:50:57.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.071+0000 7fef04c19640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef0015da50 con 0x7fef000d0e70 2023-10-18T10:50:57.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.071+0000 7fef04c19640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef0015ce40 con 0x7fef000d2f40 2023-10-18T10:50:57.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.071+0000 7fef04c19640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef0015d120 con 0x7fef0015c940 2023-10-18T10:50:57.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.071+0000 7feefed9d640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fef000d0e70 0x7fef000cd160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:53794/0 (socket says 172.21.15.89:53794) 2023-10-18T10:50:57.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.071+0000 7feefed9d640 1 -- 172.21.15.89:0/4076201108 learned_addr learned my addr 172.21.15.89:0/4076201108 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:50:57.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.071+0000 7feefe59c640 1 --2- 172.21.15.89:0/4076201108 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fef0015c940 0x7fef000d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:50:57.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.071+0000 7feefdd9b640 1 -- 172.21.15.89:0/4076201108 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1593527835 0 0) 0x7fef0015da50 con 0x7fef000d0e70 2023-10-18T10:50:57.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.071+0000 7feefdd9b640 1 -- 172.21.15.89:0/4076201108 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feeec003530 con 0x7fef000d0e70 2023-10-18T10:50:57.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.071+0000 7feefdd9b640 1 -- 172.21.15.89:0/4076201108 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1557327321 0 0) 0x7fef0015ce40 con 0x7fef000d2f40 2023-10-18T10:50:57.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.071+0000 7feefe59c640 1 -- 172.21.15.89:0/4076201108 >> v1:172.21.15.154:6789/0 conn(0x7fef000d2f40 legacy=0x7fef000d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:50:57.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.071+0000 7feefe59c640 1 -- 172.21.15.89:0/4076201108 >> v1:172.21.15.89:6789/0 conn(0x7fef000d0e70 legacy=0x7fef000cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:50:57.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.071+0000 7feefe59c640 1 -- 172.21.15.89:0/4076201108 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef0015d400 con 0x7fef0015c940 2023-10-18T10:50:57.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.075+0000 7feefe59c640 1 --2- 172.21.15.89:0/4076201108 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fef0015c940 0x7fef000d2a00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7feef0009820 tx=0x7feef001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27e1d5821789e9c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:50:57.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.075+0000 7feefdd9b640 1 -- 172.21.15.89:0/4076201108 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef00aa5f0 con 0x7fef0015c940 2023-10-18T10:50:57.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.075+0000 7feefdd9b640 1 -- 172.21.15.89:0/4076201108 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef00aa750 con 0x7fef0015c940 2023-10-18T10:50:57.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.075+0000 7feefdd9b640 1 -- 172.21.15.89:0/4076201108 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef00aaae0 con 0x7fef0015c940 2023-10-18T10:50:57.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.075+0000 7fef04c19640 1 -- 172.21.15.89:0/4076201108 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fef0015c940 msgr2=0x7fef000d2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:50:57.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.075+0000 7fef04c19640 1 --2- 172.21.15.89:0/4076201108 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fef0015c940 0x7fef000d2a00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7feef0009820 tx=0x7feef001c470 comp rx=0 tx=0).stop 2023-10-18T10:50:57.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.075+0000 7fef04c19640 1 -- 172.21.15.89:0/4076201108 shutdown_connections 2023-10-18T10:50:57.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.075+0000 7fef04c19640 1 --2- 172.21.15.89:0/4076201108 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fef0015c940 0x7fef000d2a00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:50:57.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.075+0000 7fef04c19640 1 -- 172.21.15.89:0/4076201108 >> 172.21.15.89:0/4076201108 conn(0x7fef00089fd0 msgr2=0x7fef0009b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:50:57.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.075+0000 7fef04c19640 1 -- 172.21.15.89:0/4076201108 shutdown_connections 2023-10-18T10:50:57.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.075+0000 7fef04c19640 1 -- 172.21.15.89:0/4076201108 wait complete. 2023-10-18T10:50:57.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.075+0000 7fef04c19640 1 Processor -- start 2023-10-18T10:50:57.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.075+0000 7fef04c19640 1 -- start start 2023-10-18T10:50:57.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.075+0000 7fef04c19640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fef000d0e70 0x7fef00159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:50:57.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.075+0000 7fef04c19640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef00150620 0x7fef0015aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:50:57.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.075+0000 7fef04c19640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef0015afa0 0x7fef00156750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:50:57.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.079+0000 7fef04c19640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef00158e80 con 0x7fef00150620 2023-10-18T10:50:57.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.079+0000 7fef04c19640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef00159160 con 0x7fef0015afa0 2023-10-18T10:50:57.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.079+0000 7fef04c19640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef00159440 con 0x7fef000d0e70 2023-10-18T10:50:57.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.079+0000 7feef5d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef00150620 0x7fef0015aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:50:57.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.079+0000 7feef5d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef00150620 0x7fef0015aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38280/0 (socket says 172.21.15.89:38280) 2023-10-18T10:50:57.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.079+0000 7feef5d9b640 1 -- 172.21.15.89:0/975117939 learned_addr learned my addr 172.21.15.89:0/975117939 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:50:57.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.079+0000 7feefed9d640 1 --2- 172.21.15.89:0/975117939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef0015afa0 0x7fef00156750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:50:57.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.079+0000 7feefe59c640 1 --2- 172.21.15.89:0/975117939 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fef000d0e70 0x7fef00159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:50:57.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.079+0000 7feef5d9b640 1 -- 172.21.15.89:0/975117939 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fef000d0e70 msgr2=0x7fef00159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:50:57.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.079+0000 7feef5d9b640 1 --2- 172.21.15.89:0/975117939 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fef000d0e70 0x7fef00159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:50:57.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.079+0000 7feef5d9b640 1 -- 172.21.15.89:0/975117939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef0015afa0 msgr2=0x7fef00156750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:50:57.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.079+0000 7feef5d9b640 1 --2- 172.21.15.89:0/975117939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef0015afa0 0x7fef00156750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:50:57.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.079+0000 7feef5d9b640 1 -- 172.21.15.89:0/975117939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef0009420 con 0x7fef00150620 2023-10-18T10:50:57.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.079+0000 7feef5d9b640 1 --2- 172.21.15.89:0/975117939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef00150620 0x7fef0015aa60 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7feee800c4d0 tx=0x7feee800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:50:57.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.079+0000 7feef77fe640 1 -- 172.21.15.89:0/975117939 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee8005480 con 0x7fef00150620 2023-10-18T10:50:57.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.079+0000 7feef77fe640 1 -- 172.21.15.89:0/975117939 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee8003600 con 0x7fef00150620 2023-10-18T10:50:57.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.079+0000 7feef77fe640 1 -- 172.21.15.89:0/975117939 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee80039b0 con 0x7fef00150620 2023-10-18T10:50:57.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.079+0000 7fef04c19640 1 -- 172.21.15.89:0/975117939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef00159720 con 0x7fef00150620 2023-10-18T10:50:57.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.079+0000 7fef04c19640 1 -- 172.21.15.89:0/975117939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef00158b50 con 0x7fef00150620 2023-10-18T10:50:57.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.079+0000 7fef04c19640 1 -- 172.21.15.89:0/975117939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef00159c40 con 0x7fef00150620 2023-10-18T10:50:57.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.079+0000 7feef77fe640 1 -- 172.21.15.89:0/975117939 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7feee8003b10 con 0x7fef00150620 2023-10-18T10:50:57.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.083+0000 7feef77fe640 1 --2- 172.21.15.89:0/975117939 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7feee00656b0 0x7feee0067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:50:57.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.083+0000 7feef77fe640 1 -- 172.21.15.89:0/975117939 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6440+0+0 (secure 0 0 0) 0x7feee8082440 con 0x7fef00150620 2023-10-18T10:50:57.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.083+0000 7feefe59c640 1 --2- 172.21.15.89:0/975117939 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7feee00656b0 0x7feee0067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:50:57.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.083+0000 7feefe59c640 1 --2- 172.21.15.89:0/975117939 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7feee00656b0 0x7feee0067b70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7feef00ae000 tx=0x7feef00a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:50:57.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.087+0000 7feef77fe640 1 -- 172.21.15.89:0/975117939 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fef00159c40 con 0x7fef00150620 2023-10-18T10:50:57.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.095+0000 7f422745f640 1 Processor -- start 2023-10-18T10:50:57.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.095+0000 7f422745f640 1 -- start start 2023-10-18T10:50:57.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.099+0000 7f422745f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4220160f20 0x7f42200d0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:50:57.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.099+0000 7f422745f640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42200cced0 con 0x7f42200b15e0 2023-10-18T10:50:57.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.099+0000 7f422745f640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42200b2dc0 con 0x7f42200b33b0 2023-10-18T10:50:57.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.099+0000 7f422745f640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42200b1d30 con 0x7f4220160f20 2023-10-18T10:50:57.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.099+0000 7f42251fb640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4220160f20 0x7f42200d0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:50:57.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.099+0000 7f42251fb640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4220160f20 0x7f42200d0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38292/0 (socket says 172.21.15.89:38292) 2023-10-18T10:50:57.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.099+0000 7f42251fb640 1 -- 172.21.15.89:0/1402725254 learned_addr learned my addr 172.21.15.89:0/1402725254 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:50:57.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.099+0000 7f42259fc640 1 --1- 172.21.15.89:0/1402725254 >> v1:172.21.15.89:6789/0 conn(0x7f42200b15e0 0x7f42200b26b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:53804/0 (socket says 172.21.15.89:53804) 2023-10-18T10:50:57.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.099+0000 7f42251fb640 1 -- 172.21.15.89:0/1402725254 >> v1:172.21.15.154:6789/0 conn(0x7f42200b33b0 legacy=0x7f42200b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:50:57.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.099+0000 7f42251fb640 1 -- 172.21.15.89:0/1402725254 >> v1:172.21.15.89:6789/0 conn(0x7f42200b15e0 legacy=0x7f42200b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:50:57.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.099+0000 7f42251fb640 1 -- 172.21.15.89:0/1402725254 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42200b3bc0 con 0x7f4220160f20 2023-10-18T10:50:57.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.099+0000 7f4217fff640 1 -- 172.21.15.89:0/1402725254 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3047431854 0 0) 0x7f42200cced0 con 0x7f42200b15e0 2023-10-18T10:50:57.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.103+0000 7f42251fb640 1 --2- 172.21.15.89:0/1402725254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4220160f20 0x7f42200d0750 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f421c009870 tx=0x7f421c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d809ee5d89078 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:50:57.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.103+0000 7f4217fff640 1 -- 172.21.15.89:0/1402725254 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421c0ac620 con 0x7f4220160f20 2023-10-18T10:50:57.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.103+0000 7f4217fff640 1 -- 172.21.15.89:0/1402725254 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421c0ac780 con 0x7f4220160f20 2023-10-18T10:50:57.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.103+0000 7f4217fff640 1 -- 172.21.15.89:0/1402725254 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421c0acb10 con 0x7f4220160f20 2023-10-18T10:50:57.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.103+0000 7f422745f640 1 -- 172.21.15.89:0/1402725254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4220160f20 msgr2=0x7f42200d0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:50:57.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.103+0000 7f422745f640 1 --2- 172.21.15.89:0/1402725254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4220160f20 0x7f42200d0750 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f421c009870 tx=0x7f421c01c4d0 comp rx=0 tx=0).stop 2023-10-18T10:50:57.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.103+0000 7f422745f640 1 -- 172.21.15.89:0/1402725254 shutdown_connections 2023-10-18T10:50:57.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.103+0000 7f422745f640 1 --2- 172.21.15.89:0/1402725254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4220160f20 0x7f42200d0750 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:50:57.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.103+0000 7f422745f640 1 -- 172.21.15.89:0/1402725254 >> 172.21.15.89:0/1402725254 conn(0x7f422009b930 msgr2=0x7f42200d26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:50:57.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.103+0000 7f422745f640 1 -- 172.21.15.89:0/1402725254 shutdown_connections 2023-10-18T10:50:57.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.103+0000 7f422745f640 1 -- 172.21.15.89:0/1402725254 wait complete. 2023-10-18T10:50:57.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.103+0000 7f422745f640 1 Processor -- start 2023-10-18T10:50:57.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.103+0000 7f422745f640 1 -- start start 2023-10-18T10:50:57.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.107+0000 7f422745f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42200b15e0 0x7f4220153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:50:57.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.107+0000 7f422745f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42200b33b0 0x7f42200c8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:50:57.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.107+0000 7f422745f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4220160f20 0x7f42200c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:50:57.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.107+0000 7f422745f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42200b6a20 con 0x7f4220160f20 2023-10-18T10:50:57.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.107+0000 7f422745f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42200c79c0 con 0x7f42200b33b0 2023-10-18T10:50:57.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.107+0000 7f422745f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42200c6db0 con 0x7f42200b15e0 2023-10-18T10:50:57.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.107+0000 7f42259fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4220160f20 0x7f42200c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:50:57.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.107+0000 7f42259fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4220160f20 0x7f42200c8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38298/0 (socket says 172.21.15.89:38298) 2023-10-18T10:50:57.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.107+0000 7f42259fc640 1 -- 172.21.15.89:0/2631225717 learned_addr learned my addr 172.21.15.89:0/2631225717 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:50:57.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.107+0000 7f42249fa640 1 --2- 172.21.15.89:0/2631225717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42200b33b0 0x7f42200c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:50:57.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.107+0000 7f42251fb640 1 --2- 172.21.15.89:0/2631225717 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42200b15e0 0x7f4220153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:50:57.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.107+0000 7f42259fc640 1 -- 172.21.15.89:0/2631225717 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42200b15e0 msgr2=0x7f4220153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:50:57.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.107+0000 7f42259fc640 1 --2- 172.21.15.89:0/2631225717 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42200b15e0 0x7f4220153a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:50:57.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.107+0000 7f42259fc640 1 -- 172.21.15.89:0/2631225717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42200b33b0 msgr2=0x7f42200c8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:50:57.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.107+0000 7f42259fc640 1 --2- 172.21.15.89:0/2631225717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42200b33b0 0x7f42200c8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:50:57.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.107+0000 7f42259fc640 1 -- 172.21.15.89:0/2631225717 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f421c0094d0 con 0x7f4220160f20 2023-10-18T10:50:57.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.107+0000 7f42249fa640 1 --2- 172.21.15.89:0/2631225717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42200b33b0 0x7f42200c8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:50:57.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.107+0000 7f42251fb640 1 --2- 172.21.15.89:0/2631225717 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f42200b15e0 0x7f4220153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:50:57.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.107+0000 7f42259fc640 1 --2- 172.21.15.89:0/2631225717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4220160f20 0x7f42200c8a90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f421800c710 tx=0x7f421800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:50:57.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.107+0000 7f4215ffb640 1 -- 172.21.15.89:0/2631225717 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421800ce20 con 0x7f4220160f20 2023-10-18T10:50:57.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.107+0000 7f4215ffb640 1 -- 172.21.15.89:0/2631225717 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421800a640 con 0x7f4220160f20 2023-10-18T10:50:57.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.107+0000 7f422745f640 1 -- 172.21.15.89:0/2631225717 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42200c7090 con 0x7f4220160f20 2023-10-18T10:50:57.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.107+0000 7f4215ffb640 1 -- 172.21.15.89:0/2631225717 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421800aa30 con 0x7f4220160f20 2023-10-18T10:50:57.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.107+0000 7f422745f640 1 -- 172.21.15.89:0/2631225717 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42200c7650 con 0x7f4220160f20 2023-10-18T10:50:57.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.107+0000 7f422745f640 1 -- 172.21.15.89:0/2631225717 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f422004e430 con 0x7f4220160f20 2023-10-18T10:50:57.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.107+0000 7f4215ffb640 1 -- 172.21.15.89:0/2631225717 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f421800ab90 con 0x7f4220160f20 2023-10-18T10:50:57.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.111+0000 7f4215ffb640 1 --2- 172.21.15.89:0/2631225717 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4204065680 0x7f4204067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:50:57.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.111+0000 7f4215ffb640 1 -- 172.21.15.89:0/2631225717 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f4218082720 con 0x7f4220160f20 2023-10-18T10:50:57.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.111+0000 7f4215ffb640 1 -- 172.21.15.89:0/2631225717 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+447 (secure 0 0 0) 0x7f4218089070 con 0x7f4220160f20 2023-10-18T10:50:57.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.111+0000 7f42251fb640 1 --2- 172.21.15.89:0/2631225717 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4204065680 0x7f4204067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:50:57.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.111+0000 7f422745f640 1 -- 172.21.15.89:0/2631225717 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f4220089fd0 con 0x7f4220160f20 2023-10-18T10:50:57.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.111+0000 7f42251fb640 1 --2- 172.21.15.89:0/2631225717 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4204065680 0x7f4204067b40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f421c0ae000 tx=0x7f421c0a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:50:57.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.271+0000 7fef04c19640 1 -- 172.21.15.89:0/975117939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef00159c40 con 0x7fef00150620 2023-10-18T10:50:57.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.271+0000 7feef77fe640 1 -- 172.21.15.89:0/975117939 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4875 (secure 0 0 0) 0x7fef00159c40 con 0x7fef00150620 2023-10-18T10:50:57.274 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:50:57.275 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/654729641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":654729641},{"type":"v1","addr":"172.21.15.154:6836","nonce":654729641}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/964566480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":964566480},{"type":"v1","addr":"172.21.15.154:6837","nonce":964566480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2261878538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2261878538},{"type":"v1","addr":"172.21.15.89:6833","nonce":2261878538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:47:15.199575+0000","modified":"2023-10-18T10:47:18.846656+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5517":{"gid":5517,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6835/2092274731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2092274731},{"type":"v1","addr":"172.21.15.89:6835","nonce":2092274731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T10:50:57.276 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 18 2023-10-18T10:50:57.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.275+0000 7fef04c19640 1 -- 172.21.15.89:0/975117939 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7feee00656b0 msgr2=0x7feee0067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:50:57.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.275+0000 7fef04c19640 1 --2- 172.21.15.89:0/975117939 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7feee00656b0 0x7feee0067b70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7feef00ae000 tx=0x7feef00a7000 comp rx=0 tx=0).stop 2023-10-18T10:50:57.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.275+0000 7fef04c19640 1 -- 172.21.15.89:0/975117939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef00150620 msgr2=0x7fef0015aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:50:57.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.275+0000 7fef04c19640 1 --2- 172.21.15.89:0/975117939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef00150620 0x7fef0015aa60 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7feee800c4d0 tx=0x7feee800c990 comp rx=0 tx=0).stop 2023-10-18T10:50:57.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.275+0000 7fef04c19640 1 -- 172.21.15.89:0/975117939 shutdown_connections 2023-10-18T10:50:57.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.275+0000 7fef04c19640 1 --2- 172.21.15.89:0/975117939 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7feee00656b0 0x7feee0067b70 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:50:57.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.275+0000 7fef04c19640 1 --2- 172.21.15.89:0/975117939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef0015afa0 0x7fef00156750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:50:57.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.275+0000 7fef04c19640 1 --2- 172.21.15.89:0/975117939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef00150620 0x7fef0015aa60 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:50:57.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.275+0000 7fef04c19640 1 --2- 172.21.15.89:0/975117939 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fef000d0e70 0x7fef00159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:50:57.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.275+0000 7fef04c19640 1 -- 172.21.15.89:0/975117939 >> 172.21.15.89:0/975117939 conn(0x7fef00089fd0 msgr2=0x7fef0015fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:50:57.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.275+0000 7fef04c19640 1 -- 172.21.15.89:0/975117939 shutdown_connections 2023-10-18T10:50:57.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:50:57.275+0000 7fef04c19640 1 -- 172.21.15.89:0/975117939 wait complete. 2023-10-18T10:50:57.295 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-18T10:50:57.451 INFO:teuthology.orchestra.run.smithi089.stdout:[{"id":5183,"entity":{"name":{"type":"client","num":5183},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":4272238049}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":213.74104122899999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.61027435111846096,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":3,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5183 v1:192.168.0.1:0/4272238049","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi154","kernel_version":"6.6.0-rc5-g47672119149b","root":"/"}},{"id":5461,"entity":{"name":{"type":"client","num":5461},"addr":{"type":"v1","addr":"172.21.15.89:0","nonce":330049992}},"state":"stale","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":214.83701130599999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.60873657765691813,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":25,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5461 v1:172.21.15.89:0/330049992","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi089","kernel_version":"6.6.0-rc5-g47672119149b","root":"/"}}] 2023-10-18T10:50:57.452 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "1", "hostname": "smithi154", "kernel_version": "6.6.0-rc5-g47672119149b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 4272238049, "type": "v1" }, "name": { "num": 5183, "type": "client" } }, "id": 5183, "inst": "client.5183 v1:192.168.0.1:0/4272238049", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 3, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.610274351118461 }, "state": "open", "uptime": 213.741041229 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "0", "hostname": "smithi089", "kernel_version": "6.6.0-rc5-g47672119149b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.89:0", "nonce": 330049992, "type": "v1" }, "name": { "num": 5461, "type": "client" } }, "id": 5461, "inst": "client.5461 v1:172.21.15.89:0/330049992", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 25, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.6087365776569181 }, "state": "stale", "uptime": 214.837011306 } ] 2023-10-18T10:50:57.554 DEBUG:tasks.ceph_test_case:No log hits yet, waiting... 2023-10-18T10:51:19.385 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:51:19.390 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:51:49.474 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:51:49.478 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:51:57.572 ERROR:tasks.ceph_test_case:Log output: 2023-10-18T10:51:57.572 DEBUG:tasks.cephfs.test_client_recovery:Expected log message found: '1 client(s) laggy due to laggy OSDs', retrying 2023-10-18T10:52:02.574 DEBUG:teuthology.orchestra.run.smithi089:> sudo daemon-helper kill ceph --cluster ceph -w 2023-10-18T10:52:02.579 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:52:02.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.727+0000 7f5d1360b640 1 Processor -- start 2023-10-18T10:52:02.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.727+0000 7f5d1360b640 1 -- start start 2023-10-18T10:52:02.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.731+0000 7f5d1360b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5d0c15c940 0x7f5d0c0d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:52:02.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.731+0000 7f5d1360b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5d0c0d0e70 0x7f5d0c0cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:52:02.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.731+0000 7f5d1360b640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d0c15da50 con 0x7f5d0c0d2f40 2023-10-18T10:52:02.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.731+0000 7f5d1360b640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d0c15ce40 con 0x7f5d0c0d0e70 2023-10-18T10:52:02.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.731+0000 7f5d1360b640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d0c15d120 con 0x7f5d0c15c940 2023-10-18T10:52:02.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.731+0000 7f5d11ba8640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5d0c0d0e70 0x7f5d0c0cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:52:02.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.731+0000 7f5d11ba8640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5d0c0d0e70 0x7f5d0c0cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42872/0 (socket says 172.21.15.89:42872) 2023-10-18T10:52:02.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.731+0000 7f5d11ba8640 1 -- 172.21.15.89:0/2085156014 learned_addr learned my addr 172.21.15.89:0/2085156014 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:52:02.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.731+0000 7f5d113a7640 1 --2- 172.21.15.89:0/2085156014 >> v2:172.21.15.154:3300/0 conn(0x7f5d0c15c940 0x7f5d0c0d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:52:02.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.731+0000 7f5d11ba8640 1 -- 172.21.15.89:0/2085156014 >> v2:172.21.15.154:3300/0 conn(0x7f5d0c15c940 msgr2=0x7f5d0c0d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:52:02.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.731+0000 7f5d11ba8640 1 --2- 172.21.15.89:0/2085156014 >> v2:172.21.15.154:3300/0 conn(0x7f5d0c15c940 0x7f5d0c0d2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:52:02.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.731+0000 7f5d11ba8640 1 -- 172.21.15.89:0/2085156014 >> v1:172.21.15.154:6789/0 conn(0x7f5d0c0d2f40 legacy=0x7f5d0c0d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:52:02.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.731+0000 7f5d11ba8640 1 -- 172.21.15.89:0/2085156014 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0c15d400 con 0x7f5d0c0d0e70 2023-10-18T10:52:02.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.735+0000 7f5d11ba8640 1 --2- 172.21.15.89:0/2085156014 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d0c0d0e70 0x7f5d0c0cd160 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0725b0 tx=0x7f5d0401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d3ad868afc0b966 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:52:02.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.735+0000 7f5d03fff640 1 -- 172.21.15.89:0/2085156014 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d040a5800 con 0x7f5d0c0d0e70 2023-10-18T10:52:02.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.735+0000 7f5d03fff640 1 -- 172.21.15.89:0/2085156014 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d040a5960 con 0x7f5d0c0d0e70 2023-10-18T10:52:02.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.735+0000 7f5d03fff640 1 -- 172.21.15.89:0/2085156014 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d040a5cf0 con 0x7f5d0c0d0e70 2023-10-18T10:52:02.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.735+0000 7f5d1360b640 1 -- 172.21.15.89:0/2085156014 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d0c0d0e70 msgr2=0x7f5d0c0cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:52:02.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.735+0000 7f5d1360b640 1 --2- 172.21.15.89:0/2085156014 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d0c0d0e70 0x7f5d0c0cd160 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0725b0 tx=0x7f5d0401c9a0 comp rx=0 tx=0).stop 2023-10-18T10:52:02.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.735+0000 7f5d1360b640 1 -- 172.21.15.89:0/2085156014 shutdown_connections 2023-10-18T10:52:02.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.735+0000 7f5d1360b640 1 --2- 172.21.15.89:0/2085156014 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d0c0d0e70 0x7f5d0c0cd160 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:52:02.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.735+0000 7f5d1360b640 1 --2- 172.21.15.89:0/2085156014 >> v2:172.21.15.154:3300/0 conn(0x7f5d0c15c940 0x7f5d0c0d2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:52:02.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.735+0000 7f5d1360b640 1 -- 172.21.15.89:0/2085156014 >> 172.21.15.89:0/2085156014 conn(0x7f5d0c089fd0 msgr2=0x7f5d0c09b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:52:02.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.735+0000 7f5d1360b640 1 -- 172.21.15.89:0/2085156014 shutdown_connections 2023-10-18T10:52:02.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.735+0000 7f5d1360b640 1 -- 172.21.15.89:0/2085156014 wait complete. 2023-10-18T10:52:02.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.735+0000 7f5d1360b640 1 Processor -- start 2023-10-18T10:52:02.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.735+0000 7f5d1360b640 1 -- start start 2023-10-18T10:52:02.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.739+0000 7f5d1360b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d0c0d0e70 0x7f5d0c1563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:52:02.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.739+0000 7f5d1360b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d0c157760 0x7f5d0c0cf040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:52:02.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.739+0000 7f5d1360b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d0c0cf580 0x7f5d0c1f4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:52:02.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.739+0000 7f5d1360b640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c0b5850 con 0x7f5d0c0d0e70 2023-10-18T10:52:02.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.739+0000 7f5d1360b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c0cfa00 con 0x7f5d0c0cf580 2023-10-18T10:52:02.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.739+0000 7f5d1360b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c1572d0 con 0x7f5d0c157760 2023-10-18T10:52:02.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.739+0000 7f5d113a7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d0c0d0e70 0x7f5d0c1563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:52:02.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.739+0000 7f5d113a7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d0c0d0e70 0x7f5d0c1563d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42876/0 (socket says 172.21.15.89:42876) 2023-10-18T10:52:02.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.739+0000 7f5d113a7640 1 -- 172.21.15.89:0/1017311698 learned_addr learned my addr 172.21.15.89:0/1017311698 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:52:02.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.739+0000 7f5d11ba8640 1 --2- 172.21.15.89:0/1017311698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d0c0cf580 0x7f5d0c1f4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:52:02.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.739+0000 7f5d10ba6640 1 --2- 172.21.15.89:0/1017311698 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d0c157760 0x7f5d0c0cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:52:02.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.739+0000 7f5d113a7640 1 -- 172.21.15.89:0/1017311698 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d0c157760 msgr2=0x7f5d0c0cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:52:02.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.739+0000 7f5d113a7640 1 --2- 172.21.15.89:0/1017311698 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d0c157760 0x7f5d0c0cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:52:02.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.739+0000 7f5d113a7640 1 -- 172.21.15.89:0/1017311698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d0c0cf580 msgr2=0x7f5d0c1f4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:52:02.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.739+0000 7f5d113a7640 1 --2- 172.21.15.89:0/1017311698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d0c0cf580 0x7f5d0c1f4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:52:02.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.739+0000 7f5d113a7640 1 -- 172.21.15.89:0/1017311698 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d04009b30 con 0x7f5d0c0d0e70 2023-10-18T10:52:02.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.739+0000 7f5d10ba6640 1 --2- 172.21.15.89:0/1017311698 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d0c157760 0x7f5d0c0cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:52:02.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.739+0000 7f5d113a7640 1 --2- 172.21.15.89:0/1017311698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d0c0d0e70 0x7f5d0c1563d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800d620 tx=0x7f5d0800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:52:02.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.739+0000 7f5d01ffb640 1 -- 172.21.15.89:0/1017311698 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0800b620 con 0x7f5d0c0d0e70 2023-10-18T10:52:02.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.739+0000 7f5d01ffb640 1 -- 172.21.15.89:0/1017311698 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d0800b780 con 0x7f5d0c0d0e70 2023-10-18T10:52:02.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.739+0000 7f5d01ffb640 1 -- 172.21.15.89:0/1017311698 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0800bb10 con 0x7f5d0c0d0e70 2023-10-18T10:52:02.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.739+0000 7f5d1360b640 1 -- 172.21.15.89:0/1017311698 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d0c1f5d10 con 0x7f5d0c0d0e70 2023-10-18T10:52:02.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.739+0000 7f5d1360b640 1 -- 172.21.15.89:0/1017311698 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d0c156c80 con 0x7f5d0c0d0e70 2023-10-18T10:52:02.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.739+0000 7f5d1360b640 1 -- 172.21.15.89:0/1017311698 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d0c04dbe0 con 0x7f5d0c0d0e70 2023-10-18T10:52:02.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.739+0000 7f5d01ffb640 1 -- 172.21.15.89:0/1017311698 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f5d0800bc70 con 0x7f5d0c0d0e70 2023-10-18T10:52:02.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.743+0000 7f5d01ffb640 1 --2- 172.21.15.89:0/1017311698 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5cf0065680 0x7f5cf0067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:52:02.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.743+0000 7f5d01ffb640 1 -- 172.21.15.89:0/1017311698 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f5d08083370 con 0x7f5d0c0d0e70 2023-10-18T10:52:02.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.743+0000 7f5d10ba6640 1 --2- 172.21.15.89:0/1017311698 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5cf0065680 0x7f5cf0067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:52:02.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.743+0000 7f5d10ba6640 1 --2- 172.21.15.89:0/1017311698 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5cf0065680 0x7f5cf0067b40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0b7590 tx=0x7f5cfc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:52:02.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.747+0000 7f5d01ffb640 1 -- 172.21.15.89:0/1017311698 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f5d08050d40 con 0x7f5d0c0d0e70 2023-10-18T10:52:02.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.755+0000 7fc65352c640 1 Processor -- start 2023-10-18T10:52:02.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.755+0000 7fc65352c640 1 -- start start 2023-10-18T10:52:02.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.759+0000 7fc65352c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc64c160f20 0x7fc64c0d0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:52:02.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.759+0000 7fc65352c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc64c0b15e0 0x7fc64c0d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:52:02.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.759+0000 7fc65352c640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc64c0cced0 con 0x7fc64c0b33b0 2023-10-18T10:52:02.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.759+0000 7fc65352c640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc64c0b1d30 con 0x7fc64c0b15e0 2023-10-18T10:52:02.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.759+0000 7fc65352c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c0b3bc0 con 0x7fc64c160f20 2023-10-18T10:52:02.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.759+0000 7fc650ac7640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc64c0b33b0 0x7fc64c0b1fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48070/0 (socket says 172.21.15.89:48070) 2023-10-18T10:52:02.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.759+0000 7fc650ac7640 1 -- 172.21.15.89:0/1208643010 learned_addr learned my addr 172.21.15.89:0/1208643010 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:52:02.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.759+0000 7fc651ac9640 1 --2- 172.21.15.89:0/1208643010 >> v2:172.21.15.89:3300/0 conn(0x7fc64c0b15e0 0x7fc64c0d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:52:02.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.759+0000 7fc6512c8640 1 --2- 172.21.15.89:0/1208643010 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc64c160f20 0x7fc64c0d0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:52:02.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.759+0000 7fc643fff640 1 -- 172.21.15.89:0/1208643010 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3299258728 0 0) 0x7fc64c0cced0 con 0x7fc64c0b33b0 2023-10-18T10:52:02.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.759+0000 7fc643fff640 1 -- 172.21.15.89:0/1208643010 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc638003530 con 0x7fc64c0b33b0 2023-10-18T10:52:02.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.759+0000 7fc651ac9640 1 -- 172.21.15.89:0/1208643010 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc64c160f20 msgr2=0x7fc64c0d0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:52:02.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.759+0000 7fc651ac9640 1 --2- 172.21.15.89:0/1208643010 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc64c160f20 0x7fc64c0d0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:52:02.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.759+0000 7fc651ac9640 1 -- 172.21.15.89:0/1208643010 >> v1:172.21.15.89:6789/0 conn(0x7fc64c0b33b0 legacy=0x7fc64c0b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:52:02.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.759+0000 7fc651ac9640 1 -- 172.21.15.89:0/1208643010 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc64c0afdd0 con 0x7fc64c0b15e0 2023-10-18T10:52:02.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.763+0000 7fc651ac9640 1 --2- 172.21.15.89:0/1208643010 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc64c0b15e0 0x7fc64c0d0c90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc644009d90 tx=0x7fc64401c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=389e1b3ac5a3cd6e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:52:02.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.763+0000 7fc643fff640 1 -- 172.21.15.89:0/1208643010 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6440a5850 con 0x7fc64c0b15e0 2023-10-18T10:52:02.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.763+0000 7fc643fff640 1 -- 172.21.15.89:0/1208643010 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6440a59b0 con 0x7fc64c0b15e0 2023-10-18T10:52:02.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.763+0000 7fc643fff640 1 -- 172.21.15.89:0/1208643010 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6440a5d40 con 0x7fc64c0b15e0 2023-10-18T10:52:02.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.763+0000 7fc65352c640 1 -- 172.21.15.89:0/1208643010 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc64c0b15e0 msgr2=0x7fc64c0d0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:52:02.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.763+0000 7fc65352c640 1 --2- 172.21.15.89:0/1208643010 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc64c0b15e0 0x7fc64c0d0c90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc644009d90 tx=0x7fc64401c8c0 comp rx=0 tx=0).stop 2023-10-18T10:52:02.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.763+0000 7fc65352c640 1 -- 172.21.15.89:0/1208643010 shutdown_connections 2023-10-18T10:52:02.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.763+0000 7fc65352c640 1 --2- 172.21.15.89:0/1208643010 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc64c0b15e0 0x7fc64c0d0c90 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:52:02.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.763+0000 7fc65352c640 1 --2- 172.21.15.89:0/1208643010 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc64c160f20 0x7fc64c0d0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:52:02.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.763+0000 7fc65352c640 1 -- 172.21.15.89:0/1208643010 >> 172.21.15.89:0/1208643010 conn(0x7fc64c09b930 msgr2=0x7fc64c0d26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:52:02.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.763+0000 7fc65352c640 1 -- 172.21.15.89:0/1208643010 shutdown_connections 2023-10-18T10:52:02.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.763+0000 7fc65352c640 1 -- 172.21.15.89:0/1208643010 wait complete. 2023-10-18T10:52:02.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.767+0000 7fc65352c640 1 Processor -- start 2023-10-18T10:52:02.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.767+0000 7fc65352c640 1 -- start start 2023-10-18T10:52:02.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.767+0000 7fc65352c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc64c0b15e0 0x7fc64c0c8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:52:02.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.767+0000 7fc65352c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc64c0b33b0 0x7fc64c0b5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:52:02.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.767+0000 7fc65352c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc64c160f20 0x7fc64c157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:52:02.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.767+0000 7fc65352c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c14fde0 con 0x7fc64c160f20 2023-10-18T10:52:02.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.767+0000 7fc65352c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c15eec0 con 0x7fc64c0b33b0 2023-10-18T10:52:02.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.767+0000 7fc65352c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c15f1a0 con 0x7fc64c0b15e0 2023-10-18T10:52:02.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.767+0000 7fc651ac9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc64c160f20 0x7fc64c157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:52:02.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.767+0000 7fc651ac9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc64c160f20 0x7fc64c157160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42906/0 (socket says 172.21.15.89:42906) 2023-10-18T10:52:02.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.767+0000 7fc651ac9640 1 -- 172.21.15.89:0/454833081 learned_addr learned my addr 172.21.15.89:0/454833081 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:52:02.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.767+0000 7fc650ac7640 1 --2- 172.21.15.89:0/454833081 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc64c0b33b0 0x7fc64c0b5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:52:02.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.767+0000 7fc6512c8640 1 --2- 172.21.15.89:0/454833081 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc64c0b15e0 0x7fc64c0c8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:52:02.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.767+0000 7fc651ac9640 1 -- 172.21.15.89:0/454833081 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc64c0b15e0 msgr2=0x7fc64c0c8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:52:02.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.767+0000 7fc651ac9640 1 --2- 172.21.15.89:0/454833081 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc64c0b15e0 0x7fc64c0c8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:52:02.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.767+0000 7fc651ac9640 1 -- 172.21.15.89:0/454833081 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc64c0b33b0 msgr2=0x7fc64c0b5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:52:02.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.767+0000 7fc651ac9640 1 --2- 172.21.15.89:0/454833081 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc64c0b33b0 0x7fc64c0b5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:52:02.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.767+0000 7fc651ac9640 1 -- 172.21.15.89:0/454833081 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc644009a70 con 0x7fc64c160f20 2023-10-18T10:52:02.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.767+0000 7fc6512c8640 1 --2- 172.21.15.89:0/454833081 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc64c0b15e0 0x7fc64c0c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:52:02.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.767+0000 7fc651ac9640 1 --2- 172.21.15.89:0/454833081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc64c160f20 0x7fc64c157160 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc6440b0040 tx=0x7fc644019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:52:02.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.767+0000 7fc641ffb640 1 -- 172.21.15.89:0/454833081 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6440ad640 con 0x7fc64c160f20 2023-10-18T10:52:02.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.767+0000 7fc641ffb640 1 -- 172.21.15.89:0/454833081 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6440ad7a0 con 0x7fc64c160f20 2023-10-18T10:52:02.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.767+0000 7fc641ffb640 1 -- 172.21.15.89:0/454833081 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6440adb90 con 0x7fc64c160f20 2023-10-18T10:52:02.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.767+0000 7fc65352c640 1 -- 172.21.15.89:0/454833081 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc64c15f480 con 0x7fc64c160f20 2023-10-18T10:52:02.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.771+0000 7fc65352c640 1 -- 172.21.15.89:0/454833081 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc64c0b51e0 con 0x7fc64c160f20 2023-10-18T10:52:02.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.771+0000 7fc62f7fe640 1 -- 172.21.15.89:0/454833081 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fc628005240 con 0x7fc64c160f20 2023-10-18T10:52:02.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.771+0000 7fc641ffb640 1 -- 172.21.15.89:0/454833081 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fc6440b4050 con 0x7fc64c160f20 2023-10-18T10:52:02.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.771+0000 7fc641ffb640 1 --2- 172.21.15.89:0/454833081 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc630065680 0x7fc630067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:52:02.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.771+0000 7fc641ffb640 1 -- 172.21.15.89:0/454833081 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fc64411e230 con 0x7fc64c160f20 2023-10-18T10:52:02.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.771+0000 7fc641ffb640 1 -- 172.21.15.89:0/454833081 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+446 (secure 0 0 0) 0x7fc64411e5c0 con 0x7fc64c160f20 2023-10-18T10:52:02.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.771+0000 7fc6512c8640 1 --2- 172.21.15.89:0/454833081 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc630065680 0x7fc630067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:52:02.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.771+0000 7fc62f7fe640 1 -- 172.21.15.89:0/454833081 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7fc628002340 con 0x7fc64c160f20 2023-10-18T10:52:02.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.775+0000 7fc6512c8640 1 --2- 172.21.15.89:0/454833081 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc630065680 0x7fc630067b40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc648009a40 tx=0x7fc648008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:52:02.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.935+0000 7f5d1360b640 1 -- 172.21.15.89:0/1017311698 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d0c0b85d0 con 0x7f5d0c0d0e70 2023-10-18T10:52:02.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.939+0000 7f5d01ffb640 1 -- 172.21.15.89:0/1017311698 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4875 (secure 0 0 0) 0x7f5d08054360 con 0x7f5d0c0d0e70 2023-10-18T10:52:02.938 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:52:02.938 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/654729641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":654729641},{"type":"v1","addr":"172.21.15.154:6836","nonce":654729641}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/964566480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":964566480},{"type":"v1","addr":"172.21.15.154:6837","nonce":964566480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2261878538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2261878538},{"type":"v1","addr":"172.21.15.89:6833","nonce":2261878538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:47:15.199575+0000","modified":"2023-10-18T10:47:18.846656+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5517":{"gid":5517,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6835/2092274731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2092274731},{"type":"v1","addr":"172.21.15.89:6835","nonce":2092274731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T10:52:02.939 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 18 2023-10-18T10:52:02.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.939+0000 7f5d1360b640 1 -- 172.21.15.89:0/1017311698 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5cf0065680 msgr2=0x7f5cf0067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:52:02.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.939+0000 7f5d1360b640 1 --2- 172.21.15.89:0/1017311698 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5cf0065680 0x7f5cf0067b40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0b7590 tx=0x7f5cfc009040 comp rx=0 tx=0).stop 2023-10-18T10:52:02.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.939+0000 7f5d1360b640 1 -- 172.21.15.89:0/1017311698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d0c0d0e70 msgr2=0x7f5d0c1563d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:52:02.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.939+0000 7f5d1360b640 1 --2- 172.21.15.89:0/1017311698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d0c0d0e70 0x7f5d0c1563d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800d620 tx=0x7f5d0800dae0 comp rx=0 tx=0).stop 2023-10-18T10:52:02.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.939+0000 7f5d1360b640 1 -- 172.21.15.89:0/1017311698 shutdown_connections 2023-10-18T10:52:02.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.939+0000 7f5d1360b640 1 --2- 172.21.15.89:0/1017311698 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5cf0065680 0x7f5cf0067b40 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:52:02.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.939+0000 7f5d1360b640 1 --2- 172.21.15.89:0/1017311698 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d0c157760 0x7f5d0c0cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:52:02.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.939+0000 7f5d1360b640 1 --2- 172.21.15.89:0/1017311698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d0c0cf580 0x7f5d0c1f4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:52:02.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.939+0000 7f5d1360b640 1 --2- 172.21.15.89:0/1017311698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d0c0d0e70 0x7f5d0c1563d0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:52:02.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.939+0000 7f5d1360b640 1 -- 172.21.15.89:0/1017311698 >> 172.21.15.89:0/1017311698 conn(0x7f5d0c089fd0 msgr2=0x7f5d0c15e970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:52:02.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.939+0000 7f5d1360b640 1 -- 172.21.15.89:0/1017311698 shutdown_connections 2023-10-18T10:52:02.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:52:02.939+0000 7f5d1360b640 1 -- 172.21.15.89:0/1017311698 wait complete. 2023-10-18T10:52:02.958 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-18T10:52:03.117 INFO:teuthology.orchestra.run.smithi089.stdout:[{"id":5183,"entity":{"name":{"type":"client","num":5183},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":4272238049}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":279.40837643700002,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.52436334532892726,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":3,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5183 v1:192.168.0.1:0/4272238049","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi154","kernel_version":"6.6.0-rc5-g47672119149b","root":"/"}},{"id":5461,"entity":{"name":{"type":"client","num":5461},"addr":{"type":"v1","addr":"172.21.15.89:0","nonce":330049992}},"state":"stale","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":280.50434651400002,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.52304205100421064,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":25,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5461 v1:172.21.15.89:0/330049992","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi089","kernel_version":"6.6.0-rc5-g47672119149b","root":"/"}}] 2023-10-18T10:52:03.118 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "1", "hostname": "smithi154", "kernel_version": "6.6.0-rc5-g47672119149b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 4272238049, "type": "v1" }, "name": { "num": 5183, "type": "client" } }, "id": 5183, "inst": "client.5183 v1:192.168.0.1:0/4272238049", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 3, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.5243633453289273 }, "state": "open", "uptime": 279.408376437 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "0", "hostname": "smithi089", "kernel_version": "6.6.0-rc5-g47672119149b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.89:0", "nonce": 330049992, "type": "v1" }, "name": { "num": 5461, "type": "client" } }, "id": 5461, "inst": "client.5461 v1:172.21.15.89:0/330049992", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 25, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.5230420510042106 }, "state": "stale", "uptime": 280.504346514 } ] 2023-10-18T10:52:03.220 DEBUG:tasks.ceph_test_case:No log hits yet, waiting... 2023-10-18T10:52:19.611 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:52:19.615 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:52:49.808 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:52:49.814 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:53:03.238 ERROR:tasks.ceph_test_case:Log output: 2023-10-18T10:53:03.238 DEBUG:tasks.cephfs.test_client_recovery:Expected log message found: '1 client(s) laggy due to laggy OSDs', retrying 2023-10-18T10:53:08.240 DEBUG:teuthology.orchestra.run.smithi089:> sudo daemon-helper kill ceph --cluster ceph -w 2023-10-18T10:53:08.244 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:53:08.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.394+0000 7f070c974640 1 Processor -- start 2023-10-18T10:53:08.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.394+0000 7f070c974640 1 -- start start 2023-10-18T10:53:08.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.394+0000 7f070c974640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f070806ff00 0x7f070815e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:53:08.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.394+0000 7f070c974640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07080ca340 0x7f070815b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:53:08.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.394+0000 7f070c974640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f070815b680 0x7f07080c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:53:08.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.394+0000 7f070c974640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f070815ef30 con 0x7f070815b680 2023-10-18T10:53:08.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.394+0000 7f070c974640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f070815aa10 con 0x7f070806ff00 2023-10-18T10:53:08.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.394+0000 7f070c974640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f070815acb0 con 0x7f07080ca340 2023-10-18T10:53:08.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.394+0000 7f0706d9d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f070815b680 0x7f07080c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:53:08.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.394+0000 7f0706d9d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f070815b680 0x7f07080c8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43424/0 (socket says 172.21.15.89:43424) 2023-10-18T10:53:08.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.394+0000 7f0706d9d640 1 -- 172.21.15.89:0/3496631669 learned_addr learned my addr 172.21.15.89:0/3496631669 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:53:08.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.394+0000 7f06fffff640 1 --2- 172.21.15.89:0/3496631669 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07080ca340 0x7f070815b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:53:08.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.394+0000 7f070659c640 1 --2- 172.21.15.89:0/3496631669 >> v2:172.21.15.154:3300/0 conn(0x7f070806ff00 0x7f070815e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:53:08.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.394+0000 7f0706d9d640 1 -- 172.21.15.89:0/3496631669 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07080ca340 msgr2=0x7f070815b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:53:08.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.394+0000 7f0706d9d640 1 --2- 172.21.15.89:0/3496631669 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07080ca340 0x7f070815b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:53:08.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.394+0000 7f0706d9d640 1 -- 172.21.15.89:0/3496631669 >> v2:172.21.15.154:3300/0 conn(0x7f070806ff00 msgr2=0x7f070815e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:53:08.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.394+0000 7f0706d9d640 1 --2- 172.21.15.89:0/3496631669 >> v2:172.21.15.154:3300/0 conn(0x7f070806ff00 0x7f070815e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:53:08.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.394+0000 7f0706d9d640 1 -- 172.21.15.89:0/3496631669 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07080702e0 con 0x7f070815b680 2023-10-18T10:53:08.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.394+0000 7f070659c640 1 --2- 172.21.15.89:0/3496631669 >> v2:172.21.15.154:3300/0 conn(0x7f070806ff00 0x7f070815e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:53:08.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.394+0000 7f06fffff640 1 --2- 172.21.15.89:0/3496631669 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07080ca340 0x7f070815b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:53:08.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.398+0000 7f0706d9d640 1 --2- 172.21.15.89:0/3496631669 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f070815b680 0x7f07080c8960 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f070806a830 tx=0x7f070001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27f254488d5872c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:53:08.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.398+0000 7f0705d9b640 1 -- 172.21.15.89:0/3496631669 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07000a56a0 con 0x7f070815b680 2023-10-18T10:53:08.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.398+0000 7f0705d9b640 1 -- 172.21.15.89:0/3496631669 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07000a5800 con 0x7f070815b680 2023-10-18T10:53:08.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.398+0000 7f0705d9b640 1 -- 172.21.15.89:0/3496631669 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07000a5b90 con 0x7f070815b680 2023-10-18T10:53:08.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.398+0000 7f070c974640 1 -- 172.21.15.89:0/3496631669 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f070815b680 msgr2=0x7f07080c8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:53:08.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.398+0000 7f070c974640 1 --2- 172.21.15.89:0/3496631669 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f070815b680 0x7f07080c8960 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f070806a830 tx=0x7f070001c910 comp rx=0 tx=0).stop 2023-10-18T10:53:08.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.398+0000 7f070c974640 1 -- 172.21.15.89:0/3496631669 shutdown_connections 2023-10-18T10:53:08.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.398+0000 7f070c974640 1 --2- 172.21.15.89:0/3496631669 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f070815b680 0x7f07080c8960 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:53:08.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.398+0000 7f070c974640 1 --2- 172.21.15.89:0/3496631669 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07080ca340 0x7f070815b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:53:08.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.398+0000 7f070c974640 1 --2- 172.21.15.89:0/3496631669 >> v2:172.21.15.154:3300/0 conn(0x7f070806ff00 0x7f070815e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:53:08.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.398+0000 7f070c974640 1 -- 172.21.15.89:0/3496631669 >> 172.21.15.89:0/3496631669 conn(0x7f0708069f00 msgr2=0x7f07080cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:53:08.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.398+0000 7f070c974640 1 -- 172.21.15.89:0/3496631669 shutdown_connections 2023-10-18T10:53:08.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.398+0000 7f070c974640 1 -- 172.21.15.89:0/3496631669 wait complete. 2023-10-18T10:53:08.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.402+0000 7f070c974640 1 Processor -- start 2023-10-18T10:53:08.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.402+0000 7f070c974640 1 -- start start 2023-10-18T10:53:08.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.402+0000 7f070c974640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f070806ff00 0x7f070814c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:53:08.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.402+0000 7f070c974640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07080ca340 0x7f070814d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:53:08.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.402+0000 7f070c974640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07080b1c60 0x7f070814db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:53:08.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.402+0000 7f070c974640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070814cb80 con 0x7f07080b1c60 2023-10-18T10:53:08.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.402+0000 7f070c974640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07080b2120 con 0x7f07080ca340 2023-10-18T10:53:08.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.402+0000 7f070c974640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07080b2400 con 0x7f070806ff00 2023-10-18T10:53:08.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.402+0000 7f0706d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07080b1c60 0x7f070814db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:53:08.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.402+0000 7f0706d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07080b1c60 0x7f070814db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43426/0 (socket says 172.21.15.89:43426) 2023-10-18T10:53:08.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.402+0000 7f0706d9d640 1 -- 172.21.15.89:0/683908787 learned_addr learned my addr 172.21.15.89:0/683908787 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:53:08.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.402+0000 7f06fffff640 1 --2- 172.21.15.89:0/683908787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07080ca340 0x7f070814d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:53:08.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.402+0000 7f070659c640 1 --2- 172.21.15.89:0/683908787 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f070806ff00 0x7f070814c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:53:08.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.402+0000 7f0706d9d640 1 -- 172.21.15.89:0/683908787 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f070806ff00 msgr2=0x7f070814c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:53:08.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.402+0000 7f0706d9d640 1 --2- 172.21.15.89:0/683908787 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f070806ff00 0x7f070814c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:53:08.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.402+0000 7f0706d9d640 1 -- 172.21.15.89:0/683908787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07080ca340 msgr2=0x7f070814d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:53:08.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.402+0000 7f0706d9d640 1 --2- 172.21.15.89:0/683908787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07080ca340 0x7f070814d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:53:08.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.402+0000 7f0706d9d640 1 -- 172.21.15.89:0/683908787 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0700009b30 con 0x7f07080b1c60 2023-10-18T10:53:08.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.402+0000 7f06fffff640 1 --2- 172.21.15.89:0/683908787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07080ca340 0x7f070814d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:53:08.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.402+0000 7f0706d9d640 1 --2- 172.21.15.89:0/683908787 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07080b1c60 0x7f070814db90 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f07000af040 tx=0x7f07000196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:53:08.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.402+0000 7f06feffd640 1 -- 172.21.15.89:0/683908787 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07000b8670 con 0x7f07080b1c60 2023-10-18T10:53:08.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.402+0000 7f06feffd640 1 -- 172.21.15.89:0/683908787 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07000b87f0 con 0x7f07080b1c60 2023-10-18T10:53:08.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.402+0000 7f070c974640 1 -- 172.21.15.89:0/683908787 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07080b26e0 con 0x7f07080b1c60 2023-10-18T10:53:08.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.402+0000 7f06feffd640 1 -- 172.21.15.89:0/683908787 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07000b8ba0 con 0x7f07080b1c60 2023-10-18T10:53:08.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.402+0000 7f070c974640 1 -- 172.21.15.89:0/683908787 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07080b2ca0 con 0x7f07080b1c60 2023-10-18T10:53:08.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.406+0000 7f070c974640 1 -- 172.21.15.89:0/683908787 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f070804dbe0 con 0x7f07080b1c60 2023-10-18T10:53:08.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.406+0000 7f06feffd640 1 -- 172.21.15.89:0/683908787 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f07000b8d00 con 0x7f07080b1c60 2023-10-18T10:53:08.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.406+0000 7f06feffd640 1 --2- 172.21.15.89:0/683908787 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f06e8065680 0x7f06e8067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:53:08.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.406+0000 7f06feffd640 1 -- 172.21.15.89:0/683908787 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f070011e560 con 0x7f07080b1c60 2023-10-18T10:53:08.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.406+0000 7f070659c640 1 --2- 172.21.15.89:0/683908787 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f06e8065680 0x7f06e8067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:53:08.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.406+0000 7f070659c640 1 --2- 172.21.15.89:0/683908787 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f06e8065680 0x7f06e8067b40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0708154f70 tx=0x7f06f8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:53:08.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.410+0000 7f06feffd640 1 -- 172.21.15.89:0/683908787 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f07000ebec0 con 0x7f07080b1c60 2023-10-18T10:53:08.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.422+0000 7f5c44f8f640 1 Processor -- start 2023-10-18T10:53:08.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.422+0000 7f5c44f8f640 1 -- start start 2023-10-18T10:53:08.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.422+0000 7f5c44f8f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c400ce9a0 0x7f5c40069b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:53:08.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.422+0000 7f5c44f8f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5c400b25f0 0x7f5c4015cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:53:08.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.422+0000 7f5c44f8f640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c400cdca0 con 0x7f5c4006b2c0 2023-10-18T10:53:08.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.422+0000 7f5c44f8f640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c40160540 con 0x7f5c400b25f0 2023-10-18T10:53:08.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.422+0000 7f5c44f8f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c40160820 con 0x7f5c400ce9a0 2023-10-18T10:53:08.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.422+0000 7f5c3dd9b640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5c4006b2c0 0x7f5c400b1ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58572/0 (socket says 172.21.15.89:58572) 2023-10-18T10:53:08.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.422+0000 7f5c3dd9b640 1 -- 172.21.15.89:0/3094327738 learned_addr learned my addr 172.21.15.89:0/3094327738 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:53:08.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.422+0000 7f5c3ed9d640 1 --2- 172.21.15.89:0/3094327738 >> v2:172.21.15.154:3300/0 conn(0x7f5c400b25f0 0x7f5c4015cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:53:08.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.422+0000 7f5c3e59c640 1 --2- 172.21.15.89:0/3094327738 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c400ce9a0 0x7f5c40069b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:53:08.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.422+0000 7f5c3d59a640 1 -- 172.21.15.89:0/3094327738 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1091795697 0 0) 0x7f5c400cdca0 con 0x7f5c4006b2c0 2023-10-18T10:53:08.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.422+0000 7f5c3d59a640 1 -- 172.21.15.89:0/3094327738 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c28003530 con 0x7f5c4006b2c0 2023-10-18T10:53:08.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.422+0000 7f5c3e59c640 1 -- 172.21.15.89:0/3094327738 >> v2:172.21.15.154:3300/0 conn(0x7f5c400b25f0 msgr2=0x7f5c4015cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:53:08.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.422+0000 7f5c3e59c640 1 --2- 172.21.15.89:0/3094327738 >> v2:172.21.15.154:3300/0 conn(0x7f5c400b25f0 0x7f5c4015cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:53:08.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.422+0000 7f5c3e59c640 1 -- 172.21.15.89:0/3094327738 >> v1:172.21.15.89:6789/0 conn(0x7f5c4006b2c0 legacy=0x7f5c400b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:53:08.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.422+0000 7f5c3d59a640 1 -- 172.21.15.89:0/3094327738 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2039588288 0 0) 0x7f5c28003530 con 0x7f5c4006b2c0 2023-10-18T10:53:08.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.422+0000 7f5c3e59c640 1 -- 172.21.15.89:0/3094327738 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c40160b00 con 0x7f5c400ce9a0 2023-10-18T10:53:08.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.426+0000 7f5c3e59c640 1 --2- 172.21.15.89:0/3094327738 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c400ce9a0 0x7f5c40069b10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5c300097c0 tx=0x7f5c3001c2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f161941cb50749e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:53:08.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.426+0000 7f5c3d59a640 1 -- 172.21.15.89:0/3094327738 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c34001680 con 0x7f5c400ce9a0 2023-10-18T10:53:08.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.426+0000 7f5c3d59a640 1 -- 172.21.15.89:0/3094327738 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c300a7050 con 0x7f5c400ce9a0 2023-10-18T10:53:08.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.426+0000 7f5c3d59a640 1 -- 172.21.15.89:0/3094327738 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c300ab620 con 0x7f5c400ce9a0 2023-10-18T10:53:08.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.426+0000 7f5c44f8f640 1 -- 172.21.15.89:0/3094327738 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c400ce9a0 msgr2=0x7f5c40069b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:53:08.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.426+0000 7f5c44f8f640 1 --2- 172.21.15.89:0/3094327738 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c400ce9a0 0x7f5c40069b10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5c300097c0 tx=0x7f5c3001c2e0 comp rx=0 tx=0).stop 2023-10-18T10:53:08.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.426+0000 7f5c44f8f640 1 -- 172.21.15.89:0/3094327738 shutdown_connections 2023-10-18T10:53:08.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.426+0000 7f5c44f8f640 1 --2- 172.21.15.89:0/3094327738 >> v2:172.21.15.154:3300/0 conn(0x7f5c400b25f0 0x7f5c4015cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:53:08.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.426+0000 7f5c44f8f640 1 --2- 172.21.15.89:0/3094327738 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c400ce9a0 0x7f5c40069b10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:53:08.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.426+0000 7f5c44f8f640 1 -- 172.21.15.89:0/3094327738 >> 172.21.15.89:0/3094327738 conn(0x7f5c400ccee0 msgr2=0x7f5c400cd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:53:08.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.430+0000 7f5c44f8f640 1 -- 172.21.15.89:0/3094327738 shutdown_connections 2023-10-18T10:53:08.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.430+0000 7f5c44f8f640 1 -- 172.21.15.89:0/3094327738 wait complete. 2023-10-18T10:53:08.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.430+0000 7f5c44f8f640 1 Processor -- start 2023-10-18T10:53:08.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.430+0000 7f5c44f8f640 1 -- start start 2023-10-18T10:53:08.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.430+0000 7f5c44f8f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c4006b2c0 0x7f5c40150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:53:08.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.430+0000 7f5c44f8f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c400b25f0 0x7f5c4015bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:53:08.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.430+0000 7f5c44f8f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c400ce9a0 0x7f5c4015c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:53:08.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.430+0000 7f5c44f8f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c400b3440 con 0x7f5c4006b2c0 2023-10-18T10:53:08.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.430+0000 7f5c44f8f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c400b3720 con 0x7f5c400b25f0 2023-10-18T10:53:08.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.430+0000 7f5c44f8f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c400b3a00 con 0x7f5c400ce9a0 2023-10-18T10:53:08.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.430+0000 7f5c3e59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c4006b2c0 0x7f5c40150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:53:08.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.430+0000 7f5c3e59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c4006b2c0 0x7f5c40150010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43440/0 (socket says 172.21.15.89:43440) 2023-10-18T10:53:08.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.430+0000 7f5c3e59c640 1 -- 172.21.15.89:0/1275135590 learned_addr learned my addr 172.21.15.89:0/1275135590 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:53:08.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.430+0000 7f5c3dd9b640 1 --2- 172.21.15.89:0/1275135590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c400b25f0 0x7f5c4015bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:53:08.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.430+0000 7f5c3ed9d640 1 --2- 172.21.15.89:0/1275135590 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c400ce9a0 0x7f5c4015c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:53:08.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.430+0000 7f5c3e59c640 1 -- 172.21.15.89:0/1275135590 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c400ce9a0 msgr2=0x7f5c4015c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:53:08.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.430+0000 7f5c3e59c640 1 --2- 172.21.15.89:0/1275135590 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c400ce9a0 0x7f5c4015c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:53:08.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.430+0000 7f5c3e59c640 1 -- 172.21.15.89:0/1275135590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c400b25f0 msgr2=0x7f5c4015bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:53:08.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.430+0000 7f5c3e59c640 1 --2- 172.21.15.89:0/1275135590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c400b25f0 0x7f5c4015bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:53:08.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.430+0000 7f5c3e59c640 1 -- 172.21.15.89:0/1275135590 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c30009420 con 0x7f5c4006b2c0 2023-10-18T10:53:08.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.430+0000 7f5c3e59c640 1 --2- 172.21.15.89:0/1275135590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c4006b2c0 0x7f5c40150010 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5c3001a990 tx=0x7f5c30019080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:53:08.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.434+0000 7f5c2effd640 1 -- 172.21.15.89:0/1275135590 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c300b4400 con 0x7f5c4006b2c0 2023-10-18T10:53:08.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.434+0000 7f5c2effd640 1 -- 172.21.15.89:0/1275135590 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c300b4580 con 0x7f5c4006b2c0 2023-10-18T10:53:08.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.434+0000 7f5c2effd640 1 -- 172.21.15.89:0/1275135590 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c300b4930 con 0x7f5c4006b2c0 2023-10-18T10:53:08.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.434+0000 7f5c44f8f640 1 -- 172.21.15.89:0/1275135590 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c400b3ce0 con 0x7f5c4006b2c0 2023-10-18T10:53:08.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.434+0000 7f5c44f8f640 1 -- 172.21.15.89:0/1275135590 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c40150550 con 0x7f5c4006b2c0 2023-10-18T10:53:08.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.434+0000 7f5c44f8f640 1 -- 172.21.15.89:0/1275135590 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f5c4004e430 con 0x7f5c4006b2c0 2023-10-18T10:53:08.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.434+0000 7f5c2effd640 1 -- 172.21.15.89:0/1275135590 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f5c300b2040 con 0x7f5c4006b2c0 2023-10-18T10:53:08.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.434+0000 7f5c2effd640 1 --2- 172.21.15.89:0/1275135590 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5c20065680 0x7f5c20067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:53:08.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.434+0000 7f5c2effd640 1 -- 172.21.15.89:0/1275135590 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f5c3011d0b0 con 0x7f5c4006b2c0 2023-10-18T10:53:08.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.434+0000 7f5c2effd640 1 -- 172.21.15.89:0/1275135590 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+446 (secure 0 0 0) 0x7f5c30124070 con 0x7f5c4006b2c0 2023-10-18T10:53:08.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.434+0000 7f5c3dd9b640 1 --2- 172.21.15.89:0/1275135590 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5c20065680 0x7f5c20067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:53:08.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.434+0000 7f5c44f8f640 1 -- 172.21.15.89:0/1275135590 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f5c400b63e0 con 0x7f5c4006b2c0 2023-10-18T10:53:08.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.438+0000 7f5c3dd9b640 1 --2- 172.21.15.89:0/1275135590 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5c20065680 0x7f5c20067b40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5c401f8e50 tx=0x7f5c34009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:53:08.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.590+0000 7f070c974640 1 -- 172.21.15.89:0/683908787 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07080b2e90 con 0x7f07080b1c60 2023-10-18T10:53:08.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.590+0000 7f06feffd640 1 -- 172.21.15.89:0/683908787 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4875 (secure 0 0 0) 0x7f07000ef4e0 con 0x7f07080b1c60 2023-10-18T10:53:08.591 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:53:08.592 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/654729641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":654729641},{"type":"v1","addr":"172.21.15.154:6836","nonce":654729641}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/964566480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":964566480},{"type":"v1","addr":"172.21.15.154:6837","nonce":964566480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2261878538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2261878538},{"type":"v1","addr":"172.21.15.89:6833","nonce":2261878538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:47:15.199575+0000","modified":"2023-10-18T10:47:18.846656+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5517":{"gid":5517,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6835/2092274731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2092274731},{"type":"v1","addr":"172.21.15.89:6835","nonce":2092274731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T10:53:08.593 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 18 2023-10-18T10:53:08.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.594+0000 7f070c974640 1 -- 172.21.15.89:0/683908787 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f06e8065680 msgr2=0x7f06e8067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:53:08.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.594+0000 7f070c974640 1 --2- 172.21.15.89:0/683908787 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f06e8065680 0x7f06e8067b40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0708154f70 tx=0x7f06f8008040 comp rx=0 tx=0).stop 2023-10-18T10:53:08.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.594+0000 7f070c974640 1 -- 172.21.15.89:0/683908787 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07080b1c60 msgr2=0x7f070814db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:53:08.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.594+0000 7f070c974640 1 --2- 172.21.15.89:0/683908787 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07080b1c60 0x7f070814db90 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f07000af040 tx=0x7f07000196b0 comp rx=0 tx=0).stop 2023-10-18T10:53:08.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.594+0000 7f070c974640 1 -- 172.21.15.89:0/683908787 shutdown_connections 2023-10-18T10:53:08.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.594+0000 7f070c974640 1 --2- 172.21.15.89:0/683908787 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f06e8065680 0x7f06e8067b40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:53:08.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.594+0000 7f070c974640 1 --2- 172.21.15.89:0/683908787 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07080b1c60 0x7f070814db90 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:53:08.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.594+0000 7f070c974640 1 --2- 172.21.15.89:0/683908787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07080ca340 0x7f070814d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:53:08.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.594+0000 7f070c974640 1 --2- 172.21.15.89:0/683908787 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f070806ff00 0x7f070814c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:53:08.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.594+0000 7f070c974640 1 -- 172.21.15.89:0/683908787 >> 172.21.15.89:0/683908787 conn(0x7f0708069f00 msgr2=0x7f070806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:53:08.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.594+0000 7f070c974640 1 -- 172.21.15.89:0/683908787 shutdown_connections 2023-10-18T10:53:08.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:53:08.594+0000 7f070c974640 1 -- 172.21.15.89:0/683908787 wait complete. 2023-10-18T10:53:08.612 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-18T10:53:08.769 INFO:teuthology.orchestra.run.smithi089.stdout:[{"id":5183,"entity":{"name":{"type":"client","num":5183},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":4272238049}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":345.05972690300001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.45056305001339858,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":3,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5183 v1:192.168.0.1:0/4272238049","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi154","kernel_version":"6.6.0-rc5-g47672119149b","root":"/"}},{"id":5461,"entity":{"name":{"type":"client","num":5461},"addr":{"type":"v1","addr":"172.21.15.89:0","nonce":330049992}},"state":"stale","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":346.15569698000002,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.44942771817486898,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":25,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5461 v1:172.21.15.89:0/330049992","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi089","kernel_version":"6.6.0-rc5-g47672119149b","root":"/"}}] 2023-10-18T10:53:08.770 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "1", "hostname": "smithi154", "kernel_version": "6.6.0-rc5-g47672119149b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 4272238049, "type": "v1" }, "name": { "num": 5183, "type": "client" } }, "id": 5183, "inst": "client.5183 v1:192.168.0.1:0/4272238049", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 3, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.4505630500133986 }, "state": "open", "uptime": 345.059726903 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "0", "hostname": "smithi089", "kernel_version": "6.6.0-rc5-g47672119149b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.89:0", "nonce": 330049992, "type": "v1" }, "name": { "num": 5461, "type": "client" } }, "id": 5461, "inst": "client.5461 v1:172.21.15.89:0/330049992", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 25, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.449427718174869 }, "state": "stale", "uptime": 346.15569698 } ] 2023-10-18T10:53:08.871 DEBUG:tasks.ceph_test_case:No log hits yet, waiting... 2023-10-18T10:53:19.955 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:53:19.959 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:53:50.126 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:53:50.131 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:54:08.888 ERROR:tasks.ceph_test_case:Log output: 2023-10-18T10:54:08.889 DEBUG:tasks.cephfs.test_client_recovery:Expected log message found: '1 client(s) laggy due to laggy OSDs', retrying 2023-10-18T10:54:13.890 DEBUG:teuthology.orchestra.run.smithi089:> sudo daemon-helper kill ceph --cluster ceph -w 2023-10-18T10:54:13.895 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:54:14.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.046+0000 7fd63cdba640 1 Processor -- start 2023-10-18T10:54:14.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.046+0000 7fd63cdba640 1 -- start start 2023-10-18T10:54:14.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.046+0000 7fd63cdba640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd63815c670 0x7fd6380713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:54:14.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.046+0000 7fd63cdba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd638060a30 0x7fd6380c9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:54:14.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.046+0000 7fd63cdba640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6381602a0 con 0x7fd638160a10 2023-10-18T10:54:14.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.046+0000 7fd63cdba640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd638071900 con 0x7fd63815c670 2023-10-18T10:54:14.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.046+0000 7fd63cdba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6380cd2b0 con 0x7fd638060a30 2023-10-18T10:54:14.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.046+0000 7fd63659c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd63815c670 0x7fd6380713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:54:14.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.046+0000 7fd63659c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd63815c670 0x7fd6380713c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44446/0 (socket says 172.21.15.89:44446) 2023-10-18T10:54:14.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.046+0000 7fd63659c640 1 -- 172.21.15.89:0/2936512666 learned_addr learned my addr 172.21.15.89:0/2936512666 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:54:14.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.046+0000 7fd63659c640 1 -- 172.21.15.89:0/2936512666 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd638060a30 msgr2=0x7fd6380c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:54:14.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.046+0000 7fd63659c640 1 --2- 172.21.15.89:0/2936512666 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd638060a30 0x7fd6380c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:54:14.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.046+0000 7fd63659c640 1 -- 172.21.15.89:0/2936512666 >> v1:172.21.15.154:6789/0 conn(0x7fd638160a10 legacy=0x7fd638060320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:54:14.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.046+0000 7fd63659c640 1 -- 172.21.15.89:0/2936512666 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6380cd590 con 0x7fd63815c670 2023-10-18T10:54:14.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.050+0000 7fd63659c640 1 --2- 172.21.15.89:0/2936512666 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd63815c670 0x7fd6380713c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd628009870 tx=0x7fd62801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=277c2da4f5414c7f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:54:14.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.050+0000 7fd635d9b640 1 -- 172.21.15.89:0/2936512666 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6280a89d0 con 0x7fd63815c670 2023-10-18T10:54:14.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.050+0000 7fd635d9b640 1 -- 172.21.15.89:0/2936512666 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6280a8b30 con 0x7fd63815c670 2023-10-18T10:54:14.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.050+0000 7fd635d9b640 1 -- 172.21.15.89:0/2936512666 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6280b0440 con 0x7fd63815c670 2023-10-18T10:54:14.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.050+0000 7fd63cdba640 1 -- 172.21.15.89:0/2936512666 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd63815c670 msgr2=0x7fd6380713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:54:14.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.050+0000 7fd63cdba640 1 --2- 172.21.15.89:0/2936512666 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd63815c670 0x7fd6380713c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd628009870 tx=0x7fd62801ed10 comp rx=0 tx=0).stop 2023-10-18T10:54:14.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.050+0000 7fd63cdba640 1 -- 172.21.15.89:0/2936512666 shutdown_connections 2023-10-18T10:54:14.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.050+0000 7fd63cdba640 1 --2- 172.21.15.89:0/2936512666 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd638060a30 0x7fd6380c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:54:14.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.050+0000 7fd63cdba640 1 --2- 172.21.15.89:0/2936512666 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd63815c670 0x7fd6380713c0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:54:14.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.050+0000 7fd63cdba640 1 -- 172.21.15.89:0/2936512666 >> 172.21.15.89:0/2936512666 conn(0x7fd6380cbdc0 msgr2=0x7fd6380cc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:54:14.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.050+0000 7fd63cdba640 1 -- 172.21.15.89:0/2936512666 shutdown_connections 2023-10-18T10:54:14.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.050+0000 7fd63cdba640 1 -- 172.21.15.89:0/2936512666 wait complete. 2023-10-18T10:54:14.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.054+0000 7fd63cdba640 1 Processor -- start 2023-10-18T10:54:14.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.054+0000 7fd63cdba640 1 -- start start 2023-10-18T10:54:14.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.054+0000 7fd63cdba640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd638060a30 0x7fd6381e6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:54:14.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.054+0000 7fd63cdba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd63815c670 0x7fd63815a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:54:14.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.054+0000 7fd63cdba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd638160a10 0x7fd63815d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:54:14.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.054+0000 7fd63cdba640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63815bb30 con 0x7fd638060a30 2023-10-18T10:54:14.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.054+0000 7fd63cdba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6381e6e10 con 0x7fd638160a10 2023-10-18T10:54:14.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.054+0000 7fd63cdba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6381e70f0 con 0x7fd63815c670 2023-10-18T10:54:14.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.054+0000 7fd63659c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd638060a30 0x7fd6381e6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:54:14.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.054+0000 7fd63659c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd638060a30 0x7fd6381e6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44460/0 (socket says 172.21.15.89:44460) 2023-10-18T10:54:14.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.054+0000 7fd63659c640 1 -- 172.21.15.89:0/1265069868 learned_addr learned my addr 172.21.15.89:0/1265069868 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:54:14.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.054+0000 7fd62ffff640 1 --2- 172.21.15.89:0/1265069868 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd63815c670 0x7fd63815a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:54:14.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.054+0000 7fd636d9d640 1 --2- 172.21.15.89:0/1265069868 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd638160a10 0x7fd63815d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:54:14.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.054+0000 7fd63659c640 1 -- 172.21.15.89:0/1265069868 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd63815c670 msgr2=0x7fd63815a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:54:14.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.054+0000 7fd63659c640 1 --2- 172.21.15.89:0/1265069868 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd63815c670 0x7fd63815a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:54:14.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.054+0000 7fd63659c640 1 -- 172.21.15.89:0/1265069868 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd638160a10 msgr2=0x7fd63815d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:54:14.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.054+0000 7fd63659c640 1 --2- 172.21.15.89:0/1265069868 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd638160a10 0x7fd63815d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:54:14.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.054+0000 7fd63659c640 1 -- 172.21.15.89:0/1265069868 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6280094d0 con 0x7fd638060a30 2023-10-18T10:54:14.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.054+0000 7fd63659c640 1 --2- 172.21.15.89:0/1265069868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd638060a30 0x7fd6381e6710 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd62801d3c0 tx=0x7fd62801bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:54:14.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.054+0000 7fd62effd640 1 -- 172.21.15.89:0/1265069868 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6280bb670 con 0x7fd638060a30 2023-10-18T10:54:14.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.054+0000 7fd62effd640 1 -- 172.21.15.89:0/1265069868 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6280bb7d0 con 0x7fd638060a30 2023-10-18T10:54:14.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.054+0000 7fd62effd640 1 -- 172.21.15.89:0/1265069868 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6280bbb80 con 0x7fd638060a30 2023-10-18T10:54:14.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.054+0000 7fd63cdba640 1 -- 172.21.15.89:0/1265069868 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6381e73d0 con 0x7fd638060a30 2023-10-18T10:54:14.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.054+0000 7fd63cdba640 1 -- 172.21.15.89:0/1265069868 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63815aea0 con 0x7fd638060a30 2023-10-18T10:54:14.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.058+0000 7fd63cdba640 1 -- 172.21.15.89:0/1265069868 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd63806b400 con 0x7fd638060a30 2023-10-18T10:54:14.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.058+0000 7fd62effd640 1 -- 172.21.15.89:0/1265069868 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fd6280b7070 con 0x7fd638060a30 2023-10-18T10:54:14.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.058+0000 7fd62effd640 1 --2- 172.21.15.89:0/1265069868 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd618065680 0x7fd618067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:54:14.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.058+0000 7fd62effd640 1 -- 172.21.15.89:0/1265069868 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fd628121360 con 0x7fd638060a30 2023-10-18T10:54:14.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.058+0000 7fd62ffff640 1 --2- 172.21.15.89:0/1265069868 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd618065680 0x7fd618067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:54:14.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.058+0000 7fd62ffff640 1 --2- 172.21.15.89:0/1265069868 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd618065680 0x7fd618067b40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd6381e5c20 tx=0x7fd620008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:54:14.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.062+0000 7fd62effd640 1 -- 172.21.15.89:0/1265069868 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fd6280eed10 con 0x7fd638060a30 2023-10-18T10:54:14.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.070+0000 7f64ef7fa640 1 Processor -- start 2023-10-18T10:54:14.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.070+0000 7f64ef7fa640 1 -- start start 2023-10-18T10:54:14.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.070+0000 7f64ef7fa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64e8160f20 0x7f64e80d0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:54:14.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.070+0000 7f64ef7fa640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f64e80b33b0 0x7f64e80d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:54:14.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.070+0000 7f64ef7fa640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f64e80d0270 0x7f64e80b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:54:14.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.070+0000 7f64ef7fa640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64e80cced0 con 0x7f64e80b33b0 2023-10-18T10:54:14.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.070+0000 7f64ef7fa640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64e80b1d30 con 0x7f64e80d0270 2023-10-18T10:54:14.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.070+0000 7f64ef7fa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e80b3bc0 con 0x7f64e8160f20 2023-10-18T10:54:14.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.074+0000 7f64ecd95640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f64e80b33b0 0x7f64e80d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:54:14.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.074+0000 7f64ecd95640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f64e80b33b0 0x7f64e80d0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44462/0 (socket says 172.21.15.89:44462) 2023-10-18T10:54:14.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.074+0000 7f64ecd95640 1 -- 172.21.15.89:0/1988177214 learned_addr learned my addr 172.21.15.89:0/1988177214 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:54:14.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.074+0000 7f64edd97640 1 --2- 172.21.15.89:0/1988177214 >> v2:172.21.15.154:3300/0 conn(0x7f64e80d0270 0x7f64e80b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:54:14.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.074+0000 7f64ed596640 1 --2- 172.21.15.89:0/1988177214 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64e8160f20 0x7f64e80d0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:54:14.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.074+0000 7f64ecd95640 1 -- 172.21.15.89:0/1988177214 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64e8160f20 msgr2=0x7f64e80d0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:54:14.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.074+0000 7f64ecd95640 1 --2- 172.21.15.89:0/1988177214 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64e8160f20 0x7f64e80d0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:54:14.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.074+0000 7f64ecd95640 1 -- 172.21.15.89:0/1988177214 >> v2:172.21.15.154:3300/0 conn(0x7f64e80d0270 msgr2=0x7f64e80b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:54:14.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.074+0000 7f64ecd95640 1 --2- 172.21.15.89:0/1988177214 >> v2:172.21.15.154:3300/0 conn(0x7f64e80d0270 0x7f64e80b1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:54:14.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.074+0000 7f64ecd95640 1 -- 172.21.15.89:0/1988177214 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e80afdd0 con 0x7f64e80b33b0 2023-10-18T10:54:14.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.074+0000 7f64edd97640 1 --2- 172.21.15.89:0/1988177214 >> v2:172.21.15.154:3300/0 conn(0x7f64e80d0270 0x7f64e80b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:54:14.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.074+0000 7f64ecd95640 1 --2- 172.21.15.89:0/1988177214 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64e80b33b0 0x7f64e80d0c90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f64e809bdd0 tx=0x7f64d801c1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856dc6fdbe91922f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:54:14.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.074+0000 7f64dffff640 1 -- 172.21.15.89:0/1988177214 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d80ab600 con 0x7f64e80b33b0 2023-10-18T10:54:14.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.074+0000 7f64dffff640 1 -- 172.21.15.89:0/1988177214 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d80a7050 con 0x7f64e80b33b0 2023-10-18T10:54:14.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.074+0000 7f64dffff640 1 -- 172.21.15.89:0/1988177214 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d80ab980 con 0x7f64e80b33b0 2023-10-18T10:54:14.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.078+0000 7f64ef7fa640 1 -- 172.21.15.89:0/1988177214 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64e80b33b0 msgr2=0x7f64e80d0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:54:14.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.078+0000 7f64ef7fa640 1 --2- 172.21.15.89:0/1988177214 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64e80b33b0 0x7f64e80d0c90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f64e809bdd0 tx=0x7f64d801c1b0 comp rx=0 tx=0).stop 2023-10-18T10:54:14.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.078+0000 7f64ef7fa640 1 -- 172.21.15.89:0/1988177214 shutdown_connections 2023-10-18T10:54:14.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.078+0000 7f64ef7fa640 1 --2- 172.21.15.89:0/1988177214 >> v2:172.21.15.154:3300/0 conn(0x7f64e80d0270 0x7f64e80b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:54:14.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.078+0000 7f64ef7fa640 1 --2- 172.21.15.89:0/1988177214 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64e80b33b0 0x7f64e80d0c90 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:54:14.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.078+0000 7f64ef7fa640 1 --2- 172.21.15.89:0/1988177214 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64e8160f20 0x7f64e80d0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:54:14.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.078+0000 7f64ef7fa640 1 -- 172.21.15.89:0/1988177214 >> 172.21.15.89:0/1988177214 conn(0x7f64e809b930 msgr2=0x7f64e80d26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:54:14.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.078+0000 7f64ef7fa640 1 -- 172.21.15.89:0/1988177214 shutdown_connections 2023-10-18T10:54:14.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.078+0000 7f64ef7fa640 1 -- 172.21.15.89:0/1988177214 wait complete. 2023-10-18T10:54:14.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.078+0000 7f64ef7fa640 1 Processor -- start 2023-10-18T10:54:14.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.078+0000 7f64ef7fa640 1 -- start start 2023-10-18T10:54:14.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.078+0000 7f64ef7fa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64e80b33b0 0x7f64e81581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:54:14.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.078+0000 7f64ef7fa640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64e80d0270 0x7f64e80cb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:54:14.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.078+0000 7f64ef7fa640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64e8160f20 0x7f64e80c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:54:14.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.078+0000 7f64ef7fa640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e80c90d0 con 0x7f64e80d0270 2023-10-18T10:54:14.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.078+0000 7f64ef7fa640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e8157130 con 0x7f64e8160f20 2023-10-18T10:54:14.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.078+0000 7f64ef7fa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e8157410 con 0x7f64e80b33b0 2023-10-18T10:54:14.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.078+0000 7f64ecd95640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64e80d0270 0x7f64e80cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:54:14.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.078+0000 7f64ecd95640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64e80d0270 0x7f64e80cb850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44474/0 (socket says 172.21.15.89:44474) 2023-10-18T10:54:14.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.078+0000 7f64ecd95640 1 -- 172.21.15.89:0/1164847397 learned_addr learned my addr 172.21.15.89:0/1164847397 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:54:14.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.078+0000 7f64ed596640 1 --2- 172.21.15.89:0/1164847397 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64e80b33b0 0x7f64e81581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:54:14.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.078+0000 7f64edd97640 1 --2- 172.21.15.89:0/1164847397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64e8160f20 0x7f64e80c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:54:14.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.082+0000 7f64ecd95640 1 -- 172.21.15.89:0/1164847397 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64e80b33b0 msgr2=0x7f64e81581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:54:14.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.082+0000 7f64ecd95640 1 --2- 172.21.15.89:0/1164847397 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64e80b33b0 0x7f64e81581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:54:14.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.082+0000 7f64ecd95640 1 -- 172.21.15.89:0/1164847397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64e8160f20 msgr2=0x7f64e80c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:54:14.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.082+0000 7f64ecd95640 1 --2- 172.21.15.89:0/1164847397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64e8160f20 0x7f64e80c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:54:14.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.082+0000 7f64ecd95640 1 -- 172.21.15.89:0/1164847397 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d80094d0 con 0x7f64e80d0270 2023-10-18T10:54:14.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.082+0000 7f64edd97640 1 --2- 172.21.15.89:0/1164847397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64e8160f20 0x7f64e80c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:54:14.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.082+0000 7f64ecd95640 1 --2- 172.21.15.89:0/1164847397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64e80d0270 0x7f64e80cb850 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f64d80ae040 tx=0x7f64d801a9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:54:14.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.082+0000 7f64ddffb640 1 -- 172.21.15.89:0/1164847397 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d80b6420 con 0x7f64e80d0270 2023-10-18T10:54:14.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.082+0000 7f64ddffb640 1 -- 172.21.15.89:0/1164847397 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d80b6580 con 0x7f64e80d0270 2023-10-18T10:54:14.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.082+0000 7f64ef7fa640 1 -- 172.21.15.89:0/1164847397 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e81576f0 con 0x7f64e80d0270 2023-10-18T10:54:14.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.082+0000 7f64ddffb640 1 -- 172.21.15.89:0/1164847397 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d80b6910 con 0x7f64e80d0270 2023-10-18T10:54:14.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.082+0000 7f64ef7fa640 1 -- 172.21.15.89:0/1164847397 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e8157cb0 con 0x7f64e80d0270 2023-10-18T10:54:14.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.082+0000 7f64ef7fa640 1 -- 172.21.15.89:0/1164847397 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f64e804e430 con 0x7f64e80d0270 2023-10-18T10:54:14.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.082+0000 7f64ddffb640 1 -- 172.21.15.89:0/1164847397 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f64d80b2040 con 0x7f64e80d0270 2023-10-18T10:54:14.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.082+0000 7f64ddffb640 1 --2- 172.21.15.89:0/1164847397 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f64cc0656b0 0x7f64cc067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:54:14.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.082+0000 7f64ddffb640 1 -- 172.21.15.89:0/1164847397 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f64d811d7a0 con 0x7f64e80d0270 2023-10-18T10:54:14.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.086+0000 7f64ddffb640 1 -- 172.21.15.89:0/1164847397 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+446 (secure 0 0 0) 0x7f64d811db30 con 0x7f64e80d0270 2023-10-18T10:54:14.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.086+0000 7f64ed596640 1 --2- 172.21.15.89:0/1164847397 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f64cc0656b0 0x7f64cc067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:54:14.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.086+0000 7f64ef7fa640 1 -- 172.21.15.89:0/1164847397 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f64e8089fd0 con 0x7f64e80d0270 2023-10-18T10:54:14.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.086+0000 7f64ed596640 1 --2- 172.21.15.89:0/1164847397 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f64cc0656b0 0x7f64cc067b70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009ac0 tx=0x7f64e4009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:54:14.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.246+0000 7fd63cdba640 1 -- 172.21.15.89:0/1265069868 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd63806b400 con 0x7fd638060a30 2023-10-18T10:54:14.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.246+0000 7fd62effd640 1 -- 172.21.15.89:0/1265069868 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4875 (secure 0 0 0) 0x7fd6280f2330 con 0x7fd638060a30 2023-10-18T10:54:14.246 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:54:14.246 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/654729641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":654729641},{"type":"v1","addr":"172.21.15.154:6836","nonce":654729641}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/964566480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":964566480},{"type":"v1","addr":"172.21.15.154:6837","nonce":964566480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2261878538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2261878538},{"type":"v1","addr":"172.21.15.89:6833","nonce":2261878538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:47:15.199575+0000","modified":"2023-10-18T10:47:18.846656+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5517":{"gid":5517,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6835/2092274731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2092274731},{"type":"v1","addr":"172.21.15.89:6835","nonce":2092274731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T10:54:14.247 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 18 2023-10-18T10:54:14.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.246+0000 7fd63cdba640 1 -- 172.21.15.89:0/1265069868 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd618065680 msgr2=0x7fd618067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:54:14.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.246+0000 7fd63cdba640 1 --2- 172.21.15.89:0/1265069868 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd618065680 0x7fd618067b40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd6381e5c20 tx=0x7fd620008040 comp rx=0 tx=0).stop 2023-10-18T10:54:14.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.246+0000 7fd63cdba640 1 -- 172.21.15.89:0/1265069868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd638060a30 msgr2=0x7fd6381e6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:54:14.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.246+0000 7fd63cdba640 1 --2- 172.21.15.89:0/1265069868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd638060a30 0x7fd6381e6710 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd62801d3c0 tx=0x7fd62801bab0 comp rx=0 tx=0).stop 2023-10-18T10:54:14.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.246+0000 7fd63cdba640 1 -- 172.21.15.89:0/1265069868 shutdown_connections 2023-10-18T10:54:14.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.246+0000 7fd63cdba640 1 --2- 172.21.15.89:0/1265069868 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd618065680 0x7fd618067b40 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:54:14.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.246+0000 7fd63cdba640 1 --2- 172.21.15.89:0/1265069868 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd63815c670 0x7fd63815a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:54:14.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.246+0000 7fd63cdba640 1 --2- 172.21.15.89:0/1265069868 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd638160a10 0x7fd63815d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:54:14.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.246+0000 7fd63cdba640 1 --2- 172.21.15.89:0/1265069868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd638060a30 0x7fd6381e6710 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:54:14.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.246+0000 7fd63cdba640 1 -- 172.21.15.89:0/1265069868 >> 172.21.15.89:0/1265069868 conn(0x7fd6380cbdc0 msgr2=0x7fd6380b22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:54:14.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.246+0000 7fd63cdba640 1 -- 172.21.15.89:0/1265069868 shutdown_connections 2023-10-18T10:54:14.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:14.246+0000 7fd63cdba640 1 -- 172.21.15.89:0/1265069868 wait complete. 2023-10-18T10:54:14.266 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-18T10:54:14.423 INFO:teuthology.orchestra.run.smithi089.stdout:[{"id":5183,"entity":{"name":{"type":"client","num":5183},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":4272238049}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":410.715089025,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.38714601660022357,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":3,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5183 v1:192.168.0.1:0/4272238049","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi154","kernel_version":"6.6.0-rc5-g47672119149b","root":"/"}},{"id":5461,"entity":{"name":{"type":"client","num":5461},"addr":{"type":"v1","addr":"172.21.15.89:0","nonce":330049992}},"state":"stale","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":411.811059102,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.38617048343390387,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":25,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5461 v1:172.21.15.89:0/330049992","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi089","kernel_version":"6.6.0-rc5-g47672119149b","root":"/"}}] 2023-10-18T10:54:14.424 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "1", "hostname": "smithi154", "kernel_version": "6.6.0-rc5-g47672119149b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 4272238049, "type": "v1" }, "name": { "num": 5183, "type": "client" } }, "id": 5183, "inst": "client.5183 v1:192.168.0.1:0/4272238049", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 3, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.3871460166002236 }, "state": "open", "uptime": 410.715089025 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "0", "hostname": "smithi089", "kernel_version": "6.6.0-rc5-g47672119149b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.89:0", "nonce": 330049992, "type": "v1" }, "name": { "num": 5461, "type": "client" } }, "id": 5461, "inst": "client.5461 v1:172.21.15.89:0/330049992", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 25, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.38617048343390387 }, "state": "stale", "uptime": 411.811059102 } ] 2023-10-18T10:54:14.525 DEBUG:tasks.ceph_test_case:No log hits yet, waiting... 2023-10-18T10:54:20.189 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:54:20.194 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:54:32.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:54:32.073+0000 7f64ddffb640 1 -- 172.21.15.89:0/1164847397 <== mon.0 v2:172.21.15.89:3300/0 7 ==== log(1 entries from seq 440 at 2023-10-18T10:54:32.026108+0000) v1 ==== 246+0+0 (secure 0 0 0) 0x7f64d80f2ac0 con 0x7f64e80d0270 2023-10-18T10:54:32.075 INFO:teuthology.orchestra.run.smithi089.stdout: cluster: 2023-10-18T10:54:32.075 INFO:teuthology.orchestra.run.smithi089.stdout: id: 1352f62a-2330-4e96-afcd-b8e87cd82667 2023-10-18T10:54:32.075 INFO:teuthology.orchestra.run.smithi089.stdout: health: HEALTH_OK 2023-10-18T10:54:32.075 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:54:32.076 INFO:teuthology.orchestra.run.smithi089.stdout: services: 2023-10-18T10:54:32.076 INFO:teuthology.orchestra.run.smithi089.stdout: mon: 3 daemons, quorum a,b,c (age 8m) 2023-10-18T10:54:32.076 INFO:teuthology.orchestra.run.smithi089.stdout: mgr: x(active, since 8m), standbys: y 2023-10-18T10:54:32.076 INFO:teuthology.orchestra.run.smithi089.stdout: mds: 1/1 daemons up, 3 standby 2023-10-18T10:54:32.076 INFO:teuthology.orchestra.run.smithi089.stdout: osd: 8 osds: 8 up (since 4m), 8 in (since 8m) 2023-10-18T10:54:32.076 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:54:32.077 INFO:teuthology.orchestra.run.smithi089.stdout: data: 2023-10-18T10:54:32.077 INFO:teuthology.orchestra.run.smithi089.stdout: volumes: 1/1 healthy 2023-10-18T10:54:32.077 INFO:teuthology.orchestra.run.smithi089.stdout: pools: 4 pools, 105 pgs 2023-10-18T10:54:32.077 INFO:teuthology.orchestra.run.smithi089.stdout: objects: 24 objects, 580 KiB 2023-10-18T10:54:32.077 INFO:teuthology.orchestra.run.smithi089.stdout: usage: 217 MiB used, 720 GiB / 720 GiB avail 2023-10-18T10:54:32.078 INFO:teuthology.orchestra.run.smithi089.stdout: pgs: 105 active+clean 2023-10-18T10:54:32.078 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:54:32.078 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:54:32.078 INFO:teuthology.orchestra.run.smithi089.stdout:2023-10-18T10:54:32.026108+0000 mon.a [WRN] Health check failed: 1 client(s) laggy due to laggy OSDs (MDS_CLIENTS_LAGGY) 2023-10-18T10:54:50.253 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:54:50.258 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:55:14.544 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mon mon_osd_laggy_weight 2023-10-18T10:55:14.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.733+0000 7f9dcefe1640 1 Processor -- start 2023-10-18T10:55:14.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.733+0000 7f9dcefe1640 1 -- start start 2023-10-18T10:55:14.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.737+0000 7f9dcefe1640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9dc8159490 0x7f9dc815d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:55:14.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.737+0000 7f9dcefe1640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9dc806bee0 0x7f9dc80c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:55:14.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.737+0000 7f9dcefe1640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc815ddf0 con 0x7f9dc8070600 2023-10-18T10:55:14.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.737+0000 7f9dcefe1640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc8070b90 con 0x7f9dc8159490 2023-10-18T10:55:14.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.737+0000 7f9dcefe1640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc8070e70 con 0x7f9dc806bee0 2023-10-18T10:55:14.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.737+0000 7f9dbffff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9dc8070600 0x7f9dc806b7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:50310/0 (socket says 172.21.15.89:50310) 2023-10-18T10:55:14.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.737+0000 7f9dbffff640 1 -- 172.21.15.89:0/653421405 learned_addr learned my addr 172.21.15.89:0/653421405 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:55:14.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.737+0000 7f9dccd7d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9dc8159490 0x7f9dc815d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:55:14.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.737+0000 7f9dcd57e640 1 --2- 172.21.15.89:0/653421405 >> v2:172.21.15.154:3300/0 conn(0x7f9dc806bee0 0x7f9dc80c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:55:14.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.737+0000 7f9dccd7d640 1 -- 172.21.15.89:0/653421405 >> v2:172.21.15.154:3300/0 conn(0x7f9dc806bee0 msgr2=0x7f9dc80c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:55:14.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.737+0000 7f9dccd7d640 1 --2- 172.21.15.89:0/653421405 >> v2:172.21.15.154:3300/0 conn(0x7f9dc806bee0 0x7f9dc80c8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:14.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.737+0000 7f9dccd7d640 1 -- 172.21.15.89:0/653421405 >> v1:172.21.15.89:6789/0 conn(0x7f9dc8070600 legacy=0x7f9dc806b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:55:14.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.737+0000 7f9dbf7fe640 1 -- 172.21.15.89:0/653421405 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3351133311 0 0) 0x7f9dc815ddf0 con 0x7f9dc8070600 2023-10-18T10:55:14.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.737+0000 7f9dccd7d640 1 -- 172.21.15.89:0/653421405 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc8071150 con 0x7f9dc8159490 2023-10-18T10:55:14.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.741+0000 7f9dccd7d640 1 --2- 172.21.15.89:0/653421405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9dc8159490 0x7f9dc815d8b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9dc80cbb00 tx=0x7f9dc401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e847ac0cd1c5e182 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:55:14.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.741+0000 7f9dbf7fe640 1 -- 172.21.15.89:0/653421405 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc40ac600 con 0x7f9dc8159490 2023-10-18T10:55:14.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.741+0000 7f9dbf7fe640 1 -- 172.21.15.89:0/653421405 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc40a8050 con 0x7f9dc8159490 2023-10-18T10:55:14.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.741+0000 7f9dbf7fe640 1 -- 172.21.15.89:0/653421405 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc40ac980 con 0x7f9dc8159490 2023-10-18T10:55:14.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.741+0000 7f9dcefe1640 1 -- 172.21.15.89:0/653421405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9dc8159490 msgr2=0x7f9dc815d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:55:14.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.741+0000 7f9dcefe1640 1 --2- 172.21.15.89:0/653421405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9dc8159490 0x7f9dc815d8b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9dc80cbb00 tx=0x7f9dc401c3a0 comp rx=0 tx=0).stop 2023-10-18T10:55:14.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.741+0000 7f9dcefe1640 1 -- 172.21.15.89:0/653421405 shutdown_connections 2023-10-18T10:55:14.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.741+0000 7f9dcefe1640 1 --2- 172.21.15.89:0/653421405 >> v2:172.21.15.154:3300/0 conn(0x7f9dc806bee0 0x7f9dc80c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:14.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.741+0000 7f9dcefe1640 1 --2- 172.21.15.89:0/653421405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9dc8159490 0x7f9dc815d8b0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:14.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.741+0000 7f9dcefe1640 1 -- 172.21.15.89:0/653421405 >> 172.21.15.89:0/653421405 conn(0x7f9dc8069f00 msgr2=0x7f9dc80cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:55:14.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.741+0000 7f9dcefe1640 1 -- 172.21.15.89:0/653421405 shutdown_connections 2023-10-18T10:55:14.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.741+0000 7f9dcefe1640 1 -- 172.21.15.89:0/653421405 wait complete. 2023-10-18T10:55:14.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.741+0000 7f9dcefe1640 1 Processor -- start 2023-10-18T10:55:14.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.741+0000 7f9dcefe1640 1 -- start start 2023-10-18T10:55:14.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.745+0000 7f9dcefe1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9dc806bee0 0x7f9dc81588b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:55:14.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.745+0000 7f9dcefe1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9dc8070600 0x7f9dc815c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:55:14.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.745+0000 7f9dcefe1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9dc8159490 0x7f9dc815aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:55:14.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.745+0000 7f9dcefe1640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc8158df0 con 0x7f9dc8159490 2023-10-18T10:55:14.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.745+0000 7f9dcefe1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc815b3d0 con 0x7f9dc806bee0 2023-10-18T10:55:14.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.745+0000 7f9dcefe1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc815b6b0 con 0x7f9dc8070600 2023-10-18T10:55:14.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.745+0000 7f9dcd57e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9dc8159490 0x7f9dc815aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:55:14.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.745+0000 7f9dcd57e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9dc8159490 0x7f9dc815aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56640/0 (socket says 172.21.15.89:56640) 2023-10-18T10:55:14.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.745+0000 7f9dcd57e640 1 -- 172.21.15.89:0/974331367 learned_addr learned my addr 172.21.15.89:0/974331367 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:55:14.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.745+0000 7f9dbffff640 1 --2- 172.21.15.89:0/974331367 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9dc8070600 0x7f9dc815c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:55:14.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.745+0000 7f9dccd7d640 1 --2- 172.21.15.89:0/974331367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9dc806bee0 0x7f9dc81588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:55:14.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.745+0000 7f9dcd57e640 1 -- 172.21.15.89:0/974331367 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9dc8070600 msgr2=0x7f9dc815c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:55:14.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.745+0000 7f9dcd57e640 1 --2- 172.21.15.89:0/974331367 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9dc8070600 0x7f9dc815c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:14.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.745+0000 7f9dcd57e640 1 -- 172.21.15.89:0/974331367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9dc806bee0 msgr2=0x7f9dc81588b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:55:14.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.745+0000 7f9dcd57e640 1 --2- 172.21.15.89:0/974331367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9dc806bee0 0x7f9dc81588b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:14.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.745+0000 7f9dcd57e640 1 -- 172.21.15.89:0/974331367 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc4009420 con 0x7f9dc8159490 2023-10-18T10:55:14.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.745+0000 7f9dbffff640 1 --2- 172.21.15.89:0/974331367 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9dc8070600 0x7f9dc815c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:55:14.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.745+0000 7f9dcd57e640 1 --2- 172.21.15.89:0/974331367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9dc8159490 0x7f9dc815aa10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000ef90 tx=0x7f9dc000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:55:14.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.745+0000 7f9dbd7fa640 1 -- 172.21.15.89:0/974331367 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0009bf0 con 0x7f9dc8159490 2023-10-18T10:55:14.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.745+0000 7f9dbd7fa640 1 -- 172.21.15.89:0/974331367 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc000f040 con 0x7f9dc8159490 2023-10-18T10:55:14.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.745+0000 7f9dbd7fa640 1 -- 172.21.15.89:0/974331367 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc00146c0 con 0x7f9dc8159490 2023-10-18T10:55:14.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.745+0000 7f9dcefe1640 1 -- 172.21.15.89:0/974331367 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc815b990 con 0x7f9dc8159490 2023-10-18T10:55:14.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.745+0000 7f9dcefe1640 1 -- 172.21.15.89:0/974331367 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc815bf50 con 0x7f9dc8159490 2023-10-18T10:55:14.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.745+0000 7f9dcefe1640 1 -- 172.21.15.89:0/974331367 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc804dbe0 con 0x7f9dc8159490 2023-10-18T10:55:14.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.745+0000 7f9dbd7fa640 1 -- 172.21.15.89:0/974331367 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f9dc0014820 con 0x7f9dc8159490 2023-10-18T10:55:14.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.749+0000 7f9dbd7fa640 1 --2- 172.21.15.89:0/974331367 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f9db4065680 0x7f9db4067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:55:14.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.749+0000 7f9dbd7fa640 1 -- 172.21.15.89:0/974331367 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f9dc0083d90 con 0x7f9dc8159490 2023-10-18T10:55:14.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.749+0000 7f9dccd7d640 1 --2- 172.21.15.89:0/974331367 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f9db4065680 0x7f9db4067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:55:14.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.749+0000 7f9dccd7d640 1 --2- 172.21.15.89:0/974331367 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f9db4065680 0x7f9db4067b40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40ae000 tx=0x7f9dc40a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:55:14.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.753+0000 7f9dbd7fa640 1 -- 172.21.15.89:0/974331367 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f9dc00516c0 con 0x7f9dc8159490 2023-10-18T10:55:14.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.901+0000 7f9dcefe1640 1 -- 172.21.15.89:0/974331367 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "config get", "who": "mon", "key": "mon_osd_laggy_weight"} v 0) v1 -- 0x7f9dc80cbb60 con 0x7f9dc8159490 2023-10-18T10:55:14.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.905+0000 7f9dbd7fa640 1 -- 172.21.15.89:0/974331367 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mon", "key": "mon_osd_laggy_weight"}]=0 v11) v1 ==== 103+0+9 (secure 0 0 0) 0x7f9dc0018020 con 0x7f9dc8159490 2023-10-18T10:55:14.903 INFO:teuthology.orchestra.run.smithi089.stdout:0.300000 2023-10-18T10:55:14.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.905+0000 7f9dcefe1640 1 -- 172.21.15.89:0/974331367 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f9db4065680 msgr2=0x7f9db4067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:55:14.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.905+0000 7f9dcefe1640 1 --2- 172.21.15.89:0/974331367 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f9db4065680 0x7f9db4067b40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40ae000 tx=0x7f9dc40a7000 comp rx=0 tx=0).stop 2023-10-18T10:55:14.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.905+0000 7f9dcefe1640 1 -- 172.21.15.89:0/974331367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9dc8159490 msgr2=0x7f9dc815aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:55:14.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.905+0000 7f9dcefe1640 1 --2- 172.21.15.89:0/974331367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9dc8159490 0x7f9dc815aa10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000ef90 tx=0x7f9dc000c490 comp rx=0 tx=0).stop 2023-10-18T10:55:14.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.905+0000 7f9dcefe1640 1 -- 172.21.15.89:0/974331367 shutdown_connections 2023-10-18T10:55:14.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.905+0000 7f9dcefe1640 1 --2- 172.21.15.89:0/974331367 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f9db4065680 0x7f9db4067b40 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:14.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.905+0000 7f9dcefe1640 1 --2- 172.21.15.89:0/974331367 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9dc8070600 0x7f9dc815c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:14.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.905+0000 7f9dcefe1640 1 --2- 172.21.15.89:0/974331367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9dc8159490 0x7f9dc815aa10 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:14.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.905+0000 7f9dcefe1640 1 --2- 172.21.15.89:0/974331367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9dc806bee0 0x7f9dc81588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:14.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.905+0000 7f9dcefe1640 1 -- 172.21.15.89:0/974331367 >> 172.21.15.89:0/974331367 conn(0x7f9dc8069f00 msgr2=0x7f9dc80ce090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:55:14.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.905+0000 7f9dcefe1640 1 -- 172.21.15.89:0/974331367 shutdown_connections 2023-10-18T10:55:14.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:14.905+0000 7f9dcefe1640 1 -- 172.21.15.89:0/974331367 wait complete. 2023-10-18T10:55:14.923 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_osd_laggy_weight 1 2023-10-18T10:55:15.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.065+0000 7fcde3340640 1 Processor -- start 2023-10-18T10:55:15.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.065+0000 7fcde3340640 1 -- start start 2023-10-18T10:55:15.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.065+0000 7fcde3340640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcddc143820 0x7fcddc0cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:55:15.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.065+0000 7fcde3340640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcddc143d70 0x7fcddc0ce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:55:15.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.065+0000 7fcde3340640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcddc15e0d0 con 0x7fcddc0cfc50 2023-10-18T10:55:15.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.065+0000 7fcde3340640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcddc15e320 con 0x7fcddc143d70 2023-10-18T10:55:15.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.065+0000 7fcde3340640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc15e600 con 0x7fcddc143820 2023-10-18T10:55:15.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.069+0000 7fcde10dc640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fcddc0cfc50 0x7fcddc0ccd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:50326/0 (socket says 172.21.15.89:50326) 2023-10-18T10:55:15.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.069+0000 7fcde10dc640 1 -- 172.21.15.89:0/596255008 learned_addr learned my addr 172.21.15.89:0/596255008 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:55:15.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.069+0000 7fcde18dd640 1 --2- 172.21.15.89:0/596255008 >> v2:172.21.15.89:3300/0 conn(0x7fcddc143d70 0x7fcddc0ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:55:15.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.069+0000 7fcde08db640 1 --2- 172.21.15.89:0/596255008 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcddc143820 0x7fcddc0cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:55:15.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.069+0000 7fcdd3fff640 1 -- 172.21.15.89:0/596255008 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 753365310 0 0) 0x7fcddc15e0d0 con 0x7fcddc0cfc50 2023-10-18T10:55:15.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.069+0000 7fcdd3fff640 1 -- 172.21.15.89:0/596255008 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdc8003530 con 0x7fcddc0cfc50 2023-10-18T10:55:15.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.069+0000 7fcde18dd640 1 -- 172.21.15.89:0/596255008 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcddc143820 msgr2=0x7fcddc0cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:55:15.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.069+0000 7fcde18dd640 1 --2- 172.21.15.89:0/596255008 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcddc143820 0x7fcddc0cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:15.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.069+0000 7fcde18dd640 1 -- 172.21.15.89:0/596255008 >> v1:172.21.15.89:6789/0 conn(0x7fcddc0cfc50 legacy=0x7fcddc0ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:55:15.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.069+0000 7fcde18dd640 1 -- 172.21.15.89:0/596255008 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcddc15e8e0 con 0x7fcddc143d70 2023-10-18T10:55:15.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.069+0000 7fcde18dd640 1 --2- 172.21.15.89:0/596255008 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcddc143d70 0x7fcddc0ce220 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4009d90 tx=0x7fcdd401c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d390c8fa3f73b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:55:15.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.069+0000 7fcdd3fff640 1 -- 172.21.15.89:0/596255008 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd40a59a0 con 0x7fcddc143d70 2023-10-18T10:55:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.069+0000 7fcdd3fff640 1 -- 172.21.15.89:0/596255008 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd40a5b00 con 0x7fcddc143d70 2023-10-18T10:55:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.069+0000 7fcdd3fff640 1 -- 172.21.15.89:0/596255008 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd40a5e90 con 0x7fcddc143d70 2023-10-18T10:55:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.073+0000 7fcde3340640 1 -- 172.21.15.89:0/596255008 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcddc143d70 msgr2=0x7fcddc0ce220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:55:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.073+0000 7fcde3340640 1 --2- 172.21.15.89:0/596255008 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcddc143d70 0x7fcddc0ce220 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4009d90 tx=0x7fcdd401c8c0 comp rx=0 tx=0).stop 2023-10-18T10:55:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.073+0000 7fcde3340640 1 -- 172.21.15.89:0/596255008 shutdown_connections 2023-10-18T10:55:15.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.073+0000 7fcde3340640 1 --2- 172.21.15.89:0/596255008 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcddc143d70 0x7fcddc0ce220 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:15.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.073+0000 7fcde3340640 1 --2- 172.21.15.89:0/596255008 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcddc143820 0x7fcddc0cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:15.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.073+0000 7fcde3340640 1 -- 172.21.15.89:0/596255008 >> 172.21.15.89:0/596255008 conn(0x7fcddc06dad0 msgr2=0x7fcddc06dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:55:15.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.073+0000 7fcde3340640 1 -- 172.21.15.89:0/596255008 shutdown_connections 2023-10-18T10:55:15.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.073+0000 7fcde3340640 1 -- 172.21.15.89:0/596255008 wait complete. 2023-10-18T10:55:15.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.073+0000 7fcde3340640 1 Processor -- start 2023-10-18T10:55:15.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.073+0000 7fcde3340640 1 -- start start 2023-10-18T10:55:15.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.073+0000 7fcde3340640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcddc0cfc50 0x7fcddc15d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:55:15.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.073+0000 7fcde3340640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcddc143820 0x7fcddc15ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:55:15.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.073+0000 7fcde3340640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcddc143d70 0x7fcddc0b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:55:15.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.073+0000 7fcde3340640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc15bc50 con 0x7fcddc0cfc50 2023-10-18T10:55:15.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.073+0000 7fcde3340640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc0ca7b0 con 0x7fcddc143820 2023-10-18T10:55:15.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.073+0000 7fcde3340640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc15b090 con 0x7fcddc143d70 2023-10-18T10:55:15.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.073+0000 7fcde10dc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcddc0cfc50 0x7fcddc15d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:55:15.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.073+0000 7fcde10dc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcddc0cfc50 0x7fcddc15d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56656/0 (socket says 172.21.15.89:56656) 2023-10-18T10:55:15.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.073+0000 7fcde10dc640 1 -- 172.21.15.89:0/2498438643 learned_addr learned my addr 172.21.15.89:0/2498438643 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:55:15.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.073+0000 7fcde10dc640 1 -- 172.21.15.89:0/2498438643 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcddc143d70 msgr2=0x7fcddc0b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:55:15.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.073+0000 7fcde10dc640 1 --2- 172.21.15.89:0/2498438643 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcddc143d70 0x7fcddc0b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:15.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.073+0000 7fcde10dc640 1 -- 172.21.15.89:0/2498438643 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcddc143820 msgr2=0x7fcddc15ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:55:15.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.073+0000 7fcde10dc640 1 --2- 172.21.15.89:0/2498438643 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcddc143820 0x7fcddc15ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:15.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.073+0000 7fcde10dc640 1 -- 172.21.15.89:0/2498438643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd4009a70 con 0x7fcddc0cfc50 2023-10-18T10:55:15.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.077+0000 7fcde10dc640 1 --2- 172.21.15.89:0/2498438643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcddc0cfc50 0x7fcddc15d700 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fcdd800c420 tx=0x7fcdd800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:55:15.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.077+0000 7fcdd1ffb640 1 -- 172.21.15.89:0/2498438643 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8005cf0 con 0x7fcddc0cfc50 2023-10-18T10:55:15.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.077+0000 7fcdd1ffb640 1 -- 172.21.15.89:0/2498438643 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd8005e50 con 0x7fcddc0cfc50 2023-10-18T10:55:15.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.077+0000 7fcde3340640 1 -- 172.21.15.89:0/2498438643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcddc069bb0 con 0x7fcddc0cfc50 2023-10-18T10:55:15.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.077+0000 7fcdd1ffb640 1 -- 172.21.15.89:0/2498438643 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd80072d0 con 0x7fcddc0cfc50 2023-10-18T10:55:15.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.077+0000 7fcde3340640 1 -- 172.21.15.89:0/2498438643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcddc15b590 con 0x7fcddc0cfc50 2023-10-18T10:55:15.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.077+0000 7fcde3340640 1 -- 172.21.15.89:0/2498438643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcddc04dbe0 con 0x7fcddc0cfc50 2023-10-18T10:55:15.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.077+0000 7fcdd1ffb640 1 -- 172.21.15.89:0/2498438643 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fcdd8007430 con 0x7fcddc0cfc50 2023-10-18T10:55:15.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.077+0000 7fcdd1ffb640 1 --2- 172.21.15.89:0/2498438643 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fcdc0065680 0x7fcdc0067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:55:15.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.077+0000 7fcdd1ffb640 1 -- 172.21.15.89:0/2498438643 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fcdd8082480 con 0x7fcddc0cfc50 2023-10-18T10:55:15.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.077+0000 7fcde08db640 1 --2- 172.21.15.89:0/2498438643 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fcdc0065680 0x7fcdc0067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:55:15.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.081+0000 7fcde08db640 1 --2- 172.21.15.89:0/2498438643 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fcdc0065680 0x7fcdc0067b40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc006aa0 tx=0x7fcdcc006a30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:55:15.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.081+0000 7fcdd1ffb640 1 -- 172.21.15.89:0/2498438643 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fcdd804fe60 con 0x7fcddc0cfc50 2023-10-18T10:55:15.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.233+0000 7fcde3340640 1 -- 172.21.15.89:0/2498438643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_osd_laggy_weight}] v 0) v1 -- 0x7fcddc0cfa30 con 0x7fcddc0cfc50 2023-10-18T10:55:15.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.237+0000 7fcdd1ffb640 1 -- 172.21.15.89:0/2498438643 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_osd_laggy_weight}]=0 v12)=0 v12) v1 ==== 118+0+0 (secure 0 0 0) 0x7fcdd8053480 con 0x7fcddc0cfc50 2023-10-18T10:55:15.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.241+0000 7fcde3340640 1 -- 172.21.15.89:0/2498438643 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fcdc0065680 msgr2=0x7fcdc0067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:55:15.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.241+0000 7fcde3340640 1 --2- 172.21.15.89:0/2498438643 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fcdc0065680 0x7fcdc0067b40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc006aa0 tx=0x7fcdcc006a30 comp rx=0 tx=0).stop 2023-10-18T10:55:15.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.241+0000 7fcde3340640 1 -- 172.21.15.89:0/2498438643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcddc0cfc50 msgr2=0x7fcddc15d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:55:15.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.241+0000 7fcde3340640 1 --2- 172.21.15.89:0/2498438643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcddc0cfc50 0x7fcddc15d700 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fcdd800c420 tx=0x7fcdd800c8e0 comp rx=0 tx=0).stop 2023-10-18T10:55:15.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.241+0000 7fcde3340640 1 -- 172.21.15.89:0/2498438643 shutdown_connections 2023-10-18T10:55:15.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.241+0000 7fcde3340640 1 --2- 172.21.15.89:0/2498438643 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fcdc0065680 0x7fcdc0067b40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:15.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.241+0000 7fcde3340640 1 --2- 172.21.15.89:0/2498438643 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcddc143d70 0x7fcddc0b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:15.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.241+0000 7fcde3340640 1 --2- 172.21.15.89:0/2498438643 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcddc143820 0x7fcddc15ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:15.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.241+0000 7fcde3340640 1 --2- 172.21.15.89:0/2498438643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcddc0cfc50 0x7fcddc15d700 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:15.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.241+0000 7fcde3340640 1 -- 172.21.15.89:0/2498438643 >> 172.21.15.89:0/2498438643 conn(0x7fcddc06dad0 msgr2=0x7fcddc15a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:55:15.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.241+0000 7fcde3340640 1 -- 172.21.15.89:0/2498438643 shutdown_connections 2023-10-18T10:55:15.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:15.241+0000 7fcde3340640 1 -- 172.21.15.89:0/2498438643 wait complete. 2023-10-18T10:55:15.258 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-18T10:55:15.259 INFO:tasks.ceph.osd.6:Stopping old one... 2023-10-18T10:55:15.260 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-18T10:55:15.260 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:55:15.316 INFO:tasks.ceph.osd.6:Stopped 2023-10-18T10:55:15.317 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-18T10:55:15.321 INFO:tasks.ceph.osd.6:Started 2023-10-18T10:55:15.322 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-18T10:55:15.467 INFO:teuthology.orchestra.run.smithi154.stderr:no valid command found; 10 closest matches: 2023-10-18T10:55:15.467 INFO:teuthology.orchestra.run.smithi154.stderr:0 2023-10-18T10:55:15.468 INFO:teuthology.orchestra.run.smithi154.stderr:1 2023-10-18T10:55:15.468 INFO:teuthology.orchestra.run.smithi154.stderr:2 2023-10-18T10:55:15.468 INFO:teuthology.orchestra.run.smithi154.stderr:abort 2023-10-18T10:55:15.468 INFO:teuthology.orchestra.run.smithi154.stderr:assert 2023-10-18T10:55:15.468 INFO:teuthology.orchestra.run.smithi154.stderr:config diff 2023-10-18T10:55:15.469 INFO:teuthology.orchestra.run.smithi154.stderr:config diff get 2023-10-18T10:55:15.469 INFO:teuthology.orchestra.run.smithi154.stderr:config get 2023-10-18T10:55:15.469 INFO:teuthology.orchestra.run.smithi154.stderr:config help [] 2023-10-18T10:55:15.469 INFO:teuthology.orchestra.run.smithi154.stderr:config set ... 2023-10-18T10:55:15.469 INFO:teuthology.orchestra.run.smithi154.stderr:admin_socket: invalid command 2023-10-18T10:55:15.472 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-10-18T10:55:15.473 INFO:tasks.cephfs.filesystem.ceph_manager:waiting on admin_socket for osd-6, ['dump_ops_in_flight'] 2023-10-18T10:55:15.969 INFO:tasks.ceph.osd.6.smithi154.stderr:2023-10-18T10:55:15.973+0000 7fe135bef640 -1 Falling back to public interface 2023-10-18T10:55:20.112 INFO:tasks.ceph.osd.6.smithi154.stderr:2023-10-18T10:55:20.116+0000 7fe135bef640 -1 osd.6 50 log_to_monitors true 2023-10-18T10:55:20.317 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:55:20.321 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:55:20.474 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-18T10:55:20.611 INFO:teuthology.orchestra.run.smithi154.stderr:no valid command found; 10 closest matches: 2023-10-18T10:55:20.611 INFO:teuthology.orchestra.run.smithi154.stderr:0 2023-10-18T10:55:20.611 INFO:teuthology.orchestra.run.smithi154.stderr:1 2023-10-18T10:55:20.612 INFO:teuthology.orchestra.run.smithi154.stderr:2 2023-10-18T10:55:20.612 INFO:teuthology.orchestra.run.smithi154.stderr:abort 2023-10-18T10:55:20.613 INFO:teuthology.orchestra.run.smithi154.stderr:assert 2023-10-18T10:55:20.613 INFO:teuthology.orchestra.run.smithi154.stderr:bluefs debug_inject_read_zeros 2023-10-18T10:55:20.613 INFO:teuthology.orchestra.run.smithi154.stderr:bluefs files list 2023-10-18T10:55:20.613 INFO:teuthology.orchestra.run.smithi154.stderr:bluefs stats 2023-10-18T10:55:20.614 INFO:teuthology.orchestra.run.smithi154.stderr:bluestore allocator dump block 2023-10-18T10:55:20.614 INFO:teuthology.orchestra.run.smithi154.stderr:bluestore allocator fragmentation block 2023-10-18T10:55:20.614 INFO:teuthology.orchestra.run.smithi154.stderr:admin_socket: invalid command 2023-10-18T10:55:20.616 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-10-18T10:55:20.617 INFO:tasks.cephfs.filesystem.ceph_manager:waiting on admin_socket for osd-6, ['dump_ops_in_flight'] 2023-10-18T10:55:20.936 INFO:tasks.ceph.osd.6.smithi154.stderr:2023-10-18T10:55:20.940+0000 7fe1299cc640 -1 osd.6 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T10:55:25.619 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-18T10:55:25.771 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-10-18T10:55:25.771 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2023-10-18T10:55:25.771 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2023-10-18T10:55:25.771 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-10-18T10:55:25.787 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_osd_laggy_weight 0.300000 2023-10-18T10:55:25.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.941+0000 7f16fd46d640 1 Processor -- start 2023-10-18T10:55:25.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.941+0000 7f16fd46d640 1 -- start start 2023-10-18T10:55:25.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.941+0000 7f16fd46d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16f80cb090 0x7f16f80ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:55:25.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.941+0000 7f16fd46d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f16f80cb4d0 0x7f16f80aed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:55:25.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.941+0000 7f16fd46d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16f806a620 con 0x7f16f81598b0 2023-10-18T10:55:25.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.941+0000 7f16fd46d640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16f80b59d0 con 0x7f16f80cb4d0 2023-10-18T10:55:25.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.941+0000 7f16fd46d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16f8071370 con 0x7f16f80cb090 2023-10-18T10:55:25.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.941+0000 7f16f7fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f16f80cb4d0 0x7f16f80aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:55:25.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.941+0000 7f16f7fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f16f80cb4d0 0x7f16f80aed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49820/0 (socket says 172.21.15.89:49820) 2023-10-18T10:55:25.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.941+0000 7f16f7fff640 1 -- 172.21.15.89:0/1226090225 learned_addr learned my addr 172.21.15.89:0/1226090225 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:55:25.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.941+0000 7f16ee7fc640 1 --2- 172.21.15.89:0/1226090225 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16f80cb090 0x7f16f80ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:55:25.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.941+0000 7f16f7fff640 1 -- 172.21.15.89:0/1226090225 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16f80cb090 msgr2=0x7f16f80ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:55:25.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.941+0000 7f16f7fff640 1 --2- 172.21.15.89:0/1226090225 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16f80cb090 0x7f16f80ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:25.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.941+0000 7f16f7fff640 1 -- 172.21.15.89:0/1226090225 >> v1:172.21.15.154:6789/0 conn(0x7f16f81598b0 legacy=0x7f16f80b52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:55:25.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.941+0000 7f16f7fff640 1 -- 172.21.15.89:0/1226090225 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f8071650 con 0x7f16f80cb4d0 2023-10-18T10:55:25.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.941+0000 7f16ee7fc640 1 --2- 172.21.15.89:0/1226090225 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16f80cb090 0x7f16f80ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:55:25.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.945+0000 7f16f7fff640 1 --2- 172.21.15.89:0/1226090225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16f80cb4d0 0x7f16f80aed00 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f16f0009e50 tx=0x7f16f001f1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c9b993ae46bce3 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:55:25.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.945+0000 7f16f67fc640 1 -- 172.21.15.89:0/1226090225 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f00af400 con 0x7f16f80cb4d0 2023-10-18T10:55:25.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.945+0000 7f16f67fc640 1 -- 172.21.15.89:0/1226090225 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f00ab050 con 0x7f16f80cb4d0 2023-10-18T10:55:25.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.945+0000 7f16f67fc640 1 -- 172.21.15.89:0/1226090225 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f00af780 con 0x7f16f80cb4d0 2023-10-18T10:55:25.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.945+0000 7f16fd46d640 1 -- 172.21.15.89:0/1226090225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16f80cb4d0 msgr2=0x7f16f80aed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:55:25.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.945+0000 7f16fd46d640 1 --2- 172.21.15.89:0/1226090225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16f80cb4d0 0x7f16f80aed00 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f16f0009e50 tx=0x7f16f001f1f0 comp rx=0 tx=0).stop 2023-10-18T10:55:25.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.945+0000 7f16fd46d640 1 -- 172.21.15.89:0/1226090225 shutdown_connections 2023-10-18T10:55:25.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.945+0000 7f16fd46d640 1 --2- 172.21.15.89:0/1226090225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16f80cb4d0 0x7f16f80aed00 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:25.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.945+0000 7f16fd46d640 1 --2- 172.21.15.89:0/1226090225 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16f80cb090 0x7f16f80ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:25.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.945+0000 7f16fd46d640 1 -- 172.21.15.89:0/1226090225 >> 172.21.15.89:0/1226090225 conn(0x7f16f806d060 msgr2=0x7f16f806d470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:55:25.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.945+0000 7f16fd46d640 1 -- 172.21.15.89:0/1226090225 shutdown_connections 2023-10-18T10:55:25.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.945+0000 7f16fd46d640 1 -- 172.21.15.89:0/1226090225 wait complete. 2023-10-18T10:55:25.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.949+0000 7f16fd46d640 1 Processor -- start 2023-10-18T10:55:25.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.949+0000 7f16fd46d640 1 -- start start 2023-10-18T10:55:25.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.949+0000 7f16fd46d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16f80cb090 0x7f16f8153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:55:25.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.949+0000 7f16fd46d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16f80cb4d0 0x7f16f8158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:55:25.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.949+0000 7f16fd46d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16f81598b0 0x7f16f8158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:55:25.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.949+0000 7f16fd46d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f81e3d30 con 0x7f16f81598b0 2023-10-18T10:55:25.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.949+0000 7f16fd46d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f815b560 con 0x7f16f80cb4d0 2023-10-18T10:55:25.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.949+0000 7f16fd46d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16f81e31b0 con 0x7f16f80cb090 2023-10-18T10:55:25.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.949+0000 7f16f7fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16f81598b0 0x7f16f8158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:55:25.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.949+0000 7f16ee7fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16f80cb4d0 0x7f16f8158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:55:25.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.949+0000 7f16f6ffd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16f80cb090 0x7f16f8153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:55:25.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.949+0000 7f16f6ffd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16f80cb090 0x7f16f8153150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:47014/0 (socket says 172.21.15.89:47014) 2023-10-18T10:55:25.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.949+0000 7f16ee7fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16f80cb4d0 0x7f16f8158030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:47028/0 (socket says 172.21.15.89:47028) 2023-10-18T10:55:25.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.949+0000 7f16f6ffd640 1 -- 172.21.15.89:0/503563053 learned_addr learned my addr 172.21.15.89:0/503563053 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:55:25.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.949+0000 7f16f7fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16f81598b0 0x7f16f8158570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49828/0 (socket says 172.21.15.89:49828) 2023-10-18T10:55:25.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.949+0000 7f16f7fff640 1 -- 172.21.15.89:0/503563053 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16f80cb090 msgr2=0x7f16f8153150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:55:25.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.949+0000 7f16f7fff640 1 --2- 172.21.15.89:0/503563053 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16f80cb090 0x7f16f8153150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:25.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.949+0000 7f16f7fff640 1 -- 172.21.15.89:0/503563053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16f80cb4d0 msgr2=0x7f16f8158030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:55:25.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.949+0000 7f16f7fff640 1 --2- 172.21.15.89:0/503563053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16f80cb4d0 0x7f16f8158030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:25.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.949+0000 7f16f7fff640 1 -- 172.21.15.89:0/503563053 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f0009b30 con 0x7f16f81598b0 2023-10-18T10:55:25.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.949+0000 7f16ee7fc640 1 --2- 172.21.15.89:0/503563053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16f80cb4d0 0x7f16f8158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:55:25.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.949+0000 7f16f7fff640 1 --2- 172.21.15.89:0/503563053 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16f81598b0 0x7f16f8158570 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f16f00ae000 tx=0x7f16f001bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:55:25.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.949+0000 7f16effff640 1 -- 172.21.15.89:0/503563053 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f00c0070 con 0x7f16f81598b0 2023-10-18T10:55:25.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.949+0000 7f16effff640 1 -- 172.21.15.89:0/503563053 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f0009510 con 0x7f16f81598b0 2023-10-18T10:55:25.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.949+0000 7f16effff640 1 -- 172.21.15.89:0/503563053 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f00afde0 con 0x7f16f81598b0 2023-10-18T10:55:25.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.949+0000 7f16fd46d640 1 -- 172.21.15.89:0/503563053 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16f81e3490 con 0x7f16f81598b0 2023-10-18T10:55:25.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.949+0000 7f16fd46d640 1 -- 172.21.15.89:0/503563053 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16f81e3a50 con 0x7f16f81598b0 2023-10-18T10:55:25.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.953+0000 7f16fd46d640 1 -- 172.21.15.89:0/503563053 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16f815b720 con 0x7f16f81598b0 2023-10-18T10:55:25.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.953+0000 7f16effff640 1 -- 172.21.15.89:0/503563053 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f16f00b6040 con 0x7f16f81598b0 2023-10-18T10:55:25.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.953+0000 7f16effff640 1 --2- 172.21.15.89:0/503563053 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f16d8065680 0x7f16d8067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:55:25.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.953+0000 7f16effff640 1 -- 172.21.15.89:0/503563053 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f16f01207e0 con 0x7f16f81598b0 2023-10-18T10:55:25.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.953+0000 7f16f6ffd640 1 --2- 172.21.15.89:0/503563053 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f16d8065680 0x7f16d8067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:55:25.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.953+0000 7f16f6ffd640 1 --2- 172.21.15.89:0/503563053 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f16d8065680 0x7f16d8067b40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f16e0001810 tx=0x7f16e0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:55:25.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:25.957+0000 7f16effff640 1 -- 172.21.15.89:0/503563053 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f16f00ee100 con 0x7f16f81598b0 2023-10-18T10:55:26.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.105+0000 7f16fd46d640 1 -- 172.21.15.89:0/503563053 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_osd_laggy_weight}] v 0) v1 -- 0x7f16f804dbe0 con 0x7f16f81598b0 2023-10-18T10:55:26.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.113+0000 7f16effff640 1 -- 172.21.15.89:0/503563053 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_osd_laggy_weight}]=0 v13)=0 v13) v1 ==== 125+0+0 (secure 0 0 0) 0x7f16f00f1720 con 0x7f16f81598b0 2023-10-18T10:55:26.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.113+0000 7f16fd46d640 1 -- 172.21.15.89:0/503563053 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f16d8065680 msgr2=0x7f16d8067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:55:26.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.113+0000 7f16fd46d640 1 --2- 172.21.15.89:0/503563053 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f16d8065680 0x7f16d8067b40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f16e0001810 tx=0x7f16e0009040 comp rx=0 tx=0).stop 2023-10-18T10:55:26.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.113+0000 7f16fd46d640 1 -- 172.21.15.89:0/503563053 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16f81598b0 msgr2=0x7f16f8158570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:55:26.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.113+0000 7f16fd46d640 1 --2- 172.21.15.89:0/503563053 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16f81598b0 0x7f16f8158570 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f16f00ae000 tx=0x7f16f001bf90 comp rx=0 tx=0).stop 2023-10-18T10:55:26.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.113+0000 7f16fd46d640 1 -- 172.21.15.89:0/503563053 shutdown_connections 2023-10-18T10:55:26.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.113+0000 7f16fd46d640 1 --2- 172.21.15.89:0/503563053 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f16d8065680 0x7f16d8067b40 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:26.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.113+0000 7f16fd46d640 1 --2- 172.21.15.89:0/503563053 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16f81598b0 0x7f16f8158570 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:26.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.113+0000 7f16fd46d640 1 --2- 172.21.15.89:0/503563053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16f80cb4d0 0x7f16f8158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:26.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.113+0000 7f16fd46d640 1 --2- 172.21.15.89:0/503563053 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f16f80cb090 0x7f16f8153150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:26.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.113+0000 7f16fd46d640 1 -- 172.21.15.89:0/503563053 >> 172.21.15.89:0/503563053 conn(0x7f16f806d060 msgr2=0x7f16f80ae970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:55:26.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.113+0000 7f16fd46d640 1 -- 172.21.15.89:0/503563053 shutdown_connections 2023-10-18T10:55:26.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.113+0000 7f16fd46d640 1 -- 172.21.15.89:0/503563053 wait complete. 2023-10-18T10:55:26.132 DEBUG:teuthology.orchestra.run.smithi089:> sudo daemon-helper kill ceph --cluster ceph -w 2023-10-18T10:55:26.136 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:55:26.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.277+0000 7f1f983e1640 1 Processor -- start 2023-10-18T10:55:26.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.277+0000 7f1f983e1640 1 -- start start 2023-10-18T10:55:26.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.277+0000 7f1f983e1640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1f9015c940 0x7f1f900d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:55:26.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.277+0000 7f1f983e1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f900d0e70 0x7f1f900cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:55:26.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.277+0000 7f1f983e1640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f9015da50 con 0x7f1f900d2f40 2023-10-18T10:55:26.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.277+0000 7f1f983e1640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f9015ce40 con 0x7f1f9015c940 2023-10-18T10:55:26.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.277+0000 7f1f983e1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9015d120 con 0x7f1f900d0e70 2023-10-18T10:55:26.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.277+0000 7f1f9697e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f900d0e70 0x7f1f900cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:55:26.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.277+0000 7f1f9617d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1f9015c940 0x7f1f900d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:55:26.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.277+0000 7f1f9697e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f900d0e70 0x7f1f900cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:47028/0 (socket says 172.21.15.89:47028) 2023-10-18T10:55:26.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.277+0000 7f1f9597c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f1f900d2f40 0x7f1f900d0760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.89:32882/0 (socket says 172.21.15.89:32882) 2023-10-18T10:55:26.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.277+0000 7f1f9697e640 1 -- 172.21.15.89:0/172414380 learned_addr learned my addr 172.21.15.89:0/172414380 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:55:26.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.277+0000 7f1f9697e640 1 -- 172.21.15.89:0/172414380 >> v2:172.21.15.154:3300/0 conn(0x7f1f9015c940 msgr2=0x7f1f900d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:55:26.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.277+0000 7f1f9697e640 1 --2- 172.21.15.89:0/172414380 >> v2:172.21.15.154:3300/0 conn(0x7f1f9015c940 0x7f1f900d2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:26.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.277+0000 7f1f9517b640 1 -- 172.21.15.89:0/172414380 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2974402083 0 0) 0x7f1f9015da50 con 0x7f1f900d2f40 2023-10-18T10:55:26.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.277+0000 7f1f9697e640 1 -- 172.21.15.89:0/172414380 >> v1:172.21.15.154:6789/0 conn(0x7f1f900d2f40 legacy=0x7f1f900d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:55:26.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.277+0000 7f1f9697e640 1 -- 172.21.15.89:0/172414380 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9015d400 con 0x7f1f900d0e70 2023-10-18T10:55:26.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.281+0000 7f1f9697e640 1 --2- 172.21.15.89:0/172414380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f900d0e70 0x7f1f900cd160 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1f88009e50 tx=0x7f1f8801c9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa2ad6b21c41e131 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:55:26.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.281+0000 7f1f9517b640 1 -- 172.21.15.89:0/172414380 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f880a5850 con 0x7f1f900d0e70 2023-10-18T10:55:26.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.281+0000 7f1f9517b640 1 -- 172.21.15.89:0/172414380 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f880a59b0 con 0x7f1f900d0e70 2023-10-18T10:55:26.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.281+0000 7f1f9517b640 1 -- 172.21.15.89:0/172414380 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f880a5d40 con 0x7f1f900d0e70 2023-10-18T10:55:26.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.281+0000 7f1f983e1640 1 -- 172.21.15.89:0/172414380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f900d0e70 msgr2=0x7f1f900cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:55:26.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.281+0000 7f1f983e1640 1 --2- 172.21.15.89:0/172414380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f900d0e70 0x7f1f900cd160 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1f88009e50 tx=0x7f1f8801c9e0 comp rx=0 tx=0).stop 2023-10-18T10:55:26.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.281+0000 7f1f983e1640 1 -- 172.21.15.89:0/172414380 shutdown_connections 2023-10-18T10:55:26.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.281+0000 7f1f983e1640 1 --2- 172.21.15.89:0/172414380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f900d0e70 0x7f1f900cd160 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:26.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.281+0000 7f1f983e1640 1 --2- 172.21.15.89:0/172414380 >> v2:172.21.15.154:3300/0 conn(0x7f1f9015c940 0x7f1f900d2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:26.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.281+0000 7f1f983e1640 1 -- 172.21.15.89:0/172414380 >> 172.21.15.89:0/172414380 conn(0x7f1f90089fd0 msgr2=0x7f1f9009b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:55:26.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.281+0000 7f1f983e1640 1 -- 172.21.15.89:0/172414380 shutdown_connections 2023-10-18T10:55:26.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.281+0000 7f1f983e1640 1 -- 172.21.15.89:0/172414380 wait complete. 2023-10-18T10:55:26.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.285+0000 7f1f983e1640 1 Processor -- start 2023-10-18T10:55:26.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.285+0000 7f1f983e1640 1 -- start start 2023-10-18T10:55:26.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.285+0000 7f1f983e1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f900d0e70 0x7f1f90154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:55:26.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.285+0000 7f1f983e1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f900b5270 0x7f1f9015b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:55:26.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.285+0000 7f1f983e1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f9006a950 0x7f1f9006ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:55:26.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.285+0000 7f1f983e1640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9006b340 con 0x7f1f900b5270 2023-10-18T10:55:26.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.285+0000 7f1f983e1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f900caf00 con 0x7f1f9006a950 2023-10-18T10:55:26.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.285+0000 7f1f983e1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f900b5700 con 0x7f1f900d0e70 2023-10-18T10:55:26.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.285+0000 7f1f9597c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f900b5270 0x7f1f9015b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:55:26.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.285+0000 7f1f9597c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f900b5270 0x7f1f9015b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49832/0 (socket says 172.21.15.89:49832) 2023-10-18T10:55:26.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.285+0000 7f1f9597c640 1 -- 172.21.15.89:0/3473178909 learned_addr learned my addr 172.21.15.89:0/3473178909 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:55:26.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.285+0000 7f1f9697e640 1 --2- 172.21.15.89:0/3473178909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f9006a950 0x7f1f9006ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:55:26.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.285+0000 7f1f9617d640 1 --2- 172.21.15.89:0/3473178909 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f900d0e70 0x7f1f90154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:55:26.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.285+0000 7f1f9597c640 1 -- 172.21.15.89:0/3473178909 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f900d0e70 msgr2=0x7f1f90154cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:55:26.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.285+0000 7f1f9597c640 1 --2- 172.21.15.89:0/3473178909 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f900d0e70 0x7f1f90154cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:26.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.285+0000 7f1f9597c640 1 -- 172.21.15.89:0/3473178909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f9006a950 msgr2=0x7f1f9006ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:55:26.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.285+0000 7f1f9597c640 1 --2- 172.21.15.89:0/3473178909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f9006a950 0x7f1f9006ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:26.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.285+0000 7f1f9597c640 1 -- 172.21.15.89:0/3473178909 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f88009b30 con 0x7f1f900b5270 2023-10-18T10:55:26.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.285+0000 7f1f9697e640 1 --2- 172.21.15.89:0/3473178909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f9006a950 0x7f1f9006ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:55:26.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.285+0000 7f1f9597c640 1 --2- 172.21.15.89:0/3473178909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f900b5270 0x7f1f9015b5f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f1f800011f0 tx=0x7f1f8000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:55:26.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.285+0000 7f1f86ffd640 1 -- 172.21.15.89:0/3473178909 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f800074e0 con 0x7f1f900b5270 2023-10-18T10:55:26.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.285+0000 7f1f86ffd640 1 -- 172.21.15.89:0/3473178909 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f80007640 con 0x7f1f900b5270 2023-10-18T10:55:26.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.285+0000 7f1f86ffd640 1 -- 172.21.15.89:0/3473178909 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f80004a10 con 0x7f1f900b5270 2023-10-18T10:55:26.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.285+0000 7f1f983e1640 1 -- 172.21.15.89:0/3473178909 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f900b59e0 con 0x7f1f900b5270 2023-10-18T10:55:26.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.285+0000 7f1f983e1640 1 -- 172.21.15.89:0/3473178909 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f900b5fa0 con 0x7f1f900b5270 2023-10-18T10:55:26.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.289+0000 7f1f983e1640 1 -- 172.21.15.89:0/3473178909 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f900cb0c0 con 0x7f1f900b5270 2023-10-18T10:55:26.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.289+0000 7f1f86ffd640 1 -- 172.21.15.89:0/3473178909 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f1f80004b70 con 0x7f1f900b5270 2023-10-18T10:55:26.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.289+0000 7f1f86ffd640 1 --2- 172.21.15.89:0/3473178909 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1f740656b0 0x7f1f74067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:55:26.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.289+0000 7f1f86ffd640 1 -- 172.21.15.89:0/3473178909 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f1f80082120 con 0x7f1f900b5270 2023-10-18T10:55:26.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.289+0000 7f1f9617d640 1 --2- 172.21.15.89:0/3473178909 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1f740656b0 0x7f1f74067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:55:26.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.293+0000 7f1f9617d640 1 --2- 172.21.15.89:0/3473178909 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1f740656b0 0x7f1f74067b70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c009a50 tx=0x7f1f8c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:55:26.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.293+0000 7f1f86ffd640 1 -- 172.21.15.89:0/3473178909 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f1f900cb0c0 con 0x7f1f900b5270 2023-10-18T10:55:26.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.305+0000 7f640fe0b640 1 Processor -- start 2023-10-18T10:55:26.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.305+0000 7f640fe0b640 1 -- start start 2023-10-18T10:55:26.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.305+0000 7f640fe0b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6408160f20 0x7f64080d0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:55:26.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.305+0000 7f640fe0b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64080b33b0 0x7f64080d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:55:26.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.305+0000 7f640fe0b640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64080cced0 con 0x7f64080d0270 2023-10-18T10:55:26.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.305+0000 7f640fe0b640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64080b1d30 con 0x7f6408160f20 2023-10-18T10:55:26.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.305+0000 7f640fe0b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64080b3bc0 con 0x7f64080b33b0 2023-10-18T10:55:26.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.305+0000 7f640e3a8640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f64080d0270 0x7f64080b1fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.89:32896/0 (socket says 172.21.15.89:32896) 2023-10-18T10:55:26.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.305+0000 7f640e3a8640 1 -- 172.21.15.89:0/1030429231 learned_addr learned my addr 172.21.15.89:0/1030429231 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:55:26.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.305+0000 7f640d3a6640 1 --2- 172.21.15.89:0/1030429231 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64080b33b0 0x7f64080d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:55:26.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.305+0000 7f640dba7640 1 --2- 172.21.15.89:0/1030429231 >> v2:172.21.15.154:3300/0 conn(0x7f6408160f20 0x7f64080d0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:55:26.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.305+0000 7f640cba5640 1 -- 172.21.15.89:0/1030429231 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2353574381 0 0) 0x7f64080cced0 con 0x7f64080d0270 2023-10-18T10:55:26.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.305+0000 7f640d3a6640 1 -- 172.21.15.89:0/1030429231 >> v2:172.21.15.154:3300/0 conn(0x7f6408160f20 msgr2=0x7f64080d0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:55:26.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.305+0000 7f640d3a6640 1 --2- 172.21.15.89:0/1030429231 >> v2:172.21.15.154:3300/0 conn(0x7f6408160f20 0x7f64080d0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:26.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.305+0000 7f640d3a6640 1 -- 172.21.15.89:0/1030429231 >> v1:172.21.15.154:6789/0 conn(0x7f64080d0270 legacy=0x7f64080b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:55:26.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.305+0000 7f640d3a6640 1 -- 172.21.15.89:0/1030429231 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64080afdd0 con 0x7f64080b33b0 2023-10-18T10:55:26.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.309+0000 7f640d3a6640 1 --2- 172.21.15.89:0/1030429231 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64080b33b0 0x7f64080d0c90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f63f80097c0 tx=0x7f63f801c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf48f3fd9ff452d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:55:26.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.309+0000 7f640cba5640 1 -- 172.21.15.89:0/1030429231 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f80ab600 con 0x7f64080b33b0 2023-10-18T10:55:26.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.309+0000 7f640cba5640 1 -- 172.21.15.89:0/1030429231 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f80a7050 con 0x7f64080b33b0 2023-10-18T10:55:26.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.309+0000 7f640cba5640 1 -- 172.21.15.89:0/1030429231 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f80ab980 con 0x7f64080b33b0 2023-10-18T10:55:26.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.309+0000 7f640fe0b640 1 -- 172.21.15.89:0/1030429231 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64080b33b0 msgr2=0x7f64080d0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:55:26.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.309+0000 7f640fe0b640 1 --2- 172.21.15.89:0/1030429231 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64080b33b0 0x7f64080d0c90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f63f80097c0 tx=0x7f63f801c3f0 comp rx=0 tx=0).stop 2023-10-18T10:55:26.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.309+0000 7f640fe0b640 1 -- 172.21.15.89:0/1030429231 shutdown_connections 2023-10-18T10:55:26.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.309+0000 7f640fe0b640 1 --2- 172.21.15.89:0/1030429231 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64080b33b0 0x7f64080d0c90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:26.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.309+0000 7f640fe0b640 1 --2- 172.21.15.89:0/1030429231 >> v2:172.21.15.154:3300/0 conn(0x7f6408160f20 0x7f64080d0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:26.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.309+0000 7f640fe0b640 1 -- 172.21.15.89:0/1030429231 >> 172.21.15.89:0/1030429231 conn(0x7f640809b930 msgr2=0x7f64080d26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:55:26.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.309+0000 7f640fe0b640 1 -- 172.21.15.89:0/1030429231 shutdown_connections 2023-10-18T10:55:26.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.309+0000 7f640fe0b640 1 -- 172.21.15.89:0/1030429231 wait complete. 2023-10-18T10:55:26.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.313+0000 7f640fe0b640 1 Processor -- start 2023-10-18T10:55:26.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.313+0000 7f640fe0b640 1 -- start start 2023-10-18T10:55:26.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.313+0000 7f640fe0b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64080b33b0 0x7f64080c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:55:26.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.313+0000 7f640fe0b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64080d0270 0x7f64080cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:55:26.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.313+0000 7f640fe0b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6408160f20 0x7f64080ce270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:55:26.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.313+0000 7f640fe0b640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64080c6d20 con 0x7f64080b33b0 2023-10-18T10:55:26.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.313+0000 7f640fe0b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6408160540 con 0x7f64080d0270 2023-10-18T10:55:26.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.313+0000 7f640fe0b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64080c66d0 con 0x7f6408160f20 2023-10-18T10:55:26.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.313+0000 7f640dba7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64080b33b0 0x7f64080c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:55:26.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.313+0000 7f640dba7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64080b33b0 0x7f64080c8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49848/0 (socket says 172.21.15.89:49848) 2023-10-18T10:55:26.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.313+0000 7f640dba7640 1 -- 172.21.15.89:0/3339988605 learned_addr learned my addr 172.21.15.89:0/3339988605 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:55:26.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.313+0000 7f640e3a8640 1 --2- 172.21.15.89:0/3339988605 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6408160f20 0x7f64080ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:55:26.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.313+0000 7f640d3a6640 1 --2- 172.21.15.89:0/3339988605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64080d0270 0x7f64080cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:55:26.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.313+0000 7f640dba7640 1 -- 172.21.15.89:0/3339988605 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6408160f20 msgr2=0x7f64080ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:55:26.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.313+0000 7f640dba7640 1 --2- 172.21.15.89:0/3339988605 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6408160f20 0x7f64080ce270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:26.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.313+0000 7f640dba7640 1 -- 172.21.15.89:0/3339988605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64080d0270 msgr2=0x7f64080cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:55:26.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.313+0000 7f640dba7640 1 --2- 172.21.15.89:0/3339988605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64080d0270 0x7f64080cdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:26.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.313+0000 7f640dba7640 1 -- 172.21.15.89:0/3339988605 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f8009420 con 0x7f64080b33b0 2023-10-18T10:55:26.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.313+0000 7f640dba7640 1 --2- 172.21.15.89:0/3339988605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64080b33b0 0x7f64080c8b10 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f640400e870 tx=0x7f640400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:55:26.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.313+0000 7f63fe7fc640 1 -- 172.21.15.89:0/3339988605 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6404009510 con 0x7f64080b33b0 2023-10-18T10:55:26.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.313+0000 7f63fe7fc640 1 -- 172.21.15.89:0/3339988605 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f640400eec0 con 0x7f64080b33b0 2023-10-18T10:55:26.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.313+0000 7f63fe7fc640 1 -- 172.21.15.89:0/3339988605 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640400cb40 con 0x7f64080b33b0 2023-10-18T10:55:26.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.313+0000 7f640fe0b640 1 -- 172.21.15.89:0/3339988605 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64080c69b0 con 0x7f64080b33b0 2023-10-18T10:55:26.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.313+0000 7f640fe0b640 1 -- 172.21.15.89:0/3339988605 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f640815f7d0 con 0x7f64080b33b0 2023-10-18T10:55:26.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.317+0000 7f640fe0b640 1 -- 172.21.15.89:0/3339988605 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f64080c6530 con 0x7f64080b33b0 2023-10-18T10:55:26.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.317+0000 7f63fe7fc640 1 -- 172.21.15.89:0/3339988605 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f640400cca0 con 0x7f64080b33b0 2023-10-18T10:55:26.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.317+0000 7f63fe7fc640 1 --2- 172.21.15.89:0/3339988605 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f63ec065680 0x7f63ec067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:55:26.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.317+0000 7f63fe7fc640 1 -- 172.21.15.89:0/3339988605 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f6404083350 con 0x7f64080b33b0 2023-10-18T10:55:26.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.317+0000 7f63fe7fc640 1 -- 172.21.15.89:0/3339988605 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+665 (secure 0 0 0) 0x7f64040836e0 con 0x7f64080b33b0 2023-10-18T10:55:26.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.317+0000 7f640d3a6640 1 --2- 172.21.15.89:0/3339988605 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f63ec065680 0x7f63ec067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:55:26.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.317+0000 7f640fe0b640 1 -- 172.21.15.89:0/3339988605 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f64080c6530 con 0x7f64080b33b0 2023-10-18T10:55:26.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.317+0000 7f63fe7fc640 1 -- 172.21.15.89:0/3339988605 <== mon.0 v2:172.21.15.89:3300/0 7 ==== log(2 entries from seq 494 at 2023-10-18T10:55:23.910234+0000) v1 ==== 510+0+0 (secure 0 0 0) 0x7f6404058640 con 0x7f64080b33b0 2023-10-18T10:55:26.322 INFO:teuthology.orchestra.run.smithi089.stdout: cluster: 2023-10-18T10:55:26.323 INFO:teuthology.orchestra.run.smithi089.stdout: id: 1352f62a-2330-4e96-afcd-b8e87cd82667 2023-10-18T10:55:26.323 INFO:teuthology.orchestra.run.smithi089.stdout: health: HEALTH_WARN 2023-10-18T10:55:26.323 INFO:teuthology.orchestra.run.smithi089.stdout: Reduced data availability: 4 pgs inactive, 4 pgs peering 2023-10-18T10:55:26.323 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:55:26.323 INFO:teuthology.orchestra.run.smithi089.stdout: services: 2023-10-18T10:55:26.324 INFO:teuthology.orchestra.run.smithi089.stdout: mon: 3 daemons, quorum a,b,c (age 9m) 2023-10-18T10:55:26.324 INFO:teuthology.orchestra.run.smithi089.stdout: mgr: x(active, since 9m), standbys: y 2023-10-18T10:55:26.324 INFO:teuthology.orchestra.run.smithi089.stdout: mds: 1/1 daemons up, 3 standby 2023-10-18T10:55:26.324 INFO:teuthology.orchestra.run.smithi089.stdout: osd: 8 osds: 8 up (since 4s), 8 in (since 9m) 2023-10-18T10:55:26.324 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:55:26.325 INFO:teuthology.orchestra.run.smithi089.stdout: data: 2023-10-18T10:55:26.325 INFO:teuthology.orchestra.run.smithi089.stdout: volumes: 1/1 healthy 2023-10-18T10:55:26.325 INFO:teuthology.orchestra.run.smithi089.stdout: pools: 4 pools, 105 pgs 2023-10-18T10:55:26.325 INFO:teuthology.orchestra.run.smithi089.stdout: objects: 24 objects, 580 KiB 2023-10-18T10:55:26.325 INFO:teuthology.orchestra.run.smithi089.stdout: usage: 218 MiB used, 720 GiB / 720 GiB avail 2023-10-18T10:55:26.326 INFO:teuthology.orchestra.run.smithi089.stdout: pgs: 20.952% pgs not active 2023-10-18T10:55:26.326 INFO:teuthology.orchestra.run.smithi089.stdout: 74 active+clean 2023-10-18T10:55:26.326 INFO:teuthology.orchestra.run.smithi089.stdout: 22 peering 2023-10-18T10:55:26.326 INFO:teuthology.orchestra.run.smithi089.stdout: 9 active+undersized 2023-10-18T10:55:26.327 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:55:26.327 INFO:teuthology.orchestra.run.smithi089.stdout: io: 2023-10-18T10:55:26.327 INFO:teuthology.orchestra.run.smithi089.stdout: client: 341 B/s wr, 0 op/s rd, 0 op/s wr 2023-10-18T10:55:26.327 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:55:26.327 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:55:26.328 INFO:teuthology.orchestra.run.smithi089.stdout:2023-10-18T10:55:23.910234+0000 mon.a [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4/48 objects degraded (8.333%), 2 pgs degraded) 2023-10-18T10:55:26.328 INFO:teuthology.orchestra.run.smithi089.stdout:2023-10-18T10:55:24.890281+0000 mon.a [WRN] Health check update: Reduced data availability: 4 pgs inactive, 4 pgs peering (PG_AVAILABILITY) 2023-10-18T10:55:26.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.317+0000 7f640d3a6640 1 --2- 172.21.15.89:0/3339988605 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f63ec065680 0x7f63ec067b40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f63f80ad000 tx=0x7f63f80a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:55:26.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.477+0000 7f1f983e1640 1 -- 172.21.15.89:0/3473178909 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f900cb0c0 con 0x7f1f900b5270 2023-10-18T10:55:26.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.477+0000 7f1f86ffd640 1 -- 172.21.15.89:0/3473178909 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4877 (secure 0 0 0) 0x7f1f900cb0c0 con 0x7f1f900b5270 2023-10-18T10:55:26.476 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:55:26.477 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/654729641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":654729641},{"type":"v1","addr":"172.21.15.154:6836","nonce":654729641}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/964566480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":964566480},{"type":"v1","addr":"172.21.15.154:6837","nonce":964566480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2261878538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2261878538},{"type":"v1","addr":"172.21.15.89:6833","nonce":2261878538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:47:15.199575+0000","modified":"2023-10-18T10:55:20.892655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5517":{"gid":5517,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":124,"addr":"172.21.15.89:6835/2092274731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2092274731},{"type":"v1","addr":"172.21.15.89:6835","nonce":2092274731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T10:55:26.478 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-18T10:55:26.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.477+0000 7f1f983e1640 1 -- 172.21.15.89:0/3473178909 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1f740656b0 msgr2=0x7f1f74067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:55:26.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.477+0000 7f1f983e1640 1 --2- 172.21.15.89:0/3473178909 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1f740656b0 0x7f1f74067b70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c009a50 tx=0x7f1f8c008040 comp rx=0 tx=0).stop 2023-10-18T10:55:26.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.477+0000 7f1f983e1640 1 -- 172.21.15.89:0/3473178909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f900b5270 msgr2=0x7f1f9015b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:55:26.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.477+0000 7f1f983e1640 1 --2- 172.21.15.89:0/3473178909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f900b5270 0x7f1f9015b5f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f1f800011f0 tx=0x7f1f8000c9b0 comp rx=0 tx=0).stop 2023-10-18T10:55:26.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.477+0000 7f1f983e1640 1 -- 172.21.15.89:0/3473178909 shutdown_connections 2023-10-18T10:55:26.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.477+0000 7f1f983e1640 1 --2- 172.21.15.89:0/3473178909 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1f740656b0 0x7f1f74067b70 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:26.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.477+0000 7f1f983e1640 1 --2- 172.21.15.89:0/3473178909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f9006a950 0x7f1f9006ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:26.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.477+0000 7f1f983e1640 1 --2- 172.21.15.89:0/3473178909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f900b5270 0x7f1f9015b5f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:26.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.477+0000 7f1f983e1640 1 --2- 172.21.15.89:0/3473178909 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f900d0e70 0x7f1f90154cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:55:26.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.477+0000 7f1f983e1640 1 -- 172.21.15.89:0/3473178909 >> 172.21.15.89:0/3473178909 conn(0x7f1f90089fd0 msgr2=0x7f1f900d0760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:55:26.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.477+0000 7f1f983e1640 1 -- 172.21.15.89:0/3473178909 shutdown_connections 2023-10-18T10:55:26.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.477+0000 7f1f983e1640 1 -- 172.21.15.89:0/3473178909 wait complete. 2023-10-18T10:55:26.497 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-18T10:55:26.654 INFO:teuthology.orchestra.run.smithi089.stdout:[{"id":5183,"entity":{"name":{"type":"client","num":5183},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":4272238049}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":482.94223579300001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.32764207635647902,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":3,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5183 v1:192.168.0.1:0/4272238049","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi154","kernel_version":"6.6.0-rc5-g47672119149b","root":"/"}}] 2023-10-18T10:55:26.654 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "1", "hostname": "smithi154", "kernel_version": "6.6.0-rc5-g47672119149b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 4272238049, "type": "v1" }, "name": { "num": 5183, "type": "client" } }, "id": 5183, "inst": "client.5183 v1:192.168.0.1:0/4272238049", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 3, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.327642076356479 }, "state": "open", "uptime": 482.942235793 } ] 2023-10-18T10:55:26.756 DEBUG:tasks.ceph_test_case:No log hits yet, waiting... 2023-10-18T10:55:26.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:26.921+0000 7f63fe7fc640 1 -- 172.21.15.89:0/3339988605 <== mon.0 v2:172.21.15.89:3300/0 8 ==== log(1 entries from seq 496 at 2023-10-18T10:55:26.116417+0000) v1 ==== 231+0+0 (secure 0 0 0) 0x7f64040830d0 con 0x7f64080b33b0 2023-10-18T10:55:27.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:55:27.973+0000 7f63fe7fc640 1 -- 172.21.15.89:0/3339988605 <== mon.0 v2:172.21.15.89:3300/0 9 ==== log(2 entries from seq 499 at 2023-10-18T10:55:27.923379+0000) v1 ==== 425+0+0 (secure 0 0 0) 0x7f6404052a70 con 0x7f64080b33b0 2023-10-18T10:55:27.972 INFO:teuthology.orchestra.run.smithi089.stdout:2023-10-18T10:55:27.923379+0000 mon.a [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs inactive, 4 pgs peering) 2023-10-18T10:55:27.972 INFO:teuthology.orchestra.run.smithi089.stdout:2023-10-18T10:55:27.923446+0000 mon.a [INF] Cluster is now healthy 2023-10-18T10:55:50.380 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:55:50.384 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:56:01.774 ERROR:tasks.ceph_test_case:Log output: cluster: id: 1352f62a-2330-4e96-afcd-b8e87cd82667 health: HEALTH_WARN Reduced data availability: 4 pgs inactive, 4 pgs peering services: mon: 3 daemons, quorum a,b,c (age 9m) mgr: x(active, since 9m), standbys: y mds: 1/1 daemons up, 3 standby osd: 8 osds: 8 up (since 4s), 8 in (since 9m) data: volumes: 1/1 healthy pools: 4 pools, 105 pgs objects: 24 objects, 580 KiB usage: 218 MiB used, 720 GiB / 720 GiB avail pgs: 20.952% pgs not active 74 active+clean 22 peering 9 active+undersized io: client: 341 B/s wr, 0 op/s rd, 0 op/s wr 2023-10-18T10:55:23.910234+0000 mon.a [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4/48 objects degraded (8.333%), 2 pgs degraded) 2023-10-18T10:55:24.890281+0000 mon.a [WRN] Health check update: Reduced data availability: 4 pgs inactive, 4 pgs peering (PG_AVAILABILITY) 2023-10-18T10:55:27.923379+0000 mon.a [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs inactive, 4 pgs peering) 2023-10-18T10:55:27.923446+0000 mon.a [INF] Cluster is now healthy 2023-10-18T10:56:01.774 DEBUG:tasks.cephfs.test_client_recovery:Expected log message found: 'Health check cleared: MDS_CLIENTS_LAGGY', retrying 2023-10-18T10:56:06.776 DEBUG:teuthology.orchestra.run.smithi089:> sudo daemon-helper kill ceph --cluster ceph -w 2023-10-18T10:56:06.782 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:56:06.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.932+0000 7f81855e1640 1 Processor -- start 2023-10-18T10:56:06.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.932+0000 7f81855e1640 1 -- start start 2023-10-18T10:56:06.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.932+0000 7f81855e1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81800d1a60 0x7f81800cee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:56:06.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.932+0000 7f81855e1640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f81800cf390 0x7f81800b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:56:06.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.932+0000 7f81855e1640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81800cf800 con 0x7f818015c570 2023-10-18T10:56:06.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.932+0000 7f81855e1640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81800d0c70 con 0x7f81800cf390 2023-10-18T10:56:06.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.932+0000 7f81855e1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81800d0e60 con 0x7f81800d1a60 2023-10-18T10:56:06.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.932+0000 7f817f7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f81800cf390 0x7f81800b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:56:06.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.932+0000 7f817e7fc640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81800d1a60 0x7f81800cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:56:06.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.932+0000 7f817f7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f81800cf390 0x7f81800b5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:59100/0 (socket says 172.21.15.89:59100) 2023-10-18T10:56:06.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.932+0000 7f817f7fe640 1 -- 172.21.15.89:0/19733145 learned_addr learned my addr 172.21.15.89:0/19733145 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:56:06.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.932+0000 7f817f7fe640 1 -- 172.21.15.89:0/19733145 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81800d1a60 msgr2=0x7f81800cee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:56:06.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.932+0000 7f817f7fe640 1 --2- 172.21.15.89:0/19733145 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81800d1a60 0x7f81800cee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:56:06.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.932+0000 7f817f7fe640 1 -- 172.21.15.89:0/19733145 >> v1:172.21.15.154:6789/0 conn(0x7f818015c570 legacy=0x7f81800d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:56:06.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.932+0000 7f817f7fe640 1 -- 172.21.15.89:0/19733145 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81800d1140 con 0x7f81800cf390 2023-10-18T10:56:06.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.932+0000 7f817e7fc640 1 --2- 172.21.15.89:0/19733145 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81800d1a60 0x7f81800cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:56:06.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.936+0000 7f817f7fe640 1 --2- 172.21.15.89:0/19733145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81800cf390 0x7f81800b5d00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8178009e50 tx=0x7f817801c9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17540b8a40338f1e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:56:06.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.936+0000 7f817dffb640 1 -- 172.21.15.89:0/19733145 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81780a58a0 con 0x7f81800cf390 2023-10-18T10:56:06.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.936+0000 7f817dffb640 1 -- 172.21.15.89:0/19733145 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81780a5a00 con 0x7f81800cf390 2023-10-18T10:56:06.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.936+0000 7f817dffb640 1 -- 172.21.15.89:0/19733145 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81780a5d90 con 0x7f81800cf390 2023-10-18T10:56:06.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.936+0000 7f81855e1640 1 -- 172.21.15.89:0/19733145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81800cf390 msgr2=0x7f81800b5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:56:06.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.936+0000 7f81855e1640 1 --2- 172.21.15.89:0/19733145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81800cf390 0x7f81800b5d00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8178009e50 tx=0x7f817801c9e0 comp rx=0 tx=0).stop 2023-10-18T10:56:06.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.936+0000 7f81855e1640 1 -- 172.21.15.89:0/19733145 shutdown_connections 2023-10-18T10:56:06.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.936+0000 7f81855e1640 1 --2- 172.21.15.89:0/19733145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81800cf390 0x7f81800b5d00 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:56:06.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.936+0000 7f81855e1640 1 --2- 172.21.15.89:0/19733145 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81800d1a60 0x7f81800cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:56:06.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.936+0000 7f81855e1640 1 -- 172.21.15.89:0/19733145 >> 172.21.15.89:0/19733145 conn(0x7f8180070320 msgr2=0x7f8180070730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:56:06.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.936+0000 7f81855e1640 1 -- 172.21.15.89:0/19733145 shutdown_connections 2023-10-18T10:56:06.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.936+0000 7f81855e1640 1 -- 172.21.15.89:0/19733145 wait complete. 2023-10-18T10:56:06.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.940+0000 7f81855e1640 1 Processor -- start 2023-10-18T10:56:06.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.940+0000 7f81855e1640 1 -- start start 2023-10-18T10:56:06.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.940+0000 7f81855e1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81800cf390 0x7f81800b3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:56:06.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.940+0000 7f81855e1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81800d1a60 0x7f8180155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:56:06.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.940+0000 7f81855e1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f818015c570 0x7f81800b4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:56:06.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.940+0000 7f81855e1640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8180155bb0 con 0x7f81800cf390 2023-10-18T10:56:06.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.940+0000 7f81855e1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81800b4d70 con 0x7f818015c570 2023-10-18T10:56:06.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.940+0000 7f81855e1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81800cc930 con 0x7f81800d1a60 2023-10-18T10:56:06.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.940+0000 7f817effd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81800cf390 0x7f81800b3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:56:06.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.940+0000 7f817effd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81800cf390 0x7f81800b3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46304/0 (socket says 172.21.15.89:46304) 2023-10-18T10:56:06.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.940+0000 7f817effd640 1 -- 172.21.15.89:0/1829686954 learned_addr learned my addr 172.21.15.89:0/1829686954 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:56:06.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.940+0000 7f817f7fe640 1 --2- 172.21.15.89:0/1829686954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f818015c570 0x7f81800b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:56:06.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.940+0000 7f817e7fc640 1 --2- 172.21.15.89:0/1829686954 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81800d1a60 0x7f8180155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:56:06.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.940+0000 7f817effd640 1 -- 172.21.15.89:0/1829686954 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81800d1a60 msgr2=0x7f8180155670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:56:06.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.940+0000 7f817effd640 1 --2- 172.21.15.89:0/1829686954 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81800d1a60 0x7f8180155670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:56:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.940+0000 7f817effd640 1 -- 172.21.15.89:0/1829686954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f818015c570 msgr2=0x7f81800b4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:56:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.940+0000 7f817effd640 1 --2- 172.21.15.89:0/1829686954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f818015c570 0x7f81800b4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:56:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.940+0000 7f817effd640 1 -- 172.21.15.89:0/1829686954 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8174009420 con 0x7f81800cf390 2023-10-18T10:56:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.940+0000 7f817effd640 1 --2- 172.21.15.89:0/1829686954 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81800cf390 0x7f81800b3060 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f817000c500 tx=0x7f817000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:56:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.940+0000 7f81677fe640 1 -- 172.21.15.89:0/1829686954 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8170005220 con 0x7f81800cf390 2023-10-18T10:56:06.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.940+0000 7f81677fe640 1 -- 172.21.15.89:0/1829686954 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8170005380 con 0x7f81800cf390 2023-10-18T10:56:06.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.940+0000 7f81855e1640 1 -- 172.21.15.89:0/1829686954 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8178009b30 con 0x7f81800cf390 2023-10-18T10:56:06.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.940+0000 7f81677fe640 1 -- 172.21.15.89:0/1829686954 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8170007270 con 0x7f81800cf390 2023-10-18T10:56:06.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.940+0000 7f81855e1640 1 -- 172.21.15.89:0/1829686954 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f818015cad0 con 0x7f81800cf390 2023-10-18T10:56:06.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.944+0000 7f81855e1640 1 -- 172.21.15.89:0/1829686954 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8180073b50 con 0x7f81800cf390 2023-10-18T10:56:06.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.944+0000 7f81677fe640 1 -- 172.21.15.89:0/1829686954 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f81700073d0 con 0x7f81800cf390 2023-10-18T10:56:06.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.944+0000 7f81677fe640 1 --2- 172.21.15.89:0/1829686954 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f816c0656b0 0x7f816c067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:56:06.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.944+0000 7f81677fe640 1 -- 172.21.15.89:0/1829686954 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f81700826f0 con 0x7f81800cf390 2023-10-18T10:56:06.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.944+0000 7f817e7fc640 1 --2- 172.21.15.89:0/1829686954 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f816c0656b0 0x7f816c067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:56:06.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.944+0000 7f817e7fc640 1 --2- 172.21.15.89:0/1829686954 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f816c0656b0 0x7f816c067b70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f818015ce80 tx=0x7f8174009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:56:06.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.948+0000 7f81677fe640 1 -- 172.21.15.89:0/1829686954 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f81700500c0 con 0x7f81800cf390 2023-10-18T10:56:06.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.960+0000 7f6217ebd640 1 Processor -- start 2023-10-18T10:56:06.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.960+0000 7f6217ebd640 1 -- start start 2023-10-18T10:56:06.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.960+0000 7f6217ebd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6210160f20 0x7f62100d0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:56:06.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.960+0000 7f6217ebd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f62100b15e0 0x7f62100d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:56:06.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.960+0000 7f6217ebd640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62100cced0 con 0x7f62100b33b0 2023-10-18T10:56:06.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.960+0000 7f6217ebd640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62100b1d30 con 0x7f6210160f20 2023-10-18T10:56:06.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.960+0000 7f6217ebd640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62100b3bc0 con 0x7f62100b15e0 2023-10-18T10:56:06.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.960+0000 7f6215c59640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6210160f20 0x7f62100d0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:56:06.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.960+0000 7f6215458640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f62100b33b0 0x7f62100b1fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48176/0 (socket says 172.21.15.89:48176) 2023-10-18T10:56:06.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.960+0000 7f6215458640 1 -- 172.21.15.89:0/2437198902 learned_addr learned my addr 172.21.15.89:0/2437198902 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:56:06.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.960+0000 7f621645a640 1 --2- 172.21.15.89:0/2437198902 >> v2:172.21.15.154:3300/0 conn(0x7f62100b15e0 0x7f62100d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:56:06.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.960+0000 7f6214c57640 1 -- 172.21.15.89:0/2437198902 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3414064481 0 0) 0x7f62100cced0 con 0x7f62100b33b0 2023-10-18T10:56:06.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.960+0000 7f6214c57640 1 -- 172.21.15.89:0/2437198902 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61fc003530 con 0x7f62100b33b0 2023-10-18T10:56:06.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.960+0000 7f6215c59640 1 -- 172.21.15.89:0/2437198902 >> v2:172.21.15.154:3300/0 conn(0x7f62100b15e0 msgr2=0x7f62100d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:56:06.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.960+0000 7f6215c59640 1 --2- 172.21.15.89:0/2437198902 >> v2:172.21.15.154:3300/0 conn(0x7f62100b15e0 0x7f62100d0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:56:06.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.960+0000 7f6215c59640 1 -- 172.21.15.89:0/2437198902 >> v1:172.21.15.89:6789/0 conn(0x7f62100b33b0 legacy=0x7f62100b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:56:06.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.960+0000 7f6215c59640 1 -- 172.21.15.89:0/2437198902 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62100afdd0 con 0x7f6210160f20 2023-10-18T10:56:06.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.964+0000 7f6215c59640 1 --2- 172.21.15.89:0/2437198902 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6210160f20 0x7f62100d0750 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f621008b170 tx=0x7f620c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=662080d9a7563c5f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:56:06.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.964+0000 7f6214c57640 1 -- 172.21.15.89:0/2437198902 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620c0ab600 con 0x7f6210160f20 2023-10-18T10:56:06.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.964+0000 7f6214c57640 1 -- 172.21.15.89:0/2437198902 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f620c0a7050 con 0x7f6210160f20 2023-10-18T10:56:06.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.964+0000 7f6214c57640 1 -- 172.21.15.89:0/2437198902 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620c0ab960 con 0x7f6210160f20 2023-10-18T10:56:06.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.964+0000 7f6217ebd640 1 -- 172.21.15.89:0/2437198902 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6210160f20 msgr2=0x7f62100d0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:56:06.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.964+0000 7f6217ebd640 1 --2- 172.21.15.89:0/2437198902 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6210160f20 0x7f62100d0750 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f621008b170 tx=0x7f620c01c3d0 comp rx=0 tx=0).stop 2023-10-18T10:56:06.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.964+0000 7f6217ebd640 1 -- 172.21.15.89:0/2437198902 shutdown_connections 2023-10-18T10:56:06.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.964+0000 7f6217ebd640 1 --2- 172.21.15.89:0/2437198902 >> v2:172.21.15.154:3300/0 conn(0x7f62100b15e0 0x7f62100d0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:56:06.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.964+0000 7f6217ebd640 1 --2- 172.21.15.89:0/2437198902 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6210160f20 0x7f62100d0750 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:56:06.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.964+0000 7f6217ebd640 1 -- 172.21.15.89:0/2437198902 >> 172.21.15.89:0/2437198902 conn(0x7f621009b930 msgr2=0x7f62100d26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:56:06.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.964+0000 7f6217ebd640 1 -- 172.21.15.89:0/2437198902 shutdown_connections 2023-10-18T10:56:06.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.964+0000 7f6217ebd640 1 -- 172.21.15.89:0/2437198902 wait complete. 2023-10-18T10:56:06.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.968+0000 7f6217ebd640 1 Processor -- start 2023-10-18T10:56:06.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.968+0000 7f6217ebd640 1 -- start start 2023-10-18T10:56:06.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.968+0000 7f6217ebd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62100b15e0 0x7f62100c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:56:06.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.968+0000 7f6217ebd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62100b33b0 0x7f621015f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:56:06.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.968+0000 7f6217ebd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6210160f20 0x7f621006abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:56:06.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.968+0000 7f6217ebd640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62100c8ca0 con 0x7f6210160f20 2023-10-18T10:56:06.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.968+0000 7f6217ebd640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62100ca940 con 0x7f62100b33b0 2023-10-18T10:56:06.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.968+0000 7f6217ebd640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62100c9d30 con 0x7f62100b15e0 2023-10-18T10:56:06.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.968+0000 7f621645a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6210160f20 0x7f621006abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:56:06.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.968+0000 7f621645a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6210160f20 0x7f621006abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46318/0 (socket says 172.21.15.89:46318) 2023-10-18T10:56:06.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.968+0000 7f621645a640 1 -- 172.21.15.89:0/2538481448 learned_addr learned my addr 172.21.15.89:0/2538481448 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:56:06.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.968+0000 7f6215458640 1 --2- 172.21.15.89:0/2538481448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62100b33b0 0x7f621015f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:56:06.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.968+0000 7f6215c59640 1 --2- 172.21.15.89:0/2538481448 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62100b15e0 0x7f62100c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:56:06.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.968+0000 7f621645a640 1 -- 172.21.15.89:0/2538481448 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62100b15e0 msgr2=0x7f62100c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:56:06.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.968+0000 7f621645a640 1 --2- 172.21.15.89:0/2538481448 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62100b15e0 0x7f62100c8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:56:06.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.968+0000 7f621645a640 1 -- 172.21.15.89:0/2538481448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62100b33b0 msgr2=0x7f621015f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:56:06.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.968+0000 7f621645a640 1 --2- 172.21.15.89:0/2538481448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62100b33b0 0x7f621015f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:56:06.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.968+0000 7f621645a640 1 -- 172.21.15.89:0/2538481448 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f620c009420 con 0x7f6210160f20 2023-10-18T10:56:06.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.968+0000 7f6215c59640 1 --2- 172.21.15.89:0/2538481448 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62100b15e0 0x7f62100c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:56:06.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.968+0000 7f621645a640 1 --2- 172.21.15.89:0/2538481448 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6210160f20 0x7f621006abf0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6208007e60 tx=0x7f620800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:56:06.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.968+0000 7f62067fc640 1 -- 172.21.15.89:0/2538481448 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6208014620 con 0x7f6210160f20 2023-10-18T10:56:06.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.968+0000 7f62067fc640 1 -- 172.21.15.89:0/2538481448 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62080147a0 con 0x7f6210160f20 2023-10-18T10:56:06.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.968+0000 7f62067fc640 1 -- 172.21.15.89:0/2538481448 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6208014b50 con 0x7f6210160f20 2023-10-18T10:56:06.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.968+0000 7f6217ebd640 1 -- 172.21.15.89:0/2538481448 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62100ca010 con 0x7f6210160f20 2023-10-18T10:56:06.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.968+0000 7f6217ebd640 1 -- 172.21.15.89:0/2538481448 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62100ca5d0 con 0x7f6210160f20 2023-10-18T10:56:06.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.968+0000 7f6217ebd640 1 -- 172.21.15.89:0/2538481448 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f62100b5d60 con 0x7f6210160f20 2023-10-18T10:56:06.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.972+0000 7f62067fc640 1 -- 172.21.15.89:0/2538481448 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f6208014cb0 con 0x7f6210160f20 2023-10-18T10:56:06.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.972+0000 7f62067fc640 1 --2- 172.21.15.89:0/2538481448 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f61f4065680 0x7f61f4067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:56:06.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.972+0000 7f62067fc640 1 -- 172.21.15.89:0/2538481448 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f62080843c0 con 0x7f6210160f20 2023-10-18T10:56:06.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.972+0000 7f62067fc640 1 -- 172.21.15.89:0/2538481448 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+450 (secure 0 0 0) 0x7f6208084830 con 0x7f6210160f20 2023-10-18T10:56:06.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.972+0000 7f6215c59640 1 --2- 172.21.15.89:0/2538481448 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f61f4065680 0x7f61f4067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:56:06.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.972+0000 7f6217ebd640 1 -- 172.21.15.89:0/2538481448 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f621004e430 con 0x7f6210160f20 2023-10-18T10:56:06.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:06.972+0000 7f6215c59640 1 --2- 172.21.15.89:0/2538481448 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f61f4065680 0x7f61f4067b40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f620c0ad000 tx=0x7f620c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:56:07.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:07.132+0000 7f81855e1640 1 -- 172.21.15.89:0/1829686954 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81800cca70 con 0x7f81800cf390 2023-10-18T10:56:07.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:07.136+0000 7f81677fe640 1 -- 172.21.15.89:0/1829686954 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4877 (secure 0 0 0) 0x7f81800cca70 con 0x7f81800cf390 2023-10-18T10:56:07.135 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:56:07.136 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/654729641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":654729641},{"type":"v1","addr":"172.21.15.154:6836","nonce":654729641}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/964566480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":964566480},{"type":"v1","addr":"172.21.15.154:6837","nonce":964566480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2261878538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2261878538},{"type":"v1","addr":"172.21.15.89:6833","nonce":2261878538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:47:15.199575+0000","modified":"2023-10-18T10:55:20.892655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5517":{"gid":5517,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":124,"addr":"172.21.15.89:6835/2092274731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2092274731},{"type":"v1","addr":"172.21.15.89:6835","nonce":2092274731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T10:56:07.137 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-18T10:56:07.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:07.136+0000 7f81855e1640 1 -- 172.21.15.89:0/1829686954 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f816c0656b0 msgr2=0x7f816c067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:56:07.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:07.136+0000 7f81855e1640 1 --2- 172.21.15.89:0/1829686954 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f816c0656b0 0x7f816c067b70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f818015ce80 tx=0x7f8174009190 comp rx=0 tx=0).stop 2023-10-18T10:56:07.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:07.136+0000 7f81855e1640 1 -- 172.21.15.89:0/1829686954 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81800cf390 msgr2=0x7f81800b3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:56:07.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:07.136+0000 7f81855e1640 1 --2- 172.21.15.89:0/1829686954 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81800cf390 0x7f81800b3060 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f817000c500 tx=0x7f817000c9c0 comp rx=0 tx=0).stop 2023-10-18T10:56:07.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:07.136+0000 7f81855e1640 1 -- 172.21.15.89:0/1829686954 shutdown_connections 2023-10-18T10:56:07.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:07.136+0000 7f81855e1640 1 --2- 172.21.15.89:0/1829686954 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f816c0656b0 0x7f816c067b70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:56:07.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:07.136+0000 7f81855e1640 1 --2- 172.21.15.89:0/1829686954 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f81800d1a60 0x7f8180155670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:56:07.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:07.136+0000 7f81855e1640 1 --2- 172.21.15.89:0/1829686954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f818015c570 0x7f81800b4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:56:07.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:07.136+0000 7f81855e1640 1 --2- 172.21.15.89:0/1829686954 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81800cf390 0x7f81800b3060 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:56:07.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:07.136+0000 7f81855e1640 1 -- 172.21.15.89:0/1829686954 >> 172.21.15.89:0/1829686954 conn(0x7f8180070320 msgr2=0x7f8180070700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:56:07.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:07.136+0000 7f81855e1640 1 -- 172.21.15.89:0/1829686954 shutdown_connections 2023-10-18T10:56:07.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:07.136+0000 7f81855e1640 1 -- 172.21.15.89:0/1829686954 wait complete. 2023-10-18T10:56:07.156 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-18T10:56:07.313 INFO:teuthology.orchestra.run.smithi089.stdout:[{"id":5183,"entity":{"name":{"type":"client","num":5183},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":4272238049}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":523.60137607199999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.29826406564221658,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":3,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5183 v1:192.168.0.1:0/4272238049","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi154","kernel_version":"6.6.0-rc5-g47672119149b","root":"/"}}] 2023-10-18T10:56:07.314 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "1", "hostname": "smithi154", "kernel_version": "6.6.0-rc5-g47672119149b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 4272238049, "type": "v1" }, "name": { "num": 5183, "type": "client" } }, "id": 5183, "inst": "client.5183 v1:192.168.0.1:0/4272238049", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 3, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.2982640656422166 }, "state": "open", "uptime": 523.601376072 } ] 2023-10-18T10:56:07.415 DEBUG:tasks.ceph_test_case:No log hits yet, waiting... 2023-10-18T10:56:20.504 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:56:20.510 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:56:42.433 ERROR:tasks.ceph_test_case:Log output: 2023-10-18T10:56:42.435 DEBUG:tasks.cephfs.test_client_recovery:Expected log message found: 'Health check cleared: MDS_CLIENTS_LAGGY', retrying 2023-10-18T10:56:47.436 DEBUG:teuthology.orchestra.run.smithi089:> sudo daemon-helper kill ceph --cluster ceph -w 2023-10-18T10:56:47.441 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:56:47.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.591+0000 7fd91629c640 1 Processor -- start 2023-10-18T10:56:47.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.591+0000 7fd91629c640 1 -- start start 2023-10-18T10:56:47.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.591+0000 7fd91629c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd91015c940 0x7fd9100d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:56:47.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.591+0000 7fd91629c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd9100d2f40 0x7fd9100d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:56:47.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.595+0000 7fd91629c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd91015da50 con 0x7fd9100d0ca0 2023-10-18T10:56:47.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.595+0000 7fd91629c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9100d1130 con 0x7fd9100d2f40 2023-10-18T10:56:47.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.595+0000 7fd91629c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91015ce40 con 0x7fd91015c940 2023-10-18T10:56:47.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.595+0000 7fd90f7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd9100d2f40 0x7fd9100d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:56:47.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.595+0000 7fd90ffff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd91015c940 0x7fd9100d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:56:47.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.595+0000 7fd914839640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fd9100d0ca0 0x7fd9100cf060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.89:43460/0 (socket says 172.21.15.89:43460) 2023-10-18T10:56:47.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.595+0000 7fd914839640 1 -- 172.21.15.89:0/2905298944 learned_addr learned my addr 172.21.15.89:0/2905298944 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:56:47.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.595+0000 7fd90f7fe640 1 -- 172.21.15.89:0/2905298944 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd91015c940 msgr2=0x7fd9100d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:56:47.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.595+0000 7fd90f7fe640 1 --2- 172.21.15.89:0/2905298944 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd91015c940 0x7fd9100d2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:56:47.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.595+0000 7fd90f7fe640 1 -- 172.21.15.89:0/2905298944 >> v1:172.21.15.154:6789/0 conn(0x7fd9100d0ca0 legacy=0x7fd9100cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:56:47.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.595+0000 7fd90f7fe640 1 -- 172.21.15.89:0/2905298944 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd91015d120 con 0x7fd9100d2f40 2023-10-18T10:56:47.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.595+0000 7fd90ffff640 1 --2- 172.21.15.89:0/2905298944 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd91015c940 0x7fd9100d2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:56:47.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.599+0000 7fd90f7fe640 1 --2- 172.21.15.89:0/2905298944 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9100d2f40 0x7fd9100d0760 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd9040097c0 tx=0x7fd90401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7f9b73556f14a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:56:47.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.599+0000 7fd90effd640 1 -- 172.21.15.89:0/2905298944 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9040ad660 con 0x7fd9100d2f40 2023-10-18T10:56:47.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.599+0000 7fd90effd640 1 -- 172.21.15.89:0/2905298944 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9040ad7c0 con 0x7fd9100d2f40 2023-10-18T10:56:47.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.599+0000 7fd90effd640 1 -- 172.21.15.89:0/2905298944 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9040adb50 con 0x7fd9100d2f40 2023-10-18T10:56:47.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.599+0000 7fd91629c640 1 -- 172.21.15.89:0/2905298944 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9100d2f40 msgr2=0x7fd9100d0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:56:47.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.599+0000 7fd91629c640 1 --2- 172.21.15.89:0/2905298944 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9100d2f40 0x7fd9100d0760 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd9040097c0 tx=0x7fd90401c450 comp rx=0 tx=0).stop 2023-10-18T10:56:47.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.599+0000 7fd91629c640 1 -- 172.21.15.89:0/2905298944 shutdown_connections 2023-10-18T10:56:47.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.599+0000 7fd91629c640 1 --2- 172.21.15.89:0/2905298944 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9100d2f40 0x7fd9100d0760 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:56:47.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.599+0000 7fd91629c640 1 --2- 172.21.15.89:0/2905298944 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd91015c940 0x7fd9100d2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:56:47.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.599+0000 7fd91629c640 1 -- 172.21.15.89:0/2905298944 >> 172.21.15.89:0/2905298944 conn(0x7fd910089fd0 msgr2=0x7fd91009b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:56:47.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.599+0000 7fd91629c640 1 -- 172.21.15.89:0/2905298944 shutdown_connections 2023-10-18T10:56:47.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.599+0000 7fd91629c640 1 -- 172.21.15.89:0/2905298944 wait complete. 2023-10-18T10:56:47.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.599+0000 7fd91629c640 1 Processor -- start 2023-10-18T10:56:47.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.599+0000 7fd91629c640 1 -- start start 2023-10-18T10:56:47.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.603+0000 7fd91629c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9100d0ca0 0x7fd910151220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:56:47.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.603+0000 7fd91629c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd910152f70 0x7fd9101519a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:56:47.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.603+0000 7fd91629c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd91015b130 0x7fd91015b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:56:47.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.603+0000 7fd91629c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd910152d50 con 0x7fd9100d0ca0 2023-10-18T10:56:47.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.603+0000 7fd91629c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91015baf0 con 0x7fd91015b130 2023-10-18T10:56:47.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.603+0000 7fd91629c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd910151760 con 0x7fd910152f70 2023-10-18T10:56:47.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.603+0000 7fd90ffff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9100d0ca0 0x7fd910151220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:56:47.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.603+0000 7fd90ffff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9100d0ca0 0x7fd910151220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35784/0 (socket says 172.21.15.89:35784) 2023-10-18T10:56:47.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.603+0000 7fd90ffff640 1 -- 172.21.15.89:0/1308176644 learned_addr learned my addr 172.21.15.89:0/1308176644 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:56:47.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.603+0000 7fd90f7fe640 1 --2- 172.21.15.89:0/1308176644 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd910152f70 0x7fd9101519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:56:47.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.603+0000 7fd914839640 1 --2- 172.21.15.89:0/1308176644 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd91015b130 0x7fd91015b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:56:47.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.603+0000 7fd90ffff640 1 -- 172.21.15.89:0/1308176644 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd910152f70 msgr2=0x7fd9101519a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:56:47.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.603+0000 7fd90ffff640 1 --2- 172.21.15.89:0/1308176644 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd910152f70 0x7fd9101519a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:56:47.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.603+0000 7fd90ffff640 1 -- 172.21.15.89:0/1308176644 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd91015b130 msgr2=0x7fd91015b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:56:47.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.603+0000 7fd90ffff640 1 --2- 172.21.15.89:0/1308176644 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd91015b130 0x7fd91015b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:56:47.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.603+0000 7fd90ffff640 1 -- 172.21.15.89:0/1308176644 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd900009420 con 0x7fd9100d0ca0 2023-10-18T10:56:47.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.603+0000 7fd90ffff640 1 --2- 172.21.15.89:0/1308176644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9100d0ca0 0x7fd910151220 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd90000e980 tx=0x7fd90000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:56:47.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.603+0000 7fd90cff9640 1 -- 172.21.15.89:0/1308176644 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90000c9a0 con 0x7fd9100d0ca0 2023-10-18T10:56:47.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.603+0000 7fd90cff9640 1 -- 172.21.15.89:0/1308176644 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90000cb00 con 0x7fd9100d0ca0 2023-10-18T10:56:47.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.603+0000 7fd91629c640 1 -- 172.21.15.89:0/1308176644 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd904009420 con 0x7fd9100d0ca0 2023-10-18T10:56:47.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.603+0000 7fd90cff9640 1 -- 172.21.15.89:0/1308176644 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90000ce90 con 0x7fd9100d0ca0 2023-10-18T10:56:47.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.603+0000 7fd91629c640 1 -- 172.21.15.89:0/1308176644 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9101524b0 con 0x7fd9100d0ca0 2023-10-18T10:56:47.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.603+0000 7fd91629c640 1 -- 172.21.15.89:0/1308176644 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9100cee70 con 0x7fd9100d0ca0 2023-10-18T10:56:47.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.607+0000 7fd90cff9640 1 -- 172.21.15.89:0/1308176644 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fd900013470 con 0x7fd9100d0ca0 2023-10-18T10:56:47.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.607+0000 7fd90cff9640 1 --2- 172.21.15.89:0/1308176644 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd8fc065680 0x7fd8fc067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:56:47.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.607+0000 7fd90cff9640 1 -- 172.21.15.89:0/1308176644 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fd900083180 con 0x7fd9100d0ca0 2023-10-18T10:56:47.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.607+0000 7fd90f7fe640 1 --2- 172.21.15.89:0/1308176644 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd8fc065680 0x7fd8fc067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:56:47.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.607+0000 7fd90f7fe640 1 --2- 172.21.15.89:0/1308176644 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd8fc065680 0x7fd8fc067b40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd9040af000 tx=0x7fd9040a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:56:47.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.611+0000 7fd90cff9640 1 -- 172.21.15.89:0/1308176644 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fd900050b20 con 0x7fd9100d0ca0 2023-10-18T10:56:47.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.619+0000 7f5388419640 1 Processor -- start 2023-10-18T10:56:47.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.619+0000 7f5388419640 1 -- start start 2023-10-18T10:56:47.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.623+0000 7f5388419640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f53800d0e60 0x7f53800b05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:56:47.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.623+0000 7f5388419640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53800b33b0 0x7f53800b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:56:47.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.623+0000 7f5388419640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53800cced0 con 0x7f5380160f20 2023-10-18T10:56:47.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.623+0000 7f5388419640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53800b1d30 con 0x7f53800d0e60 2023-10-18T10:56:47.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.623+0000 7f5388419640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53800b3bc0 con 0x7f53800b33b0 2023-10-18T10:56:47.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.623+0000 7f53859b4640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f53800d0e60 0x7f53800b05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:56:47.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.623+0000 7f53859b4640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f53800d0e60 0x7f53800b05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35796/0 (socket says 172.21.15.89:35796) 2023-10-18T10:56:47.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.623+0000 7f53859b4640 1 -- 172.21.15.89:0/3106692461 learned_addr learned my addr 172.21.15.89:0/3106692461 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:56:47.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.623+0000 7f53869b6640 1 --2- 172.21.15.89:0/3106692461 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53800b33b0 0x7f53800b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:56:47.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.623+0000 7f53859b4640 1 -- 172.21.15.89:0/3106692461 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53800b33b0 msgr2=0x7f53800b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:56:47.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.623+0000 7f53859b4640 1 --2- 172.21.15.89:0/3106692461 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53800b33b0 0x7f53800b0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:56:47.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.623+0000 7f53859b4640 1 -- 172.21.15.89:0/3106692461 >> v1:172.21.15.154:6789/0 conn(0x7f5380160f20 legacy=0x7f53800d0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:56:47.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.623+0000 7f53859b4640 1 -- 172.21.15.89:0/3106692461 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53800afdd0 con 0x7f53800d0e60 2023-10-18T10:56:47.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.623+0000 7f53859b4640 1 --2- 172.21.15.89:0/3106692461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53800d0e60 0x7f53800b05d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f5370009870 tx=0x7f537001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26bc6bb59d95adff server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:56:47.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.627+0000 7f53851b3640 1 -- 172.21.15.89:0/3106692461 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53800b1860 con 0x7f53800d0e60 2023-10-18T10:56:47.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.627+0000 7f53851b3640 1 -- 172.21.15.89:0/3106692461 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53700ad680 con 0x7f53800d0e60 2023-10-18T10:56:47.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.627+0000 7f53851b3640 1 -- 172.21.15.89:0/3106692461 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53700ada10 con 0x7f53800d0e60 2023-10-18T10:56:47.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.627+0000 7f5388419640 1 -- 172.21.15.89:0/3106692461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53800d0e60 msgr2=0x7f53800b05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:56:47.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.627+0000 7f5388419640 1 --2- 172.21.15.89:0/3106692461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53800d0e60 0x7f53800b05d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f5370009870 tx=0x7f537001c4b0 comp rx=0 tx=0).stop 2023-10-18T10:56:47.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.627+0000 7f5388419640 1 -- 172.21.15.89:0/3106692461 shutdown_connections 2023-10-18T10:56:47.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.627+0000 7f5388419640 1 --2- 172.21.15.89:0/3106692461 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53800b33b0 0x7f53800b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:56:47.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.627+0000 7f5388419640 1 --2- 172.21.15.89:0/3106692461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53800d0e60 0x7f53800b05d0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:56:47.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.627+0000 7f5388419640 1 -- 172.21.15.89:0/3106692461 >> 172.21.15.89:0/3106692461 conn(0x7f538009b930 msgr2=0x7f53800d26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:56:47.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.627+0000 7f5388419640 1 -- 172.21.15.89:0/3106692461 shutdown_connections 2023-10-18T10:56:47.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.627+0000 7f5388419640 1 -- 172.21.15.89:0/3106692461 wait complete. 2023-10-18T10:56:47.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.627+0000 7f5388419640 1 Processor -- start 2023-10-18T10:56:47.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.627+0000 7f5388419640 1 -- start start 2023-10-18T10:56:47.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.627+0000 7f5388419640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53800b33b0 0x7f53801524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:56:47.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.631+0000 7f5388419640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53800d0e60 0x7f5380158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:56:47.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.631+0000 7f5388419640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5380160f20 0x7f5380159530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:56:47.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.631+0000 7f5388419640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5380152a30 con 0x7f53800b33b0 2023-10-18T10:56:47.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.631+0000 7f5388419640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5380156d20 con 0x7f5380160f20 2023-10-18T10:56:47.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.631+0000 7f5388419640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53801607d0 con 0x7f53800d0e60 2023-10-18T10:56:47.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.631+0000 7f53861b5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53800b33b0 0x7f53801524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:56:47.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.631+0000 7f53861b5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53800b33b0 0x7f53801524f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35804/0 (socket says 172.21.15.89:35804) 2023-10-18T10:56:47.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.631+0000 7f53861b5640 1 -- 172.21.15.89:0/808122131 learned_addr learned my addr 172.21.15.89:0/808122131 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:56:47.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.631+0000 7f53869b6640 1 --2- 172.21.15.89:0/808122131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5380160f20 0x7f5380159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:56:47.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.631+0000 7f53859b4640 1 --2- 172.21.15.89:0/808122131 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53800d0e60 0x7f5380158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:56:47.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.631+0000 7f53861b5640 1 -- 172.21.15.89:0/808122131 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53800d0e60 msgr2=0x7f5380158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:56:47.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.631+0000 7f53861b5640 1 --2- 172.21.15.89:0/808122131 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53800d0e60 0x7f5380158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:56:47.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.631+0000 7f53861b5640 1 -- 172.21.15.89:0/808122131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5380160f20 msgr2=0x7f5380159530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:56:47.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.631+0000 7f53861b5640 1 --2- 172.21.15.89:0/808122131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5380160f20 0x7f5380159530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:56:47.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.631+0000 7f53861b5640 1 -- 172.21.15.89:0/808122131 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53700094d0 con 0x7f53800b33b0 2023-10-18T10:56:47.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.631+0000 7f53861b5640 1 --2- 172.21.15.89:0/808122131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53800b33b0 0x7f53801524f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f537c00c370 tx=0x7f537c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:56:47.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.631+0000 7f5376ffd640 1 -- 172.21.15.89:0/808122131 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c0052d0 con 0x7f53800b33b0 2023-10-18T10:56:47.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.631+0000 7f5376ffd640 1 -- 172.21.15.89:0/808122131 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f537c005430 con 0x7f53800b33b0 2023-10-18T10:56:47.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.631+0000 7f5376ffd640 1 -- 172.21.15.89:0/808122131 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c007260 con 0x7f53800b33b0 2023-10-18T10:56:47.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.631+0000 7f5388419640 1 -- 172.21.15.89:0/808122131 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5380154330 con 0x7f53800b33b0 2023-10-18T10:56:47.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.631+0000 7f5388419640 1 -- 172.21.15.89:0/808122131 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5380156800 con 0x7f53800b33b0 2023-10-18T10:56:47.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.631+0000 7f5388419640 1 -- 172.21.15.89:0/808122131 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f538004e430 con 0x7f53800b33b0 2023-10-18T10:56:47.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.631+0000 7f5376ffd640 1 -- 172.21.15.89:0/808122131 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f537c0073c0 con 0x7f53800b33b0 2023-10-18T10:56:47.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.635+0000 7f5376ffd640 1 --2- 172.21.15.89:0/808122131 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f53640656b0 0x7f5364067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:56:47.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.635+0000 7f5376ffd640 1 -- 172.21.15.89:0/808122131 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f537c082a60 con 0x7f53800b33b0 2023-10-18T10:56:47.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.635+0000 7f5376ffd640 1 -- 172.21.15.89:0/808122131 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+450 (secure 0 0 0) 0x7f537c083280 con 0x7f53800b33b0 2023-10-18T10:56:47.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.635+0000 7f53859b4640 1 --2- 172.21.15.89:0/808122131 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f53640656b0 0x7f5364067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:56:47.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.635+0000 7f5388419640 1 -- 172.21.15.89:0/808122131 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f53801569f0 con 0x7f53800b33b0 2023-10-18T10:56:47.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.635+0000 7f53859b4640 1 --2- 172.21.15.89:0/808122131 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f53640656b0 0x7f5364067b70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f53700af000 tx=0x7f53700a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:56:47.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.795+0000 7fd91629c640 1 -- 172.21.15.89:0/1308176644 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd91004dbe0 con 0x7fd9100d0ca0 2023-10-18T10:56:47.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.795+0000 7fd90cff9640 1 -- 172.21.15.89:0/1308176644 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4877 (secure 0 0 0) 0x7fd900054140 con 0x7fd9100d0ca0 2023-10-18T10:56:47.794 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:56:47.794 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/654729641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":654729641},{"type":"v1","addr":"172.21.15.154:6836","nonce":654729641}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/964566480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":964566480},{"type":"v1","addr":"172.21.15.154:6837","nonce":964566480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2261878538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2261878538},{"type":"v1","addr":"172.21.15.89:6833","nonce":2261878538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:47:15.199575+0000","modified":"2023-10-18T10:55:20.892655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5517":{"gid":5517,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":124,"addr":"172.21.15.89:6835/2092274731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2092274731},{"type":"v1","addr":"172.21.15.89:6835","nonce":2092274731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T10:56:47.795 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-18T10:56:47.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.799+0000 7fd91629c640 1 -- 172.21.15.89:0/1308176644 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd8fc065680 msgr2=0x7fd8fc067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:56:47.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.799+0000 7fd91629c640 1 --2- 172.21.15.89:0/1308176644 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd8fc065680 0x7fd8fc067b40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd9040af000 tx=0x7fd9040a7000 comp rx=0 tx=0).stop 2023-10-18T10:56:47.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.799+0000 7fd91629c640 1 -- 172.21.15.89:0/1308176644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9100d0ca0 msgr2=0x7fd910151220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:56:47.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.799+0000 7fd91629c640 1 --2- 172.21.15.89:0/1308176644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9100d0ca0 0x7fd910151220 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd90000e980 tx=0x7fd90000ee40 comp rx=0 tx=0).stop 2023-10-18T10:56:47.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.799+0000 7fd91629c640 1 -- 172.21.15.89:0/1308176644 shutdown_connections 2023-10-18T10:56:47.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.799+0000 7fd91629c640 1 --2- 172.21.15.89:0/1308176644 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd8fc065680 0x7fd8fc067b40 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:56:47.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.799+0000 7fd91629c640 1 --2- 172.21.15.89:0/1308176644 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd910152f70 0x7fd9101519a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:56:47.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.799+0000 7fd91629c640 1 --2- 172.21.15.89:0/1308176644 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd91015b130 0x7fd91015b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:56:47.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.799+0000 7fd91629c640 1 --2- 172.21.15.89:0/1308176644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9100d0ca0 0x7fd910151220 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:56:47.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.799+0000 7fd91629c640 1 -- 172.21.15.89:0/1308176644 >> 172.21.15.89:0/1308176644 conn(0x7fd910089fd0 msgr2=0x7fd910069900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:56:47.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.799+0000 7fd91629c640 1 -- 172.21.15.89:0/1308176644 shutdown_connections 2023-10-18T10:56:47.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:56:47.799+0000 7fd91629c640 1 -- 172.21.15.89:0/1308176644 wait complete. 2023-10-18T10:56:47.815 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-18T10:56:47.973 INFO:teuthology.orchestra.run.smithi089.stdout:[{"id":5183,"entity":{"name":{"type":"client","num":5183},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":4272238049}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":564.26458126700004,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.27151768272589438,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":3,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5183 v1:192.168.0.1:0/4272238049","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi154","kernel_version":"6.6.0-rc5-g47672119149b","root":"/"}}] 2023-10-18T10:56:47.974 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "1", "hostname": "smithi154", "kernel_version": "6.6.0-rc5-g47672119149b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 4272238049, "type": "v1" }, "name": { "num": 5183, "type": "client" } }, "id": 5183, "inst": "client.5183 v1:192.168.0.1:0/4272238049", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 3, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.2715176827258944 }, "state": "open", "uptime": 564.264581267 } ] 2023-10-18T10:56:48.075 DEBUG:tasks.ceph_test_case:No log hits yet, waiting... 2023-10-18T10:56:50.599 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:56:50.604 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:57:20.666 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:57:20.671 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:57:23.091 ERROR:tasks.ceph_test_case:Log output: 2023-10-18T10:57:23.092 DEBUG:tasks.cephfs.test_client_recovery:Expected log message found: 'Health check cleared: MDS_CLIENTS_LAGGY', retrying 2023-10-18T10:57:28.094 DEBUG:teuthology.orchestra.run.smithi089:> sudo daemon-helper kill ceph --cluster ceph -w 2023-10-18T10:57:28.099 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:57:28.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.250+0000 7fb322737640 1 Processor -- start 2023-10-18T10:57:28.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.250+0000 7fb322737640 1 -- start start 2023-10-18T10:57:28.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.250+0000 7fb322737640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb31c15c940 0x7fb31c0d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:57:28.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.250+0000 7fb322737640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb31c0d2f40 0x7fb31c0d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:57:28.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.250+0000 7fb322737640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb31c15da50 con 0x7fb31c0d0ca0 2023-10-18T10:57:28.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.250+0000 7fb322737640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb31c0d1130 con 0x7fb31c15c940 2023-10-18T10:57:28.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.250+0000 7fb322737640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb31c15ce40 con 0x7fb31c0d2f40 2023-10-18T10:57:28.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.250+0000 7fb320cd4640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb31c0d0ca0 0x7fb31c0cf060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52614/0 (socket says 172.21.15.89:52614) 2023-10-18T10:57:28.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.250+0000 7fb320cd4640 1 -- 172.21.15.89:0/2945092529 learned_addr learned my addr 172.21.15.89:0/2945092529 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:57:28.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.250+0000 7fb31b7fe640 1 --2- 172.21.15.89:0/2945092529 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb31c0d2f40 0x7fb31c0d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:57:28.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.250+0000 7fb31bfff640 1 --2- 172.21.15.89:0/2945092529 >> v2:172.21.15.154:3300/0 conn(0x7fb31c15c940 0x7fb31c0d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:57:28.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.250+0000 7fb31affd640 1 -- 172.21.15.89:0/2945092529 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2739436258 0 0) 0x7fb31c15da50 con 0x7fb31c0d0ca0 2023-10-18T10:57:28.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.250+0000 7fb31affd640 1 -- 172.21.15.89:0/2945092529 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb304003530 con 0x7fb31c0d0ca0 2023-10-18T10:57:28.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.250+0000 7fb31affd640 1 -- 172.21.15.89:0/2945092529 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2774468855 0 0) 0x7fb304003530 con 0x7fb31c0d0ca0 2023-10-18T10:57:28.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.250+0000 7fb31affd640 1 -- 172.21.15.89:0/2945092529 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb31c0d2f40 msgr2=0x7fb31c0d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:57:28.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.250+0000 7fb31affd640 1 --2- 172.21.15.89:0/2945092529 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb31c0d2f40 0x7fb31c0d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:57:28.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.250+0000 7fb31affd640 1 -- 172.21.15.89:0/2945092529 >> v2:172.21.15.154:3300/0 conn(0x7fb31c15c940 msgr2=0x7fb31c0d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:57:28.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.250+0000 7fb31affd640 1 --2- 172.21.15.89:0/2945092529 >> v2:172.21.15.154:3300/0 conn(0x7fb31c15c940 0x7fb31c0d2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:57:28.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.250+0000 7fb31affd640 1 -- 172.21.15.89:0/2945092529 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb31c15d120 con 0x7fb31c0d0ca0 2023-10-18T10:57:28.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.250+0000 7fb31affd640 1 -- 172.21.15.89:0/2945092529 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7fb314001a00 con 0x7fb31c0d0ca0 2023-10-18T10:57:28.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.250+0000 7fb31bfff640 1 --2- 172.21.15.89:0/2945092529 >> v2:172.21.15.154:3300/0 conn(0x7fb31c15c940 0x7fb31c0d2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:57:28.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.250+0000 7fb31b7fe640 1 --2- 172.21.15.89:0/2945092529 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb31c0d2f40 0x7fb31c0d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:57:28.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.250+0000 7fb31affd640 1 -- 172.21.15.89:0/2945092529 >> v1:172.21.15.89:6789/0 conn(0x7fb31c0d0ca0 legacy=0x7fb31c0cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:57:28.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.250+0000 7fb31affd640 1 --2- 172.21.15.89:0/2945092529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb304003980 0x7fb304003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:57:28.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.250+0000 7fb31affd640 1 --2- 172.21.15.89:0/2945092529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3040047d0 0x7fb304004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:57:28.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.250+0000 7fb31affd640 1 --2- 172.21.15.89:0/2945092529 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb304005190 0x7fb30400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:57:28.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.250+0000 7fb31affd640 1 -- 172.21.15.89:0/2945092529 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31c0d1130 con 0x7fb3040047d0 2023-10-18T10:57:28.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.254+0000 7fb31affd640 1 -- 172.21.15.89:0/2945092529 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31c15ce40 con 0x7fb304003980 2023-10-18T10:57:28.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.254+0000 7fb31affd640 1 -- 172.21.15.89:0/2945092529 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb304004520 con 0x7fb304005190 2023-10-18T10:57:28.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.254+0000 7fb31bfff640 1 --2- 172.21.15.89:0/2945092529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb304003980 0x7fb304003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:57:28.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.254+0000 7fb320cd4640 1 --2- 172.21.15.89:0/2945092529 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb304005190 0x7fb30400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:57:28.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.254+0000 7fb31b7fe640 1 --2- 172.21.15.89:0/2945092529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3040047d0 0x7fb304004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:57:28.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.254+0000 7fb31bfff640 1 -- 172.21.15.89:0/2945092529 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb304005190 msgr2=0x7fb30400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:57:28.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.254+0000 7fb31bfff640 1 --2- 172.21.15.89:0/2945092529 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb304005190 0x7fb30400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:57:28.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.254+0000 7fb31bfff640 1 -- 172.21.15.89:0/2945092529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3040047d0 msgr2=0x7fb304004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:57:28.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.254+0000 7fb31bfff640 1 --2- 172.21.15.89:0/2945092529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3040047d0 0x7fb304004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:57:28.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.254+0000 7fb31bfff640 1 -- 172.21.15.89:0/2945092529 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb310009420 con 0x7fb304003980 2023-10-18T10:57:28.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.254+0000 7fb31bfff640 1 --2- 172.21.15.89:0/2945092529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb304003980 0x7fb304003da0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb30c009cb0 tx=0x7fb30c01cc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa1da257d6a66f7a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:57:28.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.258+0000 7fb31affd640 1 -- 172.21.15.89:0/2945092529 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c0a5820 con 0x7fb304003980 2023-10-18T10:57:28.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.258+0000 7fb31affd640 1 -- 172.21.15.89:0/2945092529 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30c0a5980 con 0x7fb304003980 2023-10-18T10:57:28.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.258+0000 7fb31affd640 1 -- 172.21.15.89:0/2945092529 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c0a5db0 con 0x7fb304003980 2023-10-18T10:57:28.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.258+0000 7fb322737640 1 -- 172.21.15.89:0/2945092529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb304003980 msgr2=0x7fb304003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:57:28.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.258+0000 7fb322737640 1 --2- 172.21.15.89:0/2945092529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb304003980 0x7fb304003da0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb30c009cb0 tx=0x7fb30c01cc70 comp rx=0 tx=0).stop 2023-10-18T10:57:28.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.258+0000 7fb320cd4640 1 -- 172.21.15.89:0/2945092529 reap_dead start 2023-10-18T10:57:28.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.258+0000 7fb322737640 1 -- 172.21.15.89:0/2945092529 shutdown_connections 2023-10-18T10:57:28.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.258+0000 7fb322737640 1 -- 172.21.15.89:0/2945092529 >> 172.21.15.89:0/2945092529 conn(0x7fb31c089fd0 msgr2=0x7fb31c09b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:57:28.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.258+0000 7fb322737640 1 -- 172.21.15.89:0/2945092529 shutdown_connections 2023-10-18T10:57:28.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.258+0000 7fb322737640 1 -- 172.21.15.89:0/2945092529 wait complete. 2023-10-18T10:57:28.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.258+0000 7fb322737640 1 Processor -- start 2023-10-18T10:57:28.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.258+0000 7fb322737640 1 -- start start 2023-10-18T10:57:28.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.262+0000 7fb322737640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb304003980 0x7fb31c0b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:57:28.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.262+0000 7fb322737640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb31c0c84e0 0x7fb31c150650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:57:28.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.262+0000 7fb322737640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb31c15ee90 0x7fb31c15f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:57:28.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.262+0000 7fb322737640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31c15f880 con 0x7fb31c15ee90 2023-10-18T10:57:28.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.262+0000 7fb322737640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31c0c6530 con 0x7fb304003980 2023-10-18T10:57:28.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.262+0000 7fb322737640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb31c0c9340 con 0x7fb31c0c84e0 2023-10-18T10:57:28.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.262+0000 7fb320cd4640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb31c15ee90 0x7fb31c15f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:57:28.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.262+0000 7fb320cd4640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb31c15ee90 0x7fb31c15f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35616/0 (socket says 172.21.15.89:35616) 2023-10-18T10:57:28.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.262+0000 7fb320cd4640 1 -- 172.21.15.89:0/2056369556 learned_addr learned my addr 172.21.15.89:0/2056369556 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:57:28.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.262+0000 7fb31bfff640 1 --2- 172.21.15.89:0/2056369556 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb304003980 0x7fb31c0b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:57:28.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.262+0000 7fb31b7fe640 1 --2- 172.21.15.89:0/2056369556 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb31c0c84e0 0x7fb31c150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:57:28.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.262+0000 7fb320cd4640 1 -- 172.21.15.89:0/2056369556 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb31c0c84e0 msgr2=0x7fb31c150650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:57:28.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.262+0000 7fb320cd4640 1 --2- 172.21.15.89:0/2056369556 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb31c0c84e0 0x7fb31c150650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:57:28.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.262+0000 7fb320cd4640 1 -- 172.21.15.89:0/2056369556 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb304003980 msgr2=0x7fb31c0b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:57:28.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.262+0000 7fb320cd4640 1 --2- 172.21.15.89:0/2056369556 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb304003980 0x7fb31c0b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:57:28.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.262+0000 7fb320cd4640 1 -- 172.21.15.89:0/2056369556 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb31c0c8aa0 con 0x7fb31c15ee90 2023-10-18T10:57:28.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.262+0000 7fb31bfff640 1 --2- 172.21.15.89:0/2056369556 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb304003980 0x7fb31c0b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:57:28.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.262+0000 7fb320cd4640 1 --2- 172.21.15.89:0/2056369556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb31c15ee90 0x7fb31c15f340 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb31c0ceb80 tx=0x7fb3140115c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:57:28.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.262+0000 7fb318ff9640 1 -- 172.21.15.89:0/2056369556 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb314007ad0 con 0x7fb31c15ee90 2023-10-18T10:57:28.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.262+0000 7fb318ff9640 1 -- 172.21.15.89:0/2056369556 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb314004b70 con 0x7fb31c15ee90 2023-10-18T10:57:28.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.262+0000 7fb322737640 1 -- 172.21.15.89:0/2056369556 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb31c0c8d80 con 0x7fb31c15ee90 2023-10-18T10:57:28.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.262+0000 7fb318ff9640 1 -- 172.21.15.89:0/2056369556 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb314006070 con 0x7fb31c15ee90 2023-10-18T10:57:28.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.262+0000 7fb322737640 1 -- 172.21.15.89:0/2056369556 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb31c0c9830 con 0x7fb31c15ee90 2023-10-18T10:57:28.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.262+0000 7fb322737640 1 -- 172.21.15.89:0/2056369556 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb31c161050 con 0x7fb31c15ee90 2023-10-18T10:57:28.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.262+0000 7fb318ff9640 1 -- 172.21.15.89:0/2056369556 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fb3140061d0 con 0x7fb31c15ee90 2023-10-18T10:57:28.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.266+0000 7fb318ff9640 1 --2- 172.21.15.89:0/2056369556 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb308065680 0x7fb308067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:57:28.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.266+0000 7fb318ff9640 1 -- 172.21.15.89:0/2056369556 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fb314085cd0 con 0x7fb31c15ee90 2023-10-18T10:57:28.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.266+0000 7fb31bfff640 1 --2- 172.21.15.89:0/2056369556 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb308065680 0x7fb308067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:57:28.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.266+0000 7fb31bfff640 1 --2- 172.21.15.89:0/2056369556 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb308065680 0x7fb308067b40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb30c0af000 tx=0x7fb30c0046d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:57:28.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.270+0000 7fb318ff9640 1 -- 172.21.15.89:0/2056369556 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fb3140535f0 con 0x7fb31c15ee90 2023-10-18T10:57:28.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.278+0000 7fd960f00640 1 Processor -- start 2023-10-18T10:57:28.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.278+0000 7fd960f00640 1 -- start start 2023-10-18T10:57:28.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.278+0000 7fd960f00640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd95c0b33b0 0x7fd95c0b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:57:28.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.278+0000 7fd960f00640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd95c0cced0 con 0x7fd95c0d0e60 2023-10-18T10:57:28.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.278+0000 7fd960f00640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd95c0b1d30 con 0x7fd95c160f20 2023-10-18T10:57:28.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.278+0000 7fd960f00640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c0b3bc0 con 0x7fd95c0b33b0 2023-10-18T10:57:28.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.278+0000 7fd953fff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd95c0d0e60 0x7fd95c0b05d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52630/0 (socket says 172.21.15.89:52630) 2023-10-18T10:57:28.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.278+0000 7fd953fff640 1 -- 172.21.15.89:0/3556558246 learned_addr learned my addr 172.21.15.89:0/3556558246 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:57:28.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.278+0000 7fd95ad9d640 1 --2- 172.21.15.89:0/3556558246 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd95c0b33b0 0x7fd95c0b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:57:28.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.278+0000 7fd959d9b640 1 -- 172.21.15.89:0/3556558246 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2956699748 0 0) 0x7fd95c0cced0 con 0x7fd95c0d0e60 2023-10-18T10:57:28.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.278+0000 7fd959d9b640 1 -- 172.21.15.89:0/3556558246 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd948003530 con 0x7fd95c0d0e60 2023-10-18T10:57:28.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.278+0000 7fd959d9b640 1 -- 172.21.15.89:0/3556558246 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4088798217 0 0) 0x7fd948003530 con 0x7fd95c0d0e60 2023-10-18T10:57:28.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.278+0000 7fd959d9b640 1 -- 172.21.15.89:0/3556558246 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd95c0b33b0 msgr2=0x7fd95c0b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:57:28.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.278+0000 7fd959d9b640 1 --2- 172.21.15.89:0/3556558246 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd95c0b33b0 0x7fd95c0b0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:57:28.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.278+0000 7fd959d9b640 1 -- 172.21.15.89:0/3556558246 >> v1:172.21.15.154:6789/0 conn(0x7fd95c160f20 legacy=0x7fd95c0d0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:57:28.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.278+0000 7fd959d9b640 1 -- 172.21.15.89:0/3556558246 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd95c0afdd0 con 0x7fd95c0d0e60 2023-10-18T10:57:28.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.278+0000 7fd959d9b640 1 -- 172.21.15.89:0/3556558246 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7fd944001680 con 0x7fd95c0d0e60 2023-10-18T10:57:28.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.278+0000 7fd959d9b640 1 -- 172.21.15.89:0/3556558246 >> v1:172.21.15.89:6789/0 conn(0x7fd95c0d0e60 legacy=0x7fd95c0b05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:57:28.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.278+0000 7fd959d9b640 1 --2- 172.21.15.89:0/3556558246 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9480039b0 0x7fd948003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:57:28.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.278+0000 7fd959d9b640 1 --2- 172.21.15.89:0/3556558246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd948004360 0x7fd948004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:57:28.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.278+0000 7fd959d9b640 1 --2- 172.21.15.89:0/3556558246 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd95c0b33b0 0x7fd948005130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:57:28.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.278+0000 7fd959d9b640 1 -- 172.21.15.89:0/3556558246 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c0b3bc0 con 0x7fd948004360 2023-10-18T10:57:28.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.278+0000 7fd959d9b640 1 -- 172.21.15.89:0/3556558246 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94800ba40 con 0x7fd9480039b0 2023-10-18T10:57:28.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.278+0000 7fd959d9b640 1 -- 172.21.15.89:0/3556558246 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd94800bc00 con 0x7fd95c0b33b0 2023-10-18T10:57:28.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.278+0000 7fd953fff640 1 --2- 172.21.15.89:0/3556558246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd948004360 0x7fd948004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:57:28.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.278+0000 7fd95a59c640 1 --2- 172.21.15.89:0/3556558246 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9480039b0 0x7fd948003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:57:28.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.278+0000 7fd95ad9d640 1 --2- 172.21.15.89:0/3556558246 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd95c0b33b0 0x7fd948005130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:57:28.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.278+0000 7fd953fff640 1 -- 172.21.15.89:0/3556558246 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd95c0b33b0 msgr2=0x7fd948005130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:57:28.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.278+0000 7fd953fff640 1 --2- 172.21.15.89:0/3556558246 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd95c0b33b0 0x7fd948005130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:57:28.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.278+0000 7fd953fff640 1 -- 172.21.15.89:0/3556558246 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9480039b0 msgr2=0x7fd948003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:57:28.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.278+0000 7fd953fff640 1 --2- 172.21.15.89:0/3556558246 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9480039b0 0x7fd948003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:57:28.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.278+0000 7fd953fff640 1 -- 172.21.15.89:0/3556558246 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd948003530 con 0x7fd948004360 2023-10-18T10:57:28.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.282+0000 7fd953fff640 1 --2- 172.21.15.89:0/3556558246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd948004360 0x7fd948004830 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd94400c210 tx=0x7fd94401cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1635d99bf9eddf2 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:57:28.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.282+0000 7fd959d9b640 1 -- 172.21.15.89:0/3556558246 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9440a5b40 con 0x7fd948004360 2023-10-18T10:57:28.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.282+0000 7fd959d9b640 1 -- 172.21.15.89:0/3556558246 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9440a5cc0 con 0x7fd948004360 2023-10-18T10:57:28.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.282+0000 7fd959d9b640 1 -- 172.21.15.89:0/3556558246 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd944005b20 con 0x7fd948004360 2023-10-18T10:57:28.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.282+0000 7fd960f00640 1 -- 172.21.15.89:0/3556558246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd948004360 msgr2=0x7fd948004830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:57:28.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.282+0000 7fd960f00640 1 --2- 172.21.15.89:0/3556558246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd948004360 0x7fd948004830 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd94400c210 tx=0x7fd94401cc40 comp rx=0 tx=0).stop 2023-10-18T10:57:28.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.282+0000 7fd95ad9d640 1 -- 172.21.15.89:0/3556558246 reap_dead start 2023-10-18T10:57:28.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.282+0000 7fd960f00640 1 -- 172.21.15.89:0/3556558246 shutdown_connections 2023-10-18T10:57:28.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.282+0000 7fd960f00640 1 -- 172.21.15.89:0/3556558246 >> 172.21.15.89:0/3556558246 conn(0x7fd95c09b930 msgr2=0x7fd95c0d26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:57:28.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.282+0000 7fd960f00640 1 -- 172.21.15.89:0/3556558246 shutdown_connections 2023-10-18T10:57:28.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.282+0000 7fd960f00640 1 -- 172.21.15.89:0/3556558246 wait complete. 2023-10-18T10:57:28.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.286+0000 7fd960f00640 1 Processor -- start 2023-10-18T10:57:28.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.286+0000 7fd960f00640 1 -- start start 2023-10-18T10:57:28.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.286+0000 7fd960f00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd948004360 0x7fd95c0cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:57:28.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.286+0000 7fd960f00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd95c152e60 0x7fd95c0cb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:57:28.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.286+0000 7fd960f00640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd95c09c3a0 0x7fd95c09dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:57:28.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.286+0000 7fd960f00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c0cb990 con 0x7fd948004360 2023-10-18T10:57:28.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.286+0000 7fd960f00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c1532e0 con 0x7fd95c152e60 2023-10-18T10:57:28.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.286+0000 7fd960f00640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c0cafc0 con 0x7fd95c09c3a0 2023-10-18T10:57:28.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.286+0000 7fd95a59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd948004360 0x7fd95c0cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:57:28.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.286+0000 7fd95a59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd948004360 0x7fd95c0cc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35624/0 (socket says 172.21.15.89:35624) 2023-10-18T10:57:28.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.286+0000 7fd95a59c640 1 -- 172.21.15.89:0/134376003 learned_addr learned my addr 172.21.15.89:0/134376003 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:57:28.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.286+0000 7fd953fff640 1 --2- 172.21.15.89:0/134376003 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd95c152e60 0x7fd95c0cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:57:28.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.286+0000 7fd95ad9d640 1 --2- 172.21.15.89:0/134376003 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd95c09c3a0 0x7fd95c09dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:57:28.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.286+0000 7fd95a59c640 1 -- 172.21.15.89:0/134376003 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd95c09c3a0 msgr2=0x7fd95c09dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:57:28.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.286+0000 7fd95a59c640 1 --2- 172.21.15.89:0/134376003 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd95c09c3a0 0x7fd95c09dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:57:28.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.286+0000 7fd95a59c640 1 -- 172.21.15.89:0/134376003 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd95c152e60 msgr2=0x7fd95c0cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:57:28.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.286+0000 7fd95a59c640 1 --2- 172.21.15.89:0/134376003 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd95c152e60 0x7fd95c0cb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:57:28.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.286+0000 7fd95a59c640 1 -- 172.21.15.89:0/134376003 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd95c0ca690 con 0x7fd948004360 2023-10-18T10:57:28.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.286+0000 7fd953fff640 1 --2- 172.21.15.89:0/134376003 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd95c152e60 0x7fd95c0cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:57:28.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.286+0000 7fd95ad9d640 1 --2- 172.21.15.89:0/134376003 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd95c09c3a0 0x7fd95c09dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:57:28.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.286+0000 7fd95a59c640 1 --2- 172.21.15.89:0/134376003 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd948004360 0x7fd95c0cc0a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd94c011c40 tx=0x7fd94c00f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:57:28.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.286+0000 7fd952ffd640 1 -- 172.21.15.89:0/134376003 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94c00fcb0 con 0x7fd948004360 2023-10-18T10:57:28.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.286+0000 7fd952ffd640 1 -- 172.21.15.89:0/134376003 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd94c00fe30 con 0x7fd948004360 2023-10-18T10:57:28.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.286+0000 7fd960f00640 1 -- 172.21.15.89:0/134376003 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd95c0ca970 con 0x7fd948004360 2023-10-18T10:57:28.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.286+0000 7fd952ffd640 1 -- 172.21.15.89:0/134376003 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94c005310 con 0x7fd948004360 2023-10-18T10:57:28.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.290+0000 7fd960f00640 1 -- 172.21.15.89:0/134376003 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd95c0cc730 con 0x7fd948004360 2023-10-18T10:57:28.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.290+0000 7fd960f00640 1 -- 172.21.15.89:0/134376003 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fd95c04e430 con 0x7fd948004360 2023-10-18T10:57:28.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.290+0000 7fd952ffd640 1 -- 172.21.15.89:0/134376003 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fd94c003600 con 0x7fd948004360 2023-10-18T10:57:28.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.290+0000 7fd952ffd640 1 --2- 172.21.15.89:0/134376003 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd93c065680 0x7fd93c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:57:28.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.290+0000 7fd952ffd640 1 -- 172.21.15.89:0/134376003 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fd94c086960 con 0x7fd948004360 2023-10-18T10:57:28.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.290+0000 7fd952ffd640 1 -- 172.21.15.89:0/134376003 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+449 (secure 0 0 0) 0x7fd94c08b220 con 0x7fd948004360 2023-10-18T10:57:28.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.290+0000 7fd960f00640 1 -- 172.21.15.89:0/134376003 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7fd95c089fd0 con 0x7fd948004360 2023-10-18T10:57:28.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.290+0000 7fd953fff640 1 --2- 172.21.15.89:0/134376003 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd93c065680 0x7fd93c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:57:28.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.294+0000 7fd953fff640 1 --2- 172.21.15.89:0/134376003 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd93c065680 0x7fd93c067b40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd9440ab000 tx=0x7fd9440a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:57:28.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.450+0000 7fb322737640 1 -- 172.21.15.89:0/2056369556 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb31c04dbe0 con 0x7fb31c15ee90 2023-10-18T10:57:28.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.450+0000 7fb318ff9640 1 -- 172.21.15.89:0/2056369556 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4877 (secure 0 0 0) 0x7fb31401c070 con 0x7fb31c15ee90 2023-10-18T10:57:28.450 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:57:28.450 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/654729641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":654729641},{"type":"v1","addr":"172.21.15.154:6836","nonce":654729641}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/964566480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":964566480},{"type":"v1","addr":"172.21.15.154:6837","nonce":964566480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2261878538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2261878538},{"type":"v1","addr":"172.21.15.89:6833","nonce":2261878538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:47:15.199575+0000","modified":"2023-10-18T10:55:20.892655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5517":{"gid":5517,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":124,"addr":"172.21.15.89:6835/2092274731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2092274731},{"type":"v1","addr":"172.21.15.89:6835","nonce":2092274731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T10:57:28.451 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-18T10:57:28.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.454+0000 7fb322737640 1 -- 172.21.15.89:0/2056369556 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb308065680 msgr2=0x7fb308067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:57:28.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.454+0000 7fb322737640 1 --2- 172.21.15.89:0/2056369556 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb308065680 0x7fb308067b40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb30c0af000 tx=0x7fb30c0046d0 comp rx=0 tx=0).stop 2023-10-18T10:57:28.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.454+0000 7fb322737640 1 -- 172.21.15.89:0/2056369556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb31c15ee90 msgr2=0x7fb31c15f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:57:28.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.454+0000 7fb322737640 1 --2- 172.21.15.89:0/2056369556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb31c15ee90 0x7fb31c15f340 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb31c0ceb80 tx=0x7fb3140115c0 comp rx=0 tx=0).stop 2023-10-18T10:57:28.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.454+0000 7fb322737640 1 -- 172.21.15.89:0/2056369556 shutdown_connections 2023-10-18T10:57:28.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.454+0000 7fb322737640 1 --2- 172.21.15.89:0/2056369556 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb308065680 0x7fb308067b40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:57:28.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.454+0000 7fb322737640 1 --2- 172.21.15.89:0/2056369556 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb31c0c84e0 0x7fb31c150650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:57:28.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.454+0000 7fb322737640 1 --2- 172.21.15.89:0/2056369556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb31c15ee90 0x7fb31c15f340 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:57:28.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.454+0000 7fb322737640 1 --2- 172.21.15.89:0/2056369556 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb304003980 0x7fb31c0b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:57:28.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.454+0000 7fb322737640 1 -- 172.21.15.89:0/2056369556 >> 172.21.15.89:0/2056369556 conn(0x7fb31c089fd0 msgr2=0x7fb31c0d2550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:57:28.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.454+0000 7fb322737640 1 -- 172.21.15.89:0/2056369556 shutdown_connections 2023-10-18T10:57:28.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:57:28.454+0000 7fb322737640 1 -- 172.21.15.89:0/2056369556 wait complete. 2023-10-18T10:57:28.471 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-18T10:57:28.627 INFO:teuthology.orchestra.run.smithi089.stdout:[{"id":5183,"entity":{"name":{"type":"client","num":5183},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":4272238049}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":604.91984206100005,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.2471742785551764,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":3,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5183 v1:192.168.0.1:0/4272238049","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi154","kernel_version":"6.6.0-rc5-g47672119149b","root":"/"}}] 2023-10-18T10:57:28.628 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "1", "hostname": "smithi154", "kernel_version": "6.6.0-rc5-g47672119149b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 4272238049, "type": "v1" }, "name": { "num": 5183, "type": "client" } }, "id": 5183, "inst": "client.5183 v1:192.168.0.1:0/4272238049", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 3, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.2471742785551764 }, "state": "open", "uptime": 604.919842061 } ] 2023-10-18T10:57:28.730 DEBUG:tasks.ceph_test_case:No log hits yet, waiting... 2023-10-18T10:57:50.759 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:57:50.764 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:58:03.748 ERROR:tasks.ceph_test_case:Log output: 2023-10-18T10:58:03.748 DEBUG:tasks.cephfs.test_client_recovery:Expected log message found: 'Health check cleared: MDS_CLIENTS_LAGGY', retrying 2023-10-18T10:58:08.750 DEBUG:teuthology.orchestra.run.smithi089:> sudo daemon-helper kill ceph --cluster ceph -w 2023-10-18T10:58:08.755 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:58:08.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.905+0000 7f4424248640 1 Processor -- start 2023-10-18T10:58:08.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.905+0000 7f4424248640 1 -- start start 2023-10-18T10:58:08.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.905+0000 7f4424248640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f441c0d0bc0 0x7f441c0cd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:58:08.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.905+0000 7f4424248640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f441c15da50 con 0x7f441c15c940 2023-10-18T10:58:08.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.905+0000 7f4424248640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f441c0d3110 con 0x7f441c0d0760 2023-10-18T10:58:08.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.905+0000 7f4424248640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f441c15ce40 con 0x7f441c0d0bc0 2023-10-18T10:58:08.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.905+0000 7f44227e5640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f441c0d0bc0 0x7f441c0cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:58:08.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.905+0000 7f4421fe4640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f441c15c940 0x7f441c0d2a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49612/0 (socket says 172.21.15.89:49612) 2023-10-18T10:58:08.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.905+0000 7f4421fe4640 1 -- 172.21.15.89:0/756055321 learned_addr learned my addr 172.21.15.89:0/756055321 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:58:08.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.905+0000 7f44227e5640 1 -- 172.21.15.89:0/756055321 >> v1:172.21.15.154:6789/0 conn(0x7f441c0d0760 legacy=0x7f441c0cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:58:08.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.905+0000 7f44227e5640 1 -- 172.21.15.89:0/756055321 >> v1:172.21.15.89:6789/0 conn(0x7f441c15c940 legacy=0x7f441c0d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:58:08.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.905+0000 7f44227e5640 1 -- 172.21.15.89:0/756055321 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f441c15d120 con 0x7f441c0d0bc0 2023-10-18T10:58:08.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.909+0000 7f44227e5640 1 --2- 172.21.15.89:0/756055321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f441c0d0bc0 0x7f441c0cd870 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4414009d90 tx=0x7f441401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd21876dd4024595 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:58:08.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.909+0000 7f4420fe2640 1 -- 172.21.15.89:0/756055321 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44140a59a0 con 0x7f441c0d0bc0 2023-10-18T10:58:08.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.909+0000 7f4420fe2640 1 -- 172.21.15.89:0/756055321 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f441c0ce5c0 con 0x7f441c0d0bc0 2023-10-18T10:58:08.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.909+0000 7f4420fe2640 1 -- 172.21.15.89:0/756055321 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44140a5d20 con 0x7f441c0d0bc0 2023-10-18T10:58:08.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.909+0000 7f4424248640 1 -- 172.21.15.89:0/756055321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f441c0d0bc0 msgr2=0x7f441c0cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:58:08.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.909+0000 7f4424248640 1 --2- 172.21.15.89:0/756055321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f441c0d0bc0 0x7f441c0cd870 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4414009d90 tx=0x7f441401c8e0 comp rx=0 tx=0).stop 2023-10-18T10:58:08.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.909+0000 7f4424248640 1 -- 172.21.15.89:0/756055321 shutdown_connections 2023-10-18T10:58:08.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.909+0000 7f4424248640 1 --2- 172.21.15.89:0/756055321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f441c0d0bc0 0x7f441c0cd870 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:58:08.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.909+0000 7f4424248640 1 -- 172.21.15.89:0/756055321 >> 172.21.15.89:0/756055321 conn(0x7f441c089fd0 msgr2=0x7f441c09b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:58:08.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.909+0000 7f4424248640 1 -- 172.21.15.89:0/756055321 shutdown_connections 2023-10-18T10:58:08.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.909+0000 7f4424248640 1 -- 172.21.15.89:0/756055321 wait complete. 2023-10-18T10:58:08.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.913+0000 7f4424248640 1 Processor -- start 2023-10-18T10:58:08.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.913+0000 7f4424248640 1 -- start start 2023-10-18T10:58:08.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.913+0000 7f4424248640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f441c0d0760 0x7f441c150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:58:08.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.913+0000 7f4424248640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f441c0d0bc0 0x7f441c0ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:58:08.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.913+0000 7f4424248640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f441c0cac00 0x7f441c0b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:58:08.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.913+0000 7f4424248640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c0b7b50 con 0x7f441c0cac00 2023-10-18T10:58:08.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.913+0000 7f4424248640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c0b5d40 con 0x7f441c0d0bc0 2023-10-18T10:58:08.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.913+0000 7f4424248640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f441c0c89c0 con 0x7f441c0d0760 2023-10-18T10:58:08.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.913+0000 7f44227e5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f441c0cac00 0x7f441c0b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:58:08.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.913+0000 7f44227e5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f441c0cac00 0x7f441c0b61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54496/0 (socket says 172.21.15.89:54496) 2023-10-18T10:58:08.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.913+0000 7f44227e5640 1 -- 172.21.15.89:0/4054110552 learned_addr learned my addr 172.21.15.89:0/4054110552 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:58:08.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.913+0000 7f4421fe4640 1 --2- 172.21.15.89:0/4054110552 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f441c0d0760 0x7f441c150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:58:08.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.913+0000 7f44217e3640 1 --2- 172.21.15.89:0/4054110552 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f441c0d0bc0 0x7f441c0ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:58:08.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.913+0000 7f44227e5640 1 -- 172.21.15.89:0/4054110552 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f441c0d0760 msgr2=0x7f441c150330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:58:08.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.913+0000 7f44227e5640 1 --2- 172.21.15.89:0/4054110552 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f441c0d0760 0x7f441c150330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:58:08.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.913+0000 7f44227e5640 1 -- 172.21.15.89:0/4054110552 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f441c0d0bc0 msgr2=0x7f441c0ca6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:58:08.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.913+0000 7f44227e5640 1 --2- 172.21.15.89:0/4054110552 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f441c0d0bc0 0x7f441c0ca6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:58:08.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.913+0000 7f44227e5640 1 -- 172.21.15.89:0/4054110552 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4414009a70 con 0x7f441c0cac00 2023-10-18T10:58:08.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.913+0000 7f44227e5640 1 --2- 172.21.15.89:0/4054110552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f441c0cac00 0x7f441c0b61d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f441401a510 tx=0x7f4414019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:58:08.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.913+0000 7f44127fc640 1 -- 172.21.15.89:0/4054110552 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44140b9860 con 0x7f441c0cac00 2023-10-18T10:58:08.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.913+0000 7f44127fc640 1 -- 172.21.15.89:0/4054110552 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44140b99c0 con 0x7f441c0cac00 2023-10-18T10:58:08.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.913+0000 7f4424248640 1 -- 172.21.15.89:0/4054110552 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f441c0b7da0 con 0x7f441c0cac00 2023-10-18T10:58:08.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.913+0000 7f44127fc640 1 -- 172.21.15.89:0/4054110552 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44140b9d50 con 0x7f441c0cac00 2023-10-18T10:58:08.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.913+0000 7f4424248640 1 -- 172.21.15.89:0/4054110552 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f441c150ae0 con 0x7f441c0cac00 2023-10-18T10:58:08.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.917+0000 7f4424248640 1 -- 172.21.15.89:0/4054110552 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f441c04dbe0 con 0x7f441c0cac00 2023-10-18T10:58:08.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.917+0000 7f44127fc640 1 -- 172.21.15.89:0/4054110552 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f44140ad440 con 0x7f441c0cac00 2023-10-18T10:58:08.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.917+0000 7f44127fc640 1 --2- 172.21.15.89:0/4054110552 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f44000656b0 0x7f4400067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:58:08.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.917+0000 7f44127fc640 1 -- 172.21.15.89:0/4054110552 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f441411de00 con 0x7f441c0cac00 2023-10-18T10:58:08.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.917+0000 7f4421fe4640 1 --2- 172.21.15.89:0/4054110552 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f44000656b0 0x7f4400067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:58:08.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.917+0000 7f4421fe4640 1 --2- 172.21.15.89:0/4054110552 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f44000656b0 0x7f4400067b70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4418005890 tx=0x7f4418009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:58:08.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.925+0000 7f44127fc640 1 -- 172.21.15.89:0/4054110552 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f44140eb8e0 con 0x7f441c0cac00 2023-10-18T10:58:08.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.933+0000 7f88e0619640 1 Processor -- start 2023-10-18T10:58:08.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.933+0000 7f88e0619640 1 -- start start 2023-10-18T10:58:08.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.937+0000 7f88e0619640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88d8160f20 0x7f88d80d0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:58:08.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.937+0000 7f88e0619640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f88d80b15e0 0x7f88d80d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:58:08.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.937+0000 7f88e0619640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88d80cced0 con 0x7f88d80b33b0 2023-10-18T10:58:08.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.937+0000 7f88e0619640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88d80b1d30 con 0x7f88d80b15e0 2023-10-18T10:58:08.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.937+0000 7f88e0619640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d80b3bc0 con 0x7f88d8160f20 2023-10-18T10:58:08.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.937+0000 7f88ddbb4640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f88d80b33b0 0x7f88d80b1fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49628/0 (socket says 172.21.15.89:49628) 2023-10-18T10:58:08.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.937+0000 7f88ddbb4640 1 -- 172.21.15.89:0/2465709996 learned_addr learned my addr 172.21.15.89:0/2465709996 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:58:08.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.937+0000 7f88de3b5640 1 --2- 172.21.15.89:0/2465709996 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88d8160f20 0x7f88d80d0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:58:08.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.937+0000 7f88debb6640 1 --2- 172.21.15.89:0/2465709996 >> v2:172.21.15.154:3300/0 conn(0x7f88d80b15e0 0x7f88d80d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:58:08.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.937+0000 7f88dd3b3640 1 -- 172.21.15.89:0/2465709996 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1883241403 0 0) 0x7f88d80cced0 con 0x7f88d80b33b0 2023-10-18T10:58:08.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.937+0000 7f88dd3b3640 1 -- 172.21.15.89:0/2465709996 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88c4003530 con 0x7f88d80b33b0 2023-10-18T10:58:08.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.937+0000 7f88de3b5640 1 -- 172.21.15.89:0/2465709996 >> v2:172.21.15.154:3300/0 conn(0x7f88d80b15e0 msgr2=0x7f88d80d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:58:08.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.937+0000 7f88de3b5640 1 --2- 172.21.15.89:0/2465709996 >> v2:172.21.15.154:3300/0 conn(0x7f88d80b15e0 0x7f88d80d0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:58:08.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.937+0000 7f88de3b5640 1 -- 172.21.15.89:0/2465709996 >> v1:172.21.15.89:6789/0 conn(0x7f88d80b33b0 legacy=0x7f88d80b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:58:08.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.937+0000 7f88de3b5640 1 -- 172.21.15.89:0/2465709996 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d80afdd0 con 0x7f88d8160f20 2023-10-18T10:58:08.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.941+0000 7f88de3b5640 1 --2- 172.21.15.89:0/2465709996 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88d8160f20 0x7f88d80d0750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f88d40097c0 tx=0x7f88d401c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=771618f23b290fff server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:58:08.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.941+0000 7f88dd3b3640 1 -- 172.21.15.89:0/2465709996 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d40ab600 con 0x7f88d8160f20 2023-10-18T10:58:08.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.941+0000 7f88dd3b3640 1 -- 172.21.15.89:0/2465709996 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d40a7050 con 0x7f88d8160f20 2023-10-18T10:58:08.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.941+0000 7f88dd3b3640 1 -- 172.21.15.89:0/2465709996 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d40ab980 con 0x7f88d8160f20 2023-10-18T10:58:08.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.941+0000 7f88e0619640 1 -- 172.21.15.89:0/2465709996 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88d8160f20 msgr2=0x7f88d80d0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:58:08.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.941+0000 7f88e0619640 1 --2- 172.21.15.89:0/2465709996 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88d8160f20 0x7f88d80d0750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f88d40097c0 tx=0x7f88d401c3f0 comp rx=0 tx=0).stop 2023-10-18T10:58:08.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.941+0000 7f88e0619640 1 -- 172.21.15.89:0/2465709996 shutdown_connections 2023-10-18T10:58:08.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.941+0000 7f88e0619640 1 --2- 172.21.15.89:0/2465709996 >> v2:172.21.15.154:3300/0 conn(0x7f88d80b15e0 0x7f88d80d0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:58:08.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.941+0000 7f88e0619640 1 --2- 172.21.15.89:0/2465709996 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88d8160f20 0x7f88d80d0750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:58:08.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.941+0000 7f88e0619640 1 -- 172.21.15.89:0/2465709996 >> 172.21.15.89:0/2465709996 conn(0x7f88d809b930 msgr2=0x7f88d80d26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:58:08.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.941+0000 7f88e0619640 1 -- 172.21.15.89:0/2465709996 shutdown_connections 2023-10-18T10:58:08.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.941+0000 7f88e0619640 1 -- 172.21.15.89:0/2465709996 wait complete. 2023-10-18T10:58:08.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.941+0000 7f88e0619640 1 Processor -- start 2023-10-18T10:58:08.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.941+0000 7f88e0619640 1 -- start start 2023-10-18T10:58:08.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.945+0000 7f88e0619640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88d80b15e0 0x7f88d8159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:58:08.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.945+0000 7f88e0619640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88d80b33b0 0x7f88d80cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:58:08.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.945+0000 7f88e0619640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88d8160f20 0x7f88d80d2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:58:08.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.945+0000 7f88e0619640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d815a4d0 con 0x7f88d8160f20 2023-10-18T10:58:08.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.945+0000 7f88e0619640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d815a130 con 0x7f88d80b33b0 2023-10-18T10:58:08.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.945+0000 7f88e0619640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d80b7220 con 0x7f88d80b15e0 2023-10-18T10:58:08.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.945+0000 7f88debb6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88d8160f20 0x7f88d80d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:58:08.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.945+0000 7f88debb6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88d8160f20 0x7f88d80d2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54500/0 (socket says 172.21.15.89:54500) 2023-10-18T10:58:08.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.945+0000 7f88debb6640 1 -- 172.21.15.89:0/3651319467 learned_addr learned my addr 172.21.15.89:0/3651319467 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:58:08.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.945+0000 7f88ddbb4640 1 --2- 172.21.15.89:0/3651319467 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88d80b33b0 0x7f88d80cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:58:08.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.945+0000 7f88de3b5640 1 --2- 172.21.15.89:0/3651319467 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88d80b15e0 0x7f88d8159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:58:08.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.945+0000 7f88debb6640 1 -- 172.21.15.89:0/3651319467 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88d80b15e0 msgr2=0x7f88d8159bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:58:08.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.945+0000 7f88debb6640 1 --2- 172.21.15.89:0/3651319467 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88d80b15e0 0x7f88d8159bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:58:08.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.945+0000 7f88debb6640 1 -- 172.21.15.89:0/3651319467 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88d80b33b0 msgr2=0x7f88d80cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:58:08.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.945+0000 7f88debb6640 1 --2- 172.21.15.89:0/3651319467 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88d80b33b0 0x7f88d80cbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:58:08.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.945+0000 7f88debb6640 1 -- 172.21.15.89:0/3651319467 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d4009420 con 0x7f88d8160f20 2023-10-18T10:58:08.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.945+0000 7f88de3b5640 1 --2- 172.21.15.89:0/3651319467 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88d80b15e0 0x7f88d8159bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:58:08.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.945+0000 7f88debb6640 1 --2- 172.21.15.89:0/3651319467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88d8160f20 0x7f88d80d2680 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f88d0009dd0 tx=0x7f88d000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:58:08.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.945+0000 7f88ceffd640 1 -- 172.21.15.89:0/3651319467 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d0014480 con 0x7f88d8160f20 2023-10-18T10:58:08.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.945+0000 7f88ceffd640 1 -- 172.21.15.89:0/3651319467 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d000f040 con 0x7f88d8160f20 2023-10-18T10:58:08.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.945+0000 7f88ceffd640 1 -- 172.21.15.89:0/3651319467 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d0014840 con 0x7f88d8160f20 2023-10-18T10:58:08.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.945+0000 7f88e0619640 1 -- 172.21.15.89:0/3651319467 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88d80b7500 con 0x7f88d8160f20 2023-10-18T10:58:08.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.945+0000 7f88e0619640 1 -- 172.21.15.89:0/3651319467 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d80b7ac0 con 0x7f88d8160f20 2023-10-18T10:58:08.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.945+0000 7f88e0619640 1 -- 172.21.15.89:0/3651319467 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f88d804e430 con 0x7f88d8160f20 2023-10-18T10:58:08.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.949+0000 7f88ceffd640 1 -- 172.21.15.89:0/3651319467 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f88d00149a0 con 0x7f88d8160f20 2023-10-18T10:58:08.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.949+0000 7f88ceffd640 1 --2- 172.21.15.89:0/3651319467 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f88bc065680 0x7f88bc067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:58:08.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.949+0000 7f88ceffd640 1 -- 172.21.15.89:0/3651319467 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f88d0084740 con 0x7f88d8160f20 2023-10-18T10:58:08.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.949+0000 7f88ceffd640 1 -- 172.21.15.89:0/3651319467 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+449 (secure 0 0 0) 0x7f88d0084ad0 con 0x7f88d8160f20 2023-10-18T10:58:08.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.949+0000 7f88de3b5640 1 --2- 172.21.15.89:0/3651319467 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f88bc065680 0x7f88bc067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:58:08.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.949+0000 7f88e0619640 1 -- 172.21.15.89:0/3651319467 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f88d8089fd0 con 0x7f88d8160f20 2023-10-18T10:58:08.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:08.949+0000 7f88de3b5640 1 --2- 172.21.15.89:0/3651319467 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f88bc065680 0x7f88bc067b40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f88d40ad000 tx=0x7f88d40a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:58:09.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:09.109+0000 7f4424248640 1 -- 172.21.15.89:0/4054110552 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f441c04dbe0 con 0x7f441c0cac00 2023-10-18T10:58:09.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:09.109+0000 7f44127fc640 1 -- 172.21.15.89:0/4054110552 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4877 (secure 0 0 0) 0x7f44140eef00 con 0x7f441c0cac00 2023-10-18T10:58:09.109 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:58:09.109 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/654729641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":654729641},{"type":"v1","addr":"172.21.15.154:6836","nonce":654729641}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/964566480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":964566480},{"type":"v1","addr":"172.21.15.154:6837","nonce":964566480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2261878538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2261878538},{"type":"v1","addr":"172.21.15.89:6833","nonce":2261878538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:47:15.199575+0000","modified":"2023-10-18T10:55:20.892655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5517":{"gid":5517,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":124,"addr":"172.21.15.89:6835/2092274731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2092274731},{"type":"v1","addr":"172.21.15.89:6835","nonce":2092274731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T10:58:09.110 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-18T10:58:09.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:09.109+0000 7f4424248640 1 -- 172.21.15.89:0/4054110552 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f44000656b0 msgr2=0x7f4400067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:58:09.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:09.109+0000 7f4424248640 1 --2- 172.21.15.89:0/4054110552 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f44000656b0 0x7f4400067b70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4418005890 tx=0x7f4418009040 comp rx=0 tx=0).stop 2023-10-18T10:58:09.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:09.109+0000 7f4424248640 1 -- 172.21.15.89:0/4054110552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f441c0cac00 msgr2=0x7f441c0b61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:58:09.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:09.109+0000 7f4424248640 1 --2- 172.21.15.89:0/4054110552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f441c0cac00 0x7f441c0b61d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f441401a510 tx=0x7f4414019680 comp rx=0 tx=0).stop 2023-10-18T10:58:09.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:09.109+0000 7f4424248640 1 -- 172.21.15.89:0/4054110552 shutdown_connections 2023-10-18T10:58:09.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:09.109+0000 7f4424248640 1 --2- 172.21.15.89:0/4054110552 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f44000656b0 0x7f4400067b70 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:58:09.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:09.109+0000 7f4424248640 1 --2- 172.21.15.89:0/4054110552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f441c0cac00 0x7f441c0b61d0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:58:09.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:09.109+0000 7f4424248640 1 --2- 172.21.15.89:0/4054110552 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f441c0d0bc0 0x7f441c0ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:58:09.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:09.109+0000 7f4424248640 1 --2- 172.21.15.89:0/4054110552 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f441c0d0760 0x7f441c150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:58:09.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:09.109+0000 7f4424248640 1 -- 172.21.15.89:0/4054110552 >> 172.21.15.89:0/4054110552 conn(0x7f441c089fd0 msgr2=0x7f441c158650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:58:09.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:09.109+0000 7f4424248640 1 -- 172.21.15.89:0/4054110552 shutdown_connections 2023-10-18T10:58:09.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:09.109+0000 7f4424248640 1 -- 172.21.15.89:0/4054110552 wait complete. 2023-10-18T10:58:09.129 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-18T10:58:09.286 INFO:teuthology.orchestra.run.smithi089.stdout:[{"id":5183,"entity":{"name":{"type":"client","num":5183},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":4272238049}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":645.57876830500004,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.22501152025992219,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":3,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5183 v1:192.168.0.1:0/4272238049","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi154","kernel_version":"6.6.0-rc5-g47672119149b","root":"/"}}] 2023-10-18T10:58:09.288 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "1", "hostname": "smithi154", "kernel_version": "6.6.0-rc5-g47672119149b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 4272238049, "type": "v1" }, "name": { "num": 5183, "type": "client" } }, "id": 5183, "inst": "client.5183 v1:192.168.0.1:0/4272238049", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 3, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.2250115202599222 }, "state": "open", "uptime": 645.578768305 } ] 2023-10-18T10:58:09.389 DEBUG:tasks.ceph_test_case:No log hits yet, waiting... 2023-10-18T10:58:20.988 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:58:20.992 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:58:44.407 ERROR:tasks.ceph_test_case:Log output: 2023-10-18T10:58:44.407 DEBUG:tasks.cephfs.test_client_recovery:Expected log message found: 'Health check cleared: MDS_CLIENTS_LAGGY', retrying 2023-10-18T10:58:49.409 DEBUG:teuthology.orchestra.run.smithi089:> sudo daemon-helper kill ceph --cluster ceph -w 2023-10-18T10:58:49.413 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:58:49.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.560+0000 7f4eaea0b640 1 Processor -- start 2023-10-18T10:58:49.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.560+0000 7f4eaea0b640 1 -- start start 2023-10-18T10:58:49.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.564+0000 7f4eaea0b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4ea80cd4f0 0x7f4ea8143470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:58:49.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.564+0000 7f4eaea0b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4ea80ce070 0x7f4ea80ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:58:49.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.564+0000 7f4eaea0b640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea80cd8d0 con 0x7f4ea8145b70 2023-10-18T10:58:49.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.564+0000 7f4eaea0b640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ea8069900 con 0x7f4ea80ce070 2023-10-18T10:58:49.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.564+0000 7f4eaea0b640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ea80ce9e0 con 0x7f4ea80cd4f0 2023-10-18T10:58:49.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.564+0000 7f4eacfa8640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4ea80ce070 0x7f4ea80ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:58:49.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.564+0000 7f4eacfa8640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4ea80ce070 0x7f4ea80ce4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53834/0 (socket says 172.21.15.89:53834) 2023-10-18T10:58:49.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.564+0000 7f4eacfa8640 1 -- 172.21.15.89:0/1648800772 learned_addr learned my addr 172.21.15.89:0/1648800772 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:58:49.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.564+0000 7f4ea7fff640 1 --2- 172.21.15.89:0/1648800772 >> v2:172.21.15.154:3300/0 conn(0x7f4ea80cd4f0 0x7f4ea8143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:58:49.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.564+0000 7f4eacfa8640 1 -- 172.21.15.89:0/1648800772 >> v2:172.21.15.154:3300/0 conn(0x7f4ea80cd4f0 msgr2=0x7f4ea8143470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:58:49.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.564+0000 7f4eacfa8640 1 --2- 172.21.15.89:0/1648800772 >> v2:172.21.15.154:3300/0 conn(0x7f4ea80cd4f0 0x7f4ea8143470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:58:49.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.564+0000 7f4eacfa8640 1 -- 172.21.15.89:0/1648800772 >> v1:172.21.15.154:6789/0 conn(0x7f4ea8145b70 legacy=0x7f4ea8160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:58:49.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.564+0000 7f4eacfa8640 1 -- 172.21.15.89:0/1648800772 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea815cbf0 con 0x7f4ea80ce070 2023-10-18T10:58:49.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.568+0000 7f4eacfa8640 1 --2- 172.21.15.89:0/1648800772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ea80ce070 0x7f4ea80ce4a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4ea80716d0 tx=0x7f4ea001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a83f9ab9ae925fe4 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:58:49.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.568+0000 7f4ea6ffd640 1 -- 172.21.15.89:0/1648800772 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea00a5860 con 0x7f4ea80ce070 2023-10-18T10:58:49.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.568+0000 7f4ea6ffd640 1 -- 172.21.15.89:0/1648800772 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea00a59c0 con 0x7f4ea80ce070 2023-10-18T10:58:49.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.568+0000 7f4ea6ffd640 1 -- 172.21.15.89:0/1648800772 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea00a5d50 con 0x7f4ea80ce070 2023-10-18T10:58:49.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.568+0000 7f4eaea0b640 1 -- 172.21.15.89:0/1648800772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ea80ce070 msgr2=0x7f4ea80ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:58:49.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.568+0000 7f4eaea0b640 1 --2- 172.21.15.89:0/1648800772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ea80ce070 0x7f4ea80ce4a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4ea80716d0 tx=0x7f4ea001c9a0 comp rx=0 tx=0).stop 2023-10-18T10:58:49.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.568+0000 7f4eaea0b640 1 -- 172.21.15.89:0/1648800772 shutdown_connections 2023-10-18T10:58:49.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.568+0000 7f4eaea0b640 1 --2- 172.21.15.89:0/1648800772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ea80ce070 0x7f4ea80ce4a0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:58:49.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.568+0000 7f4eaea0b640 1 --2- 172.21.15.89:0/1648800772 >> v2:172.21.15.154:3300/0 conn(0x7f4ea80cd4f0 0x7f4ea8143470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:58:49.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.568+0000 7f4eaea0b640 1 -- 172.21.15.89:0/1648800772 >> 172.21.15.89:0/1648800772 conn(0x7f4ea806dad0 msgr2=0x7f4ea806dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:58:49.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.568+0000 7f4eaea0b640 1 -- 172.21.15.89:0/1648800772 shutdown_connections 2023-10-18T10:58:49.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.568+0000 7f4eaea0b640 1 -- 172.21.15.89:0/1648800772 wait complete. 2023-10-18T10:58:49.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.568+0000 7f4eaea0b640 1 Processor -- start 2023-10-18T10:58:49.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.572+0000 7f4eaea0b640 1 -- start start 2023-10-18T10:58:49.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.572+0000 7f4eaea0b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ea80cd4f0 0x7f4ea80c5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:58:49.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.572+0000 7f4eaea0b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ea80ce070 0x7f4ea80cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:58:49.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.572+0000 7f4eaea0b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ea8145b70 0x7f4ea80c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:58:49.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.572+0000 7f4eaea0b640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea80c5900 con 0x7f4ea8145b70 2023-10-18T10:58:49.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.572+0000 7f4eaea0b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea80c6b30 con 0x7f4ea80ce070 2023-10-18T10:58:49.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.572+0000 7f4eaea0b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea80c5b50 con 0x7f4ea80cd4f0 2023-10-18T10:58:49.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.572+0000 7f4eacfa8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ea8145b70 0x7f4ea80c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:58:49.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.572+0000 7f4eacfa8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ea8145b70 0x7f4ea80c3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53838/0 (socket says 172.21.15.89:53838) 2023-10-18T10:58:49.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.572+0000 7f4eacfa8640 1 -- 172.21.15.89:0/3735784753 learned_addr learned my addr 172.21.15.89:0/3735784753 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:58:49.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.572+0000 7f4ea7fff640 1 --2- 172.21.15.89:0/3735784753 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ea80cd4f0 0x7f4ea80c5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:58:49.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.572+0000 7f4ea77fe640 1 --2- 172.21.15.89:0/3735784753 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ea80ce070 0x7f4ea80cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:58:49.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.572+0000 7f4eacfa8640 1 -- 172.21.15.89:0/3735784753 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ea80cd4f0 msgr2=0x7f4ea80c5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:58:49.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.572+0000 7f4eacfa8640 1 --2- 172.21.15.89:0/3735784753 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ea80cd4f0 0x7f4ea80c5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:58:49.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.572+0000 7f4eacfa8640 1 -- 172.21.15.89:0/3735784753 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ea80ce070 msgr2=0x7f4ea80cbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:58:49.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.572+0000 7f4eacfa8640 1 --2- 172.21.15.89:0/3735784753 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ea80ce070 0x7f4ea80cbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:58:49.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.572+0000 7f4eacfa8640 1 -- 172.21.15.89:0/3735784753 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea0009b30 con 0x7f4ea8145b70 2023-10-18T10:58:49.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.572+0000 7f4eacfa8640 1 --2- 172.21.15.89:0/3735784753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ea8145b70 0x7f4ea80c3fd0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4ea00af040 tx=0x7f4ea0019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:58:49.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.572+0000 7f4ea4ff9640 1 -- 172.21.15.89:0/3735784753 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea00b8670 con 0x7f4ea8145b70 2023-10-18T10:58:49.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.572+0000 7f4ea4ff9640 1 -- 172.21.15.89:0/3735784753 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea00b87d0 con 0x7f4ea8145b70 2023-10-18T10:58:49.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.572+0000 7f4eaea0b640 1 -- 172.21.15.89:0/3735784753 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea80c5e30 con 0x7f4ea8145b70 2023-10-18T10:58:49.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.572+0000 7f4ea4ff9640 1 -- 172.21.15.89:0/3735784753 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea00b8b80 con 0x7f4ea8145b70 2023-10-18T10:58:49.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.572+0000 7f4eaea0b640 1 -- 172.21.15.89:0/3735784753 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea81527a0 con 0x7f4ea8145b70 2023-10-18T10:58:49.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.572+0000 7f4eaea0b640 1 -- 172.21.15.89:0/3735784753 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ea804dbe0 con 0x7f4ea8145b70 2023-10-18T10:58:49.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.576+0000 7f4ea4ff9640 1 -- 172.21.15.89:0/3735784753 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f4ea00b8ce0 con 0x7f4ea8145b70 2023-10-18T10:58:49.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.576+0000 7f4ea4ff9640 1 --2- 172.21.15.89:0/3735784753 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4e940656b0 0x7f4e94067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:58:49.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.576+0000 7f4ea4ff9640 1 -- 172.21.15.89:0/3735784753 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f4ea011e610 con 0x7f4ea8145b70 2023-10-18T10:58:49.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.576+0000 7f4ea7fff640 1 --2- 172.21.15.89:0/3735784753 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4e940656b0 0x7f4e94067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:58:49.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.576+0000 7f4ea7fff640 1 --2- 172.21.15.89:0/3735784753 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4e940656b0 0x7f4e94067b70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4e98006aa0 tx=0x7f4e98008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:58:49.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.580+0000 7f4ea4ff9640 1 -- 172.21.15.89:0/3735784753 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f4ea00ec100 con 0x7f4ea8145b70 2023-10-18T10:58:49.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.588+0000 7f54110cb640 1 Processor -- start 2023-10-18T10:58:49.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.588+0000 7f54110cb640 1 -- start start 2023-10-18T10:58:49.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.592+0000 7f54110cb640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f540c160f20 0x7f540c0d0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:58:49.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.592+0000 7f54110cb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f540c0b15e0 0x7f540c0d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:58:49.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.592+0000 7f54110cb640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f540c0cced0 con 0x7f540c0b33b0 2023-10-18T10:58:49.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.592+0000 7f54110cb640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f540c0b1d30 con 0x7f540c160f20 2023-10-18T10:58:49.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.592+0000 7f54110cb640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f540c0b3bc0 con 0x7f540c0b15e0 2023-10-18T10:58:49.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.592+0000 7f540ad9d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f540c160f20 0x7f540c0d0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:58:49.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.592+0000 7f540ad9d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f540c160f20 0x7f540c0d0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53842/0 (socket says 172.21.15.89:53842) 2023-10-18T10:58:49.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.592+0000 7f540a59c640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f540c0b33b0 0x7f540c0b1fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57534/0 (socket says 172.21.15.89:57534) 2023-10-18T10:58:49.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.592+0000 7f540ad9d640 1 -- 172.21.15.89:0/119985020 learned_addr learned my addr 172.21.15.89:0/119985020 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:58:49.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.592+0000 7f540ad9d640 1 -- 172.21.15.89:0/119985020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f540c0b15e0 msgr2=0x7f540c0d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:58:49.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.592+0000 7f540b59e640 1 --2- 172.21.15.89:0/119985020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f540c0b15e0 0x7f540c0d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:58:49.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.592+0000 7f5409d9b640 1 -- 172.21.15.89:0/119985020 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3050530106 0 0) 0x7f540c0cced0 con 0x7f540c0b33b0 2023-10-18T10:58:49.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.592+0000 7f540ad9d640 1 --2- 172.21.15.89:0/119985020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f540c0b15e0 0x7f540c0d0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:58:49.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.592+0000 7f540ad9d640 1 -- 172.21.15.89:0/119985020 >> v1:172.21.15.89:6789/0 conn(0x7f540c0b33b0 legacy=0x7f540c0b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:58:49.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.592+0000 7f540ad9d640 1 -- 172.21.15.89:0/119985020 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f540c0afdd0 con 0x7f540c160f20 2023-10-18T10:58:49.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.592+0000 7f540b59e640 1 --2- 172.21.15.89:0/119985020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f540c0b15e0 0x7f540c0d0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:58:49.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.596+0000 7f540ad9d640 1 --2- 172.21.15.89:0/119985020 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f540c160f20 0x7f540c0d0750 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f53fc009870 tx=0x7f53fc01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae3aa75f609d0003 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:58:49.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.596+0000 7f5409d9b640 1 -- 172.21.15.89:0/119985020 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc0ad680 con 0x7f540c160f20 2023-10-18T10:58:49.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.596+0000 7f5409d9b640 1 -- 172.21.15.89:0/119985020 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53fc0ad7e0 con 0x7f540c160f20 2023-10-18T10:58:49.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.596+0000 7f5409d9b640 1 -- 172.21.15.89:0/119985020 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc0adb70 con 0x7f540c160f20 2023-10-18T10:58:49.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.596+0000 7f54110cb640 1 -- 172.21.15.89:0/119985020 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f540c160f20 msgr2=0x7f540c0d0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:58:49.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.596+0000 7f54110cb640 1 --2- 172.21.15.89:0/119985020 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f540c160f20 0x7f540c0d0750 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f53fc009870 tx=0x7f53fc01c4b0 comp rx=0 tx=0).stop 2023-10-18T10:58:49.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.596+0000 7f54110cb640 1 -- 172.21.15.89:0/119985020 shutdown_connections 2023-10-18T10:58:49.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.596+0000 7f54110cb640 1 --2- 172.21.15.89:0/119985020 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f540c0b15e0 0x7f540c0d0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:58:49.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.596+0000 7f54110cb640 1 --2- 172.21.15.89:0/119985020 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f540c160f20 0x7f540c0d0750 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:58:49.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.596+0000 7f54110cb640 1 -- 172.21.15.89:0/119985020 >> 172.21.15.89:0/119985020 conn(0x7f540c09b930 msgr2=0x7f540c0d26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:58:49.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.596+0000 7f54110cb640 1 -- 172.21.15.89:0/119985020 shutdown_connections 2023-10-18T10:58:49.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.596+0000 7f54110cb640 1 -- 172.21.15.89:0/119985020 wait complete. 2023-10-18T10:58:49.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.596+0000 7f54110cb640 1 Processor -- start 2023-10-18T10:58:49.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.600+0000 7f54110cb640 1 -- start start 2023-10-18T10:58:49.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.600+0000 7f54110cb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f540c0b15e0 0x7f540c0c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:58:49.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.600+0000 7f54110cb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f540c0b33b0 0x7f540c06a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:58:49.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.600+0000 7f54110cb640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f540c160f20 0x7f540c1595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:58:49.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.600+0000 7f54110cb640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540c0c9770 con 0x7f540c160f20 2023-10-18T10:58:49.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.600+0000 7f54110cb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540c0cbf00 con 0x7f540c0b15e0 2023-10-18T10:58:49.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.600+0000 7f54110cb640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f540c0cb5d0 con 0x7f540c0b33b0 2023-10-18T10:58:49.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.600+0000 7f540b59e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f540c160f20 0x7f540c1595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:58:49.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.600+0000 7f540ad9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f540c0b15e0 0x7f540c0c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:58:49.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.600+0000 7f540b59e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f540c160f20 0x7f540c1595d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53844/0 (socket says 172.21.15.89:53844) 2023-10-18T10:58:49.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.600+0000 7f540b59e640 1 -- 172.21.15.89:0/251691521 learned_addr learned my addr 172.21.15.89:0/251691521 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:58:49.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.600+0000 7f540a59c640 1 --2- 172.21.15.89:0/251691521 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f540c0b33b0 0x7f540c06a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:58:49.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.600+0000 7f540b59e640 1 -- 172.21.15.89:0/251691521 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f540c0b33b0 msgr2=0x7f540c06a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:58:49.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.600+0000 7f540b59e640 1 --2- 172.21.15.89:0/251691521 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f540c0b33b0 0x7f540c06a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:58:49.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.600+0000 7f540b59e640 1 -- 172.21.15.89:0/251691521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f540c0b15e0 msgr2=0x7f540c0c89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:58:49.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.600+0000 7f540b59e640 1 --2- 172.21.15.89:0/251691521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f540c0b15e0 0x7f540c0c89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:58:49.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.600+0000 7f540b59e640 1 -- 172.21.15.89:0/251691521 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53fc0094d0 con 0x7f540c160f20 2023-10-18T10:58:49.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.600+0000 7f540ad9d640 1 --2- 172.21.15.89:0/251691521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f540c0b15e0 0x7f540c0c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:58:49.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.600+0000 7f540b59e640 1 --2- 172.21.15.89:0/251691521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f540c160f20 0x7f540c1595d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f540400cea0 tx=0x7f540400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:58:49.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.600+0000 7f53f37fe640 1 -- 172.21.15.89:0/251691521 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5404003ba0 con 0x7f540c160f20 2023-10-18T10:58:49.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.600+0000 7f53f37fe640 1 -- 172.21.15.89:0/251691521 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5404003d20 con 0x7f540c160f20 2023-10-18T10:58:49.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.600+0000 7f53f37fe640 1 -- 172.21.15.89:0/251691521 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540400d880 con 0x7f540c160f20 2023-10-18T10:58:49.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.600+0000 7f54110cb640 1 -- 172.21.15.89:0/251691521 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f540c0cb8b0 con 0x7f540c160f20 2023-10-18T10:58:49.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.600+0000 7f54110cb640 1 -- 172.21.15.89:0/251691521 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f540c0c8f00 con 0x7f540c160f20 2023-10-18T10:58:49.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.600+0000 7f54110cb640 1 -- 172.21.15.89:0/251691521 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f540c04e430 con 0x7f540c160f20 2023-10-18T10:58:49.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.604+0000 7f53f37fe640 1 -- 172.21.15.89:0/251691521 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f540400d9e0 con 0x7f540c160f20 2023-10-18T10:58:49.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.604+0000 7f53f37fe640 1 --2- 172.21.15.89:0/251691521 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f53f80656b0 0x7f53f8067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:58:49.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.604+0000 7f53f37fe640 1 -- 172.21.15.89:0/251691521 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f5404083770 con 0x7f540c160f20 2023-10-18T10:58:49.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.604+0000 7f53f37fe640 1 -- 172.21.15.89:0/251691521 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+449 (secure 0 0 0) 0x7f540408a070 con 0x7f540c160f20 2023-10-18T10:58:49.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.604+0000 7f540ad9d640 1 --2- 172.21.15.89:0/251691521 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f53f80656b0 0x7f53f8067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:58:49.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.604+0000 7f54110cb640 1 -- 172.21.15.89:0/251691521 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f540c089fd0 con 0x7f540c160f20 2023-10-18T10:58:49.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.604+0000 7f540ad9d640 1 --2- 172.21.15.89:0/251691521 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f53f80656b0 0x7f53f8067b70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f53fc0af000 tx=0x7f53fc0a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:58:49.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.764+0000 7f4eaea0b640 1 -- 172.21.15.89:0/3735784753 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ea80b0a10 con 0x7f4ea8145b70 2023-10-18T10:58:49.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.764+0000 7f4ea4ff9640 1 -- 172.21.15.89:0/3735784753 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4877 (secure 0 0 0) 0x7f4ea00ef720 con 0x7f4ea8145b70 2023-10-18T10:58:49.764 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:58:49.764 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/654729641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":654729641},{"type":"v1","addr":"172.21.15.154:6836","nonce":654729641}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/964566480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":964566480},{"type":"v1","addr":"172.21.15.154:6837","nonce":964566480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2261878538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2261878538},{"type":"v1","addr":"172.21.15.89:6833","nonce":2261878538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:47:15.199575+0000","modified":"2023-10-18T10:55:20.892655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5517":{"gid":5517,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":124,"addr":"172.21.15.89:6835/2092274731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2092274731},{"type":"v1","addr":"172.21.15.89:6835","nonce":2092274731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T10:58:49.765 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-18T10:58:49.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.764+0000 7f4eaea0b640 1 -- 172.21.15.89:0/3735784753 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4e940656b0 msgr2=0x7f4e94067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:58:49.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.764+0000 7f4eaea0b640 1 --2- 172.21.15.89:0/3735784753 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4e940656b0 0x7f4e94067b70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4e98006aa0 tx=0x7f4e98008040 comp rx=0 tx=0).stop 2023-10-18T10:58:49.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.764+0000 7f4eaea0b640 1 -- 172.21.15.89:0/3735784753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ea8145b70 msgr2=0x7f4ea80c3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:58:49.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.764+0000 7f4eaea0b640 1 --2- 172.21.15.89:0/3735784753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ea8145b70 0x7f4ea80c3fd0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4ea00af040 tx=0x7f4ea0019740 comp rx=0 tx=0).stop 2023-10-18T10:58:49.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.764+0000 7f4eaea0b640 1 -- 172.21.15.89:0/3735784753 shutdown_connections 2023-10-18T10:58:49.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.764+0000 7f4eaea0b640 1 --2- 172.21.15.89:0/3735784753 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4e940656b0 0x7f4e94067b70 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:58:49.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.764+0000 7f4eaea0b640 1 --2- 172.21.15.89:0/3735784753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ea8145b70 0x7f4ea80c3fd0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:58:49.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.764+0000 7f4eaea0b640 1 --2- 172.21.15.89:0/3735784753 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ea80ce070 0x7f4ea80cbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:58:49.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.764+0000 7f4eaea0b640 1 --2- 172.21.15.89:0/3735784753 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ea80cd4f0 0x7f4ea80c5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:58:49.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.764+0000 7f4eaea0b640 1 -- 172.21.15.89:0/3735784753 >> 172.21.15.89:0/3735784753 conn(0x7f4ea806dad0 msgr2=0x7f4ea8159210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:58:49.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.764+0000 7f4eaea0b640 1 -- 172.21.15.89:0/3735784753 shutdown_connections 2023-10-18T10:58:49.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:58:49.764+0000 7f4eaea0b640 1 -- 172.21.15.89:0/3735784753 wait complete. 2023-10-18T10:58:49.786 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-18T10:58:49.944 INFO:teuthology.orchestra.run.smithi089.stdout:[{"id":5183,"entity":{"name":{"type":"client","num":5183},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":4272238049}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":686.23360131799996,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.20483791184250577,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":3,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5183 v1:192.168.0.1:0/4272238049","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi154","kernel_version":"6.6.0-rc5-g47672119149b","root":"/"}}] 2023-10-18T10:58:49.945 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "1", "hostname": "smithi154", "kernel_version": "6.6.0-rc5-g47672119149b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 4272238049, "type": "v1" }, "name": { "num": 5183, "type": "client" } }, "id": 5183, "inst": "client.5183 v1:192.168.0.1:0/4272238049", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 3, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.20483791184250577 }, "state": "open", "uptime": 686.233601318 } ] 2023-10-18T10:58:50.046 DEBUG:tasks.ceph_test_case:No log hits yet, waiting... 2023-10-18T10:58:51.111 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:58:51.115 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:59:21.167 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:59:21.172 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:59:25.064 ERROR:tasks.ceph_test_case:Log output: 2023-10-18T10:59:25.065 DEBUG:tasks.cephfs.test_client_recovery:Expected log message found: 'Health check cleared: MDS_CLIENTS_LAGGY', retrying 2023-10-18T10:59:30.066 DEBUG:teuthology.orchestra.run.smithi089:> sudo daemon-helper kill ceph --cluster ceph -w 2023-10-18T10:59:30.071 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:59:30.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.223+0000 7faa3406a640 1 Processor -- start 2023-10-18T10:59:30.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.223+0000 7faa3406a640 1 -- start start 2023-10-18T10:59:30.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.223+0000 7faa3406a640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faa2c15c940 0x7faa2c0d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:59:30.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.223+0000 7faa3406a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa2c0d0e70 0x7faa2c0cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:59:30.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.223+0000 7faa3406a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa2c15da50 con 0x7faa2c0d2f40 2023-10-18T10:59:30.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.223+0000 7faa3406a640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa2c15ce40 con 0x7faa2c15c940 2023-10-18T10:59:30.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.223+0000 7faa3406a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c15d120 con 0x7faa2c0d0e70 2023-10-18T10:59:30.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.223+0000 7faa31e06640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faa2c15c940 0x7faa2c0d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:59:30.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.223+0000 7faa31e06640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faa2c15c940 0x7faa2c0d2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53150/0 (socket says 172.21.15.89:53150) 2023-10-18T10:59:30.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.223+0000 7faa31e06640 1 -- 172.21.15.89:0/1265903026 learned_addr learned my addr 172.21.15.89:0/1265903026 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:59:30.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.227+0000 7faa32607640 1 --2- 172.21.15.89:0/1265903026 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa2c0d0e70 0x7faa2c0cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:59:30.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.227+0000 7faa31e06640 1 -- 172.21.15.89:0/1265903026 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa2c0d0e70 msgr2=0x7faa2c0cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:59:30.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.227+0000 7faa31e06640 1 --2- 172.21.15.89:0/1265903026 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa2c0d0e70 0x7faa2c0cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:59:30.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.227+0000 7faa31e06640 1 -- 172.21.15.89:0/1265903026 >> v1:172.21.15.154:6789/0 conn(0x7faa2c0d2f40 legacy=0x7faa2c0d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:59:30.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.227+0000 7faa31e06640 1 -- 172.21.15.89:0/1265903026 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2c15d400 con 0x7faa2c15c940 2023-10-18T10:59:30.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.227+0000 7faa31e06640 1 --2- 172.21.15.89:0/1265903026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa2c15c940 0x7faa2c0d2a00 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7faa28009870 tx=0x7faa2801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8339b904785f1b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:59:30.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.227+0000 7faa30e04640 1 -- 172.21.15.89:0/1265903026 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa280aa610 con 0x7faa2c15c940 2023-10-18T10:59:30.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.227+0000 7faa30e04640 1 -- 172.21.15.89:0/1265903026 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa280a6040 con 0x7faa2c15c940 2023-10-18T10:59:30.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.227+0000 7faa30e04640 1 -- 172.21.15.89:0/1265903026 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa280aa990 con 0x7faa2c15c940 2023-10-18T10:59:30.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.231+0000 7faa3406a640 1 -- 172.21.15.89:0/1265903026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa2c15c940 msgr2=0x7faa2c0d2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:59:30.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.231+0000 7faa3406a640 1 --2- 172.21.15.89:0/1265903026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa2c15c940 0x7faa2c0d2a00 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7faa28009870 tx=0x7faa2801c510 comp rx=0 tx=0).stop 2023-10-18T10:59:30.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.231+0000 7faa3406a640 1 -- 172.21.15.89:0/1265903026 shutdown_connections 2023-10-18T10:59:30.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.231+0000 7faa3406a640 1 --2- 172.21.15.89:0/1265903026 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa2c0d0e70 0x7faa2c0cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:59:30.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.231+0000 7faa3406a640 1 --2- 172.21.15.89:0/1265903026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa2c15c940 0x7faa2c0d2a00 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:59:30.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.231+0000 7faa3406a640 1 -- 172.21.15.89:0/1265903026 >> 172.21.15.89:0/1265903026 conn(0x7faa2c089fd0 msgr2=0x7faa2c09b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:59:30.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.231+0000 7faa3406a640 1 -- 172.21.15.89:0/1265903026 shutdown_connections 2023-10-18T10:59:30.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.231+0000 7faa3406a640 1 -- 172.21.15.89:0/1265903026 wait complete. 2023-10-18T10:59:30.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.231+0000 7faa3406a640 1 Processor -- start 2023-10-18T10:59:30.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.231+0000 7faa3406a640 1 -- start start 2023-10-18T10:59:30.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.231+0000 7faa3406a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa2c0d0e70 0x7faa2c0ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:59:30.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.231+0000 7faa3406a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa2c09efb0 0x7faa2c15b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:59:30.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.231+0000 7faa3406a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa2c0b7440 0x7faa2c0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:59:30.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.231+0000 7faa3406a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c0b7e30 con 0x7faa2c09efb0 2023-10-18T10:59:30.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.231+0000 7faa3406a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c09f800 con 0x7faa2c0d0e70 2023-10-18T10:59:30.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.231+0000 7faa3406a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c09fae0 con 0x7faa2c0b7440 2023-10-18T10:59:30.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.231+0000 7faa31605640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa2c09efb0 0x7faa2c15b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:59:30.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.231+0000 7faa31605640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa2c09efb0 0x7faa2c15b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53160/0 (socket says 172.21.15.89:53160) 2023-10-18T10:59:30.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.231+0000 7faa31605640 1 -- 172.21.15.89:0/3353470863 learned_addr learned my addr 172.21.15.89:0/3353470863 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:59:30.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.231+0000 7faa32607640 1 --2- 172.21.15.89:0/3353470863 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa2c0b7440 0x7faa2c0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:59:30.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.231+0000 7faa31e06640 1 --2- 172.21.15.89:0/3353470863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa2c0d0e70 0x7faa2c0ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:59:30.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.235+0000 7faa31605640 1 -- 172.21.15.89:0/3353470863 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa2c0b7440 msgr2=0x7faa2c0b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:59:30.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.235+0000 7faa31605640 1 --2- 172.21.15.89:0/3353470863 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa2c0b7440 0x7faa2c0b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:59:30.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.235+0000 7faa31605640 1 -- 172.21.15.89:0/3353470863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa2c0d0e70 msgr2=0x7faa2c0ca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:59:30.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.235+0000 7faa31605640 1 --2- 172.21.15.89:0/3353470863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa2c0d0e70 0x7faa2c0ca2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:59:30.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.235+0000 7faa31605640 1 -- 172.21.15.89:0/3353470863 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa280094d0 con 0x7faa2c09efb0 2023-10-18T10:59:30.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.235+0000 7faa31605640 1 --2- 172.21.15.89:0/3353470863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa2c09efb0 0x7faa2c15b550 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7faa1c00c500 tx=0x7faa1c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:59:30.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.235+0000 7faa227fc640 1 -- 172.21.15.89:0/3353470863 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1c005220 con 0x7faa2c09efb0 2023-10-18T10:59:30.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.235+0000 7faa227fc640 1 -- 172.21.15.89:0/3353470863 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa1c005380 con 0x7faa2c09efb0 2023-10-18T10:59:30.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.235+0000 7faa227fc640 1 -- 172.21.15.89:0/3353470863 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1c007270 con 0x7faa2c09efb0 2023-10-18T10:59:30.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.235+0000 7faa3406a640 1 -- 172.21.15.89:0/3353470863 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa2c09fdc0 con 0x7faa2c09efb0 2023-10-18T10:59:30.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.235+0000 7faa3406a640 1 -- 172.21.15.89:0/3353470863 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa2c15ba90 con 0x7faa2c09efb0 2023-10-18T10:59:30.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.235+0000 7faa3406a640 1 -- 172.21.15.89:0/3353470863 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa2c0a02e0 con 0x7faa2c09efb0 2023-10-18T10:59:30.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.235+0000 7faa227fc640 1 -- 172.21.15.89:0/3353470863 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7faa1c0073d0 con 0x7faa2c09efb0 2023-10-18T10:59:30.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.235+0000 7faa227fc640 1 --2- 172.21.15.89:0/3353470863 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7faa100656b0 0x7faa10067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:59:30.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.235+0000 7faa227fc640 1 -- 172.21.15.89:0/3353470863 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7faa1c0822b0 con 0x7faa2c09efb0 2023-10-18T10:59:30.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.235+0000 7faa31e06640 1 --2- 172.21.15.89:0/3353470863 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7faa100656b0 0x7faa10067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:59:30.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.239+0000 7faa31e06640 1 --2- 172.21.15.89:0/3353470863 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7faa100656b0 0x7faa10067b70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7faa280ae000 tx=0x7faa280b5040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:59:30.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.239+0000 7faa227fc640 1 -- 172.21.15.89:0/3353470863 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7faa2c0a02e0 con 0x7faa2c09efb0 2023-10-18T10:59:30.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.251+0000 7fbdc346a640 1 Processor -- start 2023-10-18T10:59:30.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.251+0000 7fbdc346a640 1 -- start start 2023-10-18T10:59:30.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.251+0000 7fbdc346a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdbc160f20 0x7fbdbc0d0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:59:30.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.251+0000 7fbdc346a640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdbc0cced0 con 0x7fbdbc0b15e0 2023-10-18T10:59:30.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.251+0000 7fbdc346a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdbc0b2dc0 con 0x7fbdbc0b33b0 2023-10-18T10:59:30.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.251+0000 7fbdc346a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc0b1d30 con 0x7fbdbc160f20 2023-10-18T10:59:30.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.251+0000 7fbdc1a07640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fbdbc0b15e0 0x7fbdbc0b26b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39154/0 (socket says 172.21.15.89:39154) 2023-10-18T10:59:30.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.251+0000 7fbdc1a07640 1 -- 172.21.15.89:0/16286988 learned_addr learned my addr 172.21.15.89:0/16286988 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T10:59:30.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.251+0000 7fbdc1206640 1 --2- 172.21.15.89:0/16286988 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdbc160f20 0x7fbdbc0d0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:59:30.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.251+0000 7fbdb3fff640 1 -- 172.21.15.89:0/16286988 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3307134327 0 0) 0x7fbdbc0cced0 con 0x7fbdbc0b15e0 2023-10-18T10:59:30.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.251+0000 7fbdb3fff640 1 -- 172.21.15.89:0/16286988 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbda8003530 con 0x7fbdbc0b15e0 2023-10-18T10:59:30.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.251+0000 7fbdb3fff640 1 -- 172.21.15.89:0/16286988 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1144287256 0 0) 0x7fbdbc0b2dc0 con 0x7fbdbc0b33b0 2023-10-18T10:59:30.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.251+0000 7fbdb3fff640 1 -- 172.21.15.89:0/16286988 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdbc0cced0 con 0x7fbdbc0b33b0 2023-10-18T10:59:30.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.251+0000 7fbdc1206640 1 -- 172.21.15.89:0/16286988 >> v1:172.21.15.154:6789/0 conn(0x7fbdbc0b33b0 legacy=0x7fbdbc0b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:59:30.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.251+0000 7fbdc1206640 1 -- 172.21.15.89:0/16286988 >> v1:172.21.15.89:6789/0 conn(0x7fbdbc0b15e0 legacy=0x7fbdbc0b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:59:30.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.251+0000 7fbdc1206640 1 -- 172.21.15.89:0/16286988 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdbc0b3bc0 con 0x7fbdbc160f20 2023-10-18T10:59:30.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.255+0000 7fbdc1206640 1 --2- 172.21.15.89:0/16286988 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdbc160f20 0x7fbdbc0d0750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbdb80097f0 tx=0x7fbdb801c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6f7032a4a0872a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:59:30.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.255+0000 7fbdb3fff640 1 -- 172.21.15.89:0/16286988 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb80ab600 con 0x7fbdbc160f20 2023-10-18T10:59:30.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.255+0000 7fbdb3fff640 1 -- 172.21.15.89:0/16286988 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb80a7050 con 0x7fbdbc160f20 2023-10-18T10:59:30.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.255+0000 7fbdb3fff640 1 -- 172.21.15.89:0/16286988 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb80ab960 con 0x7fbdbc160f20 2023-10-18T10:59:30.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.255+0000 7fbdc346a640 1 -- 172.21.15.89:0/16286988 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdbc160f20 msgr2=0x7fbdbc0d0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:59:30.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.255+0000 7fbdc346a640 1 --2- 172.21.15.89:0/16286988 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdbc160f20 0x7fbdbc0d0750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbdb80097f0 tx=0x7fbdb801c430 comp rx=0 tx=0).stop 2023-10-18T10:59:30.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.255+0000 7fbdc346a640 1 -- 172.21.15.89:0/16286988 shutdown_connections 2023-10-18T10:59:30.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.255+0000 7fbdc346a640 1 --2- 172.21.15.89:0/16286988 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdbc160f20 0x7fbdbc0d0750 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:59:30.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.255+0000 7fbdc346a640 1 -- 172.21.15.89:0/16286988 >> 172.21.15.89:0/16286988 conn(0x7fbdbc09b930 msgr2=0x7fbdbc0d26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:59:30.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.255+0000 7fbdc346a640 1 -- 172.21.15.89:0/16286988 shutdown_connections 2023-10-18T10:59:30.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.255+0000 7fbdc346a640 1 -- 172.21.15.89:0/16286988 wait complete. 2023-10-18T10:59:30.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.259+0000 7fbdc346a640 1 Processor -- start 2023-10-18T10:59:30.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.259+0000 7fbdc346a640 1 -- start start 2023-10-18T10:59:30.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.259+0000 7fbdc346a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdbc0b15e0 0x7fbdbc1536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:59:30.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.259+0000 7fbdc346a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdbc0b33b0 0x7fbdbc0cde50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:59:30.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.259+0000 7fbdc346a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdbc160f20 0x7fbdbc0ce390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:59:30.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.259+0000 7fbdc346a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc0a0360 con 0x7fbdbc0b33b0 2023-10-18T10:59:30.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.259+0000 7fbdc346a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc0c9810 con 0x7fbdbc160f20 2023-10-18T10:59:30.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.259+0000 7fbdc346a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc0c9af0 con 0x7fbdbc0b15e0 2023-10-18T10:59:30.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.259+0000 7fbdc0a05640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdbc0b33b0 0x7fbdbc0cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:59:30.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.259+0000 7fbdc0a05640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdbc0b33b0 0x7fbdbc0cde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53168/0 (socket says 172.21.15.89:53168) 2023-10-18T10:59:30.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.259+0000 7fbdc0a05640 1 -- 172.21.15.89:0/2752418439 learned_addr learned my addr 172.21.15.89:0/2752418439 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T10:59:30.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.259+0000 7fbdc1a07640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdbc160f20 0x7fbdbc0ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:59:30.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.259+0000 7fbdc1206640 1 --2- 172.21.15.89:0/2752418439 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdbc0b15e0 0x7fbdbc1536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:59:30.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.259+0000 7fbdc0a05640 1 -- 172.21.15.89:0/2752418439 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdbc0b15e0 msgr2=0x7fbdbc1536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:59:30.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.259+0000 7fbdc0a05640 1 --2- 172.21.15.89:0/2752418439 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdbc0b15e0 0x7fbdbc1536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:59:30.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.259+0000 7fbdc0a05640 1 -- 172.21.15.89:0/2752418439 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdbc160f20 msgr2=0x7fbdbc0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:59:30.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.259+0000 7fbdc0a05640 1 --2- 172.21.15.89:0/2752418439 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdbc160f20 0x7fbdbc0ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:59:30.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.259+0000 7fbdc0a05640 1 -- 172.21.15.89:0/2752418439 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb8009420 con 0x7fbdbc0b33b0 2023-10-18T10:59:30.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.259+0000 7fbdc0a05640 1 --2- 172.21.15.89:0/2752418439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdbc0b33b0 0x7fbdbc0cde50 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fbdac00c410 tx=0x7fbdac00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:59:30.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.259+0000 7fbdb1ffb640 1 -- 172.21.15.89:0/2752418439 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdac0052d0 con 0x7fbdbc0b33b0 2023-10-18T10:59:30.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.259+0000 7fbdb1ffb640 1 -- 172.21.15.89:0/2752418439 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdac005450 con 0x7fbdbc0b33b0 2023-10-18T10:59:30.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.259+0000 7fbdb1ffb640 1 -- 172.21.15.89:0/2752418439 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdac003840 con 0x7fbdbc0b33b0 2023-10-18T10:59:30.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.259+0000 7fbdc346a640 1 -- 172.21.15.89:0/2752418439 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdbc0c9dd0 con 0x7fbdbc0b33b0 2023-10-18T10:59:30.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.263+0000 7fbdc346a640 1 -- 172.21.15.89:0/2752418439 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdbc0b75f0 con 0x7fbdbc0b33b0 2023-10-18T10:59:30.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.263+0000 7fbdc346a640 1 -- 172.21.15.89:0/2752418439 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fbdbc04e430 con 0x7fbdbc0b33b0 2023-10-18T10:59:30.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.263+0000 7fbdb1ffb640 1 -- 172.21.15.89:0/2752418439 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fbdac0039a0 con 0x7fbdbc0b33b0 2023-10-18T10:59:30.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.263+0000 7fbdb1ffb640 1 --2- 172.21.15.89:0/2752418439 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fbda0065680 0x7fbda0067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:59:30.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.263+0000 7fbdb1ffb640 1 -- 172.21.15.89:0/2752418439 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fbdac082680 con 0x7fbdbc0b33b0 2023-10-18T10:59:30.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.263+0000 7fbdb1ffb640 1 -- 172.21.15.89:0/2752418439 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+449 (secure 0 0 0) 0x7fbdac089070 con 0x7fbdbc0b33b0 2023-10-18T10:59:30.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.263+0000 7fbdc1206640 1 --2- 172.21.15.89:0/2752418439 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fbda0065680 0x7fbda0067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:59:30.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.263+0000 7fbdc346a640 1 -- 172.21.15.89:0/2752418439 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7fbdbc089fd0 con 0x7fbdbc0b33b0 2023-10-18T10:59:30.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.263+0000 7fbdc1206640 1 --2- 172.21.15.89:0/2752418439 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fbda0065680 0x7fbda0067b40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbdb80ad000 tx=0x7fbdb80a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:59:30.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.423+0000 7faa3406a640 1 -- 172.21.15.89:0/3353470863 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa2c0a02e0 con 0x7faa2c09efb0 2023-10-18T10:59:30.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.423+0000 7faa227fc640 1 -- 172.21.15.89:0/3353470863 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4877 (secure 0 0 0) 0x7faa2c0a02e0 con 0x7faa2c09efb0 2023-10-18T10:59:30.423 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T10:59:30.423 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/654729641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":654729641},{"type":"v1","addr":"172.21.15.154:6836","nonce":654729641}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/964566480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":964566480},{"type":"v1","addr":"172.21.15.154:6837","nonce":964566480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2261878538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2261878538},{"type":"v1","addr":"172.21.15.89:6833","nonce":2261878538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:47:15.199575+0000","modified":"2023-10-18T10:55:20.892655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5517":{"gid":5517,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":124,"addr":"172.21.15.89:6835/2092274731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2092274731},{"type":"v1","addr":"172.21.15.89:6835","nonce":2092274731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T10:59:30.424 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-18T10:59:30.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.427+0000 7faa3406a640 1 -- 172.21.15.89:0/3353470863 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7faa100656b0 msgr2=0x7faa10067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:59:30.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.427+0000 7faa3406a640 1 --2- 172.21.15.89:0/3353470863 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7faa100656b0 0x7faa10067b70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7faa280ae000 tx=0x7faa280b5040 comp rx=0 tx=0).stop 2023-10-18T10:59:30.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.427+0000 7faa3406a640 1 -- 172.21.15.89:0/3353470863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa2c09efb0 msgr2=0x7faa2c15b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:59:30.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.427+0000 7faa3406a640 1 --2- 172.21.15.89:0/3353470863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa2c09efb0 0x7faa2c15b550 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7faa1c00c500 tx=0x7faa1c00c9c0 comp rx=0 tx=0).stop 2023-10-18T10:59:30.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.427+0000 7faa3406a640 1 -- 172.21.15.89:0/3353470863 shutdown_connections 2023-10-18T10:59:30.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.427+0000 7faa3406a640 1 --2- 172.21.15.89:0/3353470863 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7faa100656b0 0x7faa10067b70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:59:30.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.427+0000 7faa3406a640 1 --2- 172.21.15.89:0/3353470863 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa2c0b7440 0x7faa2c0b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:59:30.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.427+0000 7faa3406a640 1 --2- 172.21.15.89:0/3353470863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa2c09efb0 0x7faa2c15b550 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:59:30.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.427+0000 7faa3406a640 1 --2- 172.21.15.89:0/3353470863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa2c0d0e70 0x7faa2c0ca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:59:30.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.427+0000 7faa3406a640 1 -- 172.21.15.89:0/3353470863 >> 172.21.15.89:0/3353470863 conn(0x7faa2c089fd0 msgr2=0x7faa2c0d0760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:59:30.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.427+0000 7faa3406a640 1 -- 172.21.15.89:0/3353470863 shutdown_connections 2023-10-18T10:59:30.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T10:59:30.427+0000 7faa3406a640 1 -- 172.21.15.89:0/3353470863 wait complete. 2023-10-18T10:59:30.444 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-18T10:59:30.600 INFO:teuthology.orchestra.run.smithi089.stdout:[{"id":5183,"entity":{"name":{"type":"client","num":5183},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":4272238049}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":726.89223001100004,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.18647135093080613,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":3,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5183 v1:192.168.0.1:0/4272238049","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi154","kernel_version":"6.6.0-rc5-g47672119149b","root":"/"}}] 2023-10-18T10:59:30.601 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "1", "hostname": "smithi154", "kernel_version": "6.6.0-rc5-g47672119149b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 4272238049, "type": "v1" }, "name": { "num": 5183, "type": "client" } }, "id": 5183, "inst": "client.5183 v1:192.168.0.1:0/4272238049", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 3, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.18647135093080613 }, "state": "open", "uptime": 726.892230011 } ] 2023-10-18T10:59:30.702 DEBUG:tasks.ceph_test_case:No log hits yet, waiting... 2023-10-18T10:59:51.240 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:59:51.244 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:00:00.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:00.970+0000 7fbdb1ffb640 1 -- 172.21.15.89:0/2752418439 <== mon.0 v2:172.21.15.89:3300/0 7 ==== log(1 entries from seq 513 at 2023-10-18T11:00:00.000146+0000) v1 ==== 187+0+0 (secure 0 0 0) 0x7fbdac0578f0 con 0x7fbdbc0b33b0 2023-10-18T11:00:00.969 INFO:teuthology.orchestra.run.smithi089.stdout: cluster: 2023-10-18T11:00:00.969 INFO:teuthology.orchestra.run.smithi089.stdout: id: 1352f62a-2330-4e96-afcd-b8e87cd82667 2023-10-18T11:00:00.970 INFO:teuthology.orchestra.run.smithi089.stdout: health: HEALTH_OK 2023-10-18T11:00:00.970 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T11:00:00.970 INFO:teuthology.orchestra.run.smithi089.stdout: services: 2023-10-18T11:00:00.970 INFO:teuthology.orchestra.run.smithi089.stdout: mon: 3 daemons, quorum a,b,c (age 14m) 2023-10-18T11:00:00.970 INFO:teuthology.orchestra.run.smithi089.stdout: mgr: x(active, since 13m), standbys: y 2023-10-18T11:00:00.971 INFO:teuthology.orchestra.run.smithi089.stdout: mds: 1/1 daemons up, 3 standby 2023-10-18T11:00:00.971 INFO:teuthology.orchestra.run.smithi089.stdout: osd: 8 osds: 8 up (since 4m), 8 in (since 13m) 2023-10-18T11:00:00.971 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T11:00:00.971 INFO:teuthology.orchestra.run.smithi089.stdout: data: 2023-10-18T11:00:00.971 INFO:teuthology.orchestra.run.smithi089.stdout: volumes: 1/1 healthy 2023-10-18T11:00:00.972 INFO:teuthology.orchestra.run.smithi089.stdout: pools: 4 pools, 105 pgs 2023-10-18T11:00:00.972 INFO:teuthology.orchestra.run.smithi089.stdout: objects: 24 objects, 580 KiB 2023-10-18T11:00:00.972 INFO:teuthology.orchestra.run.smithi089.stdout: usage: 218 MiB used, 720 GiB / 720 GiB avail 2023-10-18T11:00:00.972 INFO:teuthology.orchestra.run.smithi089.stdout: pgs: 105 active+clean 2023-10-18T11:00:00.973 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T11:00:00.973 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T11:00:00.973 INFO:teuthology.orchestra.run.smithi089.stdout:2023-10-18T11:00:00.000146+0000 mon.a [INF] overall HEALTH_OK 2023-10-18T11:00:05.720 ERROR:tasks.ceph_test_case:Log output: cluster: id: 1352f62a-2330-4e96-afcd-b8e87cd82667 health: HEALTH_OK services: mon: 3 daemons, quorum a,b,c (age 14m) mgr: x(active, since 13m), standbys: y mds: 1/1 daemons up, 3 standby osd: 8 osds: 8 up (since 4m), 8 in (since 13m) data: volumes: 1/1 healthy pools: 4 pools, 105 pgs objects: 24 objects, 580 KiB usage: 218 MiB used, 720 GiB / 720 GiB avail pgs: 105 active+clean 2023-10-18T11:00:00.000146+0000 mon.a [INF] overall HEALTH_OK 2023-10-18T11:00:05.721 DEBUG:tasks.cephfs.test_client_recovery:Expected log message found: 'Health check cleared: MDS_CLIENTS_LAGGY', retrying 2023-10-18T11:00:10.723 DEBUG:teuthology.orchestra.run.smithi089:> sudo daemon-helper kill ceph --cluster ceph -w 2023-10-18T11:00:10.727 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T11:00:10.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.877+0000 7f122a26d640 1 Processor -- start 2023-10-18T11:00:10.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.877+0000 7f122a26d640 1 -- start start 2023-10-18T11:00:10.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.877+0000 7f122a26d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f122415c940 0x7f12240d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:00:10.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.877+0000 7f122a26d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f12240d0e70 0x7f12240cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:00:10.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.877+0000 7f122a26d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f122415da50 con 0x7f12240d2f40 2023-10-18T11:00:10.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.877+0000 7f122a26d640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f122415ce40 con 0x7f122415c940 2023-10-18T11:00:10.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.877+0000 7f122a26d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f122415d120 con 0x7f12240d0e70 2023-10-18T11:00:10.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.881+0000 7f1223fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f122415c940 0x7f12240d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:00:10.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.881+0000 7f1223fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f122415c940 0x7f12240d2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55500/0 (socket says 172.21.15.89:55500) 2023-10-18T11:00:10.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.881+0000 7f1223fff640 1 -- 172.21.15.89:0/3981520322 learned_addr learned my addr 172.21.15.89:0/3981520322 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:00:10.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.881+0000 7f122880a640 1 --2- 172.21.15.89:0/3981520322 >> v2:172.21.15.154:3300/0 conn(0x7f12240d0e70 0x7f12240cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:00:10.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.881+0000 7f1223fff640 1 -- 172.21.15.89:0/3981520322 >> v2:172.21.15.154:3300/0 conn(0x7f12240d0e70 msgr2=0x7f12240cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:00:10.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.881+0000 7f1223fff640 1 --2- 172.21.15.89:0/3981520322 >> v2:172.21.15.154:3300/0 conn(0x7f12240d0e70 0x7f12240cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:00:10.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.881+0000 7f1223fff640 1 -- 172.21.15.89:0/3981520322 >> v1:172.21.15.154:6789/0 conn(0x7f12240d2f40 legacy=0x7f12240d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:00:10.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.881+0000 7f1223fff640 1 -- 172.21.15.89:0/3981520322 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122415d400 con 0x7f122415c940 2023-10-18T11:00:10.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.881+0000 7f1223fff640 1 --2- 172.21.15.89:0/3981520322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f122415c940 0x7f12240d2a00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f122408cb20 tx=0x7f121401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c224c31cee911f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:00:10.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.881+0000 7f1222ffd640 1 -- 172.21.15.89:0/3981520322 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12140a7070 con 0x7f122415c940 2023-10-18T11:00:10.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.881+0000 7f1222ffd640 1 -- 172.21.15.89:0/3981520322 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12140ab430 con 0x7f122415c940 2023-10-18T11:00:10.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.881+0000 7f1222ffd640 1 -- 172.21.15.89:0/3981520322 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12140ab7c0 con 0x7f122415c940 2023-10-18T11:00:10.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.885+0000 7f122a26d640 1 -- 172.21.15.89:0/3981520322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f122415c940 msgr2=0x7f12240d2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:00:10.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.885+0000 7f122a26d640 1 --2- 172.21.15.89:0/3981520322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f122415c940 0x7f12240d2a00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f122408cb20 tx=0x7f121401c490 comp rx=0 tx=0).stop 2023-10-18T11:00:10.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.885+0000 7f122a26d640 1 -- 172.21.15.89:0/3981520322 shutdown_connections 2023-10-18T11:00:10.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.885+0000 7f122a26d640 1 --2- 172.21.15.89:0/3981520322 >> v2:172.21.15.154:3300/0 conn(0x7f12240d0e70 0x7f12240cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:00:10.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.885+0000 7f122a26d640 1 --2- 172.21.15.89:0/3981520322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f122415c940 0x7f12240d2a00 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:00:10.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.885+0000 7f122a26d640 1 -- 172.21.15.89:0/3981520322 >> 172.21.15.89:0/3981520322 conn(0x7f1224089fd0 msgr2=0x7f122409b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:00:10.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.885+0000 7f122a26d640 1 -- 172.21.15.89:0/3981520322 shutdown_connections 2023-10-18T11:00:10.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.885+0000 7f122a26d640 1 -- 172.21.15.89:0/3981520322 wait complete. 2023-10-18T11:00:10.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.885+0000 7f122a26d640 1 Processor -- start 2023-10-18T11:00:10.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.885+0000 7f122a26d640 1 -- start start 2023-10-18T11:00:10.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.885+0000 7f122a26d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12240d0e70 0x7f12240c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:00:10.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.885+0000 7f122a26d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12240b7c60 0x7f12241545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:00:10.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.885+0000 7f122a26d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f122415e920 0x7f122415eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:00:10.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.885+0000 7f122a26d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12240c9cf0 con 0x7f12240b7c60 2023-10-18T11:00:10.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.885+0000 7f122a26d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122415f2e0 con 0x7f12240d0e70 2023-10-18T11:00:10.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.885+0000 7f122a26d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12240b53b0 con 0x7f122415e920 2023-10-18T11:00:10.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.885+0000 7f12237fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12240b7c60 0x7f12241545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:00:10.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.885+0000 7f12237fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12240b7c60 0x7f12241545e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55510/0 (socket says 172.21.15.89:55510) 2023-10-18T11:00:10.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.885+0000 7f12237fe640 1 -- 172.21.15.89:0/3095909473 learned_addr learned my addr 172.21.15.89:0/3095909473 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:00:10.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.885+0000 7f1223fff640 1 --2- 172.21.15.89:0/3095909473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12240d0e70 0x7f12240c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:00:10.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.889+0000 7f122880a640 1 --2- 172.21.15.89:0/3095909473 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f122415e920 0x7f122415eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:00:10.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.889+0000 7f12237fe640 1 -- 172.21.15.89:0/3095909473 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f122415e920 msgr2=0x7f122415eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:00:10.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.889+0000 7f12237fe640 1 --2- 172.21.15.89:0/3095909473 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f122415e920 0x7f122415eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:00:10.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.889+0000 7f12237fe640 1 -- 172.21.15.89:0/3095909473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12240d0e70 msgr2=0x7f12240c97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:00:10.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.889+0000 7f12237fe640 1 --2- 172.21.15.89:0/3095909473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12240d0e70 0x7f12240c97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:00:10.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.889+0000 7f12237fe640 1 -- 172.21.15.89:0/3095909473 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12140094d0 con 0x7f12240b7c60 2023-10-18T11:00:10.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.889+0000 7f12237fe640 1 --2- 172.21.15.89:0/3095909473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12240b7c60 0x7f12241545e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f121800c450 tx=0x7f121800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:00:10.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.889+0000 7f1220ff9640 1 -- 172.21.15.89:0/3095909473 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1218012830 con 0x7f12240b7c60 2023-10-18T11:00:10.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.889+0000 7f1220ff9640 1 -- 172.21.15.89:0/3095909473 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12180129b0 con 0x7f12240b7c60 2023-10-18T11:00:10.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.889+0000 7f122a26d640 1 -- 172.21.15.89:0/3095909473 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12240cbc80 con 0x7f12240b7c60 2023-10-18T11:00:10.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.889+0000 7f1220ff9640 1 -- 172.21.15.89:0/3095909473 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1218012d60 con 0x7f12240b7c60 2023-10-18T11:00:10.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.889+0000 7f122a26d640 1 -- 172.21.15.89:0/3095909473 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12240cc200 con 0x7f12240b7c60 2023-10-18T11:00:10.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.889+0000 7f122a26d640 1 -- 172.21.15.89:0/3095909473 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f122404dbe0 con 0x7f12240b7c60 2023-10-18T11:00:10.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.889+0000 7f1220ff9640 1 -- 172.21.15.89:0/3095909473 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f121801c460 con 0x7f12240b7c60 2023-10-18T11:00:10.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.889+0000 7f1220ff9640 1 --2- 172.21.15.89:0/3095909473 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1210065680 0x7f1210067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:00:10.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.889+0000 7f1220ff9640 1 -- 172.21.15.89:0/3095909473 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f1218082e00 con 0x7f12240b7c60 2023-10-18T11:00:10.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.893+0000 7f1223fff640 1 --2- 172.21.15.89:0/3095909473 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1210065680 0x7f1210067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:00:10.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.893+0000 7f1223fff640 1 --2- 172.21.15.89:0/3095909473 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1210065680 0x7f1210067b40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f12140af000 tx=0x7f1214008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:00:10.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.901+0000 7f1220ff9640 1 -- 172.21.15.89:0/3095909473 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f12180506a0 con 0x7f12240b7c60 2023-10-18T11:00:10.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.909+0000 7f1533700640 1 Processor -- start 2023-10-18T11:00:10.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.909+0000 7f1533700640 1 -- start start 2023-10-18T11:00:10.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.909+0000 7f1533700640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f152c160f20 0x7f152c0d0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:00:10.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.909+0000 7f1533700640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f152c0b33b0 0x7f152c0d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:00:10.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.909+0000 7f1533700640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f152c0cced0 con 0x7f152c0d0270 2023-10-18T11:00:10.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.909+0000 7f1533700640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f152c0b1d30 con 0x7f152c0b33b0 2023-10-18T11:00:10.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.909+0000 7f1533700640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f152c0b3bc0 con 0x7f152c160f20 2023-10-18T11:00:10.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.909+0000 7f1530c9b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f152c0b33b0 0x7f152c0d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:00:10.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.909+0000 7f1530c9b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f152c0b33b0 0x7f152c0d0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55516/0 (socket says 172.21.15.89:55516) 2023-10-18T11:00:10.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.909+0000 7f1530c9b640 1 -- 172.21.15.89:0/4256608428 learned_addr learned my addr 172.21.15.89:0/4256608428 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:00:10.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.909+0000 7f153149c640 1 --2- 172.21.15.89:0/4256608428 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f152c160f20 0x7f152c0d0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:00:10.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.909+0000 7f1530c9b640 1 -- 172.21.15.89:0/4256608428 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f152c160f20 msgr2=0x7f152c0d0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:00:10.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.909+0000 7f1530c9b640 1 --2- 172.21.15.89:0/4256608428 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f152c160f20 0x7f152c0d0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:00:10.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.909+0000 7f1530c9b640 1 -- 172.21.15.89:0/4256608428 >> v1:172.21.15.154:6789/0 conn(0x7f152c0d0270 legacy=0x7f152c0b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:00:10.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.909+0000 7f1530c9b640 1 -- 172.21.15.89:0/4256608428 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f152c0afdd0 con 0x7f152c0b33b0 2023-10-18T11:00:10.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.913+0000 7f1530c9b640 1 --2- 172.21.15.89:0/4256608428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f152c0b33b0 0x7f152c0d0c90 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f151c009870 tx=0x7f151c01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcbbc2cea2f90b9f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:00:10.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.913+0000 7f1523fff640 1 -- 172.21.15.89:0/4256608428 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151c0ab600 con 0x7f152c0b33b0 2023-10-18T11:00:10.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.913+0000 7f1523fff640 1 -- 172.21.15.89:0/4256608428 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f151c0a7050 con 0x7f152c0b33b0 2023-10-18T11:00:10.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.913+0000 7f1523fff640 1 -- 172.21.15.89:0/4256608428 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151c0ab980 con 0x7f152c0b33b0 2023-10-18T11:00:10.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.913+0000 7f1533700640 1 -- 172.21.15.89:0/4256608428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f152c0b33b0 msgr2=0x7f152c0d0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:00:10.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.913+0000 7f1533700640 1 --2- 172.21.15.89:0/4256608428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f152c0b33b0 0x7f152c0d0c90 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f151c009870 tx=0x7f151c01c4b0 comp rx=0 tx=0).stop 2023-10-18T11:00:10.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.913+0000 7f1533700640 1 -- 172.21.15.89:0/4256608428 shutdown_connections 2023-10-18T11:00:10.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.913+0000 7f1533700640 1 --2- 172.21.15.89:0/4256608428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f152c0b33b0 0x7f152c0d0c90 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:00:10.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.913+0000 7f1533700640 1 --2- 172.21.15.89:0/4256608428 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f152c160f20 0x7f152c0d0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:00:10.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.913+0000 7f1533700640 1 -- 172.21.15.89:0/4256608428 >> 172.21.15.89:0/4256608428 conn(0x7f152c09b930 msgr2=0x7f152c0d26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:00:10.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.913+0000 7f1533700640 1 -- 172.21.15.89:0/4256608428 shutdown_connections 2023-10-18T11:00:10.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.913+0000 7f1533700640 1 -- 172.21.15.89:0/4256608428 wait complete. 2023-10-18T11:00:10.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.917+0000 7f1533700640 1 Processor -- start 2023-10-18T11:00:10.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.917+0000 7f1533700640 1 -- start start 2023-10-18T11:00:10.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.917+0000 7f1533700640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f152c0b33b0 0x7f152c0ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:00:10.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.917+0000 7f1533700640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f152c0d0270 0x7f152c15e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:00:10.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.917+0000 7f1533700640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f152c160f20 0x7f152c152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:00:10.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.917+0000 7f1533700640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c0cbef0 con 0x7f152c0b33b0 2023-10-18T11:00:10.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.917+0000 7f1533700640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c15e920 con 0x7f152c160f20 2023-10-18T11:00:10.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.917+0000 7f1533700640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f152c0cae50 con 0x7f152c0d0270 2023-10-18T11:00:10.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.917+0000 7f153149c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f152c0b33b0 0x7f152c0ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:00:10.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.917+0000 7f153149c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f152c0b33b0 0x7f152c0ca7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55524/0 (socket says 172.21.15.89:55524) 2023-10-18T11:00:10.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.917+0000 7f153149c640 1 -- 172.21.15.89:0/3071369623 learned_addr learned my addr 172.21.15.89:0/3071369623 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:00:10.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.917+0000 7f1530c9b640 1 --2- 172.21.15.89:0/3071369623 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f152c0d0270 0x7f152c15e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:00:10.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.917+0000 7f1531c9d640 1 --2- 172.21.15.89:0/3071369623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f152c160f20 0x7f152c152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:00:10.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.917+0000 7f153149c640 1 -- 172.21.15.89:0/3071369623 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f152c0d0270 msgr2=0x7f152c15e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:00:10.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.917+0000 7f153149c640 1 --2- 172.21.15.89:0/3071369623 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f152c0d0270 0x7f152c15e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:00:10.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.917+0000 7f153149c640 1 -- 172.21.15.89:0/3071369623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f152c160f20 msgr2=0x7f152c152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:00:10.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.917+0000 7f153149c640 1 --2- 172.21.15.89:0/3071369623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f152c160f20 0x7f152c152cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:00:10.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.917+0000 7f153149c640 1 -- 172.21.15.89:0/3071369623 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f151c0094d0 con 0x7f152c0b33b0 2023-10-18T11:00:10.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.917+0000 7f1531c9d640 1 --2- 172.21.15.89:0/3071369623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f152c160f20 0x7f152c152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:00:10.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.917+0000 7f153149c640 1 --2- 172.21.15.89:0/3071369623 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f152c0b33b0 0x7f152c0ca7b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f152800d5f0 tx=0x7f152800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:00:10.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.917+0000 7f1521ffb640 1 -- 172.21.15.89:0/3071369623 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15280125f0 con 0x7f152c0b33b0 2023-10-18T11:00:10.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.917+0000 7f1521ffb640 1 -- 172.21.15.89:0/3071369623 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1528012750 con 0x7f152c0b33b0 2023-10-18T11:00:10.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.917+0000 7f1521ffb640 1 -- 172.21.15.89:0/3071369623 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1528012ae0 con 0x7f152c0b33b0 2023-10-18T11:00:10.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.917+0000 7f1533700640 1 -- 172.21.15.89:0/3071369623 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f152c0cb130 con 0x7f152c0b33b0 2023-10-18T11:00:10.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.917+0000 7f1533700640 1 -- 172.21.15.89:0/3071369623 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f152c0cb6f0 con 0x7f152c0b33b0 2023-10-18T11:00:10.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.917+0000 7f1533700640 1 -- 172.21.15.89:0/3071369623 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f152c0725d0 con 0x7f152c0b33b0 2023-10-18T11:00:10.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.921+0000 7f1521ffb640 1 -- 172.21.15.89:0/3071369623 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f1528012c40 con 0x7f152c0b33b0 2023-10-18T11:00:10.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.921+0000 7f1521ffb640 1 --2- 172.21.15.89:0/3071369623 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1510065680 0x7f1510067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:00:10.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.921+0000 7f1521ffb640 1 -- 172.21.15.89:0/3071369623 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f1528083540 con 0x7f152c0b33b0 2023-10-18T11:00:10.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.921+0000 7f1521ffb640 1 -- 172.21.15.89:0/3071369623 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+449 (secure 0 0 0) 0x7f152808a070 con 0x7f152c0b33b0 2023-10-18T11:00:10.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.921+0000 7f1530c9b640 1 --2- 172.21.15.89:0/3071369623 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1510065680 0x7f1510067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:00:10.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.921+0000 7f1533700640 1 -- 172.21.15.89:0/3071369623 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f152c04e430 con 0x7f152c0b33b0 2023-10-18T11:00:10.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:10.925+0000 7f1530c9b640 1 --2- 172.21.15.89:0/3071369623 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1510065680 0x7f1510067b40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f151c0ad000 tx=0x7f151c0a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:00:11.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:11.085+0000 7f122a26d640 1 -- 172.21.15.89:0/3095909473 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f12240b85d0 con 0x7f12240b7c60 2023-10-18T11:00:11.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:11.085+0000 7f1220ff9640 1 -- 172.21.15.89:0/3095909473 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4877 (secure 0 0 0) 0x7f1218053cc0 con 0x7f12240b7c60 2023-10-18T11:00:11.087 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T11:00:11.087 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/654729641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":654729641},{"type":"v1","addr":"172.21.15.154:6836","nonce":654729641}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/964566480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":964566480},{"type":"v1","addr":"172.21.15.154:6837","nonce":964566480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2261878538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2261878538},{"type":"v1","addr":"172.21.15.89:6833","nonce":2261878538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:47:15.199575+0000","modified":"2023-10-18T10:55:20.892655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5517":{"gid":5517,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":124,"addr":"172.21.15.89:6835/2092274731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2092274731},{"type":"v1","addr":"172.21.15.89:6835","nonce":2092274731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T11:00:11.088 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-18T11:00:11.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:11.089+0000 7f122a26d640 1 -- 172.21.15.89:0/3095909473 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1210065680 msgr2=0x7f1210067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:00:11.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:11.089+0000 7f122a26d640 1 --2- 172.21.15.89:0/3095909473 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1210065680 0x7f1210067b40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f12140af000 tx=0x7f1214008040 comp rx=0 tx=0).stop 2023-10-18T11:00:11.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:11.089+0000 7f122a26d640 1 -- 172.21.15.89:0/3095909473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12240b7c60 msgr2=0x7f12241545e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:00:11.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:11.089+0000 7f122a26d640 1 --2- 172.21.15.89:0/3095909473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12240b7c60 0x7f12241545e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f121800c450 tx=0x7f121800c910 comp rx=0 tx=0).stop 2023-10-18T11:00:11.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:11.089+0000 7f122a26d640 1 -- 172.21.15.89:0/3095909473 shutdown_connections 2023-10-18T11:00:11.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:11.089+0000 7f122a26d640 1 --2- 172.21.15.89:0/3095909473 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1210065680 0x7f1210067b40 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:00:11.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:11.089+0000 7f122a26d640 1 --2- 172.21.15.89:0/3095909473 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f122415e920 0x7f122415eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:00:11.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:11.089+0000 7f122a26d640 1 --2- 172.21.15.89:0/3095909473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12240b7c60 0x7f12241545e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:00:11.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:11.089+0000 7f122a26d640 1 --2- 172.21.15.89:0/3095909473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12240d0e70 0x7f12240c97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:00:11.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:11.089+0000 7f122a26d640 1 -- 172.21.15.89:0/3095909473 >> 172.21.15.89:0/3095909473 conn(0x7f1224089fd0 msgr2=0x7f1224158b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:00:11.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:11.089+0000 7f122a26d640 1 -- 172.21.15.89:0/3095909473 shutdown_connections 2023-10-18T11:00:11.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:11.089+0000 7f122a26d640 1 -- 172.21.15.89:0/3095909473 wait complete. 2023-10-18T11:00:11.108 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-18T11:00:11.265 INFO:teuthology.orchestra.run.smithi089.stdout:[{"id":5183,"entity":{"name":{"type":"client","num":5183},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":4272238049}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":767.55499263499996,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.16974998568064883,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":3,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5183 v1:192.168.0.1:0/4272238049","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi154","kernel_version":"6.6.0-rc5-g47672119149b","root":"/"}}] 2023-10-18T11:00:11.265 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "1", "hostname": "smithi154", "kernel_version": "6.6.0-rc5-g47672119149b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 4272238049, "type": "v1" }, "name": { "num": 5183, "type": "client" } }, "id": 5183, "inst": "client.5183 v1:192.168.0.1:0/4272238049", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 3, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.16974998568064883 }, "state": "open", "uptime": 767.554992635 } ] 2023-10-18T11:00:11.367 DEBUG:tasks.ceph_test_case:No log hits yet, waiting... 2023-10-18T11:00:21.597 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:00:21.602 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:00:46.385 ERROR:tasks.ceph_test_case:Log output: 2023-10-18T11:00:46.385 DEBUG:tasks.cephfs.test_client_recovery:Expected log message found: 'Health check cleared: MDS_CLIENTS_LAGGY', retrying 2023-10-18T11:00:51.387 DEBUG:teuthology.orchestra.run.smithi089:> sudo daemon-helper kill ceph --cluster ceph -w 2023-10-18T11:00:51.392 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T11:00:51.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.540+0000 7f46b90bd640 1 Processor -- start 2023-10-18T11:00:51.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.540+0000 7f46b90bd640 1 -- start start 2023-10-18T11:00:51.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.540+0000 7f46b90bd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46b415c940 0x7f46b40d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:00:51.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.540+0000 7f46b90bd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f46b40d0e70 0x7f46b40cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:00:51.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.540+0000 7f46b90bd640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46b415da50 con 0x7f46b40d2f40 2023-10-18T11:00:51.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.540+0000 7f46b90bd640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46b415ce40 con 0x7f46b40d0e70 2023-10-18T11:00:51.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.540+0000 7f46b90bd640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46b415d120 con 0x7f46b415c940 2023-10-18T11:00:51.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.540+0000 7f46abfff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f46b40d2f40 0x7f46b40d0760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54644/0 (socket says 172.21.15.89:54644) 2023-10-18T11:00:51.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.540+0000 7f46abfff640 1 -- 172.21.15.89:0/3413245392 learned_addr learned my addr 172.21.15.89:0/3413245392 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:00:51.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.540+0000 7f46b2d9d640 1 --2- 172.21.15.89:0/3413245392 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46b415c940 0x7f46b40d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:00:51.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.540+0000 7f46b359e640 1 --2- 172.21.15.89:0/3413245392 >> v2:172.21.15.154:3300/0 conn(0x7f46b40d0e70 0x7f46b40cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:00:51.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.544+0000 7f46b259c640 1 -- 172.21.15.89:0/3413245392 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2094580991 0 0) 0x7f46b415da50 con 0x7f46b40d2f40 2023-10-18T11:00:51.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.544+0000 7f46b259c640 1 -- 172.21.15.89:0/3413245392 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46a0003530 con 0x7f46b40d2f40 2023-10-18T11:00:51.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.544+0000 7f46b259c640 1 -- 172.21.15.89:0/3413245392 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 842049646 0 0) 0x7f46a0003530 con 0x7f46b40d2f40 2023-10-18T11:00:51.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.544+0000 7f46b259c640 1 -- 172.21.15.89:0/3413245392 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46b415c940 msgr2=0x7f46b40d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:00:51.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.544+0000 7f46b259c640 1 --2- 172.21.15.89:0/3413245392 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46b415c940 0x7f46b40d2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:00:51.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.544+0000 7f46b259c640 1 -- 172.21.15.89:0/3413245392 >> v2:172.21.15.154:3300/0 conn(0x7f46b40d0e70 msgr2=0x7f46b40cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:00:51.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.544+0000 7f46b259c640 1 --2- 172.21.15.89:0/3413245392 >> v2:172.21.15.154:3300/0 conn(0x7f46b40d0e70 0x7f46b40cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:00:51.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.544+0000 7f46b259c640 1 -- 172.21.15.89:0/3413245392 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b415d400 con 0x7f46b40d2f40 2023-10-18T11:00:51.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.544+0000 7f46b259c640 1 -- 172.21.15.89:0/3413245392 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7f469c001680 con 0x7f46b40d2f40 2023-10-18T11:00:51.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.544+0000 7f46b2d9d640 1 --2- 172.21.15.89:0/3413245392 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46b415c940 0x7f46b40d2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T11:00:51.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.544+0000 7f46b259c640 1 -- 172.21.15.89:0/3413245392 >> v1:172.21.15.89:6789/0 conn(0x7f46b40d2f40 legacy=0x7f46b40d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:00:51.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.544+0000 7f46b259c640 1 --2- 172.21.15.89:0/3413245392 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46a0003980 0x7f46a0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:00:51.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.544+0000 7f46b259c640 1 --2- 172.21.15.89:0/3413245392 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46a00047d0 0x7f46a0004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:00:51.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.544+0000 7f46b259c640 1 --2- 172.21.15.89:0/3413245392 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46a0005110 0x7f46a000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:00:51.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.544+0000 7f46b259c640 1 -- 172.21.15.89:0/3413245392 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b415ce40 con 0x7f46a0005110 2023-10-18T11:00:51.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.544+0000 7f46b259c640 1 -- 172.21.15.89:0/3413245392 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b415d120 con 0x7f46a0003980 2023-10-18T11:00:51.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.544+0000 7f46b259c640 1 -- 172.21.15.89:0/3413245392 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a0004520 con 0x7f46a00047d0 2023-10-18T11:00:51.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.544+0000 7f46b2d9d640 1 --2- 172.21.15.89:0/3413245392 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46a0003980 0x7f46a0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:00:51.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.544+0000 7f46b359e640 1 --2- 172.21.15.89:0/3413245392 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46a0005110 0x7f46a000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:00:51.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.544+0000 7f46abfff640 1 --2- 172.21.15.89:0/3413245392 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46a00047d0 0x7f46a0004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:00:51.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.544+0000 7f46b359e640 1 -- 172.21.15.89:0/3413245392 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46a00047d0 msgr2=0x7f46a0004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:00:51.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.544+0000 7f46b359e640 1 --2- 172.21.15.89:0/3413245392 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46a00047d0 0x7f46a0004bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:00:51.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.544+0000 7f46b359e640 1 -- 172.21.15.89:0/3413245392 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46a0003980 msgr2=0x7f46a0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:00:51.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.544+0000 7f46b359e640 1 --2- 172.21.15.89:0/3413245392 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46a0003980 0x7f46a0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:00:51.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.544+0000 7f46b359e640 1 -- 172.21.15.89:0/3413245392 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a4009420 con 0x7f46a0005110 2023-10-18T11:00:51.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.548+0000 7f46b359e640 1 --2- 172.21.15.89:0/3413245392 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46a0005110 0x7f46a000b990 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f46ac009f90 tx=0x7f46ac01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a33cb918c247bf36 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:00:51.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.548+0000 7f46b259c640 1 -- 172.21.15.89:0/3413245392 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac0a5c70 con 0x7f46a0005110 2023-10-18T11:00:51.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.548+0000 7f46b259c640 1 -- 172.21.15.89:0/3413245392 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ac00f040 con 0x7f46a0005110 2023-10-18T11:00:51.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.548+0000 7f46b259c640 1 -- 172.21.15.89:0/3413245392 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac0ad440 con 0x7f46a0005110 2023-10-18T11:00:51.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.548+0000 7f46b90bd640 1 -- 172.21.15.89:0/3413245392 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46a0005110 msgr2=0x7f46a000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:00:51.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.548+0000 7f46b90bd640 1 --2- 172.21.15.89:0/3413245392 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46a0005110 0x7f46a000b990 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f46ac009f90 tx=0x7f46ac01cc60 comp rx=0 tx=0).stop 2023-10-18T11:00:51.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.548+0000 7f46b359e640 1 -- 172.21.15.89:0/3413245392 reap_dead start 2023-10-18T11:00:51.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.548+0000 7f46b90bd640 1 -- 172.21.15.89:0/3413245392 shutdown_connections 2023-10-18T11:00:51.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.548+0000 7f46b90bd640 1 -- 172.21.15.89:0/3413245392 >> 172.21.15.89:0/3413245392 conn(0x7f46b4089fd0 msgr2=0x7f46b409b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:00:51.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.548+0000 7f46b90bd640 1 -- 172.21.15.89:0/3413245392 shutdown_connections 2023-10-18T11:00:51.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.548+0000 7f46b90bd640 1 -- 172.21.15.89:0/3413245392 wait complete. 2023-10-18T11:00:51.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.548+0000 7f46b90bd640 1 Processor -- start 2023-10-18T11:00:51.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.548+0000 7f46b90bd640 1 -- start start 2023-10-18T11:00:51.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.552+0000 7f46b90bd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46a0005110 0x7f46b4151e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:00:51.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.552+0000 7f46b90bd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46b40c7ce0 0x7f46b40c6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:00:51.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.552+0000 7f46b90bd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46b40c66c0 0x7f46b415fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:00:51.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.552+0000 7f46b90bd640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b40c6c40 con 0x7f46b40c66c0 2023-10-18T11:00:51.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.552+0000 7f46b90bd640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b40c6f20 con 0x7f46b40c7ce0 2023-10-18T11:00:51.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.552+0000 7f46b90bd640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46b40c7200 con 0x7f46a0005110 2023-10-18T11:00:51.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.552+0000 7f46b359e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46b40c66c0 0x7f46b415fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:00:51.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.552+0000 7f46b359e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46b40c66c0 0x7f46b415fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40672/0 (socket says 172.21.15.89:40672) 2023-10-18T11:00:51.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.552+0000 7f46b359e640 1 -- 172.21.15.89:0/1792144534 learned_addr learned my addr 172.21.15.89:0/1792144534 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:00:51.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.552+0000 7f46abfff640 1 --2- 172.21.15.89:0/1792144534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46b40c7ce0 0x7f46b40c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:00:51.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.552+0000 7f46b2d9d640 1 --2- 172.21.15.89:0/1792144534 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46a0005110 0x7f46b4151e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:00:51.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.552+0000 7f46b359e640 1 -- 172.21.15.89:0/1792144534 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46a0005110 msgr2=0x7f46b4151e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:00:51.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.552+0000 7f46b359e640 1 --2- 172.21.15.89:0/1792144534 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46a0005110 0x7f46b4151e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:00:51.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.552+0000 7f46b359e640 1 -- 172.21.15.89:0/1792144534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46b40c7ce0 msgr2=0x7f46b40c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:00:51.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.552+0000 7f46b359e640 1 --2- 172.21.15.89:0/1792144534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46b40c7ce0 0x7f46b40c6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:00:51.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.552+0000 7f46b359e640 1 -- 172.21.15.89:0/1792144534 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b40c74e0 con 0x7f46b40c66c0 2023-10-18T11:00:51.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.552+0000 7f46b359e640 1 --2- 172.21.15.89:0/1792144534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46b40c66c0 0x7f46b415fb60 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f46ac002240 tx=0x7f46ac009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:00:51.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.552+0000 7f46ab7fe640 1 -- 172.21.15.89:0/1792144534 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac0039a0 con 0x7f46b40c66c0 2023-10-18T11:00:51.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.552+0000 7f46ab7fe640 1 -- 172.21.15.89:0/1792144534 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ac003b00 con 0x7f46b40c66c0 2023-10-18T11:00:51.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.552+0000 7f46b90bd640 1 -- 172.21.15.89:0/1792144534 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46b40c77c0 con 0x7f46b40c66c0 2023-10-18T11:00:51.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.552+0000 7f46ab7fe640 1 -- 172.21.15.89:0/1792144534 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac003e90 con 0x7f46b40c66c0 2023-10-18T11:00:51.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.552+0000 7f46b90bd640 1 -- 172.21.15.89:0/1792144534 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46b40b70f0 con 0x7f46b40c66c0 2023-10-18T11:00:51.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.552+0000 7f46b90bd640 1 -- 172.21.15.89:0/1792144534 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46b404dbe0 con 0x7f46b40c66c0 2023-10-18T11:00:51.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.552+0000 7f46ab7fe640 1 -- 172.21.15.89:0/1792144534 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f46ac0b4040 con 0x7f46b40c66c0 2023-10-18T11:00:51.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.556+0000 7f46ab7fe640 1 --2- 172.21.15.89:0/1792144534 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4694065680 0x7f4694067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:00:51.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.556+0000 7f46ab7fe640 1 -- 172.21.15.89:0/1792144534 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f46ac11ce80 con 0x7f46b40c66c0 2023-10-18T11:00:51.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.556+0000 7f46b2d9d640 1 --2- 172.21.15.89:0/1792144534 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4694065680 0x7f4694067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:00:51.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.556+0000 7f46b2d9d640 1 --2- 172.21.15.89:0/1792144534 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4694065680 0x7f4694067b40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f46a4009d60 tx=0x7f46a4009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:00:51.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.560+0000 7f46ab7fe640 1 -- 172.21.15.89:0/1792144534 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f46ac0ea680 con 0x7f46b40c66c0 2023-10-18T11:00:51.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.568+0000 7f631f540640 1 Processor -- start 2023-10-18T11:00:51.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.568+0000 7f631f540640 1 -- start start 2023-10-18T11:00:51.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.568+0000 7f631f540640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f63180d0e60 0x7f63180b05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:00:51.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.568+0000 7f631f540640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63180b33b0 0x7f63180b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:00:51.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.568+0000 7f631f540640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63180cced0 con 0x7f6318160f20 2023-10-18T11:00:51.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.568+0000 7f631f540640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63180b1d30 con 0x7f63180d0e60 2023-10-18T11:00:51.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.568+0000 7f631f540640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63180b3bc0 con 0x7f63180b33b0 2023-10-18T11:00:51.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.568+0000 7f631cadb640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f63180d0e60 0x7f63180b05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:00:51.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.568+0000 7f631dadd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63180b33b0 0x7f63180b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:00:51.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.572+0000 7f631d2dc640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f6318160f20 0x7f63180d0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.89:47532/0 (socket says 172.21.15.89:47532) 2023-10-18T11:00:51.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.572+0000 7f631cadb640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f63180d0e60 0x7f63180b05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:40216/0 (socket says 172.21.15.89:40216) 2023-10-18T11:00:51.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.572+0000 7f631d2dc640 1 -- 172.21.15.89:0/97523429 learned_addr learned my addr 172.21.15.89:0/97523429 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:00:51.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.572+0000 7f630ffff640 1 -- 172.21.15.89:0/97523429 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 193827489 0 0) 0x7f63180cced0 con 0x7f6318160f20 2023-10-18T11:00:51.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.572+0000 7f631dadd640 1 -- 172.21.15.89:0/97523429 >> v2:172.21.15.154:3300/0 conn(0x7f63180d0e60 msgr2=0x7f63180b05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:00:51.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.572+0000 7f631dadd640 1 --2- 172.21.15.89:0/97523429 >> v2:172.21.15.154:3300/0 conn(0x7f63180d0e60 0x7f63180b05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:00:51.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.572+0000 7f631dadd640 1 -- 172.21.15.89:0/97523429 >> v1:172.21.15.154:6789/0 conn(0x7f6318160f20 legacy=0x7f63180d0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:00:51.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.572+0000 7f631dadd640 1 -- 172.21.15.89:0/97523429 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63180afdd0 con 0x7f63180b33b0 2023-10-18T11:00:51.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.572+0000 7f631cadb640 1 --2- 172.21.15.89:0/97523429 >> v2:172.21.15.154:3300/0 conn(0x7f63180d0e60 0x7f63180b05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T11:00:51.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.576+0000 7f631dadd640 1 --2- 172.21.15.89:0/97523429 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63180b33b0 0x7f63180b0b10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6310009d90 tx=0x7f631001c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44ccf4aa4473044f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:00:51.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.576+0000 7f630ffff640 1 -- 172.21.15.89:0/97523429 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63180b1860 con 0x7f63180b33b0 2023-10-18T11:00:51.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.576+0000 7f630ffff640 1 -- 172.21.15.89:0/97523429 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63100a5850 con 0x7f63180b33b0 2023-10-18T11:00:51.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.576+0000 7f630ffff640 1 -- 172.21.15.89:0/97523429 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63100a5be0 con 0x7f63180b33b0 2023-10-18T11:00:51.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.576+0000 7f631f540640 1 -- 172.21.15.89:0/97523429 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63180b33b0 msgr2=0x7f63180b0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:00:51.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.576+0000 7f631f540640 1 --2- 172.21.15.89:0/97523429 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63180b33b0 0x7f63180b0b10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6310009d90 tx=0x7f631001c8c0 comp rx=0 tx=0).stop 2023-10-18T11:00:51.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.576+0000 7f631f540640 1 -- 172.21.15.89:0/97523429 shutdown_connections 2023-10-18T11:00:51.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.576+0000 7f631f540640 1 --2- 172.21.15.89:0/97523429 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63180b33b0 0x7f63180b0b10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:00:51.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.576+0000 7f631f540640 1 --2- 172.21.15.89:0/97523429 >> v2:172.21.15.154:3300/0 conn(0x7f63180d0e60 0x7f63180b05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:00:51.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.576+0000 7f631f540640 1 -- 172.21.15.89:0/97523429 >> 172.21.15.89:0/97523429 conn(0x7f631809b930 msgr2=0x7f63180d26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:00:51.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.576+0000 7f631f540640 1 -- 172.21.15.89:0/97523429 shutdown_connections 2023-10-18T11:00:51.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.576+0000 7f631f540640 1 -- 172.21.15.89:0/97523429 wait complete. 2023-10-18T11:00:51.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.580+0000 7f631f540640 1 Processor -- start 2023-10-18T11:00:51.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.580+0000 7f631f540640 1 -- start start 2023-10-18T11:00:51.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.580+0000 7f631f540640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63180b33b0 0x7f631806b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:00:51.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.580+0000 7f631f540640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63180d0e60 0x7f63181f8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:00:51.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.580+0000 7f631f540640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6318160f20 0x7f63180b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:00:51.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.580+0000 7f631f540640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63181f9220 con 0x7f63180b33b0 2023-10-18T11:00:51.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.580+0000 7f631f540640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63180cc740 con 0x7f6318160f20 2023-10-18T11:00:51.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.580+0000 7f631f540640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63180cca20 con 0x7f63180d0e60 2023-10-18T11:00:51.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.580+0000 7f631d2dc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63180b33b0 0x7f631806b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:00:51.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.580+0000 7f631d2dc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63180b33b0 0x7f631806b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40682/0 (socket says 172.21.15.89:40682) 2023-10-18T11:00:51.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.580+0000 7f631d2dc640 1 -- 172.21.15.89:0/361293206 learned_addr learned my addr 172.21.15.89:0/361293206 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:00:51.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.580+0000 7f631cadb640 1 --2- 172.21.15.89:0/361293206 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63180d0e60 0x7f63181f8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:00:51.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.580+0000 7f631dadd640 1 --2- 172.21.15.89:0/361293206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6318160f20 0x7f63180b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:00:51.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.580+0000 7f631d2dc640 1 -- 172.21.15.89:0/361293206 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63180d0e60 msgr2=0x7f63181f8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:00:51.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.580+0000 7f631d2dc640 1 --2- 172.21.15.89:0/361293206 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63180d0e60 0x7f63181f8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:00:51.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.580+0000 7f631d2dc640 1 -- 172.21.15.89:0/361293206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6318160f20 msgr2=0x7f63180b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:00:51.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.580+0000 7f631d2dc640 1 --2- 172.21.15.89:0/361293206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6318160f20 0x7f63180b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:00:51.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.580+0000 7f631d2dc640 1 -- 172.21.15.89:0/361293206 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6308009420 con 0x7f63180b33b0 2023-10-18T11:00:51.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.580+0000 7f631d2dc640 1 --2- 172.21.15.89:0/361293206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63180b33b0 0x7f631806b220 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f631400c480 tx=0x7f631400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:00:51.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.580+0000 7f630dffb640 1 -- 172.21.15.89:0/361293206 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63140074e0 con 0x7f63180b33b0 2023-10-18T11:00:51.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.580+0000 7f630dffb640 1 -- 172.21.15.89:0/361293206 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6314007660 con 0x7f63180b33b0 2023-10-18T11:00:51.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.580+0000 7f630dffb640 1 -- 172.21.15.89:0/361293206 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6314004a30 con 0x7f63180b33b0 2023-10-18T11:00:51.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.580+0000 7f631f540640 1 -- 172.21.15.89:0/361293206 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6310009a70 con 0x7f63180b33b0 2023-10-18T11:00:51.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.580+0000 7f631f540640 1 -- 172.21.15.89:0/361293206 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63180cb700 con 0x7f63180b33b0 2023-10-18T11:00:51.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.580+0000 7f631f540640 1 -- 172.21.15.89:0/361293206 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f631804e430 con 0x7f63180b33b0 2023-10-18T11:00:51.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.584+0000 7f630dffb640 1 -- 172.21.15.89:0/361293206 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f631401b360 con 0x7f63180b33b0 2023-10-18T11:00:51.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.584+0000 7f630dffb640 1 --2- 172.21.15.89:0/361293206 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f62fc065680 0x7f62fc067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:00:51.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.584+0000 7f630dffb640 1 -- 172.21.15.89:0/361293206 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f6314082710 con 0x7f63180b33b0 2023-10-18T11:00:51.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.584+0000 7f630dffb640 1 -- 172.21.15.89:0/361293206 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+449 (secure 0 0 0) 0x7f6314082aa0 con 0x7f63180b33b0 2023-10-18T11:00:51.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.584+0000 7f631cadb640 1 --2- 172.21.15.89:0/361293206 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f62fc065680 0x7f62fc067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:00:51.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.584+0000 7f631f540640 1 -- 172.21.15.89:0/361293206 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f6318089fd0 con 0x7f63180b33b0 2023-10-18T11:00:51.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.584+0000 7f631cadb640 1 --2- 172.21.15.89:0/361293206 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f62fc065680 0x7f62fc067b40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f63180b2c70 tx=0x7f6308008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:00:51.652 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:00:51.656 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:00:51.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.744+0000 7f46b90bd640 1 -- 172.21.15.89:0/1792144534 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46b40d3330 con 0x7f46b40c66c0 2023-10-18T11:00:51.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.744+0000 7f46ab7fe640 1 -- 172.21.15.89:0/1792144534 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4877 (secure 0 0 0) 0x7f46ac0bd090 con 0x7f46b40c66c0 2023-10-18T11:00:51.748 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T11:00:51.748 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/654729641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":654729641},{"type":"v1","addr":"172.21.15.154:6836","nonce":654729641}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/964566480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":964566480},{"type":"v1","addr":"172.21.15.154:6837","nonce":964566480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2261878538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2261878538},{"type":"v1","addr":"172.21.15.89:6833","nonce":2261878538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:47:15.199575+0000","modified":"2023-10-18T10:55:20.892655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5517":{"gid":5517,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":124,"addr":"172.21.15.89:6835/2092274731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2092274731},{"type":"v1","addr":"172.21.15.89:6835","nonce":2092274731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T11:00:51.749 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-18T11:00:51.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.748+0000 7f46b90bd640 1 -- 172.21.15.89:0/1792144534 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4694065680 msgr2=0x7f4694067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:00:51.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.748+0000 7f46b90bd640 1 --2- 172.21.15.89:0/1792144534 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4694065680 0x7f4694067b40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f46a4009d60 tx=0x7f46a4009190 comp rx=0 tx=0).stop 2023-10-18T11:00:51.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.748+0000 7f46b90bd640 1 -- 172.21.15.89:0/1792144534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46b40c66c0 msgr2=0x7f46b415fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:00:51.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.748+0000 7f46b90bd640 1 --2- 172.21.15.89:0/1792144534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46b40c66c0 0x7f46b415fb60 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f46ac002240 tx=0x7f46ac009750 comp rx=0 tx=0).stop 2023-10-18T11:00:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.748+0000 7f46b90bd640 1 -- 172.21.15.89:0/1792144534 shutdown_connections 2023-10-18T11:00:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.748+0000 7f46b90bd640 1 --2- 172.21.15.89:0/1792144534 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4694065680 0x7f4694067b40 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:00:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.748+0000 7f46b90bd640 1 --2- 172.21.15.89:0/1792144534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46b40c66c0 0x7f46b415fb60 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:00:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.748+0000 7f46b90bd640 1 --2- 172.21.15.89:0/1792144534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46b40c7ce0 0x7f46b40c6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:00:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.748+0000 7f46b90bd640 1 --2- 172.21.15.89:0/1792144534 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46a0005110 0x7f46b4151e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:00:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.748+0000 7f46b90bd640 1 -- 172.21.15.89:0/1792144534 >> 172.21.15.89:0/1792144534 conn(0x7f46b4089fd0 msgr2=0x7f46b415bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:00:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.748+0000 7f46b90bd640 1 -- 172.21.15.89:0/1792144534 shutdown_connections 2023-10-18T11:00:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:00:51.748+0000 7f46b90bd640 1 -- 172.21.15.89:0/1792144534 wait complete. 2023-10-18T11:00:51.768 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-18T11:00:51.928 INFO:teuthology.orchestra.run.smithi089.stdout:[{"id":5183,"entity":{"name":{"type":"client","num":5183},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":4272238049}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":808.21786961600003,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.15452802739514163,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":3,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5183 v1:192.168.0.1:0/4272238049","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi154","kernel_version":"6.6.0-rc5-g47672119149b","root":"/"}}] 2023-10-18T11:00:51.928 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "1", "hostname": "smithi154", "kernel_version": "6.6.0-rc5-g47672119149b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 4272238049, "type": "v1" }, "name": { "num": 5183, "type": "client" } }, "id": 5183, "inst": "client.5183 v1:192.168.0.1:0/4272238049", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 3, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.15452802739514163 }, "state": "open", "uptime": 808.217869616 } ] 2023-10-18T11:00:52.029 DEBUG:tasks.ceph_test_case:No log hits yet, waiting... 2023-10-18T11:01:21.931 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:01:21.935 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:01:27.046 ERROR:tasks.ceph_test_case:Log output: 2023-10-18T11:01:27.046 DEBUG:tasks.cephfs.test_client_recovery:Expected log message found: 'Health check cleared: MDS_CLIENTS_LAGGY', retrying 2023-10-18T11:01:32.047 DEBUG:teuthology.orchestra.run.smithi089:> sudo daemon-helper kill ceph --cluster ceph -w 2023-10-18T11:01:32.052 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T11:01:32.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.203+0000 7fa65e100640 1 Processor -- start 2023-10-18T11:01:32.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.203+0000 7fa65e100640 1 -- start start 2023-10-18T11:01:32.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.203+0000 7fa65e100640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa65815c940 0x7fa6580d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:01:32.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.203+0000 7fa65e100640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6580d0e70 0x7fa6580cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:01:32.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.203+0000 7fa65e100640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa65815da50 con 0x7fa6580d2f40 2023-10-18T11:01:32.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.203+0000 7fa65e100640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa65815ce40 con 0x7fa65815c940 2023-10-18T11:01:32.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.203+0000 7fa65e100640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa65815d120 con 0x7fa6580d0e70 2023-10-18T11:01:32.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.203+0000 7fa64ffff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa6580d2f40 0x7fa6580d0760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41844/0 (socket says 172.21.15.89:41844) 2023-10-18T11:01:32.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.203+0000 7fa64ffff640 1 -- 172.21.15.89:0/2057104659 learned_addr learned my addr 172.21.15.89:0/2057104659 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:01:32.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.207+0000 7fa6577fe640 1 --2- 172.21.15.89:0/2057104659 >> v2:172.21.15.154:3300/0 conn(0x7fa65815c940 0x7fa6580d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:01:32.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.207+0000 7fa657fff640 1 --2- 172.21.15.89:0/2057104659 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6580d0e70 0x7fa6580cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:01:32.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.207+0000 7fa656ffd640 1 -- 172.21.15.89:0/2057104659 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1971483723 0 0) 0x7fa65815da50 con 0x7fa6580d2f40 2023-10-18T11:01:32.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.207+0000 7fa656ffd640 1 -- 172.21.15.89:0/2057104659 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa644003530 con 0x7fa6580d2f40 2023-10-18T11:01:32.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.207+0000 7fa6577fe640 1 -- 172.21.15.89:0/2057104659 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6580d0e70 msgr2=0x7fa6580cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:01:32.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.207+0000 7fa6577fe640 1 --2- 172.21.15.89:0/2057104659 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6580d0e70 0x7fa6580cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:01:32.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.207+0000 7fa6577fe640 1 -- 172.21.15.89:0/2057104659 >> v1:172.21.15.89:6789/0 conn(0x7fa6580d2f40 legacy=0x7fa6580d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:01:32.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.207+0000 7fa6577fe640 1 -- 172.21.15.89:0/2057104659 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa65815d400 con 0x7fa65815c940 2023-10-18T11:01:32.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.207+0000 7fa657fff640 1 --2- 172.21.15.89:0/2057104659 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6580d0e70 0x7fa6580cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T11:01:32.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.207+0000 7fa6577fe640 1 --2- 172.21.15.89:0/2057104659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa65815c940 0x7fa6580d2a00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa6480097c0 tx=0x7fa64801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e979aae9bfa4a59e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:01:32.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.207+0000 7fa656ffd640 1 -- 172.21.15.89:0/2057104659 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6480ab5f0 con 0x7fa65815c940 2023-10-18T11:01:32.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.207+0000 7fa656ffd640 1 -- 172.21.15.89:0/2057104659 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6480ab750 con 0x7fa65815c940 2023-10-18T11:01:32.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.207+0000 7fa656ffd640 1 -- 172.21.15.89:0/2057104659 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6480abae0 con 0x7fa65815c940 2023-10-18T11:01:32.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.211+0000 7fa65e100640 1 -- 172.21.15.89:0/2057104659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa65815c940 msgr2=0x7fa6580d2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:01:32.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.211+0000 7fa65e100640 1 --2- 172.21.15.89:0/2057104659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa65815c940 0x7fa6580d2a00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa6480097c0 tx=0x7fa64801c450 comp rx=0 tx=0).stop 2023-10-18T11:01:32.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.211+0000 7fa65e100640 1 -- 172.21.15.89:0/2057104659 shutdown_connections 2023-10-18T11:01:32.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.211+0000 7fa65e100640 1 --2- 172.21.15.89:0/2057104659 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6580d0e70 0x7fa6580cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:01:32.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.211+0000 7fa65e100640 1 --2- 172.21.15.89:0/2057104659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa65815c940 0x7fa6580d2a00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:01:32.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.211+0000 7fa65e100640 1 -- 172.21.15.89:0/2057104659 >> 172.21.15.89:0/2057104659 conn(0x7fa658089fd0 msgr2=0x7fa65809b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:01:32.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.211+0000 7fa65e100640 1 -- 172.21.15.89:0/2057104659 shutdown_connections 2023-10-18T11:01:32.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.211+0000 7fa65e100640 1 -- 172.21.15.89:0/2057104659 wait complete. 2023-10-18T11:01:32.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.211+0000 7fa65e100640 1 Processor -- start 2023-10-18T11:01:32.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.211+0000 7fa65e100640 1 -- start start 2023-10-18T11:01:32.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.211+0000 7fa65e100640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6580d0e70 0x7fa6580cad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:01:32.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.211+0000 7fa65e100640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6580c77f0 0x7fa658159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:01:32.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.211+0000 7fa65e100640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6580b7280 0x7fa6580b7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:01:32.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.211+0000 7fa65e100640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6580b7c40 con 0x7fa6580c77f0 2023-10-18T11:01:32.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.211+0000 7fa65e100640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6580c64d0 con 0x7fa6580b7280 2023-10-18T11:01:32.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.211+0000 7fa65e100640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6580cbf60 con 0x7fa6580d0e70 2023-10-18T11:01:32.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.211+0000 7fa64ffff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6580c77f0 0x7fa658159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:01:32.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.211+0000 7fa64ffff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6580c77f0 0x7fa658159420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58250/0 (socket says 172.21.15.89:58250) 2023-10-18T11:01:32.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.211+0000 7fa64ffff640 1 -- 172.21.15.89:0/828462681 learned_addr learned my addr 172.21.15.89:0/828462681 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:01:32.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.211+0000 7fa6577fe640 1 --2- 172.21.15.89:0/828462681 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6580d0e70 0x7fa6580cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:01:32.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.211+0000 7fa657fff640 1 --2- 172.21.15.89:0/828462681 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6580b7280 0x7fa6580b7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:01:32.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.215+0000 7fa64ffff640 1 -- 172.21.15.89:0/828462681 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6580d0e70 msgr2=0x7fa6580cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:01:32.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.215+0000 7fa64ffff640 1 --2- 172.21.15.89:0/828462681 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6580d0e70 0x7fa6580cad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:01:32.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.215+0000 7fa64ffff640 1 -- 172.21.15.89:0/828462681 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6580b7280 msgr2=0x7fa6580b7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:01:32.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.215+0000 7fa64ffff640 1 --2- 172.21.15.89:0/828462681 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6580b7280 0x7fa6580b7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:01:32.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.215+0000 7fa64ffff640 1 -- 172.21.15.89:0/828462681 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa650009a70 con 0x7fa6580c77f0 2023-10-18T11:01:32.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.215+0000 7fa64ffff640 1 --2- 172.21.15.89:0/828462681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6580c77f0 0x7fa658159420 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa64000c4d0 tx=0x7fa64000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:01:32.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.215+0000 7fa654ff9640 1 -- 172.21.15.89:0/828462681 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa640005cf0 con 0x7fa6580c77f0 2023-10-18T11:01:32.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.215+0000 7fa654ff9640 1 -- 172.21.15.89:0/828462681 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa640005e50 con 0x7fa6580c77f0 2023-10-18T11:01:32.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.215+0000 7fa654ff9640 1 -- 172.21.15.89:0/828462681 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6400072d0 con 0x7fa6580c77f0 2023-10-18T11:01:32.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.215+0000 7fa65e100640 1 -- 172.21.15.89:0/828462681 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa648009420 con 0x7fa6580c77f0 2023-10-18T11:01:32.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.215+0000 7fa65e100640 1 -- 172.21.15.89:0/828462681 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6580cb350 con 0x7fa6580c77f0 2023-10-18T11:01:32.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.215+0000 7fa65e100640 1 -- 172.21.15.89:0/828462681 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa65804dbe0 con 0x7fa6580c77f0 2023-10-18T11:01:32.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.215+0000 7fa654ff9640 1 -- 172.21.15.89:0/828462681 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fa640007430 con 0x7fa6580c77f0 2023-10-18T11:01:32.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.215+0000 7fa654ff9640 1 --2- 172.21.15.89:0/828462681 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa6380656b0 0x7fa638067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:01:32.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.219+0000 7fa654ff9640 1 -- 172.21.15.89:0/828462681 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fa640082500 con 0x7fa6580c77f0 2023-10-18T11:01:32.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.219+0000 7fa6577fe640 1 --2- 172.21.15.89:0/828462681 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa6380656b0 0x7fa638067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:01:32.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.219+0000 7fa6577fe640 1 --2- 172.21.15.89:0/828462681 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa6380656b0 0x7fa638067b70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa6480af000 tx=0x7fa6480a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:01:32.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.223+0000 7fa654ff9640 1 -- 172.21.15.89:0/828462681 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fa64004fec0 con 0x7fa6580c77f0 2023-10-18T11:01:32.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.231+0000 7f88f8c48640 1 Processor -- start 2023-10-18T11:01:32.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.231+0000 7f88f8c48640 1 -- start start 2023-10-18T11:01:32.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.231+0000 7f88f8c48640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88f40b33b0 0x7f88f40b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:01:32.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.231+0000 7f88f8c48640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88f40cced0 con 0x7f88f4160f20 2023-10-18T11:01:32.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.231+0000 7f88f8c48640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88f40b1d30 con 0x7f88f40d0e60 2023-10-18T11:01:32.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.231+0000 7f88f8c48640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88f40b3bc0 con 0x7f88f40b33b0 2023-10-18T11:01:32.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.231+0000 7f88f259c640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f88f4160f20 0x7f88f40d0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41858/0 (socket says 172.21.15.89:41858) 2023-10-18T11:01:32.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.231+0000 7f88f259c640 1 -- 172.21.15.89:0/4197215148 learned_addr learned my addr 172.21.15.89:0/4197215148 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:01:32.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.235+0000 7f88f2d9d640 1 --2- 172.21.15.89:0/4197215148 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88f40b33b0 0x7f88f40b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:01:32.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.235+0000 7f88f1d9b640 1 -- 172.21.15.89:0/4197215148 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2218490772 0 0) 0x7f88f40cced0 con 0x7f88f4160f20 2023-10-18T11:01:32.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.235+0000 7f88f1d9b640 1 -- 172.21.15.89:0/4197215148 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88e0003530 con 0x7f88f4160f20 2023-10-18T11:01:32.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.235+0000 7f88f1d9b640 1 -- 172.21.15.89:0/4197215148 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2985349145 0 0) 0x7f88f40b1d30 con 0x7f88f40d0e60 2023-10-18T11:01:32.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.235+0000 7f88f1d9b640 1 -- 172.21.15.89:0/4197215148 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88f40cced0 con 0x7f88f40d0e60 2023-10-18T11:01:32.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.235+0000 7f88f2d9d640 1 -- 172.21.15.89:0/4197215148 >> v1:172.21.15.154:6789/0 conn(0x7f88f40d0e60 legacy=0x7f88f40b05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:01:32.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.235+0000 7f88f2d9d640 1 -- 172.21.15.89:0/4197215148 >> v1:172.21.15.89:6789/0 conn(0x7f88f4160f20 legacy=0x7f88f40d0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:01:32.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.235+0000 7f88f2d9d640 1 -- 172.21.15.89:0/4197215148 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f40afdd0 con 0x7f88f40b33b0 2023-10-18T11:01:32.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.235+0000 7f88f2d9d640 1 --2- 172.21.15.89:0/4197215148 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88f40b33b0 0x7f88f40b0ce0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f88ec009dc0 tx=0x7f88ec01c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca9d34f6038e916e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:01:32.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.239+0000 7f88f1d9b640 1 -- 172.21.15.89:0/4197215148 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec0a5870 con 0x7f88f40b33b0 2023-10-18T11:01:32.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.239+0000 7f88f1d9b640 1 -- 172.21.15.89:0/4197215148 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88ec0a59d0 con 0x7f88f40b33b0 2023-10-18T11:01:32.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.239+0000 7f88f1d9b640 1 -- 172.21.15.89:0/4197215148 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec0a5d40 con 0x7f88f40b33b0 2023-10-18T11:01:32.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.239+0000 7f88f8c48640 1 -- 172.21.15.89:0/4197215148 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88f40b33b0 msgr2=0x7f88f40b0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:01:32.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.239+0000 7f88f8c48640 1 --2- 172.21.15.89:0/4197215148 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88f40b33b0 0x7f88f40b0ce0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f88ec009dc0 tx=0x7f88ec01c8e0 comp rx=0 tx=0).stop 2023-10-18T11:01:32.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.239+0000 7f88f8c48640 1 -- 172.21.15.89:0/4197215148 shutdown_connections 2023-10-18T11:01:32.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.239+0000 7f88f8c48640 1 --2- 172.21.15.89:0/4197215148 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88f40b33b0 0x7f88f40b0ce0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:01:32.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.239+0000 7f88f8c48640 1 -- 172.21.15.89:0/4197215148 >> 172.21.15.89:0/4197215148 conn(0x7f88f409b930 msgr2=0x7f88f40d26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:01:32.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.239+0000 7f88f8c48640 1 -- 172.21.15.89:0/4197215148 shutdown_connections 2023-10-18T11:01:32.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.239+0000 7f88f8c48640 1 -- 172.21.15.89:0/4197215148 wait complete. 2023-10-18T11:01:32.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.239+0000 7f88f8c48640 1 Processor -- start 2023-10-18T11:01:32.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.239+0000 7f88f8c48640 1 -- start start 2023-10-18T11:01:32.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.239+0000 7f88f8c48640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88f40b33b0 0x7f88f4150440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:01:32.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.239+0000 7f88f8c48640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88f40d0e60 0x7f88f41556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:01:32.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.239+0000 7f88f8c48640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88f4160f20 0x7f88f4155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:01:32.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.239+0000 7f88f8c48640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f415fda0 con 0x7f88f4160f20 2023-10-18T11:01:32.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.239+0000 7f88f8c48640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f4156250 con 0x7f88f40d0e60 2023-10-18T11:01:32.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.239+0000 7f88f8c48640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88f4160040 con 0x7f88f40b33b0 2023-10-18T11:01:32.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.239+0000 7f88f2d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88f4160f20 0x7f88f4155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:01:32.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.239+0000 7f88f2d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88f4160f20 0x7f88f4155c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58254/0 (socket says 172.21.15.89:58254) 2023-10-18T11:01:32.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.239+0000 7f88f2d9d640 1 -- 172.21.15.89:0/2012571894 learned_addr learned my addr 172.21.15.89:0/2012571894 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:01:32.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.243+0000 7f88ebfff640 1 --2- 172.21.15.89:0/2012571894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88f40d0e60 0x7f88f41556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:01:32.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.243+0000 7f88f259c640 1 --2- 172.21.15.89:0/2012571894 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88f40b33b0 0x7f88f4150440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:01:32.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.243+0000 7f88f2d9d640 1 -- 172.21.15.89:0/2012571894 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88f40b33b0 msgr2=0x7f88f4150440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:01:32.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.243+0000 7f88f2d9d640 1 --2- 172.21.15.89:0/2012571894 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88f40b33b0 0x7f88f4150440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:01:32.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.243+0000 7f88f2d9d640 1 -- 172.21.15.89:0/2012571894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88f40d0e60 msgr2=0x7f88f41556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:01:32.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.243+0000 7f88f2d9d640 1 --2- 172.21.15.89:0/2012571894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88f40d0e60 0x7f88f41556e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:01:32.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.243+0000 7f88f2d9d640 1 -- 172.21.15.89:0/2012571894 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ec009a70 con 0x7f88f4160f20 2023-10-18T11:01:32.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.243+0000 7f88f259c640 1 --2- 172.21.15.89:0/2012571894 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88f40b33b0 0x7f88f4150440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:01:32.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.243+0000 7f88f2d9d640 1 --2- 172.21.15.89:0/2012571894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88f4160f20 0x7f88f4155c20 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f88ec0b0040 tx=0x7f88ec019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:01:32.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.243+0000 7f88eaffd640 1 -- 172.21.15.89:0/2012571894 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec0ad640 con 0x7f88f4160f20 2023-10-18T11:01:32.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.243+0000 7f88eaffd640 1 -- 172.21.15.89:0/2012571894 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88ec0ad7a0 con 0x7f88f4160f20 2023-10-18T11:01:32.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.243+0000 7f88eaffd640 1 -- 172.21.15.89:0/2012571894 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec0adb70 con 0x7f88f4160f20 2023-10-18T11:01:32.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.243+0000 7f88f8c48640 1 -- 172.21.15.89:0/2012571894 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88f4160320 con 0x7f88f4160f20 2023-10-18T11:01:32.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.243+0000 7f88f8c48640 1 -- 172.21.15.89:0/2012571894 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f4157a70 con 0x7f88f4160f20 2023-10-18T11:01:32.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.243+0000 7f88f8c48640 1 -- 172.21.15.89:0/2012571894 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f88f404e430 con 0x7f88f4160f20 2023-10-18T11:01:32.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.243+0000 7f88eaffd640 1 -- 172.21.15.89:0/2012571894 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f88ec0b4050 con 0x7f88f4160f20 2023-10-18T11:01:32.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.243+0000 7f88eaffd640 1 --2- 172.21.15.89:0/2012571894 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f88d4065680 0x7f88d4067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:01:32.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.247+0000 7f88eaffd640 1 -- 172.21.15.89:0/2012571894 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f88ec11e160 con 0x7f88f4160f20 2023-10-18T11:01:32.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.247+0000 7f88eaffd640 1 -- 172.21.15.89:0/2012571894 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+449 (secure 0 0 0) 0x7f88ec11e4f0 con 0x7f88f4160f20 2023-10-18T11:01:32.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.247+0000 7f88f259c640 1 --2- 172.21.15.89:0/2012571894 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f88d4065680 0x7f88d4067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:01:32.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.247+0000 7f88f8c48640 1 -- 172.21.15.89:0/2012571894 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f88f41607d0 con 0x7f88f4160f20 2023-10-18T11:01:32.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.247+0000 7f88f259c640 1 --2- 172.21.15.89:0/2012571894 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f88d4065680 0x7f88d4067b40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f88e4000f70 tx=0x7f88e4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:01:32.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.403+0000 7fa65e100640 1 -- 172.21.15.89:0/828462681 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6580cc0a0 con 0x7fa6580c77f0 2023-10-18T11:01:32.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.403+0000 7fa654ff9640 1 -- 172.21.15.89:0/828462681 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4877 (secure 0 0 0) 0x7fa6400534e0 con 0x7fa6580c77f0 2023-10-18T11:01:32.402 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T11:01:32.402 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/654729641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":654729641},{"type":"v1","addr":"172.21.15.154:6836","nonce":654729641}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/964566480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":964566480},{"type":"v1","addr":"172.21.15.154:6837","nonce":964566480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2261878538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2261878538},{"type":"v1","addr":"172.21.15.89:6833","nonce":2261878538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:47:15.199575+0000","modified":"2023-10-18T10:55:20.892655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5517":{"gid":5517,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":124,"addr":"172.21.15.89:6835/2092274731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2092274731},{"type":"v1","addr":"172.21.15.89:6835","nonce":2092274731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T11:01:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-18T11:01:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.403+0000 7fa65e100640 1 -- 172.21.15.89:0/828462681 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa6380656b0 msgr2=0x7fa638067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:01:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.403+0000 7fa65e100640 1 --2- 172.21.15.89:0/828462681 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa6380656b0 0x7fa638067b70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa6480af000 tx=0x7fa6480a7000 comp rx=0 tx=0).stop 2023-10-18T11:01:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.403+0000 7fa65e100640 1 -- 172.21.15.89:0/828462681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6580c77f0 msgr2=0x7fa658159420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:01:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.403+0000 7fa65e100640 1 --2- 172.21.15.89:0/828462681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6580c77f0 0x7fa658159420 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa64000c4d0 tx=0x7fa64000c8d0 comp rx=0 tx=0).stop 2023-10-18T11:01:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.403+0000 7fa65e100640 1 -- 172.21.15.89:0/828462681 shutdown_connections 2023-10-18T11:01:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.403+0000 7fa65e100640 1 --2- 172.21.15.89:0/828462681 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa6380656b0 0x7fa638067b70 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:01:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.403+0000 7fa65e100640 1 --2- 172.21.15.89:0/828462681 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6580b7280 0x7fa6580b7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:01:32.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.403+0000 7fa65e100640 1 --2- 172.21.15.89:0/828462681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6580c77f0 0x7fa658159420 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:01:32.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.403+0000 7fa65e100640 1 --2- 172.21.15.89:0/828462681 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6580d0e70 0x7fa6580cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:01:32.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.403+0000 7fa65e100640 1 -- 172.21.15.89:0/828462681 >> 172.21.15.89:0/828462681 conn(0x7fa658089fd0 msgr2=0x7fa65815ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:01:32.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.407+0000 7fa65e100640 1 -- 172.21.15.89:0/828462681 shutdown_connections 2023-10-18T11:01:32.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:01:32.407+0000 7fa65e100640 1 -- 172.21.15.89:0/828462681 wait complete. 2023-10-18T11:01:32.423 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-18T11:01:32.580 INFO:teuthology.orchestra.run.smithi089.stdout:[{"id":5183,"entity":{"name":{"type":"client","num":5183},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":4272238049}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":848.87284432000001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.14067363330424623,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":3,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5183 v1:192.168.0.1:0/4272238049","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi154","kernel_version":"6.6.0-rc5-g47672119149b","root":"/"}}] 2023-10-18T11:01:32.581 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "1", "hostname": "smithi154", "kernel_version": "6.6.0-rc5-g47672119149b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 4272238049, "type": "v1" }, "name": { "num": 5183, "type": "client" } }, "id": 5183, "inst": "client.5183 v1:192.168.0.1:0/4272238049", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 3, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.14067363330424623 }, "state": "open", "uptime": 848.87284432 } ] 2023-10-18T11:01:32.682 DEBUG:tasks.ceph_test_case:No log hits yet, waiting... 2023-10-18T11:01:52.065 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:01:52.070 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:02:07.699 ERROR:tasks.ceph_test_case:Log output: 2023-10-18T11:02:07.700 DEBUG:tasks.cephfs.test_client_recovery:Expected log message found: 'Health check cleared: MDS_CLIENTS_LAGGY', retrying 2023-10-18T11:02:12.701 DEBUG:teuthology.orchestra.run.smithi089:> sudo daemon-helper kill ceph --cluster ceph -w 2023-10-18T11:02:12.706 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T11:02:12.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.850+0000 7fed9178f640 1 Processor -- start 2023-10-18T11:02:12.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.850+0000 7fed9178f640 1 -- start start 2023-10-18T11:02:12.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.850+0000 7fed9178f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fed8c15c550 0x7fed8c0d2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:02:12.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.850+0000 7fed9178f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed8c0b1860 0x7fed8c0cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:02:12.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.850+0000 7fed9178f640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed8c0d1e50 con 0x7fed8c0b1cc0 2023-10-18T11:02:12.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.850+0000 7fed9178f640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed8c0d2130 con 0x7fed8c15c550 2023-10-18T11:02:12.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.850+0000 7fed9178f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c0d2410 con 0x7fed8c0b1860 2023-10-18T11:02:12.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.850+0000 7fed8affd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fed8c15c550 0x7fed8c0d2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:02:12.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.850+0000 7fed8b7fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fed8c0b1cc0 0x7fed8c0b55f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:56368/0 (socket says 172.21.15.89:56368) 2023-10-18T11:02:12.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.850+0000 7fed8b7fe640 1 -- 172.21.15.89:0/702479929 learned_addr learned my addr 172.21.15.89:0/702479929 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:02:12.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.850+0000 7fed83fff640 1 --2- 172.21.15.89:0/702479929 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed8c0b1860 0x7fed8c0cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:02:12.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.850+0000 7fed8affd640 1 -- 172.21.15.89:0/702479929 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed8c0b1860 msgr2=0x7fed8c0cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:02:12.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.850+0000 7fed8affd640 1 --2- 172.21.15.89:0/702479929 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed8c0b1860 0x7fed8c0cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:02:12.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.850+0000 7fed8affd640 1 -- 172.21.15.89:0/702479929 >> v1:172.21.15.89:6789/0 conn(0x7fed8c0b1cc0 legacy=0x7fed8c0b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:02:12.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.850+0000 7fed8affd640 1 -- 172.21.15.89:0/702479929 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed8c0d26f0 con 0x7fed8c15c550 2023-10-18T11:02:12.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.850+0000 7fed8a7fc640 1 -- 172.21.15.89:0/702479929 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1093448856 0 0) 0x7fed8c0d1e50 con 0x7fed8c0b1cc0 2023-10-18T11:02:12.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.854+0000 7fed8affd640 1 --2- 172.21.15.89:0/702479929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed8c15c550 0x7fed8c0d2ef0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0097c0 tx=0x7fed7c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d5006828c775b00 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:02:12.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.854+0000 7fed8a7fc640 1 -- 172.21.15.89:0/702479929 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c0ad660 con 0x7fed8c15c550 2023-10-18T11:02:12.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.854+0000 7fed8a7fc640 1 -- 172.21.15.89:0/702479929 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7c0ad7c0 con 0x7fed8c15c550 2023-10-18T11:02:12.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.854+0000 7fed8a7fc640 1 -- 172.21.15.89:0/702479929 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c0adb50 con 0x7fed8c15c550 2023-10-18T11:02:12.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.854+0000 7fed9178f640 1 -- 172.21.15.89:0/702479929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed8c15c550 msgr2=0x7fed8c0d2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:02:12.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.854+0000 7fed9178f640 1 --2- 172.21.15.89:0/702479929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed8c15c550 0x7fed8c0d2ef0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0097c0 tx=0x7fed7c01c450 comp rx=0 tx=0).stop 2023-10-18T11:02:12.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.854+0000 7fed9178f640 1 -- 172.21.15.89:0/702479929 shutdown_connections 2023-10-18T11:02:12.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.854+0000 7fed9178f640 1 --2- 172.21.15.89:0/702479929 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed8c0b1860 0x7fed8c0cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:02:12.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.854+0000 7fed9178f640 1 --2- 172.21.15.89:0/702479929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed8c15c550 0x7fed8c0d2ef0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:02:12.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.854+0000 7fed9178f640 1 -- 172.21.15.89:0/702479929 >> 172.21.15.89:0/702479929 conn(0x7fed8c069d00 msgr2=0x7fed8c06a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:02:12.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.854+0000 7fed9178f640 1 -- 172.21.15.89:0/702479929 shutdown_connections 2023-10-18T11:02:12.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.854+0000 7fed9178f640 1 -- 172.21.15.89:0/702479929 wait complete. 2023-10-18T11:02:12.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.858+0000 7fed9178f640 1 Processor -- start 2023-10-18T11:02:12.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.858+0000 7fed9178f640 1 -- start start 2023-10-18T11:02:12.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.858+0000 7fed9178f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed8c0b1860 0x7fed8c0caf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:02:12.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.858+0000 7fed9178f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed8c0b1cc0 0x7fed8c1f8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:02:12.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.858+0000 7fed9178f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed8c15c550 0x7fed8c0b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:02:12.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.858+0000 7fed9178f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c0cb850 con 0x7fed8c0b1860 2023-10-18T11:02:12.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.858+0000 7fed9178f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c0cbb30 con 0x7fed8c0b1cc0 2023-10-18T11:02:12.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.858+0000 7fed9178f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c0cbe10 con 0x7fed8c15c550 2023-10-18T11:02:12.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.858+0000 7fed8affd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed8c0b1860 0x7fed8c0caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:02:12.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.858+0000 7fed8affd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed8c0b1860 0x7fed8c0caf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48026/0 (socket says 172.21.15.89:48026) 2023-10-18T11:02:12.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.858+0000 7fed8affd640 1 -- 172.21.15.89:0/664780319 learned_addr learned my addr 172.21.15.89:0/664780319 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:02:12.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.858+0000 7fed8b7fe640 1 --2- 172.21.15.89:0/664780319 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed8c15c550 0x7fed8c0b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:02:12.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.858+0000 7fed83fff640 1 --2- 172.21.15.89:0/664780319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed8c0b1cc0 0x7fed8c1f8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:02:12.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.858+0000 7fed8affd640 1 -- 172.21.15.89:0/664780319 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed8c15c550 msgr2=0x7fed8c0b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:02:12.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.858+0000 7fed8affd640 1 --2- 172.21.15.89:0/664780319 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed8c15c550 0x7fed8c0b2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:02:12.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.858+0000 7fed8affd640 1 -- 172.21.15.89:0/664780319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed8c0b1cc0 msgr2=0x7fed8c1f8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:02:12.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.858+0000 7fed8affd640 1 --2- 172.21.15.89:0/664780319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed8c0b1cc0 0x7fed8c1f8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:02:12.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.858+0000 7fed8affd640 1 -- 172.21.15.89:0/664780319 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7c009420 con 0x7fed8c0b1860 2023-10-18T11:02:12.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.858+0000 7fed83fff640 1 --2- 172.21.15.89:0/664780319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed8c0b1cc0 0x7fed8c1f8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:02:12.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.858+0000 7fed8affd640 1 --2- 172.21.15.89:0/664780319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed8c0b1860 0x7fed8c0caf70 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fed7c01a080 tx=0x7fed7c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:02:12.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.858+0000 7fed837fe640 1 -- 172.21.15.89:0/664780319 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c004220 con 0x7fed8c0b1860 2023-10-18T11:02:12.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.858+0000 7fed837fe640 1 -- 172.21.15.89:0/664780319 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7c004380 con 0x7fed8c0b1860 2023-10-18T11:02:12.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.858+0000 7fed837fe640 1 -- 172.21.15.89:0/664780319 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c003820 con 0x7fed8c0b1860 2023-10-18T11:02:12.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.858+0000 7fed9178f640 1 -- 172.21.15.89:0/664780319 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed8c0cc0f0 con 0x7fed8c0b1860 2023-10-18T11:02:12.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.858+0000 7fed9178f640 1 -- 172.21.15.89:0/664780319 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed8c1f9280 con 0x7fed8c0b1860 2023-10-18T11:02:12.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.862+0000 7fed9178f640 1 -- 172.21.15.89:0/664780319 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed8c0cc280 con 0x7fed8c0b1860 2023-10-18T11:02:12.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.862+0000 7fed837fe640 1 -- 172.21.15.89:0/664780319 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fed7c003980 con 0x7fed8c0b1860 2023-10-18T11:02:12.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.862+0000 7fed837fe640 1 --2- 172.21.15.89:0/664780319 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fed6c065680 0x7fed6c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:02:12.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.862+0000 7fed837fe640 1 -- 172.21.15.89:0/664780319 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fed7c11d830 con 0x7fed8c0b1860 2023-10-18T11:02:12.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.862+0000 7fed83fff640 1 --2- 172.21.15.89:0/664780319 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fed6c065680 0x7fed6c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:02:12.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.862+0000 7fed83fff640 1 --2- 172.21.15.89:0/664780319 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fed6c065680 0x7fed6c067b40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fed8c0b2ea0 tx=0x7fed74008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:02:12.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.866+0000 7fed837fe640 1 -- 172.21.15.89:0/664780319 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fed7c0eb0a0 con 0x7fed8c0b1860 2023-10-18T11:02:12.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.874+0000 7efc264e6640 1 Processor -- start 2023-10-18T11:02:12.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.878+0000 7efc264e6640 1 -- start start 2023-10-18T11:02:12.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.878+0000 7efc264e6640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efc2015a4b0 0x7efc2015ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:02:12.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.878+0000 7efc264e6640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc2006f7c0 0x7efc2006fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:02:12.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.878+0000 7efc264e6640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc2015a890 con 0x7efc200caa20 2023-10-18T11:02:12.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.878+0000 7efc264e6640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc20070190 con 0x7efc2015a4b0 2023-10-18T11:02:12.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.878+0000 7efc264e6640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc2015ef40 con 0x7efc2006f7c0 2023-10-18T11:02:12.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.878+0000 7efc1ffff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efc2015a4b0 0x7efc2015ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:02:12.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.878+0000 7efc1ffff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efc2015a4b0 0x7efc2015ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48042/0 (socket says 172.21.15.89:48042) 2023-10-18T11:02:12.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.878+0000 7efc1ffff640 1 -- 172.21.15.89:0/2266736230 learned_addr learned my addr 172.21.15.89:0/2266736230 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:02:12.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.878+0000 7efc24a83640 1 --2- 172.21.15.89:0/2266736230 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc2006f7c0 0x7efc2006fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:02:12.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.878+0000 7efc1ffff640 1 -- 172.21.15.89:0/2266736230 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc2006f7c0 msgr2=0x7efc2006fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:02:12.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.878+0000 7efc1ffff640 1 --2- 172.21.15.89:0/2266736230 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc2006f7c0 0x7efc2006fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:02:12.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.878+0000 7efc1ffff640 1 -- 172.21.15.89:0/2266736230 >> v1:172.21.15.154:6789/0 conn(0x7efc200caa20 legacy=0x7efc200c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:02:12.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.878+0000 7efc1ffff640 1 -- 172.21.15.89:0/2266736230 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc200caee0 con 0x7efc2015a4b0 2023-10-18T11:02:12.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.878+0000 7efc24a83640 1 --2- 172.21.15.89:0/2266736230 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc2006f7c0 0x7efc2006fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:02:12.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.882+0000 7efc1ffff640 1 --2- 172.21.15.89:0/2266736230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc2015a4b0 0x7efc2015ea00 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7efc10009870 tx=0x7efc1001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e2ad19fc1ba6d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:02:12.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.882+0000 7efc1effd640 1 -- 172.21.15.89:0/2266736230 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc100aa5f0 con 0x7efc2015a4b0 2023-10-18T11:02:12.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.882+0000 7efc1effd640 1 -- 172.21.15.89:0/2266736230 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc100aa750 con 0x7efc2015a4b0 2023-10-18T11:02:12.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.882+0000 7efc1effd640 1 -- 172.21.15.89:0/2266736230 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc100aaae0 con 0x7efc2015a4b0 2023-10-18T11:02:12.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.882+0000 7efc264e6640 1 -- 172.21.15.89:0/2266736230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc2015a4b0 msgr2=0x7efc2015ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:02:12.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.882+0000 7efc264e6640 1 --2- 172.21.15.89:0/2266736230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc2015a4b0 0x7efc2015ea00 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7efc10009870 tx=0x7efc1001c4d0 comp rx=0 tx=0).stop 2023-10-18T11:02:12.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.882+0000 7efc264e6640 1 -- 172.21.15.89:0/2266736230 shutdown_connections 2023-10-18T11:02:12.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.882+0000 7efc264e6640 1 --2- 172.21.15.89:0/2266736230 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc2006f7c0 0x7efc2006fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:02:12.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.882+0000 7efc264e6640 1 --2- 172.21.15.89:0/2266736230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc2015a4b0 0x7efc2015ea00 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:02:12.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.882+0000 7efc264e6640 1 -- 172.21.15.89:0/2266736230 >> 172.21.15.89:0/2266736230 conn(0x7efc200afd30 msgr2=0x7efc200b0140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:02:12.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.882+0000 7efc264e6640 1 -- 172.21.15.89:0/2266736230 shutdown_connections 2023-10-18T11:02:12.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.882+0000 7efc264e6640 1 -- 172.21.15.89:0/2266736230 wait complete. 2023-10-18T11:02:12.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.886+0000 7efc264e6640 1 Processor -- start 2023-10-18T11:02:12.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.886+0000 7efc264e6640 1 -- start start 2023-10-18T11:02:12.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.886+0000 7efc264e6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc2006f7c0 0x7efc200b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:02:12.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.886+0000 7efc264e6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc200caa20 0x7efc200c1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:02:12.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.886+0000 7efc264e6640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc2015a4b0 0x7efc200c24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:02:12.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.886+0000 7efc264e6640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc20157610 con 0x7efc2006f7c0 2023-10-18T11:02:12.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.886+0000 7efc264e6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc20157270 con 0x7efc200caa20 2023-10-18T11:02:12.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.886+0000 7efc264e6640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc200c2ab0 con 0x7efc2015a4b0 2023-10-18T11:02:12.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.886+0000 7efc1ffff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc2006f7c0 0x7efc200b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:02:12.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.886+0000 7efc1ffff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc2006f7c0 0x7efc200b3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48058/0 (socket says 172.21.15.89:48058) 2023-10-18T11:02:12.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.886+0000 7efc1ffff640 1 -- 172.21.15.89:0/2385999690 learned_addr learned my addr 172.21.15.89:0/2385999690 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:02:12.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.886+0000 7efc1f7fe640 1 --2- 172.21.15.89:0/2385999690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc200caa20 0x7efc200c1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:02:12.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.886+0000 7efc24a83640 1 --2- 172.21.15.89:0/2385999690 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc2015a4b0 0x7efc200c24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:02:12.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.886+0000 7efc1ffff640 1 -- 172.21.15.89:0/2385999690 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc2015a4b0 msgr2=0x7efc200c24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:02:12.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.886+0000 7efc1ffff640 1 --2- 172.21.15.89:0/2385999690 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc2015a4b0 0x7efc200c24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:02:12.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.886+0000 7efc1ffff640 1 -- 172.21.15.89:0/2385999690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc200caa20 msgr2=0x7efc200c1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:02:12.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.886+0000 7efc1ffff640 1 --2- 172.21.15.89:0/2385999690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc200caa20 0x7efc200c1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:02:12.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.886+0000 7efc1ffff640 1 -- 172.21.15.89:0/2385999690 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc100094d0 con 0x7efc2006f7c0 2023-10-18T11:02:12.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.886+0000 7efc24a83640 1 --2- 172.21.15.89:0/2385999690 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc2015a4b0 0x7efc200c24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:02:12.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.886+0000 7efc1ffff640 1 --2- 172.21.15.89:0/2385999690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc2006f7c0 0x7efc200b3ae0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7efc1001c300 tx=0x7efc1001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:02:12.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.886+0000 7efc1cff9640 1 -- 172.21.15.89:0/2385999690 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc100b3800 con 0x7efc2006f7c0 2023-10-18T11:02:12.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.886+0000 7efc1cff9640 1 -- 172.21.15.89:0/2385999690 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc100b3960 con 0x7efc2006f7c0 2023-10-18T11:02:12.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.886+0000 7efc1cff9640 1 -- 172.21.15.89:0/2385999690 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc100b3d10 con 0x7efc2006f7c0 2023-10-18T11:02:12.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.886+0000 7efc264e6640 1 -- 172.21.15.89:0/2385999690 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc20150640 con 0x7efc2006f7c0 2023-10-18T11:02:12.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.886+0000 7efc264e6640 1 -- 172.21.15.89:0/2385999690 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc2014c410 con 0x7efc2006f7c0 2023-10-18T11:02:12.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.886+0000 7efc264e6640 1 -- 172.21.15.89:0/2385999690 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7efc2004e430 con 0x7efc2006f7c0 2023-10-18T11:02:12.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.890+0000 7efc1cff9640 1 -- 172.21.15.89:0/2385999690 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7efc100c6440 con 0x7efc2006f7c0 2023-10-18T11:02:12.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.890+0000 7efc1cff9640 1 --2- 172.21.15.89:0/2385999690 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efc0c065680 0x7efc0c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:02:12.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.890+0000 7efc1cff9640 1 -- 172.21.15.89:0/2385999690 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7efc100a7070 con 0x7efc2006f7c0 2023-10-18T11:02:12.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.890+0000 7efc1cff9640 1 -- 172.21.15.89:0/2385999690 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+449 (secure 0 0 0) 0x7efc1011e460 con 0x7efc2006f7c0 2023-10-18T11:02:12.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.890+0000 7efc1f7fe640 1 --2- 172.21.15.89:0/2385999690 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efc0c065680 0x7efc0c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:02:12.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.890+0000 7efc264e6640 1 -- 172.21.15.89:0/2385999690 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7efc200cb7b0 con 0x7efc2006f7c0 2023-10-18T11:02:12.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:12.890+0000 7efc1f7fe640 1 --2- 172.21.15.89:0/2385999690 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efc0c065680 0x7efc0c067b40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efc20155530 tx=0x7efc14007e40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:02:13.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:13.050+0000 7fed9178f640 1 -- 172.21.15.89:0/664780319 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed8c1f9530 con 0x7fed8c0b1860 2023-10-18T11:02:13.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:13.054+0000 7fed837fe640 1 -- 172.21.15.89:0/664780319 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4877 (secure 0 0 0) 0x7fed7c0044f0 con 0x7fed8c0b1860 2023-10-18T11:02:13.053 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T11:02:13.053 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/654729641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":654729641},{"type":"v1","addr":"172.21.15.154:6836","nonce":654729641}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/964566480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":964566480},{"type":"v1","addr":"172.21.15.154:6837","nonce":964566480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2261878538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2261878538},{"type":"v1","addr":"172.21.15.89:6833","nonce":2261878538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:47:15.199575+0000","modified":"2023-10-18T10:55:20.892655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5517":{"gid":5517,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":124,"addr":"172.21.15.89:6835/2092274731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2092274731},{"type":"v1","addr":"172.21.15.89:6835","nonce":2092274731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T11:02:13.054 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-18T11:02:13.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:13.054+0000 7fed9178f640 1 -- 172.21.15.89:0/664780319 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fed6c065680 msgr2=0x7fed6c067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:02:13.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:13.054+0000 7fed9178f640 1 --2- 172.21.15.89:0/664780319 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fed6c065680 0x7fed6c067b40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fed8c0b2ea0 tx=0x7fed74008040 comp rx=0 tx=0).stop 2023-10-18T11:02:13.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:13.054+0000 7fed9178f640 1 -- 172.21.15.89:0/664780319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed8c0b1860 msgr2=0x7fed8c0caf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:02:13.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:13.054+0000 7fed9178f640 1 --2- 172.21.15.89:0/664780319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed8c0b1860 0x7fed8c0caf70 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fed7c01a080 tx=0x7fed7c01ac90 comp rx=0 tx=0).stop 2023-10-18T11:02:13.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:13.054+0000 7fed9178f640 1 -- 172.21.15.89:0/664780319 shutdown_connections 2023-10-18T11:02:13.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:13.054+0000 7fed9178f640 1 --2- 172.21.15.89:0/664780319 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fed6c065680 0x7fed6c067b40 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:02:13.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:13.054+0000 7fed9178f640 1 --2- 172.21.15.89:0/664780319 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed8c15c550 0x7fed8c0b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:02:13.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:13.054+0000 7fed9178f640 1 --2- 172.21.15.89:0/664780319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed8c0b1cc0 0x7fed8c1f8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:02:13.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:13.054+0000 7fed9178f640 1 --2- 172.21.15.89:0/664780319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed8c0b1860 0x7fed8c0caf70 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:02:13.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:13.054+0000 7fed9178f640 1 -- 172.21.15.89:0/664780319 >> 172.21.15.89:0/664780319 conn(0x7fed8c069d00 msgr2=0x7fed8c15bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:02:13.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:13.054+0000 7fed9178f640 1 -- 172.21.15.89:0/664780319 shutdown_connections 2023-10-18T11:02:13.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:13.054+0000 7fed9178f640 1 -- 172.21.15.89:0/664780319 wait complete. 2023-10-18T11:02:13.074 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-18T11:02:13.231 INFO:teuthology.orchestra.run.smithi089.stdout:[{"id":5183,"entity":{"name":{"type":"client","num":5183},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":4272238049}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":889.52370570300002,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.1280625884680118,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":3,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5183 v1:192.168.0.1:0/4272238049","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi154","kernel_version":"6.6.0-rc5-g47672119149b","root":"/"}}] 2023-10-18T11:02:13.232 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "1", "hostname": "smithi154", "kernel_version": "6.6.0-rc5-g47672119149b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 4272238049, "type": "v1" }, "name": { "num": 5183, "type": "client" } }, "id": 5183, "inst": "client.5183 v1:192.168.0.1:0/4272238049", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 3, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.1280625884680118 }, "state": "open", "uptime": 889.523705703 } ] 2023-10-18T11:02:13.333 DEBUG:tasks.ceph_test_case:No log hits yet, waiting... 2023-10-18T11:02:22.214 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:02:22.218 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:02:48.350 ERROR:tasks.ceph_test_case:Log output: 2023-10-18T11:02:48.351 DEBUG:tasks.cephfs.test_client_recovery:Expected log message found: 'Health check cleared: MDS_CLIENTS_LAGGY', retrying 2023-10-18T11:02:52.336 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:02:52.341 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:02:53.352 DEBUG:teuthology.orchestra.run.smithi089:> sudo daemon-helper kill ceph --cluster ceph -w 2023-10-18T11:02:53.357 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T11:02:53.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.505+0000 7fb23c1fa640 1 Processor -- start 2023-10-18T11:02:53.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.505+0000 7fb23c1fa640 1 -- start start 2023-10-18T11:02:53.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.509+0000 7fb23c1fa640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb2340d0760 0x7fb2340d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:02:53.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.509+0000 7fb23c1fa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb2340cd160 0x7fb2340cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:02:53.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.509+0000 7fb23c1fa640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2340d1100 con 0x7fb23415c940 2023-10-18T11:02:53.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.509+0000 7fb23c1fa640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb23415da50 con 0x7fb2340d0760 2023-10-18T11:02:53.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.509+0000 7fb23c1fa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2340d3110 con 0x7fb2340cd160 2023-10-18T11:02:53.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.509+0000 7fb239795640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb2340d0760 0x7fb2340d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:02:53.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.509+0000 7fb239795640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb2340d0760 0x7fb2340d0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49406/0 (socket says 172.21.15.89:49406) 2023-10-18T11:02:53.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.509+0000 7fb239795640 1 -- 172.21.15.89:0/4089671922 learned_addr learned my addr 172.21.15.89:0/4089671922 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:02:53.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.509+0000 7fb23a797640 1 --2- 172.21.15.89:0/4089671922 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb2340cd160 0x7fb2340cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:02:53.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.509+0000 7fb239795640 1 -- 172.21.15.89:0/4089671922 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb2340cd160 msgr2=0x7fb2340cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:02:53.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.509+0000 7fb239795640 1 --2- 172.21.15.89:0/4089671922 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb2340cd160 0x7fb2340cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:02:53.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.509+0000 7fb239795640 1 -- 172.21.15.89:0/4089671922 >> v1:172.21.15.154:6789/0 conn(0x7fb23415c940 legacy=0x7fb2340d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:02:53.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.509+0000 7fb239795640 1 -- 172.21.15.89:0/4089671922 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb23415ce40 con 0x7fb2340d0760 2023-10-18T11:02:53.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.509+0000 7fb239795640 1 --2- 172.21.15.89:0/4089671922 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2340d0760 0x7fb2340d0bc0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb224009870 tx=0x7fb22401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3826206a3d9c9b2e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:02:53.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.513+0000 7fb238f94640 1 -- 172.21.15.89:0/4089671922 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2240aa670 con 0x7fb2340d0760 2023-10-18T11:02:53.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.513+0000 7fb238f94640 1 -- 172.21.15.89:0/4089671922 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2240aa7d0 con 0x7fb2340d0760 2023-10-18T11:02:53.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.513+0000 7fb238f94640 1 -- 172.21.15.89:0/4089671922 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2240aab60 con 0x7fb2340d0760 2023-10-18T11:02:53.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.513+0000 7fb23c1fa640 1 -- 172.21.15.89:0/4089671922 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2340d0760 msgr2=0x7fb2340d0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:02:53.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.513+0000 7fb23c1fa640 1 --2- 172.21.15.89:0/4089671922 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2340d0760 0x7fb2340d0bc0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb224009870 tx=0x7fb22401c510 comp rx=0 tx=0).stop 2023-10-18T11:02:53.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.513+0000 7fb23c1fa640 1 -- 172.21.15.89:0/4089671922 shutdown_connections 2023-10-18T11:02:53.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.513+0000 7fb23c1fa640 1 --2- 172.21.15.89:0/4089671922 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb2340cd160 0x7fb2340cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:02:53.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.513+0000 7fb23c1fa640 1 --2- 172.21.15.89:0/4089671922 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2340d0760 0x7fb2340d0bc0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:02:53.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.513+0000 7fb23c1fa640 1 -- 172.21.15.89:0/4089671922 >> 172.21.15.89:0/4089671922 conn(0x7fb234089fd0 msgr2=0x7fb23409b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:02:53.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.513+0000 7fb23c1fa640 1 -- 172.21.15.89:0/4089671922 shutdown_connections 2023-10-18T11:02:53.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.513+0000 7fb23c1fa640 1 -- 172.21.15.89:0/4089671922 wait complete. 2023-10-18T11:02:53.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.513+0000 7fb23c1fa640 1 Processor -- start 2023-10-18T11:02:53.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.513+0000 7fb23c1fa640 1 -- start start 2023-10-18T11:02:53.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.513+0000 7fb23c1fa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb2340cd160 0x7fb2340c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:02:53.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.513+0000 7fb23c1fa640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2340d0760 0x7fb2340c9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:02:53.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.513+0000 7fb23c1fa640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2340c9c60 0x7fb2340c7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:02:53.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.513+0000 7fb23c1fa640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2340c7ff0 con 0x7fb2340c9c60 2023-10-18T11:02:53.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.513+0000 7fb23c1fa640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2340c82d0 con 0x7fb2340d0760 2023-10-18T11:02:53.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.513+0000 7fb23c1fa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2340b5c80 con 0x7fb2340cd160 2023-10-18T11:02:53.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.517+0000 7fb23a797640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2340c9c60 0x7fb2340c7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:02:53.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.517+0000 7fb23a797640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2340c9c60 0x7fb2340c7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49410/0 (socket says 172.21.15.89:49410) 2023-10-18T11:02:53.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.517+0000 7fb23a797640 1 -- 172.21.15.89:0/3597008171 learned_addr learned my addr 172.21.15.89:0/3597008171 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:02:53.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.517+0000 7fb239795640 1 --2- 172.21.15.89:0/3597008171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2340d0760 0x7fb2340c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:02:53.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.517+0000 7fb239f96640 1 --2- 172.21.15.89:0/3597008171 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb2340cd160 0x7fb2340c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:02:53.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.517+0000 7fb23a797640 1 -- 172.21.15.89:0/3597008171 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb2340cd160 msgr2=0x7fb2340c8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:02:53.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.517+0000 7fb23a797640 1 --2- 172.21.15.89:0/3597008171 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb2340cd160 0x7fb2340c8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:02:53.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.517+0000 7fb23a797640 1 -- 172.21.15.89:0/3597008171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2340d0760 msgr2=0x7fb2340c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:02:53.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.517+0000 7fb23a797640 1 --2- 172.21.15.89:0/3597008171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2340d0760 0x7fb2340c9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:02:53.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.517+0000 7fb23a797640 1 -- 172.21.15.89:0/3597008171 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2240094d0 con 0x7fb2340c9c60 2023-10-18T11:02:53.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.517+0000 7fb23a797640 1 --2- 172.21.15.89:0/3597008171 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2340c9c60 0x7fb2340c7340 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb22c007e40 tx=0x7fb22c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:02:53.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.517+0000 7fb22a7fc640 1 -- 172.21.15.89:0/3597008171 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c014620 con 0x7fb2340c9c60 2023-10-18T11:02:53.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.517+0000 7fb22a7fc640 1 -- 172.21.15.89:0/3597008171 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22c00ee90 con 0x7fb2340c9c60 2023-10-18T11:02:53.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.517+0000 7fb22a7fc640 1 -- 172.21.15.89:0/3597008171 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c0149e0 con 0x7fb2340c9c60 2023-10-18T11:02:53.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.517+0000 7fb23c1fa640 1 -- 172.21.15.89:0/3597008171 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2340b5f60 con 0x7fb2340c9c60 2023-10-18T11:02:53.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.517+0000 7fb23c1fa640 1 -- 172.21.15.89:0/3597008171 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb234150e20 con 0x7fb2340c9c60 2023-10-18T11:02:53.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.517+0000 7fb23c1fa640 1 -- 172.21.15.89:0/3597008171 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb23404dbe0 con 0x7fb2340c9c60 2023-10-18T11:02:53.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.517+0000 7fb22a7fc640 1 -- 172.21.15.89:0/3597008171 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fb22c014b40 con 0x7fb2340c9c60 2023-10-18T11:02:53.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.521+0000 7fb22a7fc640 1 --2- 172.21.15.89:0/3597008171 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb218065680 0x7fb218067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:02:53.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.521+0000 7fb22a7fc640 1 -- 172.21.15.89:0/3597008171 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fb22c083920 con 0x7fb2340c9c60 2023-10-18T11:02:53.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.521+0000 7fb239f96640 1 --2- 172.21.15.89:0/3597008171 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb218065680 0x7fb218067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:02:53.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.521+0000 7fb239f96640 1 --2- 172.21.15.89:0/3597008171 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb218065680 0x7fb218067b40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb2340cc920 tx=0x7fb230009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:02:53.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.529+0000 7fb22a7fc640 1 -- 172.21.15.89:0/3597008171 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fb22c051240 con 0x7fb2340c9c60 2023-10-18T11:02:53.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.533+0000 7f2caf911640 1 Processor -- start 2023-10-18T11:02:53.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.537+0000 7f2caf911640 1 -- start start 2023-10-18T11:02:53.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.537+0000 7f2caf911640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ca8160f20 0x7f2ca80d0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:02:53.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.537+0000 7f2caf911640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2ca80b15e0 0x7f2ca80d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:02:53.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.537+0000 7f2caf911640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ca80cced0 con 0x7f2ca80b33b0 2023-10-18T11:02:53.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.537+0000 7f2caf911640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ca80b1d30 con 0x7f2ca80b15e0 2023-10-18T11:02:53.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.537+0000 7f2caf911640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca80b3bc0 con 0x7f2ca8160f20 2023-10-18T11:02:53.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.537+0000 7f2cadeae640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2ca80b15e0 0x7f2ca80d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:02:53.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.537+0000 7f2cadeae640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2ca80b15e0 0x7f2ca80d0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49412/0 (socket says 172.21.15.89:49412) 2023-10-18T11:02:53.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.537+0000 7f2cadeae640 1 -- 172.21.15.89:0/2176212748 learned_addr learned my addr 172.21.15.89:0/2176212748 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:02:53.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.537+0000 7f2cad6ad640 1 --2- 172.21.15.89:0/2176212748 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ca8160f20 0x7f2ca80d0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:02:53.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.537+0000 7f2cadeae640 1 -- 172.21.15.89:0/2176212748 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ca8160f20 msgr2=0x7f2ca80d0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:02:53.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.537+0000 7f2cadeae640 1 --2- 172.21.15.89:0/2176212748 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ca8160f20 0x7f2ca80d0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:02:53.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.537+0000 7f2cadeae640 1 -- 172.21.15.89:0/2176212748 >> v1:172.21.15.154:6789/0 conn(0x7f2ca80b33b0 legacy=0x7f2ca80b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:02:53.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.537+0000 7f2cadeae640 1 -- 172.21.15.89:0/2176212748 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca80afdd0 con 0x7f2ca80b15e0 2023-10-18T11:02:53.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.537+0000 7f2cad6ad640 1 --2- 172.21.15.89:0/2176212748 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ca8160f20 0x7f2ca80d0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:02:53.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.541+0000 7f2cadeae640 1 --2- 172.21.15.89:0/2176212748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ca80b15e0 0x7f2ca80d0c90 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0009e50 tx=0x7f2ca001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa16e568e23fbb9 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:02:53.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.541+0000 7f2c9ffff640 1 -- 172.21.15.89:0/2176212748 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca00a5690 con 0x7f2ca80b15e0 2023-10-18T11:02:53.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.541+0000 7f2c9ffff640 1 -- 172.21.15.89:0/2176212748 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca00a57f0 con 0x7f2ca80b15e0 2023-10-18T11:02:53.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.541+0000 7f2c9ffff640 1 -- 172.21.15.89:0/2176212748 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca00a5b80 con 0x7f2ca80b15e0 2023-10-18T11:02:53.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.541+0000 7f2caf911640 1 -- 172.21.15.89:0/2176212748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ca80b15e0 msgr2=0x7f2ca80d0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:02:53.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.541+0000 7f2caf911640 1 --2- 172.21.15.89:0/2176212748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ca80b15e0 0x7f2ca80d0c90 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0009e50 tx=0x7f2ca001c940 comp rx=0 tx=0).stop 2023-10-18T11:02:53.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.541+0000 7f2caf911640 1 -- 172.21.15.89:0/2176212748 shutdown_connections 2023-10-18T11:02:53.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.541+0000 7f2caf911640 1 --2- 172.21.15.89:0/2176212748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ca80b15e0 0x7f2ca80d0c90 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:02:53.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.541+0000 7f2caf911640 1 --2- 172.21.15.89:0/2176212748 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ca8160f20 0x7f2ca80d0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:02:53.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.541+0000 7f2caf911640 1 -- 172.21.15.89:0/2176212748 >> 172.21.15.89:0/2176212748 conn(0x7f2ca809b930 msgr2=0x7f2ca80d26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:02:53.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.541+0000 7f2caf911640 1 -- 172.21.15.89:0/2176212748 shutdown_connections 2023-10-18T11:02:53.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.541+0000 7f2caf911640 1 -- 172.21.15.89:0/2176212748 wait complete. 2023-10-18T11:02:53.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.541+0000 7f2caf911640 1 Processor -- start 2023-10-18T11:02:53.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.541+0000 7f2caf911640 1 -- start start 2023-10-18T11:02:53.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.545+0000 7f2caf911640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ca80b15e0 0x7f2ca80b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:02:53.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.545+0000 7f2caf911640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ca80b33b0 0x7f2ca815f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:02:53.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.545+0000 7f2caf911640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ca8160f20 0x7f2ca815e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:02:53.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.545+0000 7f2caf911640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca815fd50 con 0x7f2ca8160f20 2023-10-18T11:02:53.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.545+0000 7f2caf911640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca815f880 con 0x7f2ca80b33b0 2023-10-18T11:02:53.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.545+0000 7f2caf911640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca815e7c0 con 0x7f2ca80b15e0 2023-10-18T11:02:53.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.545+0000 7f2cadeae640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ca8160f20 0x7f2ca815e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:02:53.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.545+0000 7f2cadeae640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ca8160f20 0x7f2ca815e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49424/0 (socket says 172.21.15.89:49424) 2023-10-18T11:02:53.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.545+0000 7f2cadeae640 1 -- 172.21.15.89:0/3385511760 learned_addr learned my addr 172.21.15.89:0/3385511760 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:02:53.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.545+0000 7f2cad6ad640 1 --2- 172.21.15.89:0/3385511760 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ca80b15e0 0x7f2ca80b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:02:53.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.545+0000 7f2caceac640 1 --2- 172.21.15.89:0/3385511760 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ca80b33b0 0x7f2ca815f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:02:53.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.545+0000 7f2cadeae640 1 -- 172.21.15.89:0/3385511760 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ca80b15e0 msgr2=0x7f2ca80b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:02:53.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.545+0000 7f2cadeae640 1 --2- 172.21.15.89:0/3385511760 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ca80b15e0 0x7f2ca80b51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:02:53.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.545+0000 7f2cadeae640 1 -- 172.21.15.89:0/3385511760 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ca80b33b0 msgr2=0x7f2ca815f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:02:53.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.545+0000 7f2cadeae640 1 --2- 172.21.15.89:0/3385511760 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ca80b33b0 0x7f2ca815f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:02:53.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.545+0000 7f2cadeae640 1 -- 172.21.15.89:0/3385511760 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca0009b30 con 0x7f2ca8160f20 2023-10-18T11:02:53.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.545+0000 7f2cad6ad640 1 --2- 172.21.15.89:0/3385511760 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ca80b15e0 0x7f2ca80b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:02:53.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.545+0000 7f2cadeae640 1 --2- 172.21.15.89:0/3385511760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ca8160f20 0x7f2ca815e280 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f2ca00af040 tx=0x7f2ca00196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:02:53.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.545+0000 7f2c9dffb640 1 -- 172.21.15.89:0/3385511760 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca00b8670 con 0x7f2ca8160f20 2023-10-18T11:02:53.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.545+0000 7f2c9dffb640 1 -- 172.21.15.89:0/3385511760 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca00b87d0 con 0x7f2ca8160f20 2023-10-18T11:02:53.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.545+0000 7f2caf911640 1 -- 172.21.15.89:0/3385511760 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ca8157bf0 con 0x7f2ca8160f20 2023-10-18T11:02:53.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.545+0000 7f2c9dffb640 1 -- 172.21.15.89:0/3385511760 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca00b8b60 con 0x7f2ca8160f20 2023-10-18T11:02:53.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.545+0000 7f2caf911640 1 -- 172.21.15.89:0/3385511760 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ca81575a0 con 0x7f2ca8160f20 2023-10-18T11:02:53.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.545+0000 7f2caf911640 1 -- 172.21.15.89:0/3385511760 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f2ca8157400 con 0x7f2ca8160f20 2023-10-18T11:02:53.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.549+0000 7f2c9dffb640 1 -- 172.21.15.89:0/3385511760 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f2ca00b8cc0 con 0x7f2ca8160f20 2023-10-18T11:02:53.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.549+0000 7f2c9dffb640 1 --2- 172.21.15.89:0/3385511760 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2c8c065680 0x7f2c8c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:02:53.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.549+0000 7f2c9dffb640 1 -- 172.21.15.89:0/3385511760 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f2ca011efa0 con 0x7f2ca8160f20 2023-10-18T11:02:53.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.549+0000 7f2c9dffb640 1 -- 172.21.15.89:0/3385511760 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+449 (secure 0 0 0) 0x7f2ca011f330 con 0x7f2ca8160f20 2023-10-18T11:02:53.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.549+0000 7f2cad6ad640 1 --2- 172.21.15.89:0/3385511760 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2c8c065680 0x7f2c8c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:02:53.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.549+0000 7f2caf911640 1 -- 172.21.15.89:0/3385511760 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f2ca8157400 con 0x7f2ca8160f20 2023-10-18T11:02:53.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.549+0000 7f2cad6ad640 1 --2- 172.21.15.89:0/3385511760 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2c8c065680 0x7f2c8c067b40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400a9b0 tx=0x7f2ca4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:02:53.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.709+0000 7fb23c1fa640 1 -- 172.21.15.89:0/3597008171 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2340b6410 con 0x7fb2340c9c60 2023-10-18T11:02:53.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.709+0000 7fb22a7fc640 1 -- 172.21.15.89:0/3597008171 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4877 (secure 0 0 0) 0x7fb22c054860 con 0x7fb2340c9c60 2023-10-18T11:02:53.711 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T11:02:53.711 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/654729641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":654729641},{"type":"v1","addr":"172.21.15.154:6836","nonce":654729641}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/964566480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":964566480},{"type":"v1","addr":"172.21.15.154:6837","nonce":964566480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2261878538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2261878538},{"type":"v1","addr":"172.21.15.89:6833","nonce":2261878538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:47:15.199575+0000","modified":"2023-10-18T10:55:20.892655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5517":{"gid":5517,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":124,"addr":"172.21.15.89:6835/2092274731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2092274731},{"type":"v1","addr":"172.21.15.89:6835","nonce":2092274731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T11:02:53.712 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-18T11:02:53.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.713+0000 7fb23c1fa640 1 -- 172.21.15.89:0/3597008171 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb218065680 msgr2=0x7fb218067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:02:53.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.713+0000 7fb23c1fa640 1 --2- 172.21.15.89:0/3597008171 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb218065680 0x7fb218067b40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb2340cc920 tx=0x7fb230009040 comp rx=0 tx=0).stop 2023-10-18T11:02:53.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.713+0000 7fb23c1fa640 1 -- 172.21.15.89:0/3597008171 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2340c9c60 msgr2=0x7fb2340c7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:02:53.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.713+0000 7fb23c1fa640 1 --2- 172.21.15.89:0/3597008171 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2340c9c60 0x7fb2340c7340 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb22c007e40 tx=0x7fb22c00e4b0 comp rx=0 tx=0).stop 2023-10-18T11:02:53.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.713+0000 7fb23c1fa640 1 -- 172.21.15.89:0/3597008171 shutdown_connections 2023-10-18T11:02:53.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.713+0000 7fb23c1fa640 1 --2- 172.21.15.89:0/3597008171 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb218065680 0x7fb218067b40 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:02:53.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.713+0000 7fb23c1fa640 1 --2- 172.21.15.89:0/3597008171 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2340c9c60 0x7fb2340c7340 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:02:53.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.713+0000 7fb23c1fa640 1 --2- 172.21.15.89:0/3597008171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2340d0760 0x7fb2340c9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:02:53.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.713+0000 7fb23c1fa640 1 --2- 172.21.15.89:0/3597008171 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb2340cd160 0x7fb2340c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:02:53.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.713+0000 7fb23c1fa640 1 -- 172.21.15.89:0/3597008171 >> 172.21.15.89:0/3597008171 conn(0x7fb234089fd0 msgr2=0x7fb23406b180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:02:53.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.713+0000 7fb23c1fa640 1 -- 172.21.15.89:0/3597008171 shutdown_connections 2023-10-18T11:02:53.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:02:53.713+0000 7fb23c1fa640 1 -- 172.21.15.89:0/3597008171 wait complete. 2023-10-18T11:02:53.732 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-18T11:02:53.889 INFO:teuthology.orchestra.run.smithi089.stdout:[{"id":5183,"entity":{"name":{"type":"client","num":5183},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":4272238049}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":930.17841504299997,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.11658105626025057,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":3,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5183 v1:192.168.0.1:0/4272238049","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi154","kernel_version":"6.6.0-rc5-g47672119149b","root":"/"}}] 2023-10-18T11:02:53.890 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000017fbff" }, "entity_id": "1", "hostname": "smithi154", "kernel_version": "6.6.0-rc5-g47672119149b", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 4272238049, "type": "v1" }, "name": { "num": 5183, "type": "client" } }, "id": 5183, "inst": "client.5183 v1:192.168.0.1:0/4272238049", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 3, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.11658105626025057 }, "state": "open", "uptime": 930.178415043 } ] 2023-10-18T11:02:53.991 DEBUG:tasks.ceph_test_case:No log hits yet, waiting... 2023-10-18T11:03:22.392 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:03:22.397 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:03:29.008 ERROR:tasks.ceph_test_case:Log output: 2023-10-18T11:03:29.008 DEBUG:tasks.cephfs.test_client_recovery:Expected log message found: 'Health check cleared: MDS_CLIENTS_LAGGY', retrying 2023-10-18T11:03:29.009 INFO:tasks.cephfs.mount:Cleaning up killed connection on ubuntu@smithi089.front.sepia.ceph.com 2023-10-18T11:03:29.009 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T11:03:29.010 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:03:29.067 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T11:03:29.068 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:03:29.121 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T11:03:29.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.304+0000 7f64d6b2c640 1 Processor -- start 2023-10-18T11:03:29.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.308+0000 7f64d6b2c640 1 -- start start 2023-10-18T11:03:29.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.308+0000 7f64d6b2c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f64d0069900 0x7f64d0069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:29.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.308+0000 7f64d6b2c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64d00cc660 0x7f64d0070a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:29.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.308+0000 7f64d6b2c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f64d0070f60 0x7f64d006da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:29.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.308+0000 7f64d6b2c640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d00713c0 con 0x7f64d0069900 2023-10-18T11:03:29.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.308+0000 7f64d6b2c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d00c94b0 con 0x7f64d0070f60 2023-10-18T11:03:29.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.308+0000 7f64d6b2c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d015cc00 con 0x7f64d00cc660 2023-10-18T11:03:29.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.308+0000 7f64d48c8640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f64d0069900 0x7f64d0069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:29.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.308+0000 7f64d48c8640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f64d0069900 0x7f64d0069ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41998/0 (socket says 172.21.15.89:41998) 2023-10-18T11:03:29.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.308+0000 7f64d48c8640 1 -- 172.21.15.89:0/751018270 learned_addr learned my addr 172.21.15.89:0/751018270 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:29.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.308+0000 7f64d50c9640 1 --2- 172.21.15.89:0/751018270 >> v2:172.21.15.154:3300/0 conn(0x7f64d0070f60 0x7f64d006da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:29.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.308+0000 7f64c7fff640 1 --2- 172.21.15.89:0/751018270 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64d00cc660 0x7f64d0070a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:29.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.308+0000 7f64d48c8640 1 -- 172.21.15.89:0/751018270 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64d00cc660 msgr2=0x7f64d0070a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:29.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.308+0000 7f64d48c8640 1 --2- 172.21.15.89:0/751018270 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64d00cc660 0x7f64d0070a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:29.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.308+0000 7f64d48c8640 1 -- 172.21.15.89:0/751018270 >> v2:172.21.15.154:3300/0 conn(0x7f64d0070f60 msgr2=0x7f64d006da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:29.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.308+0000 7f64d48c8640 1 --2- 172.21.15.89:0/751018270 >> v2:172.21.15.154:3300/0 conn(0x7f64d0070f60 0x7f64d006da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:29.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.308+0000 7f64d48c8640 1 -- 172.21.15.89:0/751018270 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d00c9180 con 0x7f64d0069900 2023-10-18T11:03:29.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.308+0000 7f64c7fff640 1 --2- 172.21.15.89:0/751018270 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64d00cc660 0x7f64d0070a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:03:29.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.312+0000 7f64d48c8640 1 --2- 172.21.15.89:0/751018270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64d0069900 0x7f64d0069ce0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f64d00ce1a0 tx=0x7f64cc01c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd03e89fc32e3f63 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:29.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.312+0000 7f64c77fe640 1 -- 172.21.15.89:0/751018270 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64cc0ab600 con 0x7f64d0069900 2023-10-18T11:03:29.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.312+0000 7f64c77fe640 1 -- 172.21.15.89:0/751018270 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64cc0a7050 con 0x7f64d0069900 2023-10-18T11:03:29.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.312+0000 7f64c77fe640 1 -- 172.21.15.89:0/751018270 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64cc0ab980 con 0x7f64d0069900 2023-10-18T11:03:29.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.312+0000 7f64d6b2c640 1 -- 172.21.15.89:0/751018270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64d0069900 msgr2=0x7f64d0069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:29.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.312+0000 7f64d6b2c640 1 --2- 172.21.15.89:0/751018270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64d0069900 0x7f64d0069ce0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f64d00ce1a0 tx=0x7f64cc01c240 comp rx=0 tx=0).stop 2023-10-18T11:03:29.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.312+0000 7f64d6b2c640 1 -- 172.21.15.89:0/751018270 shutdown_connections 2023-10-18T11:03:29.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.312+0000 7f64d6b2c640 1 --2- 172.21.15.89:0/751018270 >> v2:172.21.15.154:3300/0 conn(0x7f64d0070f60 0x7f64d006da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:29.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.312+0000 7f64d6b2c640 1 --2- 172.21.15.89:0/751018270 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64d00cc660 0x7f64d0070a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:29.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.312+0000 7f64d6b2c640 1 --2- 172.21.15.89:0/751018270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64d0069900 0x7f64d0069ce0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:29.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.312+0000 7f64d6b2c640 1 -- 172.21.15.89:0/751018270 >> 172.21.15.89:0/751018270 conn(0x7f64d00cb5a0 msgr2=0x7f64d00cb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:29.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.312+0000 7f64d6b2c640 1 -- 172.21.15.89:0/751018270 shutdown_connections 2023-10-18T11:03:29.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.312+0000 7f64d6b2c640 1 -- 172.21.15.89:0/751018270 wait complete. 2023-10-18T11:03:29.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.312+0000 7f64d6b2c640 1 Processor -- start 2023-10-18T11:03:29.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.312+0000 7f64d6b2c640 1 -- start start 2023-10-18T11:03:29.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.316+0000 7f64d6b2c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64d0069900 0x7f64d0155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:29.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.316+0000 7f64d6b2c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64d0070f60 0x7f64d01f8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:29.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.316+0000 7f64d6b2c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64d00cc660 0x7f64d01f8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:29.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.316+0000 7f64d6b2c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d015b090 con 0x7f64d0070f60 2023-10-18T11:03:29.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.316+0000 7f64d6b2c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d015bec0 con 0x7f64d0069900 2023-10-18T11:03:29.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.316+0000 7f64d6b2c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d015b900 con 0x7f64d00cc660 2023-10-18T11:03:29.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.316+0000 7f64c7fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64d0070f60 0x7f64d01f8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:29.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.316+0000 7f64c7fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64d0070f60 0x7f64d01f8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42006/0 (socket says 172.21.15.89:42006) 2023-10-18T11:03:29.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.316+0000 7f64c7fff640 1 -- 172.21.15.89:0/2770533986 learned_addr learned my addr 172.21.15.89:0/2770533986 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:29.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.316+0000 7f64c7fff640 1 -- 172.21.15.89:0/2770533986 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64d00cc660 msgr2=0x7f64d01f8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:29.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.316+0000 7f64d50c9640 1 --2- 172.21.15.89:0/2770533986 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64d00cc660 0x7f64d01f8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:29.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.316+0000 7f64d48c8640 1 --2- 172.21.15.89:0/2770533986 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64d0069900 0x7f64d0155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:29.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.316+0000 7f64c7fff640 1 --2- 172.21.15.89:0/2770533986 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64d00cc660 0x7f64d01f8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:29.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.316+0000 7f64c7fff640 1 -- 172.21.15.89:0/2770533986 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64d0069900 msgr2=0x7f64d0155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:29.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.316+0000 7f64c7fff640 1 --2- 172.21.15.89:0/2770533986 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64d0069900 0x7f64d0155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:29.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.316+0000 7f64c7fff640 1 -- 172.21.15.89:0/2770533986 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64cc0094d0 con 0x7f64d0070f60 2023-10-18T11:03:29.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.316+0000 7f64d50c9640 1 --2- 172.21.15.89:0/2770533986 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64d00cc660 0x7f64d01f8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:03:29.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.316+0000 7f64d48c8640 1 --2- 172.21.15.89:0/2770533986 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64d0069900 0x7f64d0155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:03:29.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.316+0000 7f64c7fff640 1 --2- 172.21.15.89:0/2770533986 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64d0070f60 0x7f64d01f8530 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f64c000c7f0 tx=0x7f64c000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:29.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.316+0000 7f64c57fa640 1 -- 172.21.15.89:0/2770533986 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c0007820 con 0x7f64d0070f60 2023-10-18T11:03:29.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.316+0000 7f64c57fa640 1 -- 172.21.15.89:0/2770533986 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c0007980 con 0x7f64d0070f60 2023-10-18T11:03:29.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.316+0000 7f64d6b2c640 1 -- 172.21.15.89:0/2770533986 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d015bbe0 con 0x7f64d0070f60 2023-10-18T11:03:29.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.316+0000 7f64c57fa640 1 -- 172.21.15.89:0/2770533986 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c0007d30 con 0x7f64d0070f60 2023-10-18T11:03:29.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.316+0000 7f64d6b2c640 1 -- 172.21.15.89:0/2770533986 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d0153040 con 0x7f64d0070f60 2023-10-18T11:03:29.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.316+0000 7f64d6b2c640 1 -- 172.21.15.89:0/2770533986 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d015b240 con 0x7f64d0070f60 2023-10-18T11:03:29.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.320+0000 7f64c57fa640 1 -- 172.21.15.89:0/2770533986 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f64c00113f0 con 0x7f64d0070f60 2023-10-18T11:03:29.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.320+0000 7f64c57fa640 1 --2- 172.21.15.89:0/2770533986 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f64bc065680 0x7f64bc067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:29.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.320+0000 7f64c57fa640 1 -- 172.21.15.89:0/2770533986 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f64c0083290 con 0x7f64d0070f60 2023-10-18T11:03:29.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.320+0000 7f64d48c8640 1 --2- 172.21.15.89:0/2770533986 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f64bc065680 0x7f64bc067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:29.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.320+0000 7f64d48c8640 1 --2- 172.21.15.89:0/2770533986 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f64bc065680 0x7f64bc067b40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f64cc0ad000 tx=0x7f64cc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:29.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.324+0000 7f64c57fa640 1 -- 172.21.15.89:0/2770533986 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f64d015b240 con 0x7f64d0070f60 2023-10-18T11:03:29.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.476+0000 7f64d6b2c640 1 -- 172.21.15.89:0/2770533986 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64d004dbe0 con 0x7f64d0070f60 2023-10-18T11:03:29.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.476+0000 7f64c57fa640 1 -- 172.21.15.89:0/2770533986 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+18666 (secure 0 0 0) 0x7f64c0050c30 con 0x7f64d0070f60 2023-10-18T11:03:29.478 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T11:03:29.479 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":56,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:55:22.907577+0000","last_up_change":"2023-10-18T10:55:21.905065+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:47:13.188701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-18T10:47:14.198224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":50,"up_from":55,"up_thru":55,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":1133012599},{"type":"v1","addr":"172.21.15.154:6824","nonce":1133012599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1133012599},{"type":"v1","addr":"172.21.15.154:6827","nonce":1133012599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1133012599},{"type":"v1","addr":"172.21.15.154:6831","nonce":1133012599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1133012599},{"type":"v1","addr":"172.21.15.154:6829","nonce":1133012599}]},"public_addr":"172.21.15.154:6824/1133012599","cluster_addr":"172.21.15.154:6827/1133012599","heartbeat_back_addr":"172.21.15.154:6831/1133012599","heartbeat_front_addr":"172.21.15.154:6829/1133012599","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"2023-10-18T10:47:48.955023+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":51},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.9","mappings":[{"from":0,"to":5}]},{"pgid":"7.18","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:0/330049992":"2023-10-18T11:55:17.981058+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T11:03:29.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.476+0000 7f64d6b2c640 1 -- 172.21.15.89:0/2770533986 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f64bc065680 msgr2=0x7f64bc067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:29.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.476+0000 7f64d6b2c640 1 --2- 172.21.15.89:0/2770533986 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f64bc065680 0x7f64bc067b40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f64cc0ad000 tx=0x7f64cc008040 comp rx=0 tx=0).stop 2023-10-18T11:03:29.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.476+0000 7f64d6b2c640 1 -- 172.21.15.89:0/2770533986 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64d0070f60 msgr2=0x7f64d01f8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:29.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.476+0000 7f64d6b2c640 1 --2- 172.21.15.89:0/2770533986 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64d0070f60 0x7f64d01f8530 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f64c000c7f0 tx=0x7f64c000ccb0 comp rx=0 tx=0).stop 2023-10-18T11:03:29.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.476+0000 7f64d6b2c640 1 -- 172.21.15.89:0/2770533986 shutdown_connections 2023-10-18T11:03:29.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.476+0000 7f64d6b2c640 1 --2- 172.21.15.89:0/2770533986 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f64bc065680 0x7f64bc067b40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:29.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.476+0000 7f64d6b2c640 1 --2- 172.21.15.89:0/2770533986 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64d00cc660 0x7f64d01f8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:29.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.476+0000 7f64d6b2c640 1 --2- 172.21.15.89:0/2770533986 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64d0070f60 0x7f64d01f8530 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:29.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.476+0000 7f64d6b2c640 1 --2- 172.21.15.89:0/2770533986 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64d0069900 0x7f64d0155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:29.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.476+0000 7f64d6b2c640 1 -- 172.21.15.89:0/2770533986 >> 172.21.15.89:0/2770533986 conn(0x7f64d00cb5a0 msgr2=0x7f64d00cb980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:29.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.476+0000 7f64d6b2c640 1 -- 172.21.15.89:0/2770533986 shutdown_connections 2023-10-18T11:03:29.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.476+0000 7f64d6b2c640 1 -- 172.21.15.89:0/2770533986 wait complete. 2023-10-18T11:03:29.499 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:45:43.114184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-18T10:45:48.730968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:47:13.188701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:47:14.198224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T11:03:29.500 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T11:03:29.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.636+0000 7ff3a6dba640 1 Processor -- start 2023-10-18T11:03:29.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.636+0000 7ff3a6dba640 1 -- start start 2023-10-18T11:03:29.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.640+0000 7ff3a6dba640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff3a00cd4f0 0x7ff3a0143470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:29.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.640+0000 7ff3a6dba640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3a00cd8d0 con 0x7ff3a00ce070 2023-10-18T11:03:29.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.640+0000 7ff3a6dba640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3a015cbf0 con 0x7ff3a0145b70 2023-10-18T11:03:29.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.640+0000 7ff3a6dba640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3a0069900 con 0x7ff3a00cd4f0 2023-10-18T11:03:29.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.640+0000 7ff3a5357640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff3a00ce070 0x7ff3a006abb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:38676/0 (socket says 172.21.15.89:38676) 2023-10-18T11:03:29.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.640+0000 7ff3a5357640 1 -- 172.21.15.89:0/2613316948 learned_addr learned my addr 172.21.15.89:0/2613316948 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:29.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.640+0000 7ff3a4b56640 1 --2- 172.21.15.89:0/2613316948 >> v2:172.21.15.154:3300/0 conn(0x7ff3a00cd4f0 0x7ff3a0143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:29.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.640+0000 7ff3977fe640 1 -- 172.21.15.89:0/2613316948 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 945636616 0 0) 0x7ff3a00cd8d0 con 0x7ff3a00ce070 2023-10-18T11:03:29.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.640+0000 7ff3977fe640 1 -- 172.21.15.89:0/2613316948 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff388003530 con 0x7ff3a00ce070 2023-10-18T11:03:29.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.640+0000 7ff3a4b56640 1 -- 172.21.15.89:0/2613316948 >> v1:172.21.15.154:6789/0 conn(0x7ff3a0145b70 legacy=0x7ff3a0160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:29.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.640+0000 7ff3977fe640 1 -- 172.21.15.89:0/2613316948 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3531860704 0 0) 0x7ff388003530 con 0x7ff3a00ce070 2023-10-18T11:03:29.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.640+0000 7ff3a4b56640 1 -- 172.21.15.89:0/2613316948 >> v1:172.21.15.89:6789/0 conn(0x7ff3a00ce070 legacy=0x7ff3a006abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:29.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.640+0000 7ff3a4b56640 1 -- 172.21.15.89:0/2613316948 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a0069b10 con 0x7ff3a00cd4f0 2023-10-18T11:03:29.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.644+0000 7ff3a4b56640 1 --2- 172.21.15.89:0/2613316948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a00cd4f0 0x7ff3a0143470 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009740 tx=0x7ff39c01c1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6391e06c6e03d198 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:29.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.644+0000 7ff3977fe640 1 -- 172.21.15.89:0/2613316948 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff398001a20 con 0x7ff3a00cd4f0 2023-10-18T11:03:29.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.644+0000 7ff3977fe640 1 -- 172.21.15.89:0/2613316948 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39c0a7050 con 0x7ff3a00cd4f0 2023-10-18T11:03:29.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.644+0000 7ff3977fe640 1 -- 172.21.15.89:0/2613316948 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c0ab620 con 0x7ff3a00cd4f0 2023-10-18T11:03:29.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.644+0000 7ff3a6dba640 1 -- 172.21.15.89:0/2613316948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a00cd4f0 msgr2=0x7ff3a0143470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:29.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.644+0000 7ff3a6dba640 1 --2- 172.21.15.89:0/2613316948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a00cd4f0 0x7ff3a0143470 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009740 tx=0x7ff39c01c1d0 comp rx=0 tx=0).stop 2023-10-18T11:03:29.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.644+0000 7ff3a6dba640 1 -- 172.21.15.89:0/2613316948 shutdown_connections 2023-10-18T11:03:29.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.644+0000 7ff3a6dba640 1 --2- 172.21.15.89:0/2613316948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a00cd4f0 0x7ff3a0143470 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:29.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.644+0000 7ff3a6dba640 1 -- 172.21.15.89:0/2613316948 >> 172.21.15.89:0/2613316948 conn(0x7ff3a006dad0 msgr2=0x7ff3a006dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:29.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.644+0000 7ff3a6dba640 1 -- 172.21.15.89:0/2613316948 shutdown_connections 2023-10-18T11:03:29.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.644+0000 7ff3a6dba640 1 -- 172.21.15.89:0/2613316948 wait complete. 2023-10-18T11:03:29.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.644+0000 7ff3a6dba640 1 Processor -- start 2023-10-18T11:03:29.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.644+0000 7ff3a6dba640 1 -- start start 2023-10-18T11:03:29.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.648+0000 7ff3a6dba640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3a00cd4f0 0x7ff3a00c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:29.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.648+0000 7ff3a6dba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3a00ce070 0x7ff3a00d0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:29.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.648+0000 7ff3a6dba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a0145b70 0x7ff3a00d0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:29.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.648+0000 7ff3a6dba640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a00c78e0 con 0x7ff3a00cd4f0 2023-10-18T11:03:29.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.648+0000 7ff3a6dba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a00c7bc0 con 0x7ff3a0145b70 2023-10-18T11:03:29.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.648+0000 7ff3a6dba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a00c7ea0 con 0x7ff3a00ce070 2023-10-18T11:03:29.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.648+0000 7ff3a4b56640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3a00cd4f0 0x7ff3a00c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:29.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.648+0000 7ff3a4b56640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3a00cd4f0 0x7ff3a00c8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42020/0 (socket says 172.21.15.89:42020) 2023-10-18T11:03:29.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.648+0000 7ff3a4b56640 1 -- 172.21.15.89:0/482034774 learned_addr learned my addr 172.21.15.89:0/482034774 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:29.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.648+0000 7ff3a5357640 1 --2- 172.21.15.89:0/482034774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a0145b70 0x7ff3a00d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:29.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.648+0000 7ff397fff640 1 --2- 172.21.15.89:0/482034774 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3a00ce070 0x7ff3a00d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:29.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.648+0000 7ff3a4b56640 1 -- 172.21.15.89:0/482034774 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3a00ce070 msgr2=0x7ff3a00d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:29.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.648+0000 7ff3a4b56640 1 --2- 172.21.15.89:0/482034774 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3a00ce070 0x7ff3a00d0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:29.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.648+0000 7ff3a4b56640 1 -- 172.21.15.89:0/482034774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a0145b70 msgr2=0x7ff3a00d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:29.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.648+0000 7ff3a4b56640 1 --2- 172.21.15.89:0/482034774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a0145b70 0x7ff3a00d0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:29.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.648+0000 7ff3a4b56640 1 -- 172.21.15.89:0/482034774 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff39c009420 con 0x7ff3a00cd4f0 2023-10-18T11:03:29.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.648+0000 7ff3a4b56640 1 --2- 172.21.15.89:0/482034774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3a00cd4f0 0x7ff3a00c8ff0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009740 tx=0x7ff39c01aa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:29.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.648+0000 7ff3957fa640 1 -- 172.21.15.89:0/482034774 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c004020 con 0x7ff3a00cd4f0 2023-10-18T11:03:29.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.648+0000 7ff3957fa640 1 -- 172.21.15.89:0/482034774 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39c004180 con 0x7ff3a00cd4f0 2023-10-18T11:03:29.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.648+0000 7ff3a6dba640 1 -- 172.21.15.89:0/482034774 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3a00c8180 con 0x7ff3a00cd4f0 2023-10-18T11:03:29.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.648+0000 7ff3957fa640 1 -- 172.21.15.89:0/482034774 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c0b7660 con 0x7ff3a00cd4f0 2023-10-18T11:03:29.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.648+0000 7ff3a6dba640 1 -- 172.21.15.89:0/482034774 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a01f3360 con 0x7ff3a00cd4f0 2023-10-18T11:03:29.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.648+0000 7ff3a6dba640 1 -- 172.21.15.89:0/482034774 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3a00c86a0 con 0x7ff3a00cd4f0 2023-10-18T11:03:29.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.652+0000 7ff3957fa640 1 -- 172.21.15.89:0/482034774 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7ff39c0b77c0 con 0x7ff3a00cd4f0 2023-10-18T11:03:29.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.652+0000 7ff3957fa640 1 --2- 172.21.15.89:0/482034774 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff38c065680 0x7ff38c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:29.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.652+0000 7ff3957fa640 1 -- 172.21.15.89:0/482034774 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7ff39c0a4e40 con 0x7ff3a00cd4f0 2023-10-18T11:03:29.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.652+0000 7ff397fff640 1 --2- 172.21.15.89:0/482034774 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff38c065680 0x7ff38c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:29.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.652+0000 7ff397fff640 1 --2- 172.21.15.89:0/482034774 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff38c065680 0x7ff38c067b40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff390001ec0 tx=0x7ff39000b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:29.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.656+0000 7ff3957fa640 1 -- 172.21.15.89:0/482034774 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7ff39c0eb030 con 0x7ff3a00cd4f0 2023-10-18T11:03:29.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.804+0000 7ff3a6dba640 1 -- 172.21.15.89:0/482034774 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff3a004dbe0 con 0x7ff3a00cd4f0 2023-10-18T11:03:29.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.804+0000 7ff3957fa640 1 -- 172.21.15.89:0/482034774 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff39c0ee650 con 0x7ff3a00cd4f0 2023-10-18T11:03:29.808 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-18T11:03:29.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.808+0000 7ff3a6dba640 1 -- 172.21.15.89:0/482034774 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff38c065680 msgr2=0x7ff38c067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:29.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.808+0000 7ff3a6dba640 1 --2- 172.21.15.89:0/482034774 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff38c065680 0x7ff38c067b40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff390001ec0 tx=0x7ff39000b040 comp rx=0 tx=0).stop 2023-10-18T11:03:29.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.808+0000 7ff3a6dba640 1 -- 172.21.15.89:0/482034774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3a00cd4f0 msgr2=0x7ff3a00c8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:29.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.808+0000 7ff3a6dba640 1 --2- 172.21.15.89:0/482034774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3a00cd4f0 0x7ff3a00c8ff0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009740 tx=0x7ff39c01aa10 comp rx=0 tx=0).stop 2023-10-18T11:03:29.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.808+0000 7ff3a6dba640 1 -- 172.21.15.89:0/482034774 shutdown_connections 2023-10-18T11:03:29.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.808+0000 7ff3a6dba640 1 --2- 172.21.15.89:0/482034774 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff38c065680 0x7ff38c067b40 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:29.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.808+0000 7ff3a6dba640 1 --2- 172.21.15.89:0/482034774 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3a00ce070 0x7ff3a00d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:29.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.808+0000 7ff3a6dba640 1 --2- 172.21.15.89:0/482034774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a0145b70 0x7ff3a00d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:29.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.808+0000 7ff3a6dba640 1 --2- 172.21.15.89:0/482034774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3a00cd4f0 0x7ff3a00c8ff0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:29.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.808+0000 7ff3a6dba640 1 -- 172.21.15.89:0/482034774 >> 172.21.15.89:0/482034774 conn(0x7ff3a006dad0 msgr2=0x7ff3a0144fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:29.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.808+0000 7ff3a6dba640 1 -- 172.21.15.89:0/482034774 shutdown_connections 2023-10-18T11:03:29.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.808+0000 7ff3a6dba640 1 -- 172.21.15.89:0/482034774 wait complete. 2023-10-18T11:03:29.828 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-18T11:03:29.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.964+0000 7f2b40a9c640 1 Processor -- start 2023-10-18T11:03:29.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.964+0000 7f2b40a9c640 1 -- start start 2023-10-18T11:03:29.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.968+0000 7f2b40a9c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2b3c06ff00 0x7f2b3c15e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:29.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.968+0000 7f2b40a9c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b3c0c8960 0x7f2b3c0c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:29.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.968+0000 7f2b40a9c640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b3c15acb0 con 0x7f2b3c0ca340 2023-10-18T11:03:29.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.968+0000 7f2b40a9c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b3c15ef30 con 0x7f2b3c06ff00 2023-10-18T11:03:29.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.968+0000 7f2b40a9c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c15aa10 con 0x7f2b3c0c8960 2023-10-18T11:03:29.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.968+0000 7f2b31d9b640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2b3c0ca340 0x7f2b3c15b140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:38692/0 (socket says 172.21.15.89:38692) 2023-10-18T11:03:29.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.968+0000 7f2b31d9b640 1 -- 172.21.15.89:0/3645230193 learned_addr learned my addr 172.21.15.89:0/3645230193 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:29.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.968+0000 7f2b3ad9d640 1 --2- 172.21.15.89:0/3645230193 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b3c0c8960 0x7f2b3c0c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:29.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.968+0000 7f2b3a59c640 1 --2- 172.21.15.89:0/3645230193 >> v2:172.21.15.154:3300/0 conn(0x7f2b3c06ff00 0x7f2b3c15e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:29.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.968+0000 7f2b39d9b640 1 -- 172.21.15.89:0/3645230193 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 564098106 0 0) 0x7f2b3c15acb0 con 0x7f2b3c0ca340 2023-10-18T11:03:29.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.968+0000 7f2b39d9b640 1 -- 172.21.15.89:0/3645230193 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b28003530 con 0x7f2b3c0ca340 2023-10-18T11:03:29.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.968+0000 7f2b39d9b640 1 -- 172.21.15.89:0/3645230193 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3874116624 0 0) 0x7f2b28003530 con 0x7f2b3c0ca340 2023-10-18T11:03:29.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.968+0000 7f2b39d9b640 1 -- 172.21.15.89:0/3645230193 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b3c0c8960 msgr2=0x7f2b3c0c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:29.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.968+0000 7f2b39d9b640 1 --2- 172.21.15.89:0/3645230193 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b3c0c8960 0x7f2b3c0c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:29.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.968+0000 7f2b39d9b640 1 -- 172.21.15.89:0/3645230193 >> v2:172.21.15.154:3300/0 conn(0x7f2b3c06ff00 msgr2=0x7f2b3c15e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:29.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.968+0000 7f2b39d9b640 1 --2- 172.21.15.89:0/3645230193 >> v2:172.21.15.154:3300/0 conn(0x7f2b3c06ff00 0x7f2b3c15e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:29.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.968+0000 7f2b39d9b640 1 -- 172.21.15.89:0/3645230193 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b3c0702e0 con 0x7f2b3c0ca340 2023-10-18T11:03:29.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.968+0000 7f2b39d9b640 1 -- 172.21.15.89:0/3645230193 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7f2b24001680 con 0x7f2b3c0ca340 2023-10-18T11:03:29.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.968+0000 7f2b39d9b640 1 -- 172.21.15.89:0/3645230193 >> v1:172.21.15.89:6789/0 conn(0x7f2b3c0ca340 legacy=0x7f2b3c15b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:29.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.968+0000 7f2b39d9b640 1 --2- 172.21.15.89:0/3645230193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b28003980 0x7f2b28003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:29.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.968+0000 7f2b39d9b640 1 --2- 172.21.15.89:0/3645230193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b280047d0 0x7f2b28004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:29.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.968+0000 7f2b39d9b640 1 --2- 172.21.15.89:0/3645230193 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b28005190 0x7f2b2800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:29.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.968+0000 7f2b39d9b640 1 -- 172.21.15.89:0/3645230193 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c15ef30 con 0x7f2b28003980 2023-10-18T11:03:29.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.968+0000 7f2b39d9b640 1 -- 172.21.15.89:0/3645230193 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c15aa10 con 0x7f2b280047d0 2023-10-18T11:03:29.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.968+0000 7f2b39d9b640 1 -- 172.21.15.89:0/3645230193 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2800bf00 con 0x7f2b28005190 2023-10-18T11:03:29.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.968+0000 7f2b3ad9d640 1 --2- 172.21.15.89:0/3645230193 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b3c0c8960 0x7f2b3c0c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T11:03:29.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.968+0000 7f2b3a59c640 1 --2- 172.21.15.89:0/3645230193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b28003980 0x7f2b28003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:29.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.968+0000 7f2b31d9b640 1 --2- 172.21.15.89:0/3645230193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b280047d0 0x7f2b28004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:29.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.968+0000 7f2b3ad9d640 1 --2- 172.21.15.89:0/3645230193 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b28005190 0x7f2b2800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:29.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.968+0000 7f2b3a59c640 1 -- 172.21.15.89:0/3645230193 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b28005190 msgr2=0x7f2b2800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:29.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.968+0000 7f2b3a59c640 1 --2- 172.21.15.89:0/3645230193 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b28005190 0x7f2b2800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:29.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.968+0000 7f2b3a59c640 1 -- 172.21.15.89:0/3645230193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b280047d0 msgr2=0x7f2b28004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:29.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.968+0000 7f2b3a59c640 1 --2- 172.21.15.89:0/3645230193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b280047d0 0x7f2b28004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:29.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.968+0000 7f2b3a59c640 1 -- 172.21.15.89:0/3645230193 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b28003530 con 0x7f2b28003980 2023-10-18T11:03:29.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.972+0000 7f2b3a59c640 1 --2- 172.21.15.89:0/3645230193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b28003980 0x7f2b28003da0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009fc0 tx=0x7f2b2c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86e5b4fc8a31f7f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:29.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.972+0000 7f2b39d9b640 1 -- 172.21.15.89:0/3645230193 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c0a5c90 con 0x7f2b28003980 2023-10-18T11:03:29.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.972+0000 7f2b39d9b640 1 -- 172.21.15.89:0/3645230193 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2c00f040 con 0x7f2b28003980 2023-10-18T11:03:29.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.972+0000 7f2b39d9b640 1 -- 172.21.15.89:0/3645230193 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c0ad440 con 0x7f2b28003980 2023-10-18T11:03:29.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.972+0000 7f2b40a9c640 1 -- 172.21.15.89:0/3645230193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b28003980 msgr2=0x7f2b28003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:29.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.972+0000 7f2b40a9c640 1 --2- 172.21.15.89:0/3645230193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b28003980 0x7f2b28003da0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009fc0 tx=0x7f2b2c01cc40 comp rx=0 tx=0).stop 2023-10-18T11:03:29.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.972+0000 7f2b3ad9d640 1 -- 172.21.15.89:0/3645230193 reap_dead start 2023-10-18T11:03:29.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.972+0000 7f2b40a9c640 1 -- 172.21.15.89:0/3645230193 shutdown_connections 2023-10-18T11:03:29.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.972+0000 7f2b40a9c640 1 -- 172.21.15.89:0/3645230193 >> 172.21.15.89:0/3645230193 conn(0x7f2b3c069f00 msgr2=0x7f2b3c0cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:29.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.972+0000 7f2b40a9c640 1 -- 172.21.15.89:0/3645230193 shutdown_connections 2023-10-18T11:03:29.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.972+0000 7f2b40a9c640 1 -- 172.21.15.89:0/3645230193 wait complete. 2023-10-18T11:03:29.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.976+0000 7f2b40a9c640 1 Processor -- start 2023-10-18T11:03:29.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.976+0000 7f2b40a9c640 1 -- start start 2023-10-18T11:03:29.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.976+0000 7f2b40a9c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b28003980 0x7f2b3c14f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:29.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.976+0000 7f2b40a9c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b3c14f930 0x7f2b3c0cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:29.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.976+0000 7f2b40a9c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b3c159430 0x7f2b3c1598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:29.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.976+0000 7f2b40a9c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c159df0 con 0x7f2b28003980 2023-10-18T11:03:29.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.976+0000 7f2b40a9c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c0cd340 con 0x7f2b3c159430 2023-10-18T11:03:29.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.976+0000 7f2b40a9c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c0cc730 con 0x7f2b3c14f930 2023-10-18T11:03:29.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.976+0000 7f2b3a59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b28003980 0x7f2b3c14f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:29.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.976+0000 7f2b3a59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b28003980 0x7f2b3c14f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42036/0 (socket says 172.21.15.89:42036) 2023-10-18T11:03:29.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.976+0000 7f2b3a59c640 1 -- 172.21.15.89:0/4034277384 learned_addr learned my addr 172.21.15.89:0/4034277384 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:29.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.976+0000 7f2b3ad9d640 1 --2- 172.21.15.89:0/4034277384 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b3c159430 0x7f2b3c1598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:29.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.976+0000 7f2b31d9b640 1 --2- 172.21.15.89:0/4034277384 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b3c14f930 0x7f2b3c0cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:29.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.976+0000 7f2b3a59c640 1 -- 172.21.15.89:0/4034277384 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b3c14f930 msgr2=0x7f2b3c0cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:29.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.976+0000 7f2b3a59c640 1 --2- 172.21.15.89:0/4034277384 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b3c14f930 0x7f2b3c0cbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:29.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.976+0000 7f2b3a59c640 1 -- 172.21.15.89:0/4034277384 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b3c159430 msgr2=0x7f2b3c1598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:29.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.976+0000 7f2b3a59c640 1 --2- 172.21.15.89:0/4034277384 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b3c159430 0x7f2b3c1598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:29.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.976+0000 7f2b3a59c640 1 -- 172.21.15.89:0/4034277384 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b3c0cca10 con 0x7f2b28003980 2023-10-18T11:03:29.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.976+0000 7f2b3ad9d640 1 --2- 172.21.15.89:0/4034277384 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b3c159430 0x7f2b3c1598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:03:29.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.976+0000 7f2b3a59c640 1 --2- 172.21.15.89:0/4034277384 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b28003980 0x7f2b3c14f360 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c01c6b0 tx=0x7f2b2c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:29.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.976+0000 7f2b337fe640 1 -- 172.21.15.89:0/4034277384 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c003ba0 con 0x7f2b28003980 2023-10-18T11:03:29.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.976+0000 7f2b337fe640 1 -- 172.21.15.89:0/4034277384 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2c003d00 con 0x7f2b28003980 2023-10-18T11:03:29.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.976+0000 7f2b337fe640 1 -- 172.21.15.89:0/4034277384 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c0ad910 con 0x7f2b28003980 2023-10-18T11:03:29.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.976+0000 7f2b40a9c640 1 -- 172.21.15.89:0/4034277384 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b3c0cccf0 con 0x7f2b28003980 2023-10-18T11:03:29.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.976+0000 7f2b40a9c640 1 -- 172.21.15.89:0/4034277384 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b3c0cc420 con 0x7f2b28003980 2023-10-18T11:03:29.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.976+0000 7f2b40a9c640 1 -- 172.21.15.89:0/4034277384 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b3c04dbe0 con 0x7f2b28003980 2023-10-18T11:03:29.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.980+0000 7f2b337fe640 1 -- 172.21.15.89:0/4034277384 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f2b2c0b4040 con 0x7f2b28003980 2023-10-18T11:03:29.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.980+0000 7f2b337fe640 1 --2- 172.21.15.89:0/4034277384 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2b1c065680 0x7f2b1c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:29.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.980+0000 7f2b337fe640 1 -- 172.21.15.89:0/4034277384 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f2b2c11da40 con 0x7f2b28003980 2023-10-18T11:03:29.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.980+0000 7f2b31d9b640 1 --2- 172.21.15.89:0/4034277384 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2b1c065680 0x7f2b1c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:29.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.980+0000 7f2b31d9b640 1 --2- 172.21.15.89:0/4034277384 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2b1c065680 0x7f2b1c067b40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c15a200 tx=0x7f2b24007040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:29.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:29.984+0000 7f2b337fe640 1 -- 172.21.15.89:0/4034277384 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f2b2c0eb370 con 0x7f2b28003980 2023-10-18T11:03:30.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.136+0000 7f2b40a9c640 1 -- 172.21.15.89:0/4034277384 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2b3c159df0 con 0x7f2b28003980 2023-10-18T11:03:30.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.136+0000 7f2b337fe640 1 -- 172.21.15.89:0/4034277384 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v56) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2b2c0ee990 con 0x7f2b28003980 2023-10-18T11:03:30.138 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-10-18T11:03:30.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.136+0000 7f2b40a9c640 1 -- 172.21.15.89:0/4034277384 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2b1c065680 msgr2=0x7f2b1c067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:30.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.136+0000 7f2b40a9c640 1 --2- 172.21.15.89:0/4034277384 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2b1c065680 0x7f2b1c067b40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c15a200 tx=0x7f2b24007040 comp rx=0 tx=0).stop 2023-10-18T11:03:30.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.136+0000 7f2b40a9c640 1 -- 172.21.15.89:0/4034277384 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b28003980 msgr2=0x7f2b3c14f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:30.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.136+0000 7f2b40a9c640 1 --2- 172.21.15.89:0/4034277384 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b28003980 0x7f2b3c14f360 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c01c6b0 tx=0x7f2b2c009750 comp rx=0 tx=0).stop 2023-10-18T11:03:30.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.136+0000 7f2b40a9c640 1 -- 172.21.15.89:0/4034277384 shutdown_connections 2023-10-18T11:03:30.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.136+0000 7f2b40a9c640 1 --2- 172.21.15.89:0/4034277384 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2b1c065680 0x7f2b1c067b40 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:30.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.136+0000 7f2b40a9c640 1 --2- 172.21.15.89:0/4034277384 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b3c14f930 0x7f2b3c0cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:30.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.136+0000 7f2b40a9c640 1 --2- 172.21.15.89:0/4034277384 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b3c159430 0x7f2b3c1598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:30.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.136+0000 7f2b40a9c640 1 --2- 172.21.15.89:0/4034277384 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b28003980 0x7f2b3c14f360 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:30.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.136+0000 7f2b40a9c640 1 -- 172.21.15.89:0/4034277384 >> 172.21.15.89:0/4034277384 conn(0x7f2b3c069f00 msgr2=0x7f2b3c06b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:30.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.136+0000 7f2b40a9c640 1 -- 172.21.15.89:0/4034277384 shutdown_connections 2023-10-18T11:03:30.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.136+0000 7f2b40a9c640 1 -- 172.21.15.89:0/4034277384 wait complete. 2023-10-18T11:03:30.159 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T11:03:30.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.300+0000 7fa062a34640 1 Processor -- start 2023-10-18T11:03:30.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.300+0000 7fa062a34640 1 -- start start 2023-10-18T11:03:30.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.300+0000 7fa062a34640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa05c0c9070 0x7fa05c15e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:30.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.300+0000 7fa062a34640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa05c15acb0 con 0x7fa05c0ca340 2023-10-18T11:03:30.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.300+0000 7fa062a34640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa05c0702e0 con 0x7fa05c06ff00 2023-10-18T11:03:30.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.300+0000 7fa062a34640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa05c15ef30 con 0x7fa05c0c9070 2023-10-18T11:03:30.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.300+0000 7fa05b7fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa05c0ca340 0x7fa05c0c8960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:38702/0 (socket says 172.21.15.89:38702) 2023-10-18T11:03:30.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.300+0000 7fa05b7fe640 1 -- 172.21.15.89:0/2688896326 learned_addr learned my addr 172.21.15.89:0/2688896326 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:30.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.300+0000 7fa060fd1640 1 --2- 172.21.15.89:0/2688896326 >> v2:172.21.15.89:3300/0 conn(0x7fa05c0c9070 0x7fa05c15e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:30.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.300+0000 7fa05affd640 1 -- 172.21.15.89:0/2688896326 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2367009553 0 0) 0x7fa05c15acb0 con 0x7fa05c0ca340 2023-10-18T11:03:30.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.300+0000 7fa05affd640 1 -- 172.21.15.89:0/2688896326 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa044003530 con 0x7fa05c0ca340 2023-10-18T11:03:30.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.300+0000 7fa060fd1640 1 -- 172.21.15.89:0/2688896326 >> v1:172.21.15.154:6789/0 conn(0x7fa05c06ff00 legacy=0x7fa05c15b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:30.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.300+0000 7fa060fd1640 1 -- 172.21.15.89:0/2688896326 >> v1:172.21.15.89:6789/0 conn(0x7fa05c0ca340 legacy=0x7fa05c0c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:30.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.300+0000 7fa060fd1640 1 -- 172.21.15.89:0/2688896326 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa05c15b850 con 0x7fa05c0c9070 2023-10-18T11:03:30.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.304+0000 7fa060fd1640 1 --2- 172.21.15.89:0/2688896326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa05c0c9070 0x7fa05c15e000 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa054009d90 tx=0x7fa05401c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546d3083c764ef5b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:30.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.304+0000 7fa05affd640 1 -- 172.21.15.89:0/2688896326 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0540a5870 con 0x7fa05c0c9070 2023-10-18T11:03:30.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.304+0000 7fa05affd640 1 -- 172.21.15.89:0/2688896326 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0540a59d0 con 0x7fa05c0c9070 2023-10-18T11:03:30.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.304+0000 7fa05affd640 1 -- 172.21.15.89:0/2688896326 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0540a5d40 con 0x7fa05c0c9070 2023-10-18T11:03:30.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.304+0000 7fa062a34640 1 -- 172.21.15.89:0/2688896326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa05c0c9070 msgr2=0x7fa05c15e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:30.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.304+0000 7fa062a34640 1 --2- 172.21.15.89:0/2688896326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa05c0c9070 0x7fa05c15e000 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa054009d90 tx=0x7fa05401c8c0 comp rx=0 tx=0).stop 2023-10-18T11:03:30.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.304+0000 7fa062a34640 1 -- 172.21.15.89:0/2688896326 shutdown_connections 2023-10-18T11:03:30.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.304+0000 7fa062a34640 1 --2- 172.21.15.89:0/2688896326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa05c0c9070 0x7fa05c15e000 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:30.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.304+0000 7fa062a34640 1 -- 172.21.15.89:0/2688896326 >> 172.21.15.89:0/2688896326 conn(0x7fa05c069f00 msgr2=0x7fa05c0cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:30.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.304+0000 7fa062a34640 1 -- 172.21.15.89:0/2688896326 shutdown_connections 2023-10-18T11:03:30.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.304+0000 7fa062a34640 1 -- 172.21.15.89:0/2688896326 wait complete. 2023-10-18T11:03:30.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.308+0000 7fa062a34640 1 Processor -- start 2023-10-18T11:03:30.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.308+0000 7fa062a34640 1 -- start start 2023-10-18T11:03:30.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.308+0000 7fa062a34640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa05c06ff00 0x7fa05c15bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:30.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.308+0000 7fa062a34640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa05c0ca340 0x7fa05c0b3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:30.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.308+0000 7fa062a34640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa05c14ca70 0x7fa05c14df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:30.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.308+0000 7fa062a34640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05c0b3b90 con 0x7fa05c06ff00 2023-10-18T11:03:30.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.308+0000 7fa062a34640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05c15c220 con 0x7fa05c14ca70 2023-10-18T11:03:30.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.308+0000 7fa062a34640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa05c15c500 con 0x7fa05c0ca340 2023-10-18T11:03:30.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.308+0000 7fa05bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa05c06ff00 0x7fa05c15bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:30.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.308+0000 7fa05bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa05c06ff00 0x7fa05c15bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42062/0 (socket says 172.21.15.89:42062) 2023-10-18T11:03:30.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.308+0000 7fa05bfff640 1 -- 172.21.15.89:0/3675264776 learned_addr learned my addr 172.21.15.89:0/3675264776 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:30.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.308+0000 7fa05b7fe640 1 --2- 172.21.15.89:0/3675264776 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa05c0ca340 0x7fa05c0b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:30.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.308+0000 7fa060fd1640 1 --2- 172.21.15.89:0/3675264776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa05c14ca70 0x7fa05c14df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:30.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.308+0000 7fa05bfff640 1 -- 172.21.15.89:0/3675264776 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa05c0ca340 msgr2=0x7fa05c0b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:30.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.308+0000 7fa05bfff640 1 --2- 172.21.15.89:0/3675264776 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa05c0ca340 0x7fa05c0b3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:30.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.308+0000 7fa05bfff640 1 -- 172.21.15.89:0/3675264776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa05c14ca70 msgr2=0x7fa05c14df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:30.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.308+0000 7fa05bfff640 1 --2- 172.21.15.89:0/3675264776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa05c14ca70 0x7fa05c14df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:30.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.308+0000 7fa05bfff640 1 -- 172.21.15.89:0/3675264776 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa054009a70 con 0x7fa05c06ff00 2023-10-18T11:03:30.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.308+0000 7fa05bfff640 1 --2- 172.21.15.89:0/3675264776 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa05c06ff00 0x7fa05c15bb40 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00c420 tx=0x7fa04c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:30.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.308+0000 7fa058ff9640 1 -- 172.21.15.89:0/3675264776 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04c0052d0 con 0x7fa05c06ff00 2023-10-18T11:03:30.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.308+0000 7fa058ff9640 1 -- 172.21.15.89:0/3675264776 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa04c005430 con 0x7fa05c06ff00 2023-10-18T11:03:30.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.308+0000 7fa062a34640 1 -- 172.21.15.89:0/3675264776 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa05c15c7e0 con 0x7fa05c06ff00 2023-10-18T11:03:30.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.308+0000 7fa058ff9640 1 -- 172.21.15.89:0/3675264776 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04c007260 con 0x7fa05c06ff00 2023-10-18T11:03:30.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.308+0000 7fa062a34640 1 -- 172.21.15.89:0/3675264776 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa05c15cda0 con 0x7fa05c06ff00 2023-10-18T11:03:30.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.308+0000 7fa062a34640 1 -- 172.21.15.89:0/3675264776 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa05c04dbe0 con 0x7fa05c06ff00 2023-10-18T11:03:30.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.312+0000 7fa058ff9640 1 -- 172.21.15.89:0/3675264776 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fa04c0073c0 con 0x7fa05c06ff00 2023-10-18T11:03:30.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.312+0000 7fa058ff9640 1 --2- 172.21.15.89:0/3675264776 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa0480656b0 0x7fa048067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:30.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.312+0000 7fa058ff9640 1 -- 172.21.15.89:0/3675264776 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fa04c082410 con 0x7fa05c06ff00 2023-10-18T11:03:30.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.312+0000 7fa05b7fe640 1 --2- 172.21.15.89:0/3675264776 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa0480656b0 0x7fa048067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:30.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.312+0000 7fa05b7fe640 1 --2- 172.21.15.89:0/3675264776 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa0480656b0 0x7fa048067b70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa05c154f70 tx=0x7fa050009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:30.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.316+0000 7fa058ff9640 1 -- 172.21.15.89:0/3675264776 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fa04c04fc80 con 0x7fa05c06ff00 2023-10-18T11:03:30.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.464+0000 7fa062a34640 1 -- 172.21.15.89:0/3675264776 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa05c166180 con 0x7fa05c06ff00 2023-10-18T11:03:30.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.464+0000 7fa058ff9640 1 -- 172.21.15.89:0/3675264776 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa04c0532a0 con 0x7fa05c06ff00 2023-10-18T11:03:30.468 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-18T11:03:30.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.468+0000 7fa062a34640 1 -- 172.21.15.89:0/3675264776 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa0480656b0 msgr2=0x7fa048067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:30.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.468+0000 7fa062a34640 1 --2- 172.21.15.89:0/3675264776 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa0480656b0 0x7fa048067b70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa05c154f70 tx=0x7fa050009040 comp rx=0 tx=0).stop 2023-10-18T11:03:30.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.468+0000 7fa062a34640 1 -- 172.21.15.89:0/3675264776 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa05c06ff00 msgr2=0x7fa05c15bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:30.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.468+0000 7fa062a34640 1 --2- 172.21.15.89:0/3675264776 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa05c06ff00 0x7fa05c15bb40 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00c420 tx=0x7fa04c00c8e0 comp rx=0 tx=0).stop 2023-10-18T11:03:30.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.468+0000 7fa062a34640 1 -- 172.21.15.89:0/3675264776 shutdown_connections 2023-10-18T11:03:30.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.468+0000 7fa062a34640 1 --2- 172.21.15.89:0/3675264776 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fa0480656b0 0x7fa048067b70 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:30.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.468+0000 7fa062a34640 1 --2- 172.21.15.89:0/3675264776 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa05c0ca340 0x7fa05c0b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:30.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.468+0000 7fa062a34640 1 --2- 172.21.15.89:0/3675264776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa05c14ca70 0x7fa05c14df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:30.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.468+0000 7fa062a34640 1 --2- 172.21.15.89:0/3675264776 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa05c06ff00 0x7fa05c15bb40 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:30.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.468+0000 7fa062a34640 1 -- 172.21.15.89:0/3675264776 >> 172.21.15.89:0/3675264776 conn(0x7fa05c069f00 msgr2=0x7fa05c0ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:30.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.468+0000 7fa062a34640 1 -- 172.21.15.89:0/3675264776 shutdown_connections 2023-10-18T11:03:30.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.468+0000 7fa062a34640 1 -- 172.21.15.89:0/3675264776 wait complete. 2023-10-18T11:03:30.489 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T11:03:30.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.628+0000 7f02feffa640 1 Processor -- start 2023-10-18T11:03:30.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.628+0000 7f02feffa640 1 -- start start 2023-10-18T11:03:30.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.628+0000 7f02feffa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02f815c670 0x7f02f80713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:30.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.628+0000 7f02feffa640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f02f8160a10 0x7f02f8060320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:30.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.628+0000 7f02feffa640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02f81602a0 con 0x7f02f8160e30 2023-10-18T11:03:30.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.628+0000 7f02feffa640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02f8071900 con 0x7f02f8160a10 2023-10-18T11:03:30.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.628+0000 7f02feffa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02f80cd2b0 con 0x7f02f815c670 2023-10-18T11:03:30.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.628+0000 7f02f7fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f02f8160a10 0x7f02f8060320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:30.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.628+0000 7f02fcd96640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02f815c670 0x7f02f80713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:30.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.628+0000 7f02f7fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f02f8160a10 0x7f02f8060320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:42008/0 (socket says 172.21.15.89:42008) 2023-10-18T11:03:30.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.628+0000 7f02fd597640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f02f8160e30 0x7f02f80c9990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.89:46882/0 (socket says 172.21.15.89:46882) 2023-10-18T11:03:30.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.628+0000 7f02f7fff640 1 -- 172.21.15.89:0/3666064265 learned_addr learned my addr 172.21.15.89:0/3666064265 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:30.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.628+0000 7f02fcd96640 1 -- 172.21.15.89:0/3666064265 >> v2:172.21.15.154:3300/0 conn(0x7f02f8160a10 msgr2=0x7f02f8060320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:30.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.628+0000 7f02fcd96640 1 --2- 172.21.15.89:0/3666064265 >> v2:172.21.15.154:3300/0 conn(0x7f02f8160a10 0x7f02f8060320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:30.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.628+0000 7f02fcd96640 1 -- 172.21.15.89:0/3666064265 >> v1:172.21.15.154:6789/0 conn(0x7f02f8160e30 legacy=0x7f02f80c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:30.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.628+0000 7f02fcd96640 1 -- 172.21.15.89:0/3666064265 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f80cd590 con 0x7f02f815c670 2023-10-18T11:03:30.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.628+0000 7f02f7fff640 1 --2- 172.21.15.89:0/3666064265 >> v2:172.21.15.154:3300/0 conn(0x7f02f8160a10 0x7f02f8060320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T11:03:30.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.632+0000 7f02fcd96640 1 --2- 172.21.15.89:0/3666064265 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02f815c670 0x7f02f80713c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f02e00097c0 tx=0x7f02e001ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daf58d997273ebf5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:30.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.632+0000 7f02f77fe640 1 -- 172.21.15.89:0/3666064265 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e00a8970 con 0x7f02f815c670 2023-10-18T11:03:30.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.632+0000 7f02f77fe640 1 -- 172.21.15.89:0/3666064265 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e00a8ad0 con 0x7f02f815c670 2023-10-18T11:03:30.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.632+0000 7f02f77fe640 1 -- 172.21.15.89:0/3666064265 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e00a8e60 con 0x7f02f815c670 2023-10-18T11:03:30.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.632+0000 7f02feffa640 1 -- 172.21.15.89:0/3666064265 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02f815c670 msgr2=0x7f02f80713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:30.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.632+0000 7f02feffa640 1 --2- 172.21.15.89:0/3666064265 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02f815c670 0x7f02f80713c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f02e00097c0 tx=0x7f02e001ec70 comp rx=0 tx=0).stop 2023-10-18T11:03:30.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.632+0000 7f02feffa640 1 -- 172.21.15.89:0/3666064265 shutdown_connections 2023-10-18T11:03:30.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.632+0000 7f02feffa640 1 --2- 172.21.15.89:0/3666064265 >> v2:172.21.15.154:3300/0 conn(0x7f02f8160a10 0x7f02f8060320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:30.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.632+0000 7f02feffa640 1 --2- 172.21.15.89:0/3666064265 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02f815c670 0x7f02f80713c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:30.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.632+0000 7f02feffa640 1 -- 172.21.15.89:0/3666064265 >> 172.21.15.89:0/3666064265 conn(0x7f02f80cbdc0 msgr2=0x7f02f80cc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:30.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.632+0000 7f02feffa640 1 -- 172.21.15.89:0/3666064265 shutdown_connections 2023-10-18T11:03:30.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.632+0000 7f02feffa640 1 -- 172.21.15.89:0/3666064265 wait complete. 2023-10-18T11:03:30.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.636+0000 7f02feffa640 1 Processor -- start 2023-10-18T11:03:30.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.636+0000 7f02feffa640 1 -- start start 2023-10-18T11:03:30.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.636+0000 7f02feffa640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02f815c670 0x7f02f815e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:30.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.636+0000 7f02feffa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02f8160a10 0x7f02f81491a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:30.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.636+0000 7f02feffa640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02f8160e30 0x7f02f8145e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:30.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.636+0000 7f02feffa640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f815eaf0 con 0x7f02f815c670 2023-10-18T11:03:30.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.636+0000 7f02feffa640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f815edd0 con 0x7f02f8160e30 2023-10-18T11:03:30.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.636+0000 7f02feffa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02f815f0b0 con 0x7f02f8160a10 2023-10-18T11:03:30.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.636+0000 7f02fcd96640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02f815c670 0x7f02f815e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:30.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.636+0000 7f02fcd96640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02f815c670 0x7f02f815e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42076/0 (socket says 172.21.15.89:42076) 2023-10-18T11:03:30.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.636+0000 7f02fcd96640 1 -- 172.21.15.89:0/3884675185 learned_addr learned my addr 172.21.15.89:0/3884675185 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:30.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.636+0000 7f02f7fff640 1 --2- 172.21.15.89:0/3884675185 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02f8160a10 0x7f02f81491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:30.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.636+0000 7f02fd597640 1 --2- 172.21.15.89:0/3884675185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02f8160e30 0x7f02f8145e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:30.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.636+0000 7f02fcd96640 1 -- 172.21.15.89:0/3884675185 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02f8160a10 msgr2=0x7f02f81491a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:30.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.636+0000 7f02fcd96640 1 --2- 172.21.15.89:0/3884675185 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02f8160a10 0x7f02f81491a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:30.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.636+0000 7f02fcd96640 1 -- 172.21.15.89:0/3884675185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02f8160e30 msgr2=0x7f02f8145e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:30.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.636+0000 7f02fcd96640 1 --2- 172.21.15.89:0/3884675185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02f8160e30 0x7f02f8145e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:30.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.636+0000 7f02fcd96640 1 -- 172.21.15.89:0/3884675185 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e80094d0 con 0x7f02f815c670 2023-10-18T11:03:30.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.636+0000 7f02fcd96640 1 --2- 172.21.15.89:0/3884675185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02f815c670 0x7f02f815e450 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f02e00b3040 tx=0x7f02e001ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:30.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.636+0000 7f02f57fa640 1 -- 172.21.15.89:0/3884675185 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e00b9630 con 0x7f02f815c670 2023-10-18T11:03:30.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.636+0000 7f02f57fa640 1 -- 172.21.15.89:0/3884675185 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e00b97b0 con 0x7f02f815c670 2023-10-18T11:03:30.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.636+0000 7f02feffa640 1 -- 172.21.15.89:0/3884675185 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e0009420 con 0x7f02f815c670 2023-10-18T11:03:30.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.636+0000 7f02f57fa640 1 -- 172.21.15.89:0/3884675185 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e00b9b60 con 0x7f02f815c670 2023-10-18T11:03:30.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.640+0000 7f02feffa640 1 -- 172.21.15.89:0/3884675185 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02f815f670 con 0x7f02f815c670 2023-10-18T11:03:30.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.640+0000 7f02feffa640 1 -- 172.21.15.89:0/3884675185 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02f804dbe0 con 0x7f02f815c670 2023-10-18T11:03:30.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.640+0000 7f02f57fa640 1 -- 172.21.15.89:0/3884675185 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f02e00b7040 con 0x7f02f815c670 2023-10-18T11:03:30.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.640+0000 7f02f57fa640 1 --2- 172.21.15.89:0/3884675185 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f02d8065680 0x7f02d8067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:30.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.640+0000 7f02f57fa640 1 -- 172.21.15.89:0/3884675185 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f02e0120fa0 con 0x7f02f815c670 2023-10-18T11:03:30.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.640+0000 7f02f7fff640 1 --2- 172.21.15.89:0/3884675185 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f02d8065680 0x7f02d8067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:30.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.644+0000 7f02f7fff640 1 --2- 172.21.15.89:0/3884675185 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f02d8065680 0x7f02d8067b40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f02f80c6350 tx=0x7f02e8009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:30.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.644+0000 7f02f57fa640 1 -- 172.21.15.89:0/3884675185 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f02e00ee840 con 0x7f02f815c670 2023-10-18T11:03:30.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.792+0000 7f02feffa640 1 -- 172.21.15.89:0/3884675185 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f02f80ceec0 con 0x7f02f815c670 2023-10-18T11:03:30.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.792+0000 7f02f57fa640 1 -- 172.21.15.89:0/3884675185 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) v1 ==== 100+0+11 (secure 0 0 0) 0x7f02e00f1e60 con 0x7f02f815c670 2023-10-18T11:03:30.795 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-18T11:03:30.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.792+0000 7f02feffa640 1 -- 172.21.15.89:0/3884675185 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f02d8065680 msgr2=0x7f02d8067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:30.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.792+0000 7f02feffa640 1 --2- 172.21.15.89:0/3884675185 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f02d8065680 0x7f02d8067b40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f02f80c6350 tx=0x7f02e8009240 comp rx=0 tx=0).stop 2023-10-18T11:03:30.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.792+0000 7f02feffa640 1 -- 172.21.15.89:0/3884675185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02f815c670 msgr2=0x7f02f815e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:30.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.792+0000 7f02feffa640 1 --2- 172.21.15.89:0/3884675185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02f815c670 0x7f02f815e450 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f02e00b3040 tx=0x7f02e001ba10 comp rx=0 tx=0).stop 2023-10-18T11:03:30.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.796+0000 7f02feffa640 1 -- 172.21.15.89:0/3884675185 shutdown_connections 2023-10-18T11:03:30.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.796+0000 7f02feffa640 1 --2- 172.21.15.89:0/3884675185 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f02d8065680 0x7f02d8067b40 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:30.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.796+0000 7f02feffa640 1 --2- 172.21.15.89:0/3884675185 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f02f8160a10 0x7f02f81491a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:30.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.796+0000 7f02feffa640 1 --2- 172.21.15.89:0/3884675185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02f8160e30 0x7f02f8145e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:30.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.796+0000 7f02feffa640 1 --2- 172.21.15.89:0/3884675185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02f815c670 0x7f02f815e450 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:30.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.796+0000 7f02feffa640 1 -- 172.21.15.89:0/3884675185 >> 172.21.15.89:0/3884675185 conn(0x7f02f80cbdc0 msgr2=0x7f02f80b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:30.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.796+0000 7f02feffa640 1 -- 172.21.15.89:0/3884675185 shutdown_connections 2023-10-18T11:03:30.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.796+0000 7f02feffa640 1 -- 172.21.15.89:0/3884675185 wait complete. 2023-10-18T11:03:30.816 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T11:03:30.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.952+0000 7fda6540b640 1 Processor -- start 2023-10-18T11:03:30.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.952+0000 7fda6540b640 1 -- start start 2023-10-18T11:03:30.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.956+0000 7fda6540b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fda6015c940 0x7fda600d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:30.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.956+0000 7fda6540b640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda6015da50 con 0x7fda600d0e70 2023-10-18T11:03:30.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.956+0000 7fda6540b640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda6015ce40 con 0x7fda600d2f40 2023-10-18T11:03:30.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.956+0000 7fda6540b640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda6015d120 con 0x7fda6015c940 2023-10-18T11:03:30.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.956+0000 7fda5f7fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fda600d0e70 0x7fda600cd160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:38718/0 (socket says 172.21.15.89:38718) 2023-10-18T11:03:30.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.956+0000 7fda5f7fe640 1 -- 172.21.15.89:0/4115552966 learned_addr learned my addr 172.21.15.89:0/4115552966 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:30.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.956+0000 7fda5effd640 1 --2- 172.21.15.89:0/4115552966 >> v2:172.21.15.154:3300/0 conn(0x7fda6015c940 0x7fda600d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:30.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.956+0000 7fda5e7fc640 1 -- 172.21.15.89:0/4115552966 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1766855752 0 0) 0x7fda6015da50 con 0x7fda600d0e70 2023-10-18T11:03:30.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.956+0000 7fda5e7fc640 1 -- 172.21.15.89:0/4115552966 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda4c003530 con 0x7fda600d0e70 2023-10-18T11:03:30.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.956+0000 7fda5e7fc640 1 -- 172.21.15.89:0/4115552966 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 147856344 0 0) 0x7fda6015ce40 con 0x7fda600d2f40 2023-10-18T11:03:30.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.956+0000 7fda5e7fc640 1 -- 172.21.15.89:0/4115552966 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda6015da50 con 0x7fda600d2f40 2023-10-18T11:03:30.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.956+0000 7fda5e7fc640 1 -- 172.21.15.89:0/4115552966 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4071798789 0 0) 0x7fda4c003530 con 0x7fda600d0e70 2023-10-18T11:03:30.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.956+0000 7fda5e7fc640 1 -- 172.21.15.89:0/4115552966 >> v2:172.21.15.154:3300/0 conn(0x7fda6015c940 msgr2=0x7fda600d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:30.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.956+0000 7fda5e7fc640 1 --2- 172.21.15.89:0/4115552966 >> v2:172.21.15.154:3300/0 conn(0x7fda6015c940 0x7fda600d2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:30.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.956+0000 7fda5e7fc640 1 -- 172.21.15.89:0/4115552966 >> v1:172.21.15.154:6789/0 conn(0x7fda600d2f40 legacy=0x7fda600d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:30.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.956+0000 7fda5e7fc640 1 -- 172.21.15.89:0/4115552966 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6015d400 con 0x7fda600d0e70 2023-10-18T11:03:30.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.956+0000 7fda5e7fc640 1 -- 172.21.15.89:0/4115552966 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7fda58001680 con 0x7fda600d0e70 2023-10-18T11:03:30.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.956+0000 7fda5e7fc640 1 -- 172.21.15.89:0/4115552966 >> v1:172.21.15.89:6789/0 conn(0x7fda600d0e70 legacy=0x7fda600cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:30.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.956+0000 7fda5e7fc640 1 --2- 172.21.15.89:0/4115552966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda4c003b10 0x7fda4c003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:30.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.956+0000 7fda5e7fc640 1 --2- 172.21.15.89:0/4115552966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda4c0047d0 0x7fda4c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:30.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.956+0000 7fda5e7fc640 1 --2- 172.21.15.89:0/4115552966 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda4c005190 0x7fda4c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:30.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.956+0000 7fda5e7fc640 1 -- 172.21.15.89:0/4115552966 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6015d120 con 0x7fda4c0047d0 2023-10-18T11:03:30.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.956+0000 7fda5e7fc640 1 -- 172.21.15.89:0/4115552966 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c00c010 con 0x7fda4c003b10 2023-10-18T11:03:30.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.956+0000 7fda5e7fc640 1 -- 172.21.15.89:0/4115552966 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c00c1d0 con 0x7fda4c005190 2023-10-18T11:03:30.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.956+0000 7fda567fc640 1 --2- 172.21.15.89:0/4115552966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda4c0047d0 0x7fda4c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:30.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.956+0000 7fda5effd640 1 --2- 172.21.15.89:0/4115552966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda4c003b10 0x7fda4c003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:30.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.956+0000 7fda5f7fe640 1 --2- 172.21.15.89:0/4115552966 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda4c005190 0x7fda4c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:30.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.956+0000 7fda567fc640 1 -- 172.21.15.89:0/4115552966 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda4c005190 msgr2=0x7fda4c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:30.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.956+0000 7fda567fc640 1 --2- 172.21.15.89:0/4115552966 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda4c005190 0x7fda4c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:30.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.956+0000 7fda567fc640 1 -- 172.21.15.89:0/4115552966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda4c003b10 msgr2=0x7fda4c003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:30.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.956+0000 7fda567fc640 1 --2- 172.21.15.89:0/4115552966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda4c003b10 0x7fda4c003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:30.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.956+0000 7fda567fc640 1 -- 172.21.15.89:0/4115552966 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4c003530 con 0x7fda4c0047d0 2023-10-18T11:03:30.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.956+0000 7fda5f7fe640 1 -- 172.21.15.89:0/4115552966 reap_dead start 2023-10-18T11:03:30.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.960+0000 7fda567fc640 1 --2- 172.21.15.89:0/4115552966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda4c0047d0 0x7fda4c004c50 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fda48001fd0 tx=0x7fda4801eff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33734a50de9724f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:30.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.960+0000 7fda5e7fc640 1 -- 172.21.15.89:0/4115552966 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda480a8c10 con 0x7fda4c0047d0 2023-10-18T11:03:30.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.960+0000 7fda5e7fc640 1 -- 172.21.15.89:0/4115552966 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda480a8d90 con 0x7fda4c0047d0 2023-10-18T11:03:30.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.960+0000 7fda5e7fc640 1 -- 172.21.15.89:0/4115552966 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda48003bc0 con 0x7fda4c0047d0 2023-10-18T11:03:30.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.960+0000 7fda6540b640 1 -- 172.21.15.89:0/4115552966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda4c0047d0 msgr2=0x7fda4c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:30.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.960+0000 7fda6540b640 1 --2- 172.21.15.89:0/4115552966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda4c0047d0 0x7fda4c004c50 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fda48001fd0 tx=0x7fda4801eff0 comp rx=0 tx=0).stop 2023-10-18T11:03:30.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.960+0000 7fda6540b640 1 -- 172.21.15.89:0/4115552966 shutdown_connections 2023-10-18T11:03:30.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.960+0000 7fda6540b640 1 --2- 172.21.15.89:0/4115552966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda4c0047d0 0x7fda4c004c50 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:30.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.960+0000 7fda6540b640 1 -- 172.21.15.89:0/4115552966 >> 172.21.15.89:0/4115552966 conn(0x7fda60089fd0 msgr2=0x7fda6009b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:30.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.960+0000 7fda6540b640 1 -- 172.21.15.89:0/4115552966 shutdown_connections 2023-10-18T11:03:30.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.960+0000 7fda6540b640 1 -- 172.21.15.89:0/4115552966 wait complete. 2023-10-18T11:03:30.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.964+0000 7fda6540b640 1 Processor -- start 2023-10-18T11:03:30.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.964+0000 7fda6540b640 1 -- start start 2023-10-18T11:03:30.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.964+0000 7fda6540b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda4c0047d0 0x7fda600b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:30.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.964+0000 7fda6540b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda60159ed0 0x7fda600cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:30.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.964+0000 7fda6540b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda600c9720 0x7fda600c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:30.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.964+0000 7fda6540b640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda600ca0f0 con 0x7fda60159ed0 2023-10-18T11:03:30.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.964+0000 7fda6540b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6015a330 con 0x7fda4c0047d0 2023-10-18T11:03:30.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.964+0000 7fda6540b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda600cbf80 con 0x7fda600c9720 2023-10-18T11:03:30.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.964+0000 7fda567fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda60159ed0 0x7fda600cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:30.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.964+0000 7fda567fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda60159ed0 0x7fda600cb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42096/0 (socket says 172.21.15.89:42096) 2023-10-18T11:03:30.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.964+0000 7fda567fc640 1 -- 172.21.15.89:0/3952040042 learned_addr learned my addr 172.21.15.89:0/3952040042 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:30.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.964+0000 7fda5effd640 1 --2- 172.21.15.89:0/3952040042 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda4c0047d0 0x7fda600b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:30.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.964+0000 7fda5f7fe640 1 --2- 172.21.15.89:0/3952040042 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda600c9720 0x7fda600c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:30.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.964+0000 7fda567fc640 1 -- 172.21.15.89:0/3952040042 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda600c9720 msgr2=0x7fda600c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:30.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.964+0000 7fda567fc640 1 --2- 172.21.15.89:0/3952040042 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda600c9720 0x7fda600c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:30.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.964+0000 7fda567fc640 1 -- 172.21.15.89:0/3952040042 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda4c0047d0 msgr2=0x7fda600b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:30.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.964+0000 7fda567fc640 1 --2- 172.21.15.89:0/3952040042 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda4c0047d0 0x7fda600b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:30.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.964+0000 7fda567fc640 1 -- 172.21.15.89:0/3952040042 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda600cc260 con 0x7fda60159ed0 2023-10-18T11:03:30.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.964+0000 7fda5effd640 1 --2- 172.21.15.89:0/3952040042 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda4c0047d0 0x7fda600b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:03:30.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.964+0000 7fda567fc640 1 --2- 172.21.15.89:0/3952040042 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda60159ed0 0x7fda600cb6a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fda48005040 tx=0x7fda4c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:30.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.964+0000 7fda57fff640 1 -- 172.21.15.89:0/3952040042 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda480a8a10 con 0x7fda60159ed0 2023-10-18T11:03:30.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.964+0000 7fda57fff640 1 -- 172.21.15.89:0/3952040042 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda480a8b70 con 0x7fda60159ed0 2023-10-18T11:03:30.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.964+0000 7fda57fff640 1 -- 172.21.15.89:0/3952040042 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda48006da0 con 0x7fda60159ed0 2023-10-18T11:03:30.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.964+0000 7fda6540b640 1 -- 172.21.15.89:0/3952040042 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda600cc540 con 0x7fda60159ed0 2023-10-18T11:03:30.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.964+0000 7fda6540b640 1 -- 172.21.15.89:0/3952040042 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda600cbbe0 con 0x7fda60159ed0 2023-10-18T11:03:30.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.968+0000 7fda6540b640 1 -- 172.21.15.89:0/3952040042 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda600af370 con 0x7fda60159ed0 2023-10-18T11:03:30.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.968+0000 7fda57fff640 1 -- 172.21.15.89:0/3952040042 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fda48005070 con 0x7fda60159ed0 2023-10-18T11:03:30.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.968+0000 7fda57fff640 1 --2- 172.21.15.89:0/3952040042 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fda40065680 0x7fda40067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:30.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.968+0000 7fda57fff640 1 -- 172.21.15.89:0/3952040042 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fda4811fb00 con 0x7fda60159ed0 2023-10-18T11:03:30.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.968+0000 7fda5effd640 1 --2- 172.21.15.89:0/3952040042 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fda40065680 0x7fda40067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:30.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.968+0000 7fda5effd640 1 --2- 172.21.15.89:0/3952040042 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fda40065680 0x7fda40067b40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fda600684d0 tx=0x7fda50008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:30.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:30.972+0000 7fda57fff640 1 -- 172.21.15.89:0/3952040042 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fda600af370 con 0x7fda60159ed0 2023-10-18T11:03:31.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.132+0000 7fda6540b640 1 -- 172.21.15.89:0/3952040042 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fda600af370 con 0x7fda60159ed0 2023-10-18T11:03:31.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.132+0000 7fda57fff640 1 -- 172.21.15.89:0/3952040042 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 1 entries v56) v1 ==== 80+0+57 (secure 0 0 0) 0x7fda600af370 con 0x7fda60159ed0 2023-10-18T11:03:31.134 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/330049992 2023-10-18T11:55:17.981058+0000 2023-10-18T11:03:31.135 INFO:teuthology.orchestra.run.smithi089.stderr:listed 1 entries 2023-10-18T11:03:31.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.132+0000 7fda6540b640 1 -- 172.21.15.89:0/3952040042 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fda40065680 msgr2=0x7fda40067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:31.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.132+0000 7fda6540b640 1 --2- 172.21.15.89:0/3952040042 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fda40065680 0x7fda40067b40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fda600684d0 tx=0x7fda50008040 comp rx=0 tx=0).stop 2023-10-18T11:03:31.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.132+0000 7fda6540b640 1 -- 172.21.15.89:0/3952040042 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda60159ed0 msgr2=0x7fda600cb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:31.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.132+0000 7fda6540b640 1 --2- 172.21.15.89:0/3952040042 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda60159ed0 0x7fda600cb6a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fda48005040 tx=0x7fda4c005800 comp rx=0 tx=0).stop 2023-10-18T11:03:31.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.132+0000 7fda6540b640 1 -- 172.21.15.89:0/3952040042 shutdown_connections 2023-10-18T11:03:31.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.132+0000 7fda6540b640 1 --2- 172.21.15.89:0/3952040042 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fda40065680 0x7fda40067b40 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:31.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.132+0000 7fda6540b640 1 --2- 172.21.15.89:0/3952040042 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda600c9720 0x7fda600c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:31.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.132+0000 7fda6540b640 1 --2- 172.21.15.89:0/3952040042 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda60159ed0 0x7fda600cb6a0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:31.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.132+0000 7fda6540b640 1 --2- 172.21.15.89:0/3952040042 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda4c0047d0 0x7fda600b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:31.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.132+0000 7fda6540b640 1 -- 172.21.15.89:0/3952040042 >> 172.21.15.89:0/3952040042 conn(0x7fda60089fd0 msgr2=0x7fda600cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:31.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.136+0000 7fda6540b640 1 -- 172.21.15.89:0/3952040042 shutdown_connections 2023-10-18T11:03:31.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.136+0000 7fda6540b640 1 -- 172.21.15.89:0/3952040042 wait complete. 2023-10-18T11:03:31.156 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-18T11:03:31.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:03:31.157 DEBUG:teuthology.orchestra.run.smithi089:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-10-18T11:03:31.181 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi089.front.sepia.ceph.com 2023-10-18T11:03:31.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:03:31.182 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-18T11:03:31.233 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T11:03:31.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.412+0000 7ff5359fa640 1 Processor -- start 2023-10-18T11:03:31.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.412+0000 7ff5359fa640 1 -- start start 2023-10-18T11:03:31.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.412+0000 7ff5359fa640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff53015c2f0 0x7ff5300b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:31.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.412+0000 7ff5359fa640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff53015d1b0 con 0x7ff5300d0700 2023-10-18T11:03:31.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.412+0000 7ff5359fa640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff53015d490 con 0x7ff5300d0270 2023-10-18T11:03:31.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.412+0000 7ff5359fa640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5300d0b50 con 0x7ff53015c2f0 2023-10-18T11:03:31.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.412+0000 7ff52effd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff53015c2f0 0x7ff5300b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:31.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.412+0000 7ff52f7fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff5300d0700 0x7ff530160780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:38730/0 (socket says 172.21.15.89:38730) 2023-10-18T11:03:31.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.412+0000 7ff52f7fe640 1 -- 172.21.15.89:0/3217719265 learned_addr learned my addr 172.21.15.89:0/3217719265 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:31.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.412+0000 7ff52effd640 1 -- 172.21.15.89:0/3217719265 >> v1:172.21.15.154:6789/0 conn(0x7ff5300d0270 legacy=0x7ff53015c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:31.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.412+0000 7ff52dffb640 1 -- 172.21.15.89:0/3217719265 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2051468284 0 0) 0x7ff53015d1b0 con 0x7ff5300d0700 2023-10-18T11:03:31.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.412+0000 7ff52effd640 1 -- 172.21.15.89:0/3217719265 >> v1:172.21.15.89:6789/0 conn(0x7ff5300d0700 legacy=0x7ff530160780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:31.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.412+0000 7ff52effd640 1 -- 172.21.15.89:0/3217719265 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff53015d770 con 0x7ff53015c2f0 2023-10-18T11:03:31.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.416+0000 7ff52effd640 1 --2- 172.21.15.89:0/3217719265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff53015c2f0 0x7ff5300b18e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ff5200097f0 tx=0x7ff52001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=597d815197757873 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:31.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.416+0000 7ff52dffb640 1 -- 172.21.15.89:0/3217719265 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5200ab610 con 0x7ff53015c2f0 2023-10-18T11:03:31.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.416+0000 7ff52dffb640 1 -- 172.21.15.89:0/3217719265 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5200ab770 con 0x7ff53015c2f0 2023-10-18T11:03:31.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.416+0000 7ff52dffb640 1 -- 172.21.15.89:0/3217719265 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5200abb00 con 0x7ff53015c2f0 2023-10-18T11:03:31.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.416+0000 7ff5359fa640 1 -- 172.21.15.89:0/3217719265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff53015c2f0 msgr2=0x7ff5300b18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:31.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.416+0000 7ff5359fa640 1 --2- 172.21.15.89:0/3217719265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff53015c2f0 0x7ff5300b18e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ff5200097f0 tx=0x7ff52001c410 comp rx=0 tx=0).stop 2023-10-18T11:03:31.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.416+0000 7ff5359fa640 1 -- 172.21.15.89:0/3217719265 shutdown_connections 2023-10-18T11:03:31.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.416+0000 7ff5359fa640 1 --2- 172.21.15.89:0/3217719265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff53015c2f0 0x7ff5300b18e0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:31.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.416+0000 7ff5359fa640 1 -- 172.21.15.89:0/3217719265 >> 172.21.15.89:0/3217719265 conn(0x7ff530069ad0 msgr2=0x7ff530069eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:31.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.416+0000 7ff5359fa640 1 -- 172.21.15.89:0/3217719265 shutdown_connections 2023-10-18T11:03:31.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.416+0000 7ff5359fa640 1 -- 172.21.15.89:0/3217719265 wait complete. 2023-10-18T11:03:31.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.416+0000 7ff5359fa640 1 Processor -- start 2023-10-18T11:03:31.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.420+0000 7ff5359fa640 1 -- start start 2023-10-18T11:03:31.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.420+0000 7ff5359fa640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5300d0270 0x7ff5300c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:31.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.420+0000 7ff5359fa640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5300d0700 0x7ff53015aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:31.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.420+0000 7ff5359fa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff53015c2f0 0x7ff53015eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:31.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.420+0000 7ff5359fa640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff530149890 con 0x7ff5300d0270 2023-10-18T11:03:31.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.420+0000 7ff5359fa640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff530149e50 con 0x7ff5300d0700 2023-10-18T11:03:31.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.420+0000 7ff5359fa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff53015b330 con 0x7ff53015c2f0 2023-10-18T11:03:31.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.420+0000 7ff52effd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5300d0270 0x7ff5300c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:31.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.420+0000 7ff52effd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5300d0270 0x7ff5300c8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42120/0 (socket says 172.21.15.89:42120) 2023-10-18T11:03:31.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.420+0000 7ff52effd640 1 -- 172.21.15.89:0/2974299205 learned_addr learned my addr 172.21.15.89:0/2974299205 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:31.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.420+0000 7ff52e7fc640 1 --2- 172.21.15.89:0/2974299205 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5300d0700 0x7ff53015aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:31.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.420+0000 7ff52f7fe640 1 --2- 172.21.15.89:0/2974299205 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff53015c2f0 0x7ff53015eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:31.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.420+0000 7ff52effd640 1 -- 172.21.15.89:0/2974299205 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff53015c2f0 msgr2=0x7ff53015eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:31.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.420+0000 7ff52effd640 1 --2- 172.21.15.89:0/2974299205 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff53015c2f0 0x7ff53015eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:31.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.420+0000 7ff52effd640 1 -- 172.21.15.89:0/2974299205 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5300d0700 msgr2=0x7ff53015aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:31.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.420+0000 7ff52effd640 1 --2- 172.21.15.89:0/2974299205 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5300d0700 0x7ff53015aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:31.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.420+0000 7ff52effd640 1 -- 172.21.15.89:0/2974299205 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff520009420 con 0x7ff5300d0270 2023-10-18T11:03:31.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.420+0000 7ff52effd640 1 --2- 172.21.15.89:0/2974299205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5300d0270 0x7ff5300c8890 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff52001a040 tx=0x7ff52001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:31.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.420+0000 7ff517fff640 1 -- 172.21.15.89:0/2974299205 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5200b5810 con 0x7ff5300d0270 2023-10-18T11:03:31.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.420+0000 7ff517fff640 1 -- 172.21.15.89:0/2974299205 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5200b5970 con 0x7ff5300d0270 2023-10-18T11:03:31.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.420+0000 7ff5359fa640 1 -- 172.21.15.89:0/2974299205 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff53015b610 con 0x7ff5300d0270 2023-10-18T11:03:31.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.420+0000 7ff517fff640 1 -- 172.21.15.89:0/2974299205 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5200b5d20 con 0x7ff5300d0270 2023-10-18T11:03:31.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.420+0000 7ff5359fa640 1 -- 172.21.15.89:0/2974299205 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5300c8dd0 con 0x7ff5300d0270 2023-10-18T11:03:31.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.420+0000 7ff5359fa640 1 -- 172.21.15.89:0/2974299205 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff530066ca0 con 0x7ff5300d0270 2023-10-18T11:03:31.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.424+0000 7ff517fff640 1 -- 172.21.15.89:0/2974299205 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7ff5200bf410 con 0x7ff5300d0270 2023-10-18T11:03:31.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.424+0000 7ff517fff640 1 --2- 172.21.15.89:0/2974299205 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff51c065680 0x7ff51c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:31.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.424+0000 7ff517fff640 1 -- 172.21.15.89:0/2974299205 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7ff52011df30 con 0x7ff5300d0270 2023-10-18T11:03:31.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.424+0000 7ff52e7fc640 1 --2- 172.21.15.89:0/2974299205 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff51c065680 0x7ff51c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:31.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.424+0000 7ff52e7fc640 1 --2- 172.21.15.89:0/2974299205 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff51c065680 0x7ff51c067b40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff53015fb20 tx=0x7ff524009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:31.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.428+0000 7ff517fff640 1 -- 172.21.15.89:0/2974299205 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7ff5200eb9f0 con 0x7ff5300d0270 2023-10-18T11:03:31.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.608+0000 7ff5359fa640 1 -- 172.21.15.89:0/2974299205 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff5300d20b0 con 0x7ff5300d0270 2023-10-18T11:03:31.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.608+0000 7ff517fff640 1 -- 172.21.15.89:0/2974299205 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff5200ef010 con 0x7ff5300d0270 2023-10-18T11:03:31.610 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T11:03:31.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.608+0000 7ff5359fa640 1 -- 172.21.15.89:0/2974299205 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff51c065680 msgr2=0x7ff51c067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:31.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.608+0000 7ff5359fa640 1 --2- 172.21.15.89:0/2974299205 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff51c065680 0x7ff51c067b40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff53015fb20 tx=0x7ff524009040 comp rx=0 tx=0).stop 2023-10-18T11:03:31.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.608+0000 7ff5359fa640 1 -- 172.21.15.89:0/2974299205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5300d0270 msgr2=0x7ff5300c8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:31.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.608+0000 7ff5359fa640 1 --2- 172.21.15.89:0/2974299205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5300d0270 0x7ff5300c8890 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff52001a040 tx=0x7ff52001ac50 comp rx=0 tx=0).stop 2023-10-18T11:03:31.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.608+0000 7ff5359fa640 1 -- 172.21.15.89:0/2974299205 shutdown_connections 2023-10-18T11:03:31.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.608+0000 7ff5359fa640 1 --2- 172.21.15.89:0/2974299205 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff51c065680 0x7ff51c067b40 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:31.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.608+0000 7ff5359fa640 1 --2- 172.21.15.89:0/2974299205 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff53015c2f0 0x7ff53015eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:31.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.608+0000 7ff5359fa640 1 --2- 172.21.15.89:0/2974299205 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5300d0700 0x7ff53015aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:31.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.608+0000 7ff5359fa640 1 --2- 172.21.15.89:0/2974299205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5300d0270 0x7ff5300c8890 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:31.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.608+0000 7ff5359fa640 1 -- 172.21.15.89:0/2974299205 >> 172.21.15.89:0/2974299205 conn(0x7ff530069ad0 msgr2=0x7ff530167a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:31.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.608+0000 7ff5359fa640 1 -- 172.21.15.89:0/2974299205 shutdown_connections 2023-10-18T11:03:31.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:31.608+0000 7ff5359fa640 1 -- 172.21.15.89:0/2974299205 wait complete. 2023-10-18T11:03:31.631 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-18T11:03:31.631 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T11:03:31.631 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi089.front.sepia.ceph.com 2023-10-18T11:03:31.632 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-18T11:03:31.632 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-18T11:03:31.632 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-18T11:03:31.633 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-18T11:03:31.633 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T11:03:31.634 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T11:03:31.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:03:31.634 DEBUG:teuthology.orchestra.run.smithi089:> ip addr 2023-10-18T11:03:31.644 INFO:teuthology.orchestra.run.smithi089.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T11:03:31.644 INFO:teuthology.orchestra.run.smithi089.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T11:03:31.645 INFO:teuthology.orchestra.run.smithi089.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T11:03:31.645 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T11:03:31.645 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 ::1/128 scope host 2023-10-18T11:03:31.645 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T11:03:31.645 INFO:teuthology.orchestra.run.smithi089.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-18T11:03:31.646 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c6 brd ff:ff:ff:ff:ff:ff 2023-10-18T11:03:31.646 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f0 2023-10-18T11:03:31.646 INFO:teuthology.orchestra.run.smithi089.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-18T11:03:31.646 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c7 brd ff:ff:ff:ff:ff:ff 2023-10-18T11:03:31.646 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f1 2023-10-18T11:03:31.646 INFO:teuthology.orchestra.run.smithi089.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-18T11:03:31.647 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:00 brd ff:ff:ff:ff:ff:ff 2023-10-18T11:03:31.647 INFO:teuthology.orchestra.run.smithi089.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T11:03:31.647 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:01 brd ff:ff:ff:ff:ff:ff 2023-10-18T11:03:31.647 INFO:teuthology.orchestra.run.smithi089.stdout: inet 172.21.15.89/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-18T11:03:31.647 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft 41337sec preferred_lft 41337sec 2023-10-18T11:03:31.648 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec4:7aff:fe8f:d501/64 scope link 2023-10-18T11:03:31.648 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T11:03:31.648 INFO:teuthology.orchestra.run.smithi089.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-18T11:03:31.648 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 02:42:3f:fd:35:8d brd ff:ff:ff:ff:ff:ff 2023-10-18T11:03:31.648 INFO:teuthology.orchestra.run.smithi089.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-18T11:03:31.649 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T11:03:31.649 INFO:teuthology.orchestra.run.smithi089.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T11:03:31.649 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-18T11:03:31.649 INFO:teuthology.orchestra.run.smithi089.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T11:03:31.649 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T11:03:31.650 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-18T11:03:31.650 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T11:03:31.650 INFO:teuthology.orchestra.run.smithi089.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state DOWN group default qlen 1000 2023-10-18T11:03:31.650 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T11:03:31.650 INFO:teuthology.orchestra.run.smithi089.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T11:03:31.650 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T11:03:31.651 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-10-18T11:03:31.651 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-18T11:03:31.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:03:31.652 DEBUG:teuthology.orchestra.run.smithi089:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T11:03:31.714 INFO:teuthology.orchestra.run.smithi089.stdout:1 2023-10-18T11:03:31.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:03:31.717 DEBUG:teuthology.orchestra.run.smithi089:> route 2023-10-18T11:03:31.770 INFO:teuthology.orchestra.run.smithi089.stdout:Kernel IP routing table 2023-10-18T11:03:31.771 INFO:teuthology.orchestra.run.smithi089.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T11:03:31.771 INFO:teuthology.orchestra.run.smithi089.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-18T11:03:31.771 INFO:teuthology.orchestra.run.smithi089.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T11:03:31.771 INFO:teuthology.orchestra.run.smithi089.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-18T11:03:31.772 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T11:03:31.772 INFO:teuthology.orchestra.run.smithi089.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-18T11:03:31.772 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-18T11:03:31.772 INFO:teuthology.orchestra.run.smithi089.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-18T11:03:31.772 INFO:teuthology.orchestra.run.smithi089.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T11:03:31.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:03:31.773 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-18T11:03:31.774 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-18T11:03:31.774 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T11:03:31.774 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T11:03:31.775 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T11:03:31.775 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-18T11:03:31.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:03:31.863 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-10-18T11:03:31.873 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-18T11:03:31.874 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-18T11:03:31.875 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T11:03:31.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:03:31.876 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set brx.0 up 2023-10-18T11:03:31.937 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T11:03:31.938 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-18T11:03:31.988 INFO:teuthology.orchestra.run.smithi089.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-18T11:03:31.989 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T11:03:31.989 DEBUG:teuthology.orchestra.run.smithi089:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-18T11:03:32.040 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-18T11:03:32.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:03:32.042 DEBUG:teuthology.orchestra.run.smithi089:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-18T11:03:32.134 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-18T11:03:32.134 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-18T11:03:32.134 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-18T11:03:32.150 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-18T11:03:32.151 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-18T11:03:32.151 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-18T11:03:32.151 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.154:3300,172.21.15.154:3301" 2023-10-18T11:03:32.151 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-18T11:03:32.160 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-18T11:03:32.161 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-18T11:03:32.161 INFO:teuthology.orchestra.run.smithi089.stdout:invalid new device string format 2023-10-18T11:03:32.161 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:3300,172.21.15.154:3300,172.21.15.154:3301" 2023-10-18T11:03:32.161 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-18T11:03:32.161 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.154:6789,172.21.15.154:6790" 2023-10-18T11:03:32.161 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.89:6789,172.21.15.154:6789,172.21.15.154:6790:/ 2023-10-18T11:03:32.162 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1352f62a-2330-4e96-afcd-b8e87cd82667" will pass to kernel 2023-10-18T11:03:32.165 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-18T11:03:32.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:03:32.166 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-18T11:03:32.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:03:32.187 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T11:03:32.239 INFO:teuthology.orchestra.run.smithi089.stdout:client5755 2023-10-18T11:03:32.239 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T11:03:32.240 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:03:32.288 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T11:03:32.288 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:03:32.340 DEBUG:tasks.cephfs.mount:Creating test file 2023-10-18 11:03:32.340107 0 2023-10-18T11:03:32.340 DEBUG:teuthology.orchestra.run.smithi089:> touch '/home/ubuntu/cephtest/mnt.0/2023-10-18 11:03:32.340107 0' 2023-10-18T11:03:32.396 DEBUG:tasks.cephfs.mount:Deleting test file 2023-10-18 11:03:32.340107 0 2023-10-18T11:03:32.396 DEBUG:teuthology.orchestra.run.smithi089:> rm -f '/home/ubuntu/cephtest/mnt.0/2023-10-18 11:03:32.340107 0' 2023-10-18T11:03:32.453 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T11:03:32.454 DEBUG:teuthology.orchestra.run.smithi089:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T11:03:32.574 DEBUG:teuthology.orchestra.run.smithi154:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T11:03:32.661 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T11:03:32.661 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T11:03:32.662 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:03:32.671 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T11:03:32.671 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:03:32.720 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T11:03:32.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.908+0000 7f3fbe52c640 1 Processor -- start 2023-10-18T11:03:32.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.908+0000 7f3fbe52c640 1 -- start start 2023-10-18T11:03:32.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.908+0000 7f3fbe52c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3fb80ad5b0 0x7f3fb8160a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:32.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.908+0000 7f3fbe52c640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fb81605c0 con 0x7f3fb80afd90 2023-10-18T11:03:32.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.908+0000 7f3fbe52c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fb806c000 con 0x7f3fb815c7f0 2023-10-18T11:03:32.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.908+0000 7f3fbe52c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fb806c2e0 con 0x7f3fb80ad5b0 2023-10-18T11:03:32.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.908+0000 7f3fb7fff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f3fb80afd90 0x7f3fb80b0170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39506/0 (socket says 172.21.15.89:39506) 2023-10-18T11:03:32.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.908+0000 7f3fb7fff640 1 -- 172.21.15.89:0/3920321686 learned_addr learned my addr 172.21.15.89:0/3920321686 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:32.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.908+0000 7f3fbcac9640 1 --2- 172.21.15.89:0/3920321686 >> v2:172.21.15.154:3300/0 conn(0x7f3fb80ad5b0 0x7f3fb8160a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:32.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.908+0000 7f3fb6ffd640 1 -- 172.21.15.89:0/3920321686 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1709597903 0 0) 0x7f3fb81605c0 con 0x7f3fb80afd90 2023-10-18T11:03:32.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.908+0000 7f3fb6ffd640 1 -- 172.21.15.89:0/3920321686 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fa0003530 con 0x7f3fb80afd90 2023-10-18T11:03:32.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.908+0000 7f3fbcac9640 1 -- 172.21.15.89:0/3920321686 >> v1:172.21.15.154:6789/0 conn(0x7f3fb815c7f0 legacy=0x7f3fb806b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:32.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.908+0000 7f3fb6ffd640 1 -- 172.21.15.89:0/3920321686 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1713140697 0 0) 0x7f3fa0003530 con 0x7f3fb80afd90 2023-10-18T11:03:32.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.908+0000 7f3fbcac9640 1 -- 172.21.15.89:0/3920321686 >> v1:172.21.15.89:6789/0 conn(0x7f3fb80afd90 legacy=0x7f3fb80b0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:32.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.908+0000 7f3fbcac9640 1 -- 172.21.15.89:0/3920321686 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb806c5c0 con 0x7f3fb80ad5b0 2023-10-18T11:03:32.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.912+0000 7f3fbcac9640 1 --2- 172.21.15.89:0/3920321686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fb80ad5b0 0x7f3fb8160a50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0009d90 tx=0x7f3fb001c690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5da8be83ec4604d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:32.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.912+0000 7f3fb6ffd640 1 -- 172.21.15.89:0/3920321686 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb81611a0 con 0x7f3fb80ad5b0 2023-10-18T11:03:32.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.912+0000 7f3fb6ffd640 1 -- 172.21.15.89:0/3920321686 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb00a6430 con 0x7f3fb80ad5b0 2023-10-18T11:03:32.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.912+0000 7f3fb6ffd640 1 -- 172.21.15.89:0/3920321686 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa8001680 con 0x7f3fb80ad5b0 2023-10-18T11:03:32.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.912+0000 7f3fbe52c640 1 -- 172.21.15.89:0/3920321686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fb80ad5b0 msgr2=0x7f3fb8160a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:32.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.912+0000 7f3fbe52c640 1 --2- 172.21.15.89:0/3920321686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fb80ad5b0 0x7f3fb8160a50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0009d90 tx=0x7f3fb001c690 comp rx=0 tx=0).stop 2023-10-18T11:03:32.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.912+0000 7f3fbe52c640 1 -- 172.21.15.89:0/3920321686 shutdown_connections 2023-10-18T11:03:32.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.912+0000 7f3fbe52c640 1 --2- 172.21.15.89:0/3920321686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fb80ad5b0 0x7f3fb8160a50 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:32.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.912+0000 7f3fbe52c640 1 -- 172.21.15.89:0/3920321686 >> 172.21.15.89:0/3920321686 conn(0x7f3fb8069f00 msgr2=0x7f3fb80cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:32.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.912+0000 7f3fbe52c640 1 -- 172.21.15.89:0/3920321686 shutdown_connections 2023-10-18T11:03:32.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.912+0000 7f3fbe52c640 1 -- 172.21.15.89:0/3920321686 wait complete. 2023-10-18T11:03:32.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.916+0000 7f3fbe52c640 1 Processor -- start 2023-10-18T11:03:32.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.916+0000 7f3fbe52c640 1 -- start start 2023-10-18T11:03:32.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.916+0000 7f3fbe52c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fb80ad5b0 0x7f3fb8156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:32.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.916+0000 7f3fbe52c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3fb80afd90 0x7f3fb8155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:32.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.916+0000 7f3fbe52c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fb815c7f0 0x7f3fb814c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:32.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.916+0000 7f3fbe52c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb80b3c30 con 0x7f3fb80ad5b0 2023-10-18T11:03:32.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.916+0000 7f3fbe52c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb80c7d60 con 0x7f3fb815c7f0 2023-10-18T11:03:32.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.916+0000 7f3fbe52c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb814ec70 con 0x7f3fb80afd90 2023-10-18T11:03:32.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.916+0000 7f3fb7fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fb80ad5b0 0x7f3fb8156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:32.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.916+0000 7f3fb7fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fb80ad5b0 0x7f3fb8156b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51178/0 (socket says 172.21.15.89:51178) 2023-10-18T11:03:32.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.916+0000 7f3fb7fff640 1 -- 172.21.15.89:0/2843539045 learned_addr learned my addr 172.21.15.89:0/2843539045 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:32.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.916+0000 7f3fbcac9640 1 --2- 172.21.15.89:0/2843539045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fb815c7f0 0x7f3fb814c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:32.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.916+0000 7f3fb77fe640 1 --2- 172.21.15.89:0/2843539045 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3fb80afd90 0x7f3fb8155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:32.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.916+0000 7f3fb7fff640 1 -- 172.21.15.89:0/2843539045 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3fb80afd90 msgr2=0x7f3fb8155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:32.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.916+0000 7f3fb7fff640 1 --2- 172.21.15.89:0/2843539045 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3fb80afd90 0x7f3fb8155a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:32.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.916+0000 7f3fb7fff640 1 -- 172.21.15.89:0/2843539045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fb815c7f0 msgr2=0x7f3fb814c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:32.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.916+0000 7f3fb7fff640 1 --2- 172.21.15.89:0/2843539045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fb815c7f0 0x7f3fb814c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:32.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.916+0000 7f3fb7fff640 1 -- 172.21.15.89:0/2843539045 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb0009a70 con 0x7f3fb80ad5b0 2023-10-18T11:03:32.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.916+0000 7f3fb7fff640 1 --2- 172.21.15.89:0/2843539045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fb80ad5b0 0x7f3fb8156b50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8007fd0 tx=0x7f3fa800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:32.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.916+0000 7f3fb4ff9640 1 -- 172.21.15.89:0/2843539045 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa80072d0 con 0x7f3fb80ad5b0 2023-10-18T11:03:32.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.916+0000 7f3fb4ff9640 1 -- 172.21.15.89:0/2843539045 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa8007430 con 0x7f3fb80ad5b0 2023-10-18T11:03:32.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.916+0000 7f3fbe52c640 1 -- 172.21.15.89:0/2843539045 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fb815b910 con 0x7f3fb80ad5b0 2023-10-18T11:03:32.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.916+0000 7f3fb4ff9640 1 -- 172.21.15.89:0/2843539045 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa80047f0 con 0x7f3fb80ad5b0 2023-10-18T11:03:32.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.916+0000 7f3fbe52c640 1 -- 172.21.15.89:0/2843539045 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fb815be90 con 0x7f3fb80ad5b0 2023-10-18T11:03:32.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.920+0000 7f3fbe52c640 1 -- 172.21.15.89:0/2843539045 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fb815c080 con 0x7f3fb80ad5b0 2023-10-18T11:03:32.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.920+0000 7f3fb4ff9640 1 -- 172.21.15.89:0/2843539045 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f3fa8004950 con 0x7f3fb80ad5b0 2023-10-18T11:03:32.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.920+0000 7f3fb4ff9640 1 --2- 172.21.15.89:0/2843539045 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3fa4065680 0x7f3fa4067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:32.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.920+0000 7f3fb4ff9640 1 -- 172.21.15.89:0/2843539045 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f3fa80825f0 con 0x7f3fb80ad5b0 2023-10-18T11:03:32.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.920+0000 7f3fb77fe640 1 --2- 172.21.15.89:0/2843539045 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3fa4065680 0x7f3fa4067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:32.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.920+0000 7f3fb77fe640 1 --2- 172.21.15.89:0/2843539045 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3fa4065680 0x7f3fa4067b40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3fb80c5300 tx=0x7f3fac009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:32.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:32.924+0000 7f3fb4ff9640 1 -- 172.21.15.89:0/2843539045 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f3fa804ff10 con 0x7f3fb80ad5b0 2023-10-18T11:03:33.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.076+0000 7f3fbe52c640 1 -- 172.21.15.89:0/2843539045 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3fb804dbe0 con 0x7f3fb80ad5b0 2023-10-18T11:03:33.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.076+0000 7f3fb4ff9640 1 -- 172.21.15.89:0/2843539045 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+18666 (secure 0 0 0) 0x7f3fa8053530 con 0x7f3fb80ad5b0 2023-10-18T11:03:33.075 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T11:03:33.075 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":56,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:55:22.907577+0000","last_up_change":"2023-10-18T10:55:21.905065+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:47:13.188701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-18T10:47:14.198224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":50,"up_from":55,"up_thru":55,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":1133012599},{"type":"v1","addr":"172.21.15.154:6824","nonce":1133012599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1133012599},{"type":"v1","addr":"172.21.15.154:6827","nonce":1133012599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1133012599},{"type":"v1","addr":"172.21.15.154:6831","nonce":1133012599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1133012599},{"type":"v1","addr":"172.21.15.154:6829","nonce":1133012599}]},"public_addr":"172.21.15.154:6824/1133012599","cluster_addr":"172.21.15.154:6827/1133012599","heartbeat_back_addr":"172.21.15.154:6831/1133012599","heartbeat_front_addr":"172.21.15.154:6829/1133012599","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"2023-10-18T10:47:48.955023+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":51},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.9","mappings":[{"from":0,"to":5}]},{"pgid":"7.18","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:0/330049992":"2023-10-18T11:55:17.981058+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T11:03:33.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.076+0000 7f3fbe52c640 1 -- 172.21.15.89:0/2843539045 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3fa4065680 msgr2=0x7f3fa4067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:33.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.076+0000 7f3fbe52c640 1 --2- 172.21.15.89:0/2843539045 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3fa4065680 0x7f3fa4067b40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3fb80c5300 tx=0x7f3fac009040 comp rx=0 tx=0).stop 2023-10-18T11:03:33.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.076+0000 7f3fbe52c640 1 -- 172.21.15.89:0/2843539045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fb80ad5b0 msgr2=0x7f3fb8156b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:33.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.076+0000 7f3fbe52c640 1 --2- 172.21.15.89:0/2843539045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fb80ad5b0 0x7f3fb8156b50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8007fd0 tx=0x7f3fa800cbe0 comp rx=0 tx=0).stop 2023-10-18T11:03:33.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.076+0000 7f3fbe52c640 1 -- 172.21.15.89:0/2843539045 shutdown_connections 2023-10-18T11:03:33.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.076+0000 7f3fbe52c640 1 --2- 172.21.15.89:0/2843539045 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3fa4065680 0x7f3fa4067b40 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:33.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.076+0000 7f3fbe52c640 1 --2- 172.21.15.89:0/2843539045 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3fb80afd90 0x7f3fb8155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:33.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.076+0000 7f3fbe52c640 1 --2- 172.21.15.89:0/2843539045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fb815c7f0 0x7f3fb814c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:33.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.076+0000 7f3fbe52c640 1 --2- 172.21.15.89:0/2843539045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fb80ad5b0 0x7f3fb8156b50 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:33.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.080+0000 7f3fbe52c640 1 -- 172.21.15.89:0/2843539045 >> 172.21.15.89:0/2843539045 conn(0x7f3fb8069f00 msgr2=0x7f3fb80cbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:33.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.080+0000 7f3fbe52c640 1 -- 172.21.15.89:0/2843539045 shutdown_connections 2023-10-18T11:03:33.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.080+0000 7f3fbe52c640 1 -- 172.21.15.89:0/2843539045 wait complete. 2023-10-18T11:03:33.095 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:45:43.114184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-18T10:45:48.730968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:47:13.188701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:47:14.198224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T11:03:33.096 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T11:03:33.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.240+0000 7fb3c31e1640 1 Processor -- start 2023-10-18T11:03:33.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.240+0000 7fb3c31e1640 1 -- start start 2023-10-18T11:03:33.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.240+0000 7fb3c31e1640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb3bc15c670 0x7fb3bc0713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:33.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.240+0000 7fb3c31e1640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb3bc060a30 0x7fb3bc0c9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:33.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.240+0000 7fb3c31e1640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3bc1602a0 con 0x7fb3bc160a10 2023-10-18T11:03:33.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.240+0000 7fb3c31e1640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3bc071900 con 0x7fb3bc15c670 2023-10-18T11:03:33.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.240+0000 7fb3c31e1640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3bc0cd2b0 con 0x7fb3bc060a30 2023-10-18T11:03:33.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.240+0000 7fb3c0f7d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb3bc15c670 0x7fb3bc0713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:33.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.240+0000 7fb3b3fff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb3bc160a10 0x7fb3bc060320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39520/0 (socket says 172.21.15.89:39520) 2023-10-18T11:03:33.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.240+0000 7fb3b3fff640 1 -- 172.21.15.89:0/1959172139 learned_addr learned my addr 172.21.15.89:0/1959172139 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:33.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.240+0000 7fb3c177e640 1 --2- 172.21.15.89:0/1959172139 >> v2:172.21.15.154:3300/0 conn(0x7fb3bc060a30 0x7fb3bc0c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:33.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.240+0000 7fb3c0f7d640 1 -- 172.21.15.89:0/1959172139 >> v2:172.21.15.154:3300/0 conn(0x7fb3bc060a30 msgr2=0x7fb3bc0c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:33.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.240+0000 7fb3c0f7d640 1 --2- 172.21.15.89:0/1959172139 >> v2:172.21.15.154:3300/0 conn(0x7fb3bc060a30 0x7fb3bc0c9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:33.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.240+0000 7fb3c0f7d640 1 -- 172.21.15.89:0/1959172139 >> v1:172.21.15.89:6789/0 conn(0x7fb3bc160a10 legacy=0x7fb3bc060320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:33.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.240+0000 7fb3b37fe640 1 -- 172.21.15.89:0/1959172139 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 695141473 0 0) 0x7fb3bc1602a0 con 0x7fb3bc160a10 2023-10-18T11:03:33.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.240+0000 7fb3c0f7d640 1 -- 172.21.15.89:0/1959172139 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3bc0cd590 con 0x7fb3bc15c670 2023-10-18T11:03:33.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.244+0000 7fb3c0f7d640 1 --2- 172.21.15.89:0/1959172139 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3bc15c670 0x7fb3bc0713c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc06e500 tx=0x7fb3b801ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc79b9cb457765b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:33.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.244+0000 7fb3b37fe640 1 -- 172.21.15.89:0/1959172139 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b80a8950 con 0x7fb3bc15c670 2023-10-18T11:03:33.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.244+0000 7fb3b37fe640 1 -- 172.21.15.89:0/1959172139 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b80a8ab0 con 0x7fb3bc15c670 2023-10-18T11:03:33.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.244+0000 7fb3b37fe640 1 -- 172.21.15.89:0/1959172139 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b80a8e40 con 0x7fb3bc15c670 2023-10-18T11:03:33.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.244+0000 7fb3c31e1640 1 -- 172.21.15.89:0/1959172139 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3bc15c670 msgr2=0x7fb3bc0713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:33.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.244+0000 7fb3c31e1640 1 --2- 172.21.15.89:0/1959172139 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3bc15c670 0x7fb3bc0713c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc06e500 tx=0x7fb3b801ec50 comp rx=0 tx=0).stop 2023-10-18T11:03:33.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.244+0000 7fb3c31e1640 1 -- 172.21.15.89:0/1959172139 shutdown_connections 2023-10-18T11:03:33.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.244+0000 7fb3c31e1640 1 --2- 172.21.15.89:0/1959172139 >> v2:172.21.15.154:3300/0 conn(0x7fb3bc060a30 0x7fb3bc0c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:33.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.244+0000 7fb3c31e1640 1 --2- 172.21.15.89:0/1959172139 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3bc15c670 0x7fb3bc0713c0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:33.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.244+0000 7fb3c31e1640 1 -- 172.21.15.89:0/1959172139 >> 172.21.15.89:0/1959172139 conn(0x7fb3bc0cbdc0 msgr2=0x7fb3bc0cc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:33.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.244+0000 7fb3c31e1640 1 -- 172.21.15.89:0/1959172139 shutdown_connections 2023-10-18T11:03:33.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.244+0000 7fb3c31e1640 1 -- 172.21.15.89:0/1959172139 wait complete. 2023-10-18T11:03:33.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.244+0000 7fb3c31e1640 1 Processor -- start 2023-10-18T11:03:33.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.248+0000 7fb3c31e1640 1 -- start start 2023-10-18T11:03:33.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.248+0000 7fb3c31e1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3bc060a30 0x7fb3bc15f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:33.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.248+0000 7fb3c31e1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3bc15c670 0x7fb3bc15b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:33.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.248+0000 7fb3c31e1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3bc160a10 0x7fb3bc15d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:33.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.248+0000 7fb3c31e1640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3bc15be40 con 0x7fb3bc060a30 2023-10-18T11:03:33.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.248+0000 7fb3c31e1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3bc15d9a0 con 0x7fb3bc15c670 2023-10-18T11:03:33.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.248+0000 7fb3c31e1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3bc15dbd0 con 0x7fb3bc160a10 2023-10-18T11:03:33.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.248+0000 7fb3c0f7d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3bc060a30 0x7fb3bc15f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:33.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.248+0000 7fb3c0f7d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3bc060a30 0x7fb3bc15f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51188/0 (socket says 172.21.15.89:51188) 2023-10-18T11:03:33.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.248+0000 7fb3c0f7d640 1 -- 172.21.15.89:0/2271632807 learned_addr learned my addr 172.21.15.89:0/2271632807 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:33.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.248+0000 7fb3c177e640 1 --2- 172.21.15.89:0/2271632807 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3bc160a10 0x7fb3bc15d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:33.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.248+0000 7fb3c0f7d640 1 -- 172.21.15.89:0/2271632807 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3bc160a10 msgr2=0x7fb3bc15d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:33.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.248+0000 7fb3c0f7d640 1 --2- 172.21.15.89:0/2271632807 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3bc160a10 0x7fb3bc15d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:33.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.248+0000 7fb3c0f7d640 1 -- 172.21.15.89:0/2271632807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3bc15c670 msgr2=0x7fb3bc15b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:33.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.248+0000 7fb3c0f7d640 1 --2- 172.21.15.89:0/2271632807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3bc15c670 0x7fb3bc15b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:33.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.248+0000 7fb3c0f7d640 1 -- 172.21.15.89:0/2271632807 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b8009420 con 0x7fb3bc060a30 2023-10-18T11:03:33.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.248+0000 7fb3c177e640 1 --2- 172.21.15.89:0/2271632807 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3bc160a10 0x7fb3bc15d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:03:33.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.248+0000 7fb3c0f7d640 1 --2- 172.21.15.89:0/2271632807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3bc060a30 0x7fb3bc15f620 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb3b80b3040 tx=0x7fb3b801b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:33.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.248+0000 7fb3b17fa640 1 -- 172.21.15.89:0/2271632807 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b80b9630 con 0x7fb3bc060a30 2023-10-18T11:03:33.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.248+0000 7fb3b17fa640 1 -- 172.21.15.89:0/2271632807 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b80b97b0 con 0x7fb3bc060a30 2023-10-18T11:03:33.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.248+0000 7fb3b17fa640 1 -- 172.21.15.89:0/2271632807 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b80b9b60 con 0x7fb3bc060a30 2023-10-18T11:03:33.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.248+0000 7fb3c31e1640 1 -- 172.21.15.89:0/2271632807 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3bc15deb0 con 0x7fb3bc060a30 2023-10-18T11:03:33.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.248+0000 7fb3c31e1640 1 -- 172.21.15.89:0/2271632807 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3bc0aba40 con 0x7fb3bc060a30 2023-10-18T11:03:33.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.248+0000 7fb3c31e1640 1 -- 172.21.15.89:0/2271632807 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3bc04dbe0 con 0x7fb3bc060a30 2023-10-18T11:03:33.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.252+0000 7fb3b17fa640 1 -- 172.21.15.89:0/2271632807 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fb3b80b7040 con 0x7fb3bc060a30 2023-10-18T11:03:33.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.252+0000 7fb3b17fa640 1 --2- 172.21.15.89:0/2271632807 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb3a8065680 0x7fb3a8067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:33.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.252+0000 7fb3b17fa640 1 -- 172.21.15.89:0/2271632807 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fb3b8121090 con 0x7fb3bc060a30 2023-10-18T11:03:33.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.252+0000 7fb3b3fff640 1 --2- 172.21.15.89:0/2271632807 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb3a8065680 0x7fb3a8067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:33.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.252+0000 7fb3b3fff640 1 --2- 172.21.15.89:0/2271632807 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb3a8065680 0x7fb3a8067b40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0011f0 tx=0x7fb3ac007930 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:33.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.256+0000 7fb3b17fa640 1 -- 172.21.15.89:0/2271632807 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fb3b80ee900 con 0x7fb3bc060a30 2023-10-18T11:03:33.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.404+0000 7fb3c31e1640 1 -- 172.21.15.89:0/2271632807 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb3bc071c60 con 0x7fb3bc060a30 2023-10-18T11:03:33.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.404+0000 7fb3b17fa640 1 -- 172.21.15.89:0/2271632807 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb3b80f1f20 con 0x7fb3bc060a30 2023-10-18T11:03:33.402 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-18T11:03:33.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.404+0000 7fb3c31e1640 1 -- 172.21.15.89:0/2271632807 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb3a8065680 msgr2=0x7fb3a8067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:33.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.404+0000 7fb3c31e1640 1 --2- 172.21.15.89:0/2271632807 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb3a8065680 0x7fb3a8067b40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0011f0 tx=0x7fb3ac007930 comp rx=0 tx=0).stop 2023-10-18T11:03:33.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.404+0000 7fb3c31e1640 1 -- 172.21.15.89:0/2271632807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3bc060a30 msgr2=0x7fb3bc15f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:33.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.404+0000 7fb3c31e1640 1 --2- 172.21.15.89:0/2271632807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3bc060a30 0x7fb3bc15f620 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb3b80b3040 tx=0x7fb3b801b9f0 comp rx=0 tx=0).stop 2023-10-18T11:03:33.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.404+0000 7fb3c31e1640 1 -- 172.21.15.89:0/2271632807 shutdown_connections 2023-10-18T11:03:33.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.404+0000 7fb3c31e1640 1 --2- 172.21.15.89:0/2271632807 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb3a8065680 0x7fb3a8067b40 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:33.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.404+0000 7fb3c31e1640 1 --2- 172.21.15.89:0/2271632807 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3bc160a10 0x7fb3bc15d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:33.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.404+0000 7fb3c31e1640 1 --2- 172.21.15.89:0/2271632807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3bc15c670 0x7fb3bc15b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:33.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.404+0000 7fb3c31e1640 1 --2- 172.21.15.89:0/2271632807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3bc060a30 0x7fb3bc15f620 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:33.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.404+0000 7fb3c31e1640 1 -- 172.21.15.89:0/2271632807 >> 172.21.15.89:0/2271632807 conn(0x7fb3bc0cbdc0 msgr2=0x7fb3bc070990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:33.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.404+0000 7fb3c31e1640 1 -- 172.21.15.89:0/2271632807 shutdown_connections 2023-10-18T11:03:33.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.404+0000 7fb3c31e1640 1 -- 172.21.15.89:0/2271632807 wait complete. 2023-10-18T11:03:33.422 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-18T11:03:33.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.564+0000 7ff305cd8640 1 Processor -- start 2023-10-18T11:03:33.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.564+0000 7ff305cd8640 1 -- start start 2023-10-18T11:03:33.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.564+0000 7ff305cd8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3000ca340 0x7ff30015e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:33.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.568+0000 7ff305cd8640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff30015acb0 con 0x7ff3000c8960 2023-10-18T11:03:33.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.568+0000 7ff305cd8640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3000702e0 con 0x7ff30006ff00 2023-10-18T11:03:33.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.568+0000 7ff305cd8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff30015ef30 con 0x7ff3000ca340 2023-10-18T11:03:33.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.568+0000 7ff2fffff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff3000c8960 0x7ff3000afd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39530/0 (socket says 172.21.15.89:39530) 2023-10-18T11:03:33.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.568+0000 7ff2fffff640 1 -- 172.21.15.89:0/879503717 learned_addr learned my addr 172.21.15.89:0/879503717 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:33.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.568+0000 7ff2f6ffd640 1 --2- 172.21.15.89:0/879503717 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3000ca340 0x7ff30015e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:33.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.568+0000 7ff2feffd640 1 -- 172.21.15.89:0/879503717 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2694847362 0 0) 0x7ff30015acb0 con 0x7ff3000c8960 2023-10-18T11:03:33.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.568+0000 7ff2feffd640 1 -- 172.21.15.89:0/879503717 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2ec003530 con 0x7ff3000c8960 2023-10-18T11:03:33.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.568+0000 7ff2feffd640 1 -- 172.21.15.89:0/879503717 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2742572805 0 0) 0x7ff3000702e0 con 0x7ff30006ff00 2023-10-18T11:03:33.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.568+0000 7ff2feffd640 1 -- 172.21.15.89:0/879503717 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff30015acb0 con 0x7ff30006ff00 2023-10-18T11:03:33.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.568+0000 7ff2f6ffd640 1 -- 172.21.15.89:0/879503717 >> v1:172.21.15.154:6789/0 conn(0x7ff30006ff00 legacy=0x7ff30015b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:33.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.568+0000 7ff2f6ffd640 1 -- 172.21.15.89:0/879503717 >> v1:172.21.15.89:6789/0 conn(0x7ff3000c8960 legacy=0x7ff3000afd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:33.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.568+0000 7ff2f6ffd640 1 -- 172.21.15.89:0/879503717 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30015b850 con 0x7ff3000ca340 2023-10-18T11:03:33.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.572+0000 7ff2f6ffd640 1 --2- 172.21.15.89:0/879503717 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3000ca340 0x7ff30015e000 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff2e80097f0 tx=0x7ff2e801c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d56e26608f7de4cf server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:33.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.572+0000 7ff2feffd640 1 -- 172.21.15.89:0/879503717 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e80ab600 con 0x7ff3000ca340 2023-10-18T11:03:33.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.572+0000 7ff2feffd640 1 -- 172.21.15.89:0/879503717 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2e80a7050 con 0x7ff3000ca340 2023-10-18T11:03:33.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.572+0000 7ff2feffd640 1 -- 172.21.15.89:0/879503717 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e80ab980 con 0x7ff3000ca340 2023-10-18T11:03:33.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.572+0000 7ff305cd8640 1 -- 172.21.15.89:0/879503717 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3000ca340 msgr2=0x7ff30015e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:33.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.572+0000 7ff305cd8640 1 --2- 172.21.15.89:0/879503717 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3000ca340 0x7ff30015e000 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff2e80097f0 tx=0x7ff2e801c430 comp rx=0 tx=0).stop 2023-10-18T11:03:33.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.572+0000 7ff305cd8640 1 -- 172.21.15.89:0/879503717 shutdown_connections 2023-10-18T11:03:33.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.572+0000 7ff305cd8640 1 --2- 172.21.15.89:0/879503717 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3000ca340 0x7ff30015e000 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:33.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.572+0000 7ff305cd8640 1 -- 172.21.15.89:0/879503717 >> 172.21.15.89:0/879503717 conn(0x7ff300069f00 msgr2=0x7ff3000cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:33.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.572+0000 7ff305cd8640 1 -- 172.21.15.89:0/879503717 shutdown_connections 2023-10-18T11:03:33.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.572+0000 7ff305cd8640 1 -- 172.21.15.89:0/879503717 wait complete. 2023-10-18T11:03:33.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.572+0000 7ff305cd8640 1 Processor -- start 2023-10-18T11:03:33.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.572+0000 7ff305cd8640 1 -- start start 2023-10-18T11:03:33.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.572+0000 7ff305cd8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff30006ff00 0x7ff30014bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:33.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.572+0000 7ff305cd8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3000c8960 0x7ff300150790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:33.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.576+0000 7ff305cd8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3000ca340 0x7ff300150cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:33.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.576+0000 7ff305cd8640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30014cc00 con 0x7ff30006ff00 2023-10-18T11:03:33.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.576+0000 7ff305cd8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3000ae0c0 con 0x7ff3000ca340 2023-10-18T11:03:33.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.576+0000 7ff305cd8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3000b02d0 con 0x7ff3000c8960 2023-10-18T11:03:33.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.576+0000 7ff2ff7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff30006ff00 0x7ff30014bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:33.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.576+0000 7ff2ff7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff30006ff00 0x7ff30014bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51200/0 (socket says 172.21.15.89:51200) 2023-10-18T11:03:33.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.576+0000 7ff2ff7fe640 1 -- 172.21.15.89:0/2456411593 learned_addr learned my addr 172.21.15.89:0/2456411593 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:33.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.576+0000 7ff2fffff640 1 --2- 172.21.15.89:0/2456411593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3000ca340 0x7ff300150cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:33.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.576+0000 7ff2f6ffd640 1 --2- 172.21.15.89:0/2456411593 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3000c8960 0x7ff300150790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:33.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.576+0000 7ff2ff7fe640 1 -- 172.21.15.89:0/2456411593 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3000c8960 msgr2=0x7ff300150790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:33.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.576+0000 7ff2ff7fe640 1 --2- 172.21.15.89:0/2456411593 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3000c8960 0x7ff300150790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:33.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.576+0000 7ff2ff7fe640 1 -- 172.21.15.89:0/2456411593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3000ca340 msgr2=0x7ff300150cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:33.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.576+0000 7ff2ff7fe640 1 --2- 172.21.15.89:0/2456411593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3000ca340 0x7ff300150cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:33.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.576+0000 7ff2ff7fe640 1 -- 172.21.15.89:0/2456411593 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e8009420 con 0x7ff30006ff00 2023-10-18T11:03:33.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.576+0000 7ff2ff7fe640 1 --2- 172.21.15.89:0/2456411593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff30006ff00 0x7ff30014bb90 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000c870 tx=0x7ff2f000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:33.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.576+0000 7ff2fcff9640 1 -- 172.21.15.89:0/2456411593 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0003b80 con 0x7ff30006ff00 2023-10-18T11:03:33.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.576+0000 7ff2fcff9640 1 -- 172.21.15.89:0/2456411593 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f0003ce0 con 0x7ff30006ff00 2023-10-18T11:03:33.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.576+0000 7ff2fcff9640 1 -- 172.21.15.89:0/2456411593 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f000aa30 con 0x7ff30006ff00 2023-10-18T11:03:33.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.576+0000 7ff305cd8640 1 -- 172.21.15.89:0/2456411593 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff30014c2d0 con 0x7ff30006ff00 2023-10-18T11:03:33.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.576+0000 7ff305cd8640 1 -- 172.21.15.89:0/2456411593 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff30014c890 con 0x7ff30006ff00 2023-10-18T11:03:33.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.576+0000 7ff305cd8640 1 -- 172.21.15.89:0/2456411593 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff30004dbe0 con 0x7ff30006ff00 2023-10-18T11:03:33.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.576+0000 7ff2fcff9640 1 -- 172.21.15.89:0/2456411593 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7ff2f000ab90 con 0x7ff30006ff00 2023-10-18T11:03:33.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.580+0000 7ff2fcff9640 1 --2- 172.21.15.89:0/2456411593 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff2e0065680 0x7ff2e0067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:33.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.580+0000 7ff2fcff9640 1 -- 172.21.15.89:0/2456411593 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7ff2f0082d70 con 0x7ff30006ff00 2023-10-18T11:03:33.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.580+0000 7ff2f6ffd640 1 --2- 172.21.15.89:0/2456411593 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff2e0065680 0x7ff2e0067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:33.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.580+0000 7ff2f6ffd640 1 --2- 172.21.15.89:0/2456411593 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff2e0065680 0x7ff2e0067b40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff2e80ad000 tx=0x7ff2e8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:33.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.584+0000 7ff2fcff9640 1 -- 172.21.15.89:0/2456411593 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7ff2f00507a0 con 0x7ff30006ff00 2023-10-18T11:03:33.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.732+0000 7ff305cd8640 1 -- 172.21.15.89:0/2456411593 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff3000b0440 con 0x7ff30006ff00 2023-10-18T11:03:33.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.732+0000 7ff2fcff9640 1 -- 172.21.15.89:0/2456411593 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v56) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff2f0053dc0 con 0x7ff30006ff00 2023-10-18T11:03:33.730 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-10-18T11:03:33.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.732+0000 7ff305cd8640 1 -- 172.21.15.89:0/2456411593 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff2e0065680 msgr2=0x7ff2e0067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:33.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.732+0000 7ff305cd8640 1 --2- 172.21.15.89:0/2456411593 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff2e0065680 0x7ff2e0067b40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff2e80ad000 tx=0x7ff2e8008040 comp rx=0 tx=0).stop 2023-10-18T11:03:33.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.732+0000 7ff305cd8640 1 -- 172.21.15.89:0/2456411593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff30006ff00 msgr2=0x7ff30014bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:33.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.732+0000 7ff305cd8640 1 --2- 172.21.15.89:0/2456411593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff30006ff00 0x7ff30014bb90 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000c870 tx=0x7ff2f000cd30 comp rx=0 tx=0).stop 2023-10-18T11:03:33.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.732+0000 7ff305cd8640 1 -- 172.21.15.89:0/2456411593 shutdown_connections 2023-10-18T11:03:33.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.732+0000 7ff305cd8640 1 --2- 172.21.15.89:0/2456411593 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff2e0065680 0x7ff2e0067b40 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:33.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.732+0000 7ff305cd8640 1 --2- 172.21.15.89:0/2456411593 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3000c8960 0x7ff300150790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:33.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.732+0000 7ff305cd8640 1 --2- 172.21.15.89:0/2456411593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3000ca340 0x7ff300150cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:33.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.732+0000 7ff305cd8640 1 --2- 172.21.15.89:0/2456411593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff30006ff00 0x7ff30014bb90 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:33.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.732+0000 7ff305cd8640 1 -- 172.21.15.89:0/2456411593 >> 172.21.15.89:0/2456411593 conn(0x7ff300069f00 msgr2=0x7ff3000cd130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:33.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.732+0000 7ff305cd8640 1 -- 172.21.15.89:0/2456411593 shutdown_connections 2023-10-18T11:03:33.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.732+0000 7ff305cd8640 1 -- 172.21.15.89:0/2456411593 wait complete. 2023-10-18T11:03:33.751 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T11:03:33.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.892+0000 7efdca8e5640 1 Processor -- start 2023-10-18T11:03:33.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.892+0000 7efdca8e5640 1 -- start start 2023-10-18T11:03:33.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.896+0000 7efdca8e5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdc40afd90 0x7efdc40b01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:33.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.896+0000 7efdca8e5640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efdc4070280 0x7efdc4070710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:33.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.896+0000 7efdca8e5640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdc406fe10 con 0x7efdc4158d20 2023-10-18T11:03:33.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.896+0000 7efdca8e5640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdc40c9220 con 0x7efdc4070280 2023-10-18T11:03:33.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.896+0000 7efdca8e5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdc40c9500 con 0x7efdc40afd90 2023-10-18T11:03:33.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.896+0000 7efdc8e82640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efdc4070280 0x7efdc4070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:33.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.896+0000 7efdc3fff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7efdc4158d20 0x7efdc415d110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39546/0 (socket says 172.21.15.89:39546) 2023-10-18T11:03:33.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.896+0000 7efdc8e82640 1 --2- 172.21.15.89:0/2581532288 >> v2:172.21.15.89:3300/0 conn(0x7efdc4070280 0x7efdc4070710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51212/0 (socket says 172.21.15.89:51212) 2023-10-18T11:03:33.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.896+0000 7efdc3fff640 1 -- 172.21.15.89:0/2581532288 learned_addr learned my addr 172.21.15.89:0/2581532288 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:33.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.896+0000 7efdc37fe640 1 --2- 172.21.15.89:0/2581532288 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdc40afd90 0x7efdc40b01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:33.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.896+0000 7efdc8e82640 1 -- 172.21.15.89:0/2581532288 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdc40afd90 msgr2=0x7efdc40b01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:33.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.896+0000 7efdc8e82640 1 --2- 172.21.15.89:0/2581532288 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdc40afd90 0x7efdc40b01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:33.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.896+0000 7efdc8e82640 1 -- 172.21.15.89:0/2581532288 >> v1:172.21.15.89:6789/0 conn(0x7efdc4158d20 legacy=0x7efdc415d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:33.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.896+0000 7efdc8e82640 1 -- 172.21.15.89:0/2581532288 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc40b0730 con 0x7efdc4070280 2023-10-18T11:03:33.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.896+0000 7efdc37fe640 1 --2- 172.21.15.89:0/2581532288 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdc40afd90 0x7efdc40b01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:03:33.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.900+0000 7efdc8e82640 1 --2- 172.21.15.89:0/2581532288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efdc4070280 0x7efdc4070710 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7efdbc00bfa0 tx=0x7efdbc01eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=105075a4e375c315 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:33.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.900+0000 7efdc2ffd640 1 -- 172.21.15.89:0/2581532288 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc00b2d0 con 0x7efdc4070280 2023-10-18T11:03:33.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.900+0000 7efdc2ffd640 1 -- 172.21.15.89:0/2581532288 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdbc0a7890 con 0x7efdc4070280 2023-10-18T11:03:33.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.900+0000 7efdc2ffd640 1 -- 172.21.15.89:0/2581532288 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc0a7c20 con 0x7efdc4070280 2023-10-18T11:03:33.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.900+0000 7efdca8e5640 1 -- 172.21.15.89:0/2581532288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efdc4070280 msgr2=0x7efdc4070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:33.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.900+0000 7efdca8e5640 1 --2- 172.21.15.89:0/2581532288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efdc4070280 0x7efdc4070710 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7efdbc00bfa0 tx=0x7efdbc01eb50 comp rx=0 tx=0).stop 2023-10-18T11:03:33.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.900+0000 7efdca8e5640 1 -- 172.21.15.89:0/2581532288 shutdown_connections 2023-10-18T11:03:33.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.900+0000 7efdca8e5640 1 --2- 172.21.15.89:0/2581532288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efdc4070280 0x7efdc4070710 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:33.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.900+0000 7efdca8e5640 1 --2- 172.21.15.89:0/2581532288 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdc40afd90 0x7efdc40b01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:33.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.900+0000 7efdca8e5640 1 -- 172.21.15.89:0/2581532288 >> 172.21.15.89:0/2581532288 conn(0x7efdc4069f00 msgr2=0x7efdc40cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:33.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.900+0000 7efdca8e5640 1 -- 172.21.15.89:0/2581532288 shutdown_connections 2023-10-18T11:03:33.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.900+0000 7efdca8e5640 1 -- 172.21.15.89:0/2581532288 wait complete. 2023-10-18T11:03:33.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.900+0000 7efdca8e5640 1 Processor -- start 2023-10-18T11:03:33.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.900+0000 7efdca8e5640 1 -- start start 2023-10-18T11:03:33.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.904+0000 7efdca8e5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdc4070280 0x7efdc40c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:33.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.904+0000 7efdca8e5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efdc40afd90 0x7efdc4147570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:33.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.904+0000 7efdca8e5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdc4158d20 0x7efdc40ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:33.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.904+0000 7efdca8e5640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc40c8070 con 0x7efdc40afd90 2023-10-18T11:03:33.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.904+0000 7efdca8e5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc40c6a90 con 0x7efdc4158d20 2023-10-18T11:03:33.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.904+0000 7efdca8e5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdc40c6d70 con 0x7efdc4070280 2023-10-18T11:03:33.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.904+0000 7efdc37fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efdc40afd90 0x7efdc4147570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:33.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.904+0000 7efdc3fff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdc4070280 0x7efdc40c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:33.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.904+0000 7efdc37fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efdc40afd90 0x7efdc4147570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51222/0 (socket says 172.21.15.89:51222) 2023-10-18T11:03:33.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.904+0000 7efdc3fff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdc4070280 0x7efdc40c7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:37052/0 (socket says 172.21.15.89:37052) 2023-10-18T11:03:33.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.904+0000 7efdc37fe640 1 -- 172.21.15.89:0/3693480034 learned_addr learned my addr 172.21.15.89:0/3693480034 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:33.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.904+0000 7efdc8e82640 1 --2- 172.21.15.89:0/3693480034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdc4158d20 0x7efdc40ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:33.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.904+0000 7efdc37fe640 1 -- 172.21.15.89:0/3693480034 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdc4070280 msgr2=0x7efdc40c7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:33.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.904+0000 7efdc37fe640 1 --2- 172.21.15.89:0/3693480034 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdc4070280 0x7efdc40c7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:33.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.904+0000 7efdc37fe640 1 -- 172.21.15.89:0/3693480034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdc4158d20 msgr2=0x7efdc40ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:33.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.904+0000 7efdc37fe640 1 --2- 172.21.15.89:0/3693480034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdc4158d20 0x7efdc40ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:33.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.904+0000 7efdc37fe640 1 -- 172.21.15.89:0/3693480034 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdbc00bc80 con 0x7efdc40afd90 2023-10-18T11:03:33.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.904+0000 7efdc37fe640 1 --2- 172.21.15.89:0/3693480034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efdc40afd90 0x7efdc4147570 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7efdb800c7c0 tx=0x7efdb800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:33.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.904+0000 7efdc0ff9640 1 -- 172.21.15.89:0/3693480034 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb8007a30 con 0x7efdc40afd90 2023-10-18T11:03:33.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.904+0000 7efdc0ff9640 1 -- 172.21.15.89:0/3693480034 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb8007b90 con 0x7efdc40afd90 2023-10-18T11:03:33.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.904+0000 7efdc0ff9640 1 -- 172.21.15.89:0/3693480034 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb800ce70 con 0x7efdc40afd90 2023-10-18T11:03:33.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.904+0000 7efdca8e5640 1 -- 172.21.15.89:0/3693480034 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdc40c7050 con 0x7efdc40afd90 2023-10-18T11:03:33.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.904+0000 7efdca8e5640 1 -- 172.21.15.89:0/3693480034 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdc40c7610 con 0x7efdc40afd90 2023-10-18T11:03:33.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.904+0000 7efdca8e5640 1 -- 172.21.15.89:0/3693480034 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdc40c7830 con 0x7efdc40afd90 2023-10-18T11:03:33.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.904+0000 7efdc0ff9640 1 -- 172.21.15.89:0/3693480034 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7efdb80113f0 con 0x7efdc40afd90 2023-10-18T11:03:33.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.908+0000 7efdc0ff9640 1 --2- 172.21.15.89:0/3693480034 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efdb00656b0 0x7efdb0067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:33.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.908+0000 7efdc0ff9640 1 -- 172.21.15.89:0/3693480034 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7efdb8083410 con 0x7efdc40afd90 2023-10-18T11:03:33.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.908+0000 7efdc3fff640 1 --2- 172.21.15.89:0/3693480034 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efdb00656b0 0x7efdb0067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:33.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.908+0000 7efdc3fff640 1 --2- 172.21.15.89:0/3693480034 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efdb00656b0 0x7efdb0067b70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7efdb4005890 tx=0x7efdb4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:33.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:33.912+0000 7efdc0ff9640 1 -- 172.21.15.89:0/3693480034 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7efdb8050d70 con 0x7efdc40afd90 2023-10-18T11:03:34.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.060+0000 7efdca8e5640 1 -- 172.21.15.89:0/3693480034 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efdc404dbe0 con 0x7efdc40afd90 2023-10-18T11:03:34.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.060+0000 7efdc0ff9640 1 -- 172.21.15.89:0/3693480034 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) v1 ==== 104+0+11 (secure 0 0 0) 0x7efdb8054390 con 0x7efdc40afd90 2023-10-18T11:03:34.058 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-18T11:03:34.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.060+0000 7efdca8e5640 1 -- 172.21.15.89:0/3693480034 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efdb00656b0 msgr2=0x7efdb0067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:34.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.060+0000 7efdca8e5640 1 --2- 172.21.15.89:0/3693480034 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efdb00656b0 0x7efdb0067b70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7efdb4005890 tx=0x7efdb4009040 comp rx=0 tx=0).stop 2023-10-18T11:03:34.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.060+0000 7efdca8e5640 1 -- 172.21.15.89:0/3693480034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efdc40afd90 msgr2=0x7efdc4147570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:34.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.060+0000 7efdca8e5640 1 --2- 172.21.15.89:0/3693480034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efdc40afd90 0x7efdc4147570 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7efdb800c7c0 tx=0x7efdb800cc80 comp rx=0 tx=0).stop 2023-10-18T11:03:34.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.060+0000 7efdca8e5640 1 -- 172.21.15.89:0/3693480034 shutdown_connections 2023-10-18T11:03:34.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.060+0000 7efdca8e5640 1 --2- 172.21.15.89:0/3693480034 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efdb00656b0 0x7efdb0067b70 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:34.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.060+0000 7efdca8e5640 1 --2- 172.21.15.89:0/3693480034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdc4158d20 0x7efdc40ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:34.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.060+0000 7efdca8e5640 1 --2- 172.21.15.89:0/3693480034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efdc40afd90 0x7efdc4147570 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:34.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.060+0000 7efdca8e5640 1 --2- 172.21.15.89:0/3693480034 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efdc4070280 0x7efdc40c7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:34.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.060+0000 7efdca8e5640 1 -- 172.21.15.89:0/3693480034 >> 172.21.15.89:0/3693480034 conn(0x7efdc4069f00 msgr2=0x7efdc40cc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:34.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.060+0000 7efdca8e5640 1 -- 172.21.15.89:0/3693480034 shutdown_connections 2023-10-18T11:03:34.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.060+0000 7efdca8e5640 1 -- 172.21.15.89:0/3693480034 wait complete. 2023-10-18T11:03:34.079 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T11:03:34.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.224+0000 7fdf63e9c640 1 Processor -- start 2023-10-18T11:03:34.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.224+0000 7fdf63e9c640 1 -- start start 2023-10-18T11:03:34.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.224+0000 7fdf63e9c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdf5c0b1950 0x7fdf5c0d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:34.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.224+0000 7fdf63e9c640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf5c0b1430 con 0x7fdf5c15ff30 2023-10-18T11:03:34.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.224+0000 7fdf63e9c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf5c0d3080 con 0x7fdf5c06a010 2023-10-18T11:03:34.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.224+0000 7fdf63e9c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf5c0d3690 con 0x7fdf5c0b1950 2023-10-18T11:03:34.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.224+0000 7fdf61c38640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fdf5c15ff30 0x7fdf5c069900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39556/0 (socket says 172.21.15.89:39556) 2023-10-18T11:03:34.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.224+0000 7fdf61c38640 1 -- 172.21.15.89:0/696468866 learned_addr learned my addr 172.21.15.89:0/696468866 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:34.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.224+0000 7fdf62439640 1 --2- 172.21.15.89:0/696468866 >> v2:172.21.15.154:3300/0 conn(0x7fdf5c0b1950 0x7fdf5c0d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:34.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.224+0000 7fdf60c36640 1 -- 172.21.15.89:0/696468866 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2931661457 0 0) 0x7fdf5c0b1430 con 0x7fdf5c15ff30 2023-10-18T11:03:34.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.224+0000 7fdf60c36640 1 -- 172.21.15.89:0/696468866 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf48003530 con 0x7fdf5c15ff30 2023-10-18T11:03:34.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.224+0000 7fdf60c36640 1 -- 172.21.15.89:0/696468866 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4075522713 0 0) 0x7fdf5c0d3080 con 0x7fdf5c06a010 2023-10-18T11:03:34.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.224+0000 7fdf60c36640 1 -- 172.21.15.89:0/696468866 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf5c0b1430 con 0x7fdf5c06a010 2023-10-18T11:03:34.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.224+0000 7fdf60c36640 1 -- 172.21.15.89:0/696468866 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2295956771 0 0) 0x7fdf48003530 con 0x7fdf5c15ff30 2023-10-18T11:03:34.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.224+0000 7fdf60c36640 1 -- 172.21.15.89:0/696468866 >> v2:172.21.15.154:3300/0 conn(0x7fdf5c0b1950 msgr2=0x7fdf5c0d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:34.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.224+0000 7fdf60c36640 1 --2- 172.21.15.89:0/696468866 >> v2:172.21.15.154:3300/0 conn(0x7fdf5c0b1950 0x7fdf5c0d0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:34.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.224+0000 7fdf60c36640 1 -- 172.21.15.89:0/696468866 >> v1:172.21.15.154:6789/0 conn(0x7fdf5c06a010 legacy=0x7fdf5c0b0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:34.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.224+0000 7fdf60c36640 1 -- 172.21.15.89:0/696468866 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5c0d3320 con 0x7fdf5c15ff30 2023-10-18T11:03:34.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.224+0000 7fdf60c36640 1 -- 172.21.15.89:0/696468866 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7fdf58001800 con 0x7fdf5c15ff30 2023-10-18T11:03:34.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.224+0000 7fdf60c36640 1 -- 172.21.15.89:0/696468866 >> v1:172.21.15.89:6789/0 conn(0x7fdf5c15ff30 legacy=0x7fdf5c069900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:34.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.224+0000 7fdf60c36640 1 --2- 172.21.15.89:0/696468866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf48003b10 0x7fdf48003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:34.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.224+0000 7fdf60c36640 1 --2- 172.21.15.89:0/696468866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf480047d0 0x7fdf48004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:34.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.224+0000 7fdf60c36640 1 --2- 172.21.15.89:0/696468866 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf48005200 0x7fdf4800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:34.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.224+0000 7fdf60c36640 1 -- 172.21.15.89:0/696468866 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c0d3690 con 0x7fdf48003b10 2023-10-18T11:03:34.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.224+0000 7fdf60c36640 1 -- 172.21.15.89:0/696468866 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4800c160 con 0x7fdf480047d0 2023-10-18T11:03:34.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.224+0000 7fdf60c36640 1 -- 172.21.15.89:0/696468866 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4800c320 con 0x7fdf48005200 2023-10-18T11:03:34.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.224+0000 7fdf61c38640 1 --2- 172.21.15.89:0/696468866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf48003b10 0x7fdf48003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:34.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.228+0000 7fdf62439640 1 --2- 172.21.15.89:0/696468866 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf48005200 0x7fdf4800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:34.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.228+0000 7fdf61437640 1 --2- 172.21.15.89:0/696468866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf480047d0 0x7fdf48004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:34.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.228+0000 7fdf61c38640 1 -- 172.21.15.89:0/696468866 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf48005200 msgr2=0x7fdf4800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:34.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.228+0000 7fdf61c38640 1 --2- 172.21.15.89:0/696468866 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf48005200 0x7fdf4800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:34.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.228+0000 7fdf61c38640 1 -- 172.21.15.89:0/696468866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf480047d0 msgr2=0x7fdf48004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:34.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.228+0000 7fdf61c38640 1 --2- 172.21.15.89:0/696468866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf480047d0 0x7fdf48004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:34.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.228+0000 7fdf61c38640 1 -- 172.21.15.89:0/696468866 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf48003530 con 0x7fdf48003b10 2023-10-18T11:03:34.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.228+0000 7fdf62439640 1 -- 172.21.15.89:0/696468866 reap_dead start 2023-10-18T11:03:34.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.228+0000 7fdf61437640 1 --2- 172.21.15.89:0/696468866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf480047d0 0x7fdf48004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:03:34.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.228+0000 7fdf61c38640 1 --2- 172.21.15.89:0/696468866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf48003b10 0x7fdf48003f10 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fdf5800c160 tx=0x7fdf5801cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a0dc7411528126 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:34.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.228+0000 7fdf60c36640 1 -- 172.21.15.89:0/696468866 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf580070d0 con 0x7fdf48003b10 2023-10-18T11:03:34.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.228+0000 7fdf60c36640 1 -- 172.21.15.89:0/696468866 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf58007250 con 0x7fdf48003b10 2023-10-18T11:03:34.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.228+0000 7fdf60c36640 1 -- 172.21.15.89:0/696468866 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf58007640 con 0x7fdf48003b10 2023-10-18T11:03:34.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.228+0000 7fdf63e9c640 1 -- 172.21.15.89:0/696468866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf48003b10 msgr2=0x7fdf48003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:34.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.228+0000 7fdf63e9c640 1 --2- 172.21.15.89:0/696468866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf48003b10 0x7fdf48003f10 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fdf5800c160 tx=0x7fdf5801cb50 comp rx=0 tx=0).stop 2023-10-18T11:03:34.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.232+0000 7fdf63e9c640 1 -- 172.21.15.89:0/696468866 shutdown_connections 2023-10-18T11:03:34.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.232+0000 7fdf63e9c640 1 --2- 172.21.15.89:0/696468866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf48003b10 0x7fdf48003f10 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:34.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.232+0000 7fdf63e9c640 1 -- 172.21.15.89:0/696468866 >> 172.21.15.89:0/696468866 conn(0x7fdf5c09b9d0 msgr2=0x7fdf5c0d26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:34.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.232+0000 7fdf63e9c640 1 -- 172.21.15.89:0/696468866 shutdown_connections 2023-10-18T11:03:34.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.232+0000 7fdf63e9c640 1 -- 172.21.15.89:0/696468866 wait complete. 2023-10-18T11:03:34.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.232+0000 7fdf63e9c640 1 Processor -- start 2023-10-18T11:03:34.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.232+0000 7fdf63e9c640 1 -- start start 2023-10-18T11:03:34.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.232+0000 7fdf63e9c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf48003b10 0x7fdf5c15dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:34.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.232+0000 7fdf63e9c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf5c0d1aa0 0x7fdf5c0d1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:34.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.232+0000 7fdf63e9c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf5c09ca80 0x7fdf5c09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:34.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.232+0000 7fdf63e9c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c1f5540 con 0x7fdf5c09ca80 2023-10-18T11:03:34.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.232+0000 7fdf63e9c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c09e370 con 0x7fdf5c0d1aa0 2023-10-18T11:03:34.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.232+0000 7fdf63e9c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c0d2460 con 0x7fdf48003b10 2023-10-18T11:03:34.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.232+0000 7fdf62439640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf5c09ca80 0x7fdf5c09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:34.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.232+0000 7fdf62439640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf5c09ca80 0x7fdf5c09cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51228/0 (socket says 172.21.15.89:51228) 2023-10-18T11:03:34.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.232+0000 7fdf62439640 1 -- 172.21.15.89:0/723028121 learned_addr learned my addr 172.21.15.89:0/723028121 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:34.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.232+0000 7fdf61c38640 1 --2- 172.21.15.89:0/723028121 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf48003b10 0x7fdf5c15dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:34.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.232+0000 7fdf61437640 1 --2- 172.21.15.89:0/723028121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf5c0d1aa0 0x7fdf5c0d1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:34.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.232+0000 7fdf62439640 1 -- 172.21.15.89:0/723028121 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf48003b10 msgr2=0x7fdf5c15dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:34.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.232+0000 7fdf62439640 1 --2- 172.21.15.89:0/723028121 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf48003b10 0x7fdf5c15dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:34.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.232+0000 7fdf62439640 1 -- 172.21.15.89:0/723028121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf5c0d1aa0 msgr2=0x7fdf5c0d1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:34.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.232+0000 7fdf62439640 1 --2- 172.21.15.89:0/723028121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf5c0d1aa0 0x7fdf5c0d1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:34.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.232+0000 7fdf62439640 1 -- 172.21.15.89:0/723028121 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5c15e3e0 con 0x7fdf5c09ca80 2023-10-18T11:03:34.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.232+0000 7fdf61437640 1 --2- 172.21.15.89:0/723028121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf5c0d1aa0 0x7fdf5c0d1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:03:34.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.232+0000 7fdf62439640 1 --2- 172.21.15.89:0/723028121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf5c09ca80 0x7fdf5c09cf00 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c0b1310 tx=0x7fdf5400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:34.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.236+0000 7fdf527fc640 1 -- 172.21.15.89:0/723028121 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5400e040 con 0x7fdf5c09ca80 2023-10-18T11:03:34.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.236+0000 7fdf527fc640 1 -- 172.21.15.89:0/723028121 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5400f040 con 0x7fdf5c09ca80 2023-10-18T11:03:34.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.236+0000 7fdf527fc640 1 -- 172.21.15.89:0/723028121 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf54015430 con 0x7fdf5c09ca80 2023-10-18T11:03:34.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.236+0000 7fdf63e9c640 1 -- 172.21.15.89:0/723028121 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf5c09d7f0 con 0x7fdf5c09ca80 2023-10-18T11:03:34.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.236+0000 7fdf63e9c640 1 -- 172.21.15.89:0/723028121 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf5c09ddb0 con 0x7fdf5c09ca80 2023-10-18T11:03:34.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.236+0000 7fdf63e9c640 1 -- 172.21.15.89:0/723028121 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf5c1f5690 con 0x7fdf5c09ca80 2023-10-18T11:03:34.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.236+0000 7fdf527fc640 1 -- 172.21.15.89:0/723028121 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fdf54015590 con 0x7fdf5c09ca80 2023-10-18T11:03:34.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.236+0000 7fdf527fc640 1 --2- 172.21.15.89:0/723028121 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdf40065680 0x7fdf40067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:34.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.236+0000 7fdf527fc640 1 -- 172.21.15.89:0/723028121 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fdf54083740 con 0x7fdf5c09ca80 2023-10-18T11:03:34.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.236+0000 7fdf61c38640 1 --2- 172.21.15.89:0/723028121 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdf40065680 0x7fdf40067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:34.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.240+0000 7fdf61c38640 1 --2- 172.21.15.89:0/723028121 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdf40065680 0x7fdf40067b40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdf580aa000 tx=0x7fdf580b0040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:34.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.240+0000 7fdf527fc640 1 -- 172.21.15.89:0/723028121 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fdf5c1f5690 con 0x7fdf5c09ca80 2023-10-18T11:03:34.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.392+0000 7fdf63e9c640 1 -- 172.21.15.89:0/723028121 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdf5c1f5690 con 0x7fdf5c09ca80 2023-10-18T11:03:34.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.392+0000 7fdf527fc640 1 -- 172.21.15.89:0/723028121 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdf5c1f5690 con 0x7fdf5c09ca80 2023-10-18T11:03:34.390 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-18T11:03:34.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.392+0000 7fdf63e9c640 1 -- 172.21.15.89:0/723028121 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdf40065680 msgr2=0x7fdf40067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:34.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.392+0000 7fdf63e9c640 1 --2- 172.21.15.89:0/723028121 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdf40065680 0x7fdf40067b40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdf580aa000 tx=0x7fdf580b0040 comp rx=0 tx=0).stop 2023-10-18T11:03:34.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.392+0000 7fdf63e9c640 1 -- 172.21.15.89:0/723028121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf5c09ca80 msgr2=0x7fdf5c09cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:34.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.392+0000 7fdf63e9c640 1 --2- 172.21.15.89:0/723028121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf5c09ca80 0x7fdf5c09cf00 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c0b1310 tx=0x7fdf5400f580 comp rx=0 tx=0).stop 2023-10-18T11:03:34.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.392+0000 7fdf63e9c640 1 -- 172.21.15.89:0/723028121 shutdown_connections 2023-10-18T11:03:34.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.392+0000 7fdf63e9c640 1 --2- 172.21.15.89:0/723028121 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fdf40065680 0x7fdf40067b40 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:34.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.392+0000 7fdf63e9c640 1 --2- 172.21.15.89:0/723028121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf5c09ca80 0x7fdf5c09cf00 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:34.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.392+0000 7fdf63e9c640 1 --2- 172.21.15.89:0/723028121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf5c0d1aa0 0x7fdf5c0d1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:34.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.392+0000 7fdf63e9c640 1 --2- 172.21.15.89:0/723028121 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdf48003b10 0x7fdf5c15dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:34.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.392+0000 7fdf63e9c640 1 -- 172.21.15.89:0/723028121 >> 172.21.15.89:0/723028121 conn(0x7fdf5c09b9d0 msgr2=0x7fdf5c0b7c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:34.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.392+0000 7fdf63e9c640 1 -- 172.21.15.89:0/723028121 shutdown_connections 2023-10-18T11:03:34.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.392+0000 7fdf63e9c640 1 -- 172.21.15.89:0/723028121 wait complete. 2023-10-18T11:03:34.410 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T11:03:34.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.552+0000 7fe3117c8640 1 Processor -- start 2023-10-18T11:03:34.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.552+0000 7fe3117c8640 1 -- start start 2023-10-18T11:03:34.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.552+0000 7fe3117c8640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe30c0d0bd0 0x7fe30c167590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:34.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.552+0000 7fe3117c8640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe30c15ce40 con 0x7fe30c15ca60 2023-10-18T11:03:34.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.552+0000 7fe3117c8640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe30c0cd730 con 0x7fe30c0b17f0 2023-10-18T11:03:34.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.552+0000 7fe3117c8640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe30c0d1340 con 0x7fe30c0d0bd0 2023-10-18T11:03:34.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.552+0000 7fe30affd640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe30c15ca60 0x7fe30c0ccec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39566/0 (socket says 172.21.15.89:39566) 2023-10-18T11:03:34.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.552+0000 7fe30affd640 1 -- 172.21.15.89:0/488602154 learned_addr learned my addr 172.21.15.89:0/488602154 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:34.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.552+0000 7fe30b7fe640 1 --2- 172.21.15.89:0/488602154 >> v2:172.21.15.154:3300/0 conn(0x7fe30c0d0bd0 0x7fe30c167590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:34.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.552+0000 7fe309ffb640 1 -- 172.21.15.89:0/488602154 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 915791666 0 0) 0x7fe30c15ce40 con 0x7fe30c15ca60 2023-10-18T11:03:34.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.552+0000 7fe309ffb640 1 -- 172.21.15.89:0/488602154 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2f4003530 con 0x7fe30c15ca60 2023-10-18T11:03:34.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.552+0000 7fe309ffb640 1 -- 172.21.15.89:0/488602154 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1895698416 0 0) 0x7fe2f4003530 con 0x7fe30c15ca60 2023-10-18T11:03:34.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.552+0000 7fe309ffb640 1 -- 172.21.15.89:0/488602154 >> v2:172.21.15.154:3300/0 conn(0x7fe30c0d0bd0 msgr2=0x7fe30c167590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:34.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.552+0000 7fe309ffb640 1 --2- 172.21.15.89:0/488602154 >> v2:172.21.15.154:3300/0 conn(0x7fe30c0d0bd0 0x7fe30c167590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:34.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.552+0000 7fe309ffb640 1 -- 172.21.15.89:0/488602154 >> v1:172.21.15.154:6789/0 conn(0x7fe30c0b17f0 legacy=0x7fe30c0d04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:34.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.552+0000 7fe309ffb640 1 -- 172.21.15.89:0/488602154 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe30c0cda10 con 0x7fe30c15ca60 2023-10-18T11:03:34.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.552+0000 7fe309ffb640 1 -- 172.21.15.89:0/488602154 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7fe2fc001b80 con 0x7fe30c15ca60 2023-10-18T11:03:34.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.552+0000 7fe309ffb640 1 -- 172.21.15.89:0/488602154 >> v1:172.21.15.89:6789/0 conn(0x7fe30c15ca60 legacy=0x7fe30c0ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:34.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.552+0000 7fe309ffb640 1 --2- 172.21.15.89:0/488602154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2f4003980 0x7fe2f4003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:34.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.552+0000 7fe309ffb640 1 --2- 172.21.15.89:0/488602154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2f4004380 0x7fe2f4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:34.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.552+0000 7fe309ffb640 1 --2- 172.21.15.89:0/488602154 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe2f40050f0 0x7fe2f400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:34.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.552+0000 7fe309ffb640 1 -- 172.21.15.89:0/488602154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c0d1340 con 0x7fe2f4004380 2023-10-18T11:03:34.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.552+0000 7fe309ffb640 1 -- 172.21.15.89:0/488602154 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f400c030 con 0x7fe2f4003980 2023-10-18T11:03:34.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.552+0000 7fe309ffb640 1 -- 172.21.15.89:0/488602154 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f400c1f0 con 0x7fe2f40050f0 2023-10-18T11:03:34.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.552+0000 7fe30a7fc640 1 --2- 172.21.15.89:0/488602154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2f4004380 0x7fe2f4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:34.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.552+0000 7fe30affd640 1 --2- 172.21.15.89:0/488602154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2f4003980 0x7fe2f4003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:34.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.552+0000 7fe30b7fe640 1 --2- 172.21.15.89:0/488602154 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe2f40050f0 0x7fe2f400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:34.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.552+0000 7fe30a7fc640 1 -- 172.21.15.89:0/488602154 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe2f40050f0 msgr2=0x7fe2f400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:34.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.552+0000 7fe30a7fc640 1 --2- 172.21.15.89:0/488602154 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe2f40050f0 0x7fe2f400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:34.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.552+0000 7fe30a7fc640 1 -- 172.21.15.89:0/488602154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2f4003980 msgr2=0x7fe2f4003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:34.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.552+0000 7fe30a7fc640 1 --2- 172.21.15.89:0/488602154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2f4003980 0x7fe2f4003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:34.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.552+0000 7fe30a7fc640 1 -- 172.21.15.89:0/488602154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f4003530 con 0x7fe2f4004380 2023-10-18T11:03:34.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.552+0000 7fe30b7fe640 1 -- 172.21.15.89:0/488602154 reap_dead start 2023-10-18T11:03:34.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.556+0000 7fe30a7fc640 1 --2- 172.21.15.89:0/488602154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2f4004380 0x7fe2f4004850 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fe300001860 tx=0x7fe30001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d22a74f1fa210c05 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:34.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.556+0000 7fe309ffb640 1 -- 172.21.15.89:0/488602154 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3000ac630 con 0x7fe2f4004380 2023-10-18T11:03:34.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.556+0000 7fe309ffb640 1 -- 172.21.15.89:0/488602154 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3000ac790 con 0x7fe2f4004380 2023-10-18T11:03:34.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.556+0000 7fe309ffb640 1 -- 172.21.15.89:0/488602154 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3000acb80 con 0x7fe2f4004380 2023-10-18T11:03:34.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.556+0000 7fe3117c8640 1 -- 172.21.15.89:0/488602154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2f4004380 msgr2=0x7fe2f4004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:34.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.556+0000 7fe3117c8640 1 --2- 172.21.15.89:0/488602154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2f4004380 0x7fe2f4004850 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fe300001860 tx=0x7fe30001c490 comp rx=0 tx=0).stop 2023-10-18T11:03:34.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.556+0000 7fe3117c8640 1 -- 172.21.15.89:0/488602154 shutdown_connections 2023-10-18T11:03:34.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.556+0000 7fe3117c8640 1 --2- 172.21.15.89:0/488602154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2f4004380 0x7fe2f4004850 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:34.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.556+0000 7fe3117c8640 1 -- 172.21.15.89:0/488602154 >> 172.21.15.89:0/488602154 conn(0x7fe30c09b990 msgr2=0x7fe30c0d26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:34.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.556+0000 7fe3117c8640 1 -- 172.21.15.89:0/488602154 shutdown_connections 2023-10-18T11:03:34.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.556+0000 7fe3117c8640 1 -- 172.21.15.89:0/488602154 wait complete. 2023-10-18T11:03:34.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.560+0000 7fe3117c8640 1 Processor -- start 2023-10-18T11:03:34.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.560+0000 7fe3117c8640 1 -- start start 2023-10-18T11:03:34.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.560+0000 7fe3117c8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2f4004380 0x7fe30c0ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:34.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.560+0000 7fe3117c8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe30c0cb0a0 0x7fe30c158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:34.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.560+0000 7fe3117c8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe30c159730 0x7fe30c159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:34.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.560+0000 7fe3117c8640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c15a0f0 con 0x7fe2f4004380 2023-10-18T11:03:34.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.560+0000 7fe3117c8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c0cc490 con 0x7fe30c0cb0a0 2023-10-18T11:03:34.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.560+0000 7fe3117c8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c0cb880 con 0x7fe30c159730 2023-10-18T11:03:34.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.560+0000 7fe30affd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2f4004380 0x7fe30c0ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:34.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.560+0000 7fe30affd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2f4004380 0x7fe30c0ca6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51242/0 (socket says 172.21.15.89:51242) 2023-10-18T11:03:34.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.560+0000 7fe30affd640 1 -- 172.21.15.89:0/3622157881 learned_addr learned my addr 172.21.15.89:0/3622157881 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:34.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.560+0000 7fe30b7fe640 1 --2- 172.21.15.89:0/3622157881 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe30c159730 0x7fe30c159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:34.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.560+0000 7fe30a7fc640 1 --2- 172.21.15.89:0/3622157881 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe30c0cb0a0 0x7fe30c158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:34.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.560+0000 7fe30affd640 1 -- 172.21.15.89:0/3622157881 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe30c159730 msgr2=0x7fe30c159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:34.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.560+0000 7fe30affd640 1 --2- 172.21.15.89:0/3622157881 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe30c159730 0x7fe30c159bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:34.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.560+0000 7fe30affd640 1 -- 172.21.15.89:0/3622157881 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe30c0cb0a0 msgr2=0x7fe30c158bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:34.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.560+0000 7fe30affd640 1 --2- 172.21.15.89:0/3622157881 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe30c0cb0a0 0x7fe30c158bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:34.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.560+0000 7fe30affd640 1 -- 172.21.15.89:0/3622157881 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe30c0cbb60 con 0x7fe2f4004380 2023-10-18T11:03:34.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.560+0000 7fe30a7fc640 1 --2- 172.21.15.89:0/3622157881 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe30c0cb0a0 0x7fe30c158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:03:34.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.560+0000 7fe30affd640 1 --2- 172.21.15.89:0/3622157881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2f4004380 0x7fe30c0ca6d0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc001b70 tx=0x7fe2fc00d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:34.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.560+0000 7fe2f37fe640 1 -- 172.21.15.89:0/3622157881 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc00dbb0 con 0x7fe2f4004380 2023-10-18T11:03:34.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.560+0000 7fe2f37fe640 1 -- 172.21.15.89:0/3622157881 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2fc00dd10 con 0x7fe2f4004380 2023-10-18T11:03:34.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.560+0000 7fe2f37fe640 1 -- 172.21.15.89:0/3622157881 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc005d70 con 0x7fe2f4004380 2023-10-18T11:03:34.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.560+0000 7fe3117c8640 1 -- 172.21.15.89:0/3622157881 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe30c0cbe40 con 0x7fe2f4004380 2023-10-18T11:03:34.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.560+0000 7fe3117c8640 1 -- 172.21.15.89:0/3622157881 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe30c0cb520 con 0x7fe2f4004380 2023-10-18T11:03:34.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.564+0000 7fe3117c8640 1 -- 172.21.15.89:0/3622157881 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe30c04dbe0 con 0x7fe2f4004380 2023-10-18T11:03:34.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.564+0000 7fe2f37fe640 1 -- 172.21.15.89:0/3622157881 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fe2fc0070d0 con 0x7fe2f4004380 2023-10-18T11:03:34.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.564+0000 7fe2f37fe640 1 --2- 172.21.15.89:0/3622157881 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe2f8065680 0x7fe2f8067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:34.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.564+0000 7fe2f37fe640 1 -- 172.21.15.89:0/3622157881 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fe2fc083470 con 0x7fe2f4004380 2023-10-18T11:03:34.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.564+0000 7fe30a7fc640 1 --2- 172.21.15.89:0/3622157881 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe2f8065680 0x7fe2f8067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:34.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.564+0000 7fe30a7fc640 1 --2- 172.21.15.89:0/3622157881 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe2f8065680 0x7fe2f8067b40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe3000ae000 tx=0x7fe3000a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:34.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.568+0000 7fe2f37fe640 1 -- 172.21.15.89:0/3622157881 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fe2fc050e40 con 0x7fe2f4004380 2023-10-18T11:03:34.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.728+0000 7fe3117c8640 1 -- 172.21.15.89:0/3622157881 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe30c0b1540 con 0x7fe2f4004380 2023-10-18T11:03:34.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.728+0000 7fe2f37fe640 1 -- 172.21.15.89:0/3622157881 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 1 entries v56) v1 ==== 80+0+57 (secure 0 0 0) 0x7fe2fc054460 con 0x7fe2f4004380 2023-10-18T11:03:34.728 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/330049992 2023-10-18T11:55:17.981058+0000 2023-10-18T11:03:34.728 INFO:teuthology.orchestra.run.smithi089.stderr:listed 1 entries 2023-10-18T11:03:34.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.732+0000 7fe3117c8640 1 -- 172.21.15.89:0/3622157881 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe2f8065680 msgr2=0x7fe2f8067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:34.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.732+0000 7fe3117c8640 1 --2- 172.21.15.89:0/3622157881 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe2f8065680 0x7fe2f8067b40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe3000ae000 tx=0x7fe3000a7000 comp rx=0 tx=0).stop 2023-10-18T11:03:34.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.732+0000 7fe3117c8640 1 -- 172.21.15.89:0/3622157881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2f4004380 msgr2=0x7fe30c0ca6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:34.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.732+0000 7fe3117c8640 1 --2- 172.21.15.89:0/3622157881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2f4004380 0x7fe30c0ca6d0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc001b70 tx=0x7fe2fc00d520 comp rx=0 tx=0).stop 2023-10-18T11:03:34.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.732+0000 7fe3117c8640 1 -- 172.21.15.89:0/3622157881 shutdown_connections 2023-10-18T11:03:34.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.732+0000 7fe3117c8640 1 --2- 172.21.15.89:0/3622157881 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe2f8065680 0x7fe2f8067b40 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:34.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.732+0000 7fe3117c8640 1 --2- 172.21.15.89:0/3622157881 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe30c159730 0x7fe30c159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:34.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.732+0000 7fe3117c8640 1 --2- 172.21.15.89:0/3622157881 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe30c0cb0a0 0x7fe30c158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:34.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.732+0000 7fe3117c8640 1 --2- 172.21.15.89:0/3622157881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2f4004380 0x7fe30c0ca6d0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:34.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.732+0000 7fe3117c8640 1 -- 172.21.15.89:0/3622157881 >> 172.21.15.89:0/3622157881 conn(0x7fe30c09b990 msgr2=0x7fe30c06b060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:34.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.732+0000 7fe3117c8640 1 -- 172.21.15.89:0/3622157881 shutdown_connections 2023-10-18T11:03:34.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:34.732+0000 7fe3117c8640 1 -- 172.21.15.89:0/3622157881 wait complete. 2023-10-18T11:03:34.748 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-18T11:03:34.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:03:34.749 DEBUG:teuthology.orchestra.run.smithi089:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-18T11:03:34.778 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi089.front.sepia.ceph.com 2023-10-18T11:03:34.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:03:34.779 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-18T11:03:34.829 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T11:03:34.829 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:03:34.840 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T11:03:34.840 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:03:34.891 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T11:03:35.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.028+0000 7fee9eda3640 1 Processor -- start 2023-10-18T11:03:35.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.032+0000 7fee9eda3640 1 -- start start 2023-10-18T11:03:35.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.032+0000 7fee9eda3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fee9806ff00 0x7fee9815e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:35.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.032+0000 7fee9eda3640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fee980ca340 0x7fee9815b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:35.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.032+0000 7fee9eda3640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee9815acb0 con 0x7fee9815b680 2023-10-18T11:03:35.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.032+0000 7fee9eda3640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee9815ef30 con 0x7fee980ca340 2023-10-18T11:03:35.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.032+0000 7fee9eda3640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee9815aa10 con 0x7fee9806ff00 2023-10-18T11:03:35.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.032+0000 7fee97fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fee980ca340 0x7fee9815b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:35.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.032+0000 7fee97fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fee980ca340 0x7fee9815b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51248/0 (socket says 172.21.15.89:51248) 2023-10-18T11:03:35.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.032+0000 7fee97fff640 1 -- 172.21.15.89:0/498924072 learned_addr learned my addr 172.21.15.89:0/498924072 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:35.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.032+0000 7fee9cb3f640 1 --2- 172.21.15.89:0/498924072 >> v2:172.21.15.154:3300/0 conn(0x7fee9806ff00 0x7fee9815e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:35.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.032+0000 7fee97fff640 1 -- 172.21.15.89:0/498924072 >> v2:172.21.15.154:3300/0 conn(0x7fee9806ff00 msgr2=0x7fee9815e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:35.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.032+0000 7fee97fff640 1 --2- 172.21.15.89:0/498924072 >> v2:172.21.15.154:3300/0 conn(0x7fee9806ff00 0x7fee9815e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:35.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.032+0000 7fee97fff640 1 -- 172.21.15.89:0/498924072 >> v1:172.21.15.154:6789/0 conn(0x7fee9815b680 legacy=0x7fee980c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:35.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.032+0000 7fee97fff640 1 -- 172.21.15.89:0/498924072 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee980702e0 con 0x7fee980ca340 2023-10-18T11:03:35.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.036+0000 7fee97fff640 1 --2- 172.21.15.89:0/498924072 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee980ca340 0x7fee9815b140 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fee9806a830 tx=0x7fee8c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81bb74f86d0d084d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:35.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.036+0000 7fee977fe640 1 -- 172.21.15.89:0/498924072 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c0ab600 con 0x7fee980ca340 2023-10-18T11:03:35.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.036+0000 7fee977fe640 1 -- 172.21.15.89:0/498924072 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee8c0ab760 con 0x7fee980ca340 2023-10-18T11:03:35.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.036+0000 7fee977fe640 1 -- 172.21.15.89:0/498924072 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c0abaf0 con 0x7fee980ca340 2023-10-18T11:03:35.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.036+0000 7fee9eda3640 1 -- 172.21.15.89:0/498924072 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee980ca340 msgr2=0x7fee9815b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:35.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.036+0000 7fee9eda3640 1 --2- 172.21.15.89:0/498924072 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee980ca340 0x7fee9815b140 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fee9806a830 tx=0x7fee8c01c400 comp rx=0 tx=0).stop 2023-10-18T11:03:35.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.040+0000 7fee9eda3640 1 -- 172.21.15.89:0/498924072 shutdown_connections 2023-10-18T11:03:35.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.040+0000 7fee9eda3640 1 --2- 172.21.15.89:0/498924072 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee980ca340 0x7fee9815b140 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:35.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.040+0000 7fee9eda3640 1 --2- 172.21.15.89:0/498924072 >> v2:172.21.15.154:3300/0 conn(0x7fee9806ff00 0x7fee9815e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:35.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.040+0000 7fee9eda3640 1 -- 172.21.15.89:0/498924072 >> 172.21.15.89:0/498924072 conn(0x7fee98069f00 msgr2=0x7fee980cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:35.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.040+0000 7fee9eda3640 1 -- 172.21.15.89:0/498924072 shutdown_connections 2023-10-18T11:03:35.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.040+0000 7fee9eda3640 1 -- 172.21.15.89:0/498924072 wait complete. 2023-10-18T11:03:35.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.040+0000 7fee9eda3640 1 Processor -- start 2023-10-18T11:03:35.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.040+0000 7fee9eda3640 1 -- start start 2023-10-18T11:03:35.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.040+0000 7fee9eda3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee9806ff00 0x7fee980b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:35.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.040+0000 7fee9eda3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee980ca340 0x7fee9814c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:35.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.040+0000 7fee9eda3640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee980cdc60 0x7fee98159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:35.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.040+0000 7fee9eda3640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee980aec30 con 0x7fee980cdc60 2023-10-18T11:03:35.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.040+0000 7fee9eda3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee980aef10 con 0x7fee9806ff00 2023-10-18T11:03:35.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.040+0000 7fee9eda3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee980cc080 con 0x7fee980ca340 2023-10-18T11:03:35.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.040+0000 7fee9d340640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee980cdc60 0x7fee98159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:35.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.040+0000 7fee9d340640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee980cdc60 0x7fee98159690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51262/0 (socket says 172.21.15.89:51262) 2023-10-18T11:03:35.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.040+0000 7fee9d340640 1 -- 172.21.15.89:0/2274461974 learned_addr learned my addr 172.21.15.89:0/2274461974 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:35.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.040+0000 7fee9cb3f640 1 --2- 172.21.15.89:0/2274461974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee9806ff00 0x7fee980b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:35.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.040+0000 7fee97fff640 1 --2- 172.21.15.89:0/2274461974 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee980ca340 0x7fee9814c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:35.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.040+0000 7fee9d340640 1 -- 172.21.15.89:0/2274461974 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee980ca340 msgr2=0x7fee9814c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:35.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.040+0000 7fee9d340640 1 --2- 172.21.15.89:0/2274461974 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee980ca340 0x7fee9814c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:35.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.040+0000 7fee9d340640 1 -- 172.21.15.89:0/2274461974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee9806ff00 msgr2=0x7fee980b0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:35.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.040+0000 7fee9d340640 1 --2- 172.21.15.89:0/2274461974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee9806ff00 0x7fee980b0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:35.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.044+0000 7fee9d340640 1 -- 172.21.15.89:0/2274461974 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee8c0094d0 con 0x7fee980cdc60 2023-10-18T11:03:35.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.044+0000 7fee9d340640 1 --2- 172.21.15.89:0/2274461974 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee980cdc60 0x7fee98159690 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fee9000b410 tx=0x7fee9000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:35.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.044+0000 7fee957fa640 1 -- 172.21.15.89:0/2274461974 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee90004020 con 0x7fee980cdc60 2023-10-18T11:03:35.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.044+0000 7fee957fa640 1 -- 172.21.15.89:0/2274461974 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee90004180 con 0x7fee980cdc60 2023-10-18T11:03:35.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.044+0000 7fee9eda3640 1 -- 172.21.15.89:0/2274461974 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee9814c0a0 con 0x7fee980cdc60 2023-10-18T11:03:35.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.044+0000 7fee957fa640 1 -- 172.21.15.89:0/2274461974 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee90011630 con 0x7fee980cdc60 2023-10-18T11:03:35.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.044+0000 7fee9eda3640 1 -- 172.21.15.89:0/2274461974 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee98159da0 con 0x7fee980cdc60 2023-10-18T11:03:35.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.044+0000 7fee9eda3640 1 -- 172.21.15.89:0/2274461974 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee980cbee0 con 0x7fee980cdc60 2023-10-18T11:03:35.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.044+0000 7fee957fa640 1 -- 172.21.15.89:0/2274461974 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fee90011790 con 0x7fee980cdc60 2023-10-18T11:03:35.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.044+0000 7fee957fa640 1 --2- 172.21.15.89:0/2274461974 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fee84065680 0x7fee84067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:35.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.044+0000 7fee957fa640 1 -- 172.21.15.89:0/2274461974 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fee90082a90 con 0x7fee980cdc60 2023-10-18T11:03:35.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.044+0000 7fee9cb3f640 1 --2- 172.21.15.89:0/2274461974 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fee84065680 0x7fee84067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:35.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.048+0000 7fee9cb3f640 1 --2- 172.21.15.89:0/2274461974 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fee84065680 0x7fee84067b40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fee980b0d60 tx=0x7fee880098d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:35.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.048+0000 7fee957fa640 1 -- 172.21.15.89:0/2274461974 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fee980cbee0 con 0x7fee980cdc60 2023-10-18T11:03:35.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.200+0000 7fee9eda3640 1 -- 172.21.15.89:0/2274461974 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee980cbee0 con 0x7fee980cdc60 2023-10-18T11:03:35.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.200+0000 7fee957fa640 1 -- 172.21.15.89:0/2274461974 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+18666 (secure 0 0 0) 0x7fee980cbee0 con 0x7fee980cdc60 2023-10-18T11:03:35.200 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T11:03:35.200 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":56,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:55:22.907577+0000","last_up_change":"2023-10-18T10:55:21.905065+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:47:13.188701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-18T10:47:14.198224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":50,"up_from":55,"up_thru":55,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":1133012599},{"type":"v1","addr":"172.21.15.154:6824","nonce":1133012599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1133012599},{"type":"v1","addr":"172.21.15.154:6827","nonce":1133012599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1133012599},{"type":"v1","addr":"172.21.15.154:6831","nonce":1133012599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1133012599},{"type":"v1","addr":"172.21.15.154:6829","nonce":1133012599}]},"public_addr":"172.21.15.154:6824/1133012599","cluster_addr":"172.21.15.154:6827/1133012599","heartbeat_back_addr":"172.21.15.154:6831/1133012599","heartbeat_front_addr":"172.21.15.154:6829/1133012599","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"2023-10-18T10:47:48.955023+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":51},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.9","mappings":[{"from":0,"to":5}]},{"pgid":"7.18","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:0/330049992":"2023-10-18T11:55:17.981058+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T11:03:35.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.204+0000 7fee9eda3640 1 -- 172.21.15.89:0/2274461974 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fee84065680 msgr2=0x7fee84067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:35.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.204+0000 7fee9eda3640 1 --2- 172.21.15.89:0/2274461974 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fee84065680 0x7fee84067b40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fee980b0d60 tx=0x7fee880098d0 comp rx=0 tx=0).stop 2023-10-18T11:03:35.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.204+0000 7fee9eda3640 1 -- 172.21.15.89:0/2274461974 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee980cdc60 msgr2=0x7fee98159690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:35.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.204+0000 7fee9eda3640 1 --2- 172.21.15.89:0/2274461974 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee980cdc60 0x7fee98159690 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fee9000b410 tx=0x7fee9000b8d0 comp rx=0 tx=0).stop 2023-10-18T11:03:35.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.204+0000 7fee9eda3640 1 -- 172.21.15.89:0/2274461974 shutdown_connections 2023-10-18T11:03:35.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.204+0000 7fee9eda3640 1 --2- 172.21.15.89:0/2274461974 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fee84065680 0x7fee84067b40 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:35.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.204+0000 7fee9eda3640 1 --2- 172.21.15.89:0/2274461974 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee980ca340 0x7fee9814c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:35.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.204+0000 7fee9eda3640 1 --2- 172.21.15.89:0/2274461974 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee980cdc60 0x7fee98159690 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:35.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.204+0000 7fee9eda3640 1 --2- 172.21.15.89:0/2274461974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee9806ff00 0x7fee980b0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:35.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.204+0000 7fee9eda3640 1 -- 172.21.15.89:0/2274461974 >> 172.21.15.89:0/2274461974 conn(0x7fee98069f00 msgr2=0x7fee9815bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:35.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.204+0000 7fee9eda3640 1 -- 172.21.15.89:0/2274461974 shutdown_connections 2023-10-18T11:03:35.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.204+0000 7fee9eda3640 1 -- 172.21.15.89:0/2274461974 wait complete. 2023-10-18T11:03:35.221 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:45:43.114184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-18T10:45:48.730968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:47:13.188701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:47:14.198224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T11:03:35.222 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T11:03:35.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.364+0000 7fe4249a3640 1 Processor -- start 2023-10-18T11:03:35.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.364+0000 7fe4249a3640 1 -- start start 2023-10-18T11:03:35.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.368+0000 7fe4249a3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe420145b70 0x7fe420143470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:35.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.368+0000 7fe4249a3640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4200cd8d0 con 0x7fe4200ce070 2023-10-18T11:03:35.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.368+0000 7fe4249a3640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe42015cbf0 con 0x7fe4200cd4f0 2023-10-18T11:03:35.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.368+0000 7fe4249a3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe420069900 con 0x7fe420145b70 2023-10-18T11:03:35.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.368+0000 7fe41ed9d640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe4200ce070 0x7fe42006abb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39580/0 (socket says 172.21.15.89:39580) 2023-10-18T11:03:35.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.368+0000 7fe41ed9d640 1 -- 172.21.15.89:0/906267081 learned_addr learned my addr 172.21.15.89:0/906267081 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:35.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.368+0000 7fe41dd9b640 1 --2- 172.21.15.89:0/906267081 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe420145b70 0x7fe420143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:35.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.368+0000 7fe41d59a640 1 -- 172.21.15.89:0/906267081 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4231199865 0 0) 0x7fe4200cd8d0 con 0x7fe4200ce070 2023-10-18T11:03:35.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.368+0000 7fe41d59a640 1 -- 172.21.15.89:0/906267081 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe40c003530 con 0x7fe4200ce070 2023-10-18T11:03:35.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.368+0000 7fe41d59a640 1 -- 172.21.15.89:0/906267081 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 10802120 0 0) 0x7fe42015cbf0 con 0x7fe4200cd4f0 2023-10-18T11:03:35.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.368+0000 7fe41dd9b640 1 -- 172.21.15.89:0/906267081 >> v1:172.21.15.154:6789/0 conn(0x7fe4200cd4f0 legacy=0x7fe420160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:35.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.368+0000 7fe41dd9b640 1 -- 172.21.15.89:0/906267081 >> v1:172.21.15.89:6789/0 conn(0x7fe4200ce070 legacy=0x7fe42006abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:35.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.368+0000 7fe41dd9b640 1 -- 172.21.15.89:0/906267081 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe420069b10 con 0x7fe420145b70 2023-10-18T11:03:35.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.368+0000 7fe41dd9b640 1 --2- 172.21.15.89:0/906267081 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe420145b70 0x7fe420143470 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe4080097f0 tx=0x7fe40801c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff3fd29df13d93a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:35.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.372+0000 7fe41d59a640 1 -- 172.21.15.89:0/906267081 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4080aa5f0 con 0x7fe420145b70 2023-10-18T11:03:35.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.372+0000 7fe41d59a640 1 -- 172.21.15.89:0/906267081 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4080aa750 con 0x7fe420145b70 2023-10-18T11:03:35.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.372+0000 7fe41d59a640 1 -- 172.21.15.89:0/906267081 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4080aaae0 con 0x7fe420145b70 2023-10-18T11:03:35.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.372+0000 7fe4249a3640 1 -- 172.21.15.89:0/906267081 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe420145b70 msgr2=0x7fe420143470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:35.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.372+0000 7fe4249a3640 1 --2- 172.21.15.89:0/906267081 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe420145b70 0x7fe420143470 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe4080097f0 tx=0x7fe40801c490 comp rx=0 tx=0).stop 2023-10-18T11:03:35.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.372+0000 7fe4249a3640 1 -- 172.21.15.89:0/906267081 shutdown_connections 2023-10-18T11:03:35.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.372+0000 7fe4249a3640 1 --2- 172.21.15.89:0/906267081 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe420145b70 0x7fe420143470 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:35.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.372+0000 7fe4249a3640 1 -- 172.21.15.89:0/906267081 >> 172.21.15.89:0/906267081 conn(0x7fe42006dad0 msgr2=0x7fe42006dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:35.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.372+0000 7fe4249a3640 1 -- 172.21.15.89:0/906267081 shutdown_connections 2023-10-18T11:03:35.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.372+0000 7fe4249a3640 1 -- 172.21.15.89:0/906267081 wait complete. 2023-10-18T11:03:35.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.372+0000 7fe4249a3640 1 Processor -- start 2023-10-18T11:03:35.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.372+0000 7fe4249a3640 1 -- start start 2023-10-18T11:03:35.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.372+0000 7fe4249a3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4200cd4f0 0x7fe420159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:35.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.372+0000 7fe4249a3640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4200ce070 0x7fe4201580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:35.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.372+0000 7fe4249a3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe420145b70 0x7fe42015a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:35.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.376+0000 7fe4249a3640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4200b4130 con 0x7fe4200ce070 2023-10-18T11:03:35.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.376+0000 7fe4249a3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42015a010 con 0x7fe4200cd4f0 2023-10-18T11:03:35.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.376+0000 7fe4249a3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4201585e0 con 0x7fe420145b70 2023-10-18T11:03:35.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.376+0000 7fe41dd9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4200ce070 0x7fe4201580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:35.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.376+0000 7fe41dd9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4200ce070 0x7fe4201580a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51266/0 (socket says 172.21.15.89:51266) 2023-10-18T11:03:35.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.376+0000 7fe41dd9b640 1 -- 172.21.15.89:0/2100565331 learned_addr learned my addr 172.21.15.89:0/2100565331 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:35.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.376+0000 7fe41e59c640 1 --2- 172.21.15.89:0/2100565331 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4200cd4f0 0x7fe420159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:35.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.376+0000 7fe41ed9d640 1 --2- 172.21.15.89:0/2100565331 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe420145b70 0x7fe42015a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:35.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.376+0000 7fe41dd9b640 1 -- 172.21.15.89:0/2100565331 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe420145b70 msgr2=0x7fe42015a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:35.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.376+0000 7fe41dd9b640 1 --2- 172.21.15.89:0/2100565331 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe420145b70 0x7fe42015a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:35.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.376+0000 7fe41dd9b640 1 -- 172.21.15.89:0/2100565331 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4200cd4f0 msgr2=0x7fe420159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:35.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.376+0000 7fe41dd9b640 1 --2- 172.21.15.89:0/2100565331 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4200cd4f0 0x7fe420159ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:35.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.376+0000 7fe41dd9b640 1 -- 172.21.15.89:0/2100565331 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe408009420 con 0x7fe4200ce070 2023-10-18T11:03:35.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.376+0000 7fe41dd9b640 1 --2- 172.21.15.89:0/2100565331 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4200ce070 0x7fe4201580a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe4080097f0 tx=0x7fe40801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:35.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.376+0000 7fe416ffd640 1 -- 172.21.15.89:0/2100565331 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4080b4600 con 0x7fe4200ce070 2023-10-18T11:03:35.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.376+0000 7fe416ffd640 1 -- 172.21.15.89:0/2100565331 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4080b4760 con 0x7fe4200ce070 2023-10-18T11:03:35.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.376+0000 7fe416ffd640 1 -- 172.21.15.89:0/2100565331 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4080b4b10 con 0x7fe4200ce070 2023-10-18T11:03:35.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.376+0000 7fe4249a3640 1 -- 172.21.15.89:0/2100565331 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4200b5c50 con 0x7fe4200ce070 2023-10-18T11:03:35.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.376+0000 7fe4249a3640 1 -- 172.21.15.89:0/2100565331 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4200b3770 con 0x7fe4200ce070 2023-10-18T11:03:35.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.376+0000 7fe4249a3640 1 -- 172.21.15.89:0/2100565331 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe42004dbe0 con 0x7fe4200ce070 2023-10-18T11:03:35.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.376+0000 7fe416ffd640 1 -- 172.21.15.89:0/2100565331 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fe4080b4c70 con 0x7fe4200ce070 2023-10-18T11:03:35.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.380+0000 7fe416ffd640 1 --2- 172.21.15.89:0/2100565331 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe400065680 0x7fe400067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:35.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.380+0000 7fe416ffd640 1 -- 172.21.15.89:0/2100565331 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fe4080a7070 con 0x7fe4200ce070 2023-10-18T11:03:35.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.380+0000 7fe41e59c640 1 --2- 172.21.15.89:0/2100565331 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe400065680 0x7fe400067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:35.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.380+0000 7fe41e59c640 1 --2- 172.21.15.89:0/2100565331 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe400065680 0x7fe400067b40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe410000f70 tx=0x7fe410009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:35.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.384+0000 7fe416ffd640 1 -- 172.21.15.89:0/2100565331 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fe4080eae70 con 0x7fe4200ce070 2023-10-18T11:03:35.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.532+0000 7fe4249a3640 1 -- 172.21.15.89:0/2100565331 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe4200b2d80 con 0x7fe4200ce070 2023-10-18T11:03:35.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.532+0000 7fe416ffd640 1 -- 172.21.15.89:0/2100565331 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe4080ee490 con 0x7fe4200ce070 2023-10-18T11:03:35.532 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-18T11:03:35.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.536+0000 7fe4249a3640 1 -- 172.21.15.89:0/2100565331 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe400065680 msgr2=0x7fe400067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:35.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.536+0000 7fe4249a3640 1 --2- 172.21.15.89:0/2100565331 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe400065680 0x7fe400067b40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe410000f70 tx=0x7fe410009040 comp rx=0 tx=0).stop 2023-10-18T11:03:35.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.536+0000 7fe4249a3640 1 -- 172.21.15.89:0/2100565331 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4200ce070 msgr2=0x7fe4201580a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:35.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.536+0000 7fe4249a3640 1 --2- 172.21.15.89:0/2100565331 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4200ce070 0x7fe4201580a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe4080097f0 tx=0x7fe40801acd0 comp rx=0 tx=0).stop 2023-10-18T11:03:35.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.536+0000 7fe4249a3640 1 -- 172.21.15.89:0/2100565331 shutdown_connections 2023-10-18T11:03:35.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.536+0000 7fe4249a3640 1 --2- 172.21.15.89:0/2100565331 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe400065680 0x7fe400067b40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:35.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.536+0000 7fe4249a3640 1 --2- 172.21.15.89:0/2100565331 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe420145b70 0x7fe42015a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:35.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.536+0000 7fe4249a3640 1 --2- 172.21.15.89:0/2100565331 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4200ce070 0x7fe4201580a0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:35.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.536+0000 7fe4249a3640 1 --2- 172.21.15.89:0/2100565331 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4200cd4f0 0x7fe420159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:35.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.536+0000 7fe4249a3640 1 -- 172.21.15.89:0/2100565331 >> 172.21.15.89:0/2100565331 conn(0x7fe42006dad0 msgr2=0x7fe42015d060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:35.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.536+0000 7fe4249a3640 1 -- 172.21.15.89:0/2100565331 shutdown_connections 2023-10-18T11:03:35.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.536+0000 7fe4249a3640 1 -- 172.21.15.89:0/2100565331 wait complete. 2023-10-18T11:03:35.552 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-18T11:03:35.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.696+0000 7f093b893640 1 Processor -- start 2023-10-18T11:03:35.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.696+0000 7f093b893640 1 -- start start 2023-10-18T11:03:35.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.700+0000 7f093b893640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0934145b70 0x7f0934143470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:35.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.700+0000 7f093b893640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09340ce070 0x7f09340ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:35.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.700+0000 7f093b893640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09340cd8d0 con 0x7f09340cd4f0 2023-10-18T11:03:35.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.700+0000 7f093b893640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0934069900 con 0x7f0934145b70 2023-10-18T11:03:35.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.700+0000 7f093b893640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09340cea20 con 0x7f09340ce070 2023-10-18T11:03:35.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.700+0000 7f093962f640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f09340cd4f0 0x7f0934160cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39596/0 (socket says 172.21.15.89:39596) 2023-10-18T11:03:35.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.700+0000 7f093962f640 1 -- 172.21.15.89:0/1602158708 learned_addr learned my addr 172.21.15.89:0/1602158708 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:35.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.700+0000 7f0938e2e640 1 --2- 172.21.15.89:0/1602158708 >> v2:172.21.15.154:3300/0 conn(0x7f0934145b70 0x7f0934143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:35.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.700+0000 7f0939e30640 1 --2- 172.21.15.89:0/1602158708 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09340ce070 0x7f09340ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:35.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.700+0000 7f092bfff640 1 -- 172.21.15.89:0/1602158708 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 422574032 0 0) 0x7f09340cd8d0 con 0x7f09340cd4f0 2023-10-18T11:03:35.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.700+0000 7f092bfff640 1 -- 172.21.15.89:0/1602158708 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0920003530 con 0x7f09340cd4f0 2023-10-18T11:03:35.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.700+0000 7f092bfff640 1 -- 172.21.15.89:0/1602158708 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1039067627 0 0) 0x7f0920003530 con 0x7f09340cd4f0 2023-10-18T11:03:35.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.700+0000 7f092bfff640 1 -- 172.21.15.89:0/1602158708 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09340ce070 msgr2=0x7f09340ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:35.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.700+0000 7f092bfff640 1 --2- 172.21.15.89:0/1602158708 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09340ce070 0x7f09340ce4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:35.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.700+0000 7f092bfff640 1 -- 172.21.15.89:0/1602158708 >> v2:172.21.15.154:3300/0 conn(0x7f0934145b70 msgr2=0x7f0934143470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:35.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.700+0000 7f092bfff640 1 --2- 172.21.15.89:0/1602158708 >> v2:172.21.15.154:3300/0 conn(0x7f0934145b70 0x7f0934143470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:35.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.700+0000 7f092bfff640 1 -- 172.21.15.89:0/1602158708 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093415cbf0 con 0x7f09340cd4f0 2023-10-18T11:03:35.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.700+0000 7f092bfff640 1 -- 172.21.15.89:0/1602158708 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7f0930001a00 con 0x7f09340cd4f0 2023-10-18T11:03:35.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.700+0000 7f092bfff640 1 -- 172.21.15.89:0/1602158708 >> v1:172.21.15.89:6789/0 conn(0x7f09340cd4f0 legacy=0x7f0934160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:35.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.700+0000 7f092bfff640 1 --2- 172.21.15.89:0/1602158708 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09340ce070 0x7f09200039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:35.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.700+0000 7f092bfff640 1 --2- 172.21.15.89:0/1602158708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0920003f20 0x7f09200047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:35.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.700+0000 7f092bfff640 1 --2- 172.21.15.89:0/1602158708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0920004d10 0x7f0920005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:35.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.700+0000 7f092bfff640 1 -- 172.21.15.89:0/1602158708 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0934069900 con 0x7f0920004d10 2023-10-18T11:03:35.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.700+0000 7f092bfff640 1 -- 172.21.15.89:0/1602158708 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09340cea20 con 0x7f0920003f20 2023-10-18T11:03:35.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.700+0000 7f092bfff640 1 -- 172.21.15.89:0/1602158708 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f092000b990 con 0x7f09340ce070 2023-10-18T11:03:35.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.700+0000 7f093962f640 1 --2- 172.21.15.89:0/1602158708 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09340ce070 0x7f09200039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:35.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.700+0000 7f0939e30640 1 --2- 172.21.15.89:0/1602158708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0920004d10 0x7f0920005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:35.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.700+0000 7f0938e2e640 1 --2- 172.21.15.89:0/1602158708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0920003f20 0x7f09200047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:35.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.700+0000 7f0939e30640 1 -- 172.21.15.89:0/1602158708 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09340ce070 msgr2=0x7f09200039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:35.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.700+0000 7f0939e30640 1 --2- 172.21.15.89:0/1602158708 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09340ce070 0x7f09200039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:35.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.700+0000 7f0939e30640 1 -- 172.21.15.89:0/1602158708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0920003f20 msgr2=0x7f09200047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:35.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.700+0000 7f0939e30640 1 --2- 172.21.15.89:0/1602158708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0920003f20 0x7f09200047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:35.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.700+0000 7f0939e30640 1 -- 172.21.15.89:0/1602158708 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0920003530 con 0x7f0920004d10 2023-10-18T11:03:35.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.700+0000 7f093962f640 1 --2- 172.21.15.89:0/1602158708 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09340ce070 0x7f09200039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T11:03:35.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.704+0000 7f0939e30640 1 --2- 172.21.15.89:0/1602158708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0920004d10 0x7f0920005190 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f092c009b10 tx=0x7f092c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7751c54555cebd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:35.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.704+0000 7f092bfff640 1 -- 172.21.15.89:0/1602158708 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092c0a5640 con 0x7f0920004d10 2023-10-18T11:03:35.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.704+0000 7f092bfff640 1 -- 172.21.15.89:0/1602158708 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092c0a57a0 con 0x7f0920004d10 2023-10-18T11:03:35.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.704+0000 7f092bfff640 1 -- 172.21.15.89:0/1602158708 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092c0a5b50 con 0x7f0920004d10 2023-10-18T11:03:35.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.704+0000 7f093b893640 1 -- 172.21.15.89:0/1602158708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0920004d10 msgr2=0x7f0920005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:35.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.704+0000 7f093b893640 1 --2- 172.21.15.89:0/1602158708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0920004d10 0x7f0920005190 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f092c009b10 tx=0x7f092c01ca50 comp rx=0 tx=0).stop 2023-10-18T11:03:35.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.704+0000 7f0939e30640 1 -- 172.21.15.89:0/1602158708 reap_dead start 2023-10-18T11:03:35.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.704+0000 7f093b893640 1 -- 172.21.15.89:0/1602158708 shutdown_connections 2023-10-18T11:03:35.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.704+0000 7f093b893640 1 -- 172.21.15.89:0/1602158708 >> 172.21.15.89:0/1602158708 conn(0x7f093406dad0 msgr2=0x7f093406dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:35.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.704+0000 7f093b893640 1 -- 172.21.15.89:0/1602158708 shutdown_connections 2023-10-18T11:03:35.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.704+0000 7f093b893640 1 -- 172.21.15.89:0/1602158708 wait complete. 2023-10-18T11:03:35.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.708+0000 7f093b893640 1 Processor -- start 2023-10-18T11:03:35.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.708+0000 7f093b893640 1 -- start start 2023-10-18T11:03:35.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.708+0000 7f093b893640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0920004d10 0x7f09340c8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:35.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.708+0000 7f093b893640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0934156c10 0x7f09340c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:35.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.708+0000 7f093b893640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09340d0740 0x7f09340d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:35.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.708+0000 7f093b893640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09341e5180 con 0x7f0934156c10 2023-10-18T11:03:35.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.708+0000 7f093b893640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09340d1100 con 0x7f09340d0740 2023-10-18T11:03:35.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.708+0000 7f093b893640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09340c9ed0 con 0x7f0920004d10 2023-10-18T11:03:35.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.708+0000 7f0938e2e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0934156c10 0x7f09340c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:35.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.708+0000 7f0938e2e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0934156c10 0x7f09340c95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51280/0 (socket says 172.21.15.89:51280) 2023-10-18T11:03:35.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.708+0000 7f0938e2e640 1 -- 172.21.15.89:0/22146663 learned_addr learned my addr 172.21.15.89:0/22146663 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:35.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.708+0000 7f0939e30640 1 --2- 172.21.15.89:0/22146663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09340d0740 0x7f09340d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:35.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.708+0000 7f093962f640 1 --2- 172.21.15.89:0/22146663 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0920004d10 0x7f09340c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:35.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.708+0000 7f0938e2e640 1 -- 172.21.15.89:0/22146663 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0920004d10 msgr2=0x7f09340c8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:35.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.708+0000 7f0938e2e640 1 --2- 172.21.15.89:0/22146663 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0920004d10 0x7f09340c8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:35.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.708+0000 7f0938e2e640 1 -- 172.21.15.89:0/22146663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09340d0740 msgr2=0x7f09340d0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:35.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.708+0000 7f0938e2e640 1 --2- 172.21.15.89:0/22146663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09340d0740 0x7f09340d0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:35.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.708+0000 7f0938e2e640 1 -- 172.21.15.89:0/22146663 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09340ca1b0 con 0x7f0934156c10 2023-10-18T11:03:35.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.708+0000 7f0938e2e640 1 --2- 172.21.15.89:0/22146663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0934156c10 0x7f09340c95f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f09240119d0 tx=0x7f0924011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:35.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.708+0000 7f0929ffb640 1 -- 172.21.15.89:0/22146663 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092400faf0 con 0x7f0934156c10 2023-10-18T11:03:35.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.708+0000 7f0929ffb640 1 -- 172.21.15.89:0/22146663 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092400fc50 con 0x7f0934156c10 2023-10-18T11:03:35.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.708+0000 7f0929ffb640 1 -- 172.21.15.89:0/22146663 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0924014440 con 0x7f0934156c10 2023-10-18T11:03:35.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.708+0000 7f093b893640 1 -- 172.21.15.89:0/22146663 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09341e5360 con 0x7f0934156c10 2023-10-18T11:03:35.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.708+0000 7f093b893640 1 -- 172.21.15.89:0/22146663 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09340c9b30 con 0x7f0934156c10 2023-10-18T11:03:35.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.708+0000 7f093b893640 1 -- 172.21.15.89:0/22146663 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09340ca3d0 con 0x7f0934156c10 2023-10-18T11:03:35.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.712+0000 7f0929ffb640 1 -- 172.21.15.89:0/22146663 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f09240145a0 con 0x7f0934156c10 2023-10-18T11:03:35.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.712+0000 7f0929ffb640 1 --2- 172.21.15.89:0/22146663 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0918065680 0x7f0918067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:35.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.712+0000 7f0929ffb640 1 -- 172.21.15.89:0/22146663 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f09240838a0 con 0x7f0934156c10 2023-10-18T11:03:35.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.712+0000 7f093962f640 1 --2- 172.21.15.89:0/22146663 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0918065680 0x7f0918067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:35.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.712+0000 7f093962f640 1 --2- 172.21.15.89:0/22146663 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0918065680 0x7f0918067b40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0930004d10 tx=0x7f0930006040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:35.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.716+0000 7f0929ffb640 1 -- 172.21.15.89:0/22146663 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f0924051140 con 0x7f0934156c10 2023-10-18T11:03:35.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.864+0000 7f093b893640 1 -- 172.21.15.89:0/22146663 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f093404dbe0 con 0x7f0934156c10 2023-10-18T11:03:35.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.864+0000 7f0929ffb640 1 -- 172.21.15.89:0/22146663 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v56) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0924022020 con 0x7f0934156c10 2023-10-18T11:03:35.863 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-10-18T11:03:35.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.864+0000 7f093b893640 1 -- 172.21.15.89:0/22146663 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0918065680 msgr2=0x7f0918067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:35.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.864+0000 7f093b893640 1 --2- 172.21.15.89:0/22146663 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0918065680 0x7f0918067b40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0930004d10 tx=0x7f0930006040 comp rx=0 tx=0).stop 2023-10-18T11:03:35.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.864+0000 7f093b893640 1 -- 172.21.15.89:0/22146663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0934156c10 msgr2=0x7f09340c95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:35.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.864+0000 7f093b893640 1 --2- 172.21.15.89:0/22146663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0934156c10 0x7f09340c95f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f09240119d0 tx=0x7f0924011dd0 comp rx=0 tx=0).stop 2023-10-18T11:03:35.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.868+0000 7f093b893640 1 -- 172.21.15.89:0/22146663 shutdown_connections 2023-10-18T11:03:35.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.868+0000 7f093b893640 1 --2- 172.21.15.89:0/22146663 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0918065680 0x7f0918067b40 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:35.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.868+0000 7f093b893640 1 --2- 172.21.15.89:0/22146663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09340d0740 0x7f09340d0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:35.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.868+0000 7f093b893640 1 --2- 172.21.15.89:0/22146663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0934156c10 0x7f09340c95f0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:35.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.868+0000 7f093b893640 1 --2- 172.21.15.89:0/22146663 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0920004d10 0x7f09340c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:35.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.868+0000 7f093b893640 1 -- 172.21.15.89:0/22146663 >> 172.21.15.89:0/22146663 conn(0x7f093406dad0 msgr2=0x7f093415b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:35.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.868+0000 7f093b893640 1 -- 172.21.15.89:0/22146663 shutdown_connections 2023-10-18T11:03:35.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:35.868+0000 7f093b893640 1 -- 172.21.15.89:0/22146663 wait complete. 2023-10-18T11:03:35.883 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T11:03:36.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f676b59e640 1 Processor -- start 2023-10-18T11:03:36.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f676b59e640 1 -- start start 2023-10-18T11:03:36.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f676b59e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f676c071870 0x7f676c070980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:36.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f676b59e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f676c06b970 0x7f676c06be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:36.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f676b59e640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f676c070ec0 con 0x7f676c0c9a80 2023-10-18T11:03:36.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f676b59e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f676c06c340 con 0x7f676c06b970 2023-10-18T11:03:36.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f676b59e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f676c159070 con 0x7f676c071870 2023-10-18T11:03:36.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f676a59c640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f676c0c9a80 0x7f676c0c86a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39602/0 (socket says 172.21.15.89:39602) 2023-10-18T11:03:36.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f676a59c640 1 -- 172.21.15.89:0/2835097623 learned_addr learned my addr 172.21.15.89:0/2835097623 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:36.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f6769d9b640 1 --2- 172.21.15.89:0/2835097623 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f676c071870 0x7f676c070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:36.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f676ad9d640 1 --2- 172.21.15.89:0/2835097623 >> v2:172.21.15.154:3300/0 conn(0x7f676c06b970 0x7f676c06be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:36.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f676959a640 1 -- 172.21.15.89:0/2835097623 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1314845260 0 0) 0x7f676c070ec0 con 0x7f676c0c9a80 2023-10-18T11:03:36.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f676959a640 1 -- 172.21.15.89:0/2835097623 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6754004640 con 0x7f676c0c9a80 2023-10-18T11:03:36.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f676959a640 1 -- 172.21.15.89:0/2835097623 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3276283897 0 0) 0x7f6754004640 con 0x7f676c0c9a80 2023-10-18T11:03:36.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f676959a640 1 -- 172.21.15.89:0/2835097623 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f676c071870 msgr2=0x7f676c070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:36.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f676959a640 1 --2- 172.21.15.89:0/2835097623 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f676c071870 0x7f676c070980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:36.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f676959a640 1 -- 172.21.15.89:0/2835097623 >> v2:172.21.15.154:3300/0 conn(0x7f676c06b970 msgr2=0x7f676c06be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:36.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f676959a640 1 --2- 172.21.15.89:0/2835097623 >> v2:172.21.15.154:3300/0 conn(0x7f676c06b970 0x7f676c06be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:36.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f676959a640 1 -- 172.21.15.89:0/2835097623 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f676c0c8db0 con 0x7f676c0c9a80 2023-10-18T11:03:36.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f676959a640 1 -- 172.21.15.89:0/2835097623 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7f675c001680 con 0x7f676c0c9a80 2023-10-18T11:03:36.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f676959a640 1 -- 172.21.15.89:0/2835097623 >> v1:172.21.15.89:6789/0 conn(0x7f676c0c9a80 legacy=0x7f676c0c86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:36.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f676959a640 1 --2- 172.21.15.89:0/2835097623 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6754004b00 0x7f6754004f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:36.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f676959a640 1 --2- 172.21.15.89:0/2835097623 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f676c071870 0x7f67540058e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:36.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f676959a640 1 --2- 172.21.15.89:0/2835097623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6754005e20 0x7f67540062a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:36.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f676959a640 1 -- 172.21.15.89:0/2835097623 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c06c340 con 0x7f6754004b00 2023-10-18T11:03:36.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f676959a640 1 -- 172.21.15.89:0/2835097623 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c159070 con 0x7f6754005e20 2023-10-18T11:03:36.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f676959a640 1 -- 172.21.15.89:0/2835097623 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67540034a0 con 0x7f676c071870 2023-10-18T11:03:36.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f676a59c640 1 --2- 172.21.15.89:0/2835097623 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6754004b00 0x7f6754004f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:36.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f6769d9b640 1 --2- 172.21.15.89:0/2835097623 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f676c071870 0x7f67540058e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:36.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f676ad9d640 1 --2- 172.21.15.89:0/2835097623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6754005e20 0x7f67540062a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:36.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f676a59c640 1 -- 172.21.15.89:0/2835097623 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f676c071870 msgr2=0x7f67540058e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:36.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f676a59c640 1 --2- 172.21.15.89:0/2835097623 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f676c071870 0x7f67540058e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:36.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f676a59c640 1 -- 172.21.15.89:0/2835097623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6754005e20 msgr2=0x7f67540062a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:36.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f676a59c640 1 --2- 172.21.15.89:0/2835097623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6754005e20 0x7f67540062a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:36.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f676a59c640 1 -- 172.21.15.89:0/2835097623 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6754004640 con 0x7f6754004b00 2023-10-18T11:03:36.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.028+0000 7f6769d9b640 1 --2- 172.21.15.89:0/2835097623 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f676c071870 0x7f67540058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:03:36.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.032+0000 7f676a59c640 1 --2- 172.21.15.89:0/2835097623 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6754004b00 0x7f6754004f20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f675c00c240 tx=0x7f675c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16254339f82c7522 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:36.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.032+0000 7f676959a640 1 -- 172.21.15.89:0/2835097623 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675c0058e0 con 0x7f6754004b00 2023-10-18T11:03:36.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.032+0000 7f676959a640 1 -- 172.21.15.89:0/2835097623 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f675c005a40 con 0x7f6754004b00 2023-10-18T11:03:36.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.032+0000 7f676959a640 1 -- 172.21.15.89:0/2835097623 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675c005e10 con 0x7f6754004b00 2023-10-18T11:03:36.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.032+0000 7f676b59e640 1 -- 172.21.15.89:0/2835097623 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6754004b00 msgr2=0x7f6754004f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:36.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.032+0000 7f676b59e640 1 --2- 172.21.15.89:0/2835097623 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6754004b00 0x7f6754004f20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f675c00c240 tx=0x7f675c01cc60 comp rx=0 tx=0).stop 2023-10-18T11:03:36.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.032+0000 7f676ad9d640 1 -- 172.21.15.89:0/2835097623 reap_dead start 2023-10-18T11:03:36.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.032+0000 7f676b59e640 1 -- 172.21.15.89:0/2835097623 shutdown_connections 2023-10-18T11:03:36.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.032+0000 7f676b59e640 1 -- 172.21.15.89:0/2835097623 >> 172.21.15.89:0/2835097623 conn(0x7f676c0afbc0 msgr2=0x7f676c0affd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:36.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.032+0000 7f676b59e640 1 -- 172.21.15.89:0/2835097623 shutdown_connections 2023-10-18T11:03:36.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.032+0000 7f676b59e640 1 -- 172.21.15.89:0/2835097623 wait complete. 2023-10-18T11:03:36.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.036+0000 7f676b59e640 1 Processor -- start 2023-10-18T11:03:36.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.036+0000 7f676b59e640 1 -- start start 2023-10-18T11:03:36.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.036+0000 7f676b59e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6754004b00 0x7f676c0c0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:36.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.036+0000 7f676b59e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f676c159230 0x7f676c1596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:36.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.036+0000 7f676b59e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f676c0c61e0 0x7f676c0c6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:36.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.036+0000 7f676b59e640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c0c7a40 con 0x7f6754004b00 2023-10-18T11:03:36.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.036+0000 7f676b59e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c159bf0 con 0x7f676c0c61e0 2023-10-18T11:03:36.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.036+0000 7f676b59e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f676c0c6ba0 con 0x7f676c159230 2023-10-18T11:03:36.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.036+0000 7f676a59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6754004b00 0x7f676c0c0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:36.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.036+0000 7f676a59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6754004b00 0x7f676c0c0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51298/0 (socket says 172.21.15.89:51298) 2023-10-18T11:03:36.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.036+0000 7f676a59c640 1 -- 172.21.15.89:0/1439129658 learned_addr learned my addr 172.21.15.89:0/1439129658 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:36.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.036+0000 7f6769d9b640 1 --2- 172.21.15.89:0/1439129658 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f676c159230 0x7f676c1596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:36.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.036+0000 7f676ad9d640 1 --2- 172.21.15.89:0/1439129658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f676c0c61e0 0x7f676c0c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:36.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.036+0000 7f676a59c640 1 -- 172.21.15.89:0/1439129658 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f676c159230 msgr2=0x7f676c1596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:36.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.036+0000 7f676a59c640 1 --2- 172.21.15.89:0/1439129658 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f676c159230 0x7f676c1596b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:36.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.036+0000 7f676a59c640 1 -- 172.21.15.89:0/1439129658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f676c0c61e0 msgr2=0x7f676c0c6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:36.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.036+0000 7f676a59c640 1 --2- 172.21.15.89:0/1439129658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f676c0c61e0 0x7f676c0c6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:36.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.036+0000 7f676a59c640 1 -- 172.21.15.89:0/1439129658 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f676c154870 con 0x7f6754004b00 2023-10-18T11:03:36.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.036+0000 7f676ad9d640 1 --2- 172.21.15.89:0/1439129658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f676c0c61e0 0x7f676c0c6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:03:36.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.036+0000 7f676a59c640 1 --2- 172.21.15.89:0/1439129658 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6754004b00 0x7f676c0c0b50 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f675c0ab040 tx=0x7f6754006910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:36.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.036+0000 7f675affd640 1 -- 172.21.15.89:0/1439129658 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675c005e70 con 0x7f6754004b00 2023-10-18T11:03:36.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.036+0000 7f675affd640 1 -- 172.21.15.89:0/1439129658 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f675c0047f0 con 0x7f6754004b00 2023-10-18T11:03:36.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.036+0000 7f675affd640 1 -- 172.21.15.89:0/1439129658 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675c004b60 con 0x7f6754004b00 2023-10-18T11:03:36.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.036+0000 7f676b59e640 1 -- 172.21.15.89:0/1439129658 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f676c0c6dc0 con 0x7f6754004b00 2023-10-18T11:03:36.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.036+0000 7f676b59e640 1 -- 172.21.15.89:0/1439129658 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f676c0c7380 con 0x7f6754004b00 2023-10-18T11:03:36.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.040+0000 7f676b59e640 1 -- 172.21.15.89:0/1439129658 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f676c0cde50 con 0x7f6754004b00 2023-10-18T11:03:36.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.040+0000 7f675affd640 1 -- 172.21.15.89:0/1439129658 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f675c004cc0 con 0x7f6754004b00 2023-10-18T11:03:36.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.040+0000 7f675affd640 1 --2- 172.21.15.89:0/1439129658 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f674c065680 0x7f674c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:36.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.040+0000 7f675affd640 1 -- 172.21.15.89:0/1439129658 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f675c11c930 con 0x7f6754004b00 2023-10-18T11:03:36.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.040+0000 7f6769d9b640 1 --2- 172.21.15.89:0/1439129658 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f674c065680 0x7f674c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:36.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.040+0000 7f6769d9b640 1 --2- 172.21.15.89:0/1439129658 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f674c065680 0x7f674c067b40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f676c0ca920 tx=0x7f6760008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:36.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.044+0000 7f675affd640 1 -- 172.21.15.89:0/1439129658 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f675c0ea230 con 0x7f6754004b00 2023-10-18T11:03:36.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.196+0000 7f676b59e640 1 -- 172.21.15.89:0/1439129658 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f676c073b50 con 0x7f6754004b00 2023-10-18T11:03:36.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.200+0000 7f675affd640 1 -- 172.21.15.89:0/1439129658 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) v1 ==== 104+0+11 (secure 0 0 0) 0x7f675c0ed850 con 0x7f6754004b00 2023-10-18T11:03:36.198 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-18T11:03:36.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.200+0000 7f676b59e640 1 -- 172.21.15.89:0/1439129658 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f674c065680 msgr2=0x7f674c067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:36.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.200+0000 7f676b59e640 1 --2- 172.21.15.89:0/1439129658 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f674c065680 0x7f674c067b40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f676c0ca920 tx=0x7f6760008040 comp rx=0 tx=0).stop 2023-10-18T11:03:36.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.200+0000 7f676b59e640 1 -- 172.21.15.89:0/1439129658 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6754004b00 msgr2=0x7f676c0c0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:36.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.200+0000 7f676b59e640 1 --2- 172.21.15.89:0/1439129658 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6754004b00 0x7f676c0c0b50 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f675c0ab040 tx=0x7f6754006910 comp rx=0 tx=0).stop 2023-10-18T11:03:36.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.200+0000 7f676b59e640 1 -- 172.21.15.89:0/1439129658 shutdown_connections 2023-10-18T11:03:36.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.200+0000 7f676b59e640 1 --2- 172.21.15.89:0/1439129658 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f674c065680 0x7f674c067b40 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:36.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.200+0000 7f676b59e640 1 --2- 172.21.15.89:0/1439129658 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f676c159230 0x7f676c1596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:36.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.200+0000 7f676b59e640 1 --2- 172.21.15.89:0/1439129658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f676c0c61e0 0x7f676c0c6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:36.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.200+0000 7f676b59e640 1 --2- 172.21.15.89:0/1439129658 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6754004b00 0x7f676c0c0b50 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:36.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.200+0000 7f676b59e640 1 -- 172.21.15.89:0/1439129658 >> 172.21.15.89:0/1439129658 conn(0x7f676c0afbc0 msgr2=0x7f676c0c8680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:36.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.200+0000 7f676b59e640 1 -- 172.21.15.89:0/1439129658 shutdown_connections 2023-10-18T11:03:36.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.200+0000 7f676b59e640 1 -- 172.21.15.89:0/1439129658 wait complete. 2023-10-18T11:03:36.218 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T11:03:36.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.360+0000 7fb38885b640 1 Processor -- start 2023-10-18T11:03:36.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.360+0000 7fb38885b640 1 -- start start 2023-10-18T11:03:36.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.360+0000 7fb38885b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb38415c330 0x7fb3840d1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:36.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.360+0000 7fb38885b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb38415c710 0x7fb3840b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:36.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.360+0000 7fb38885b640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3840d0070 con 0x7fb3840b44f0 2023-10-18T11:03:36.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.360+0000 7fb38885b640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3840d1650 con 0x7fb38415c710 2023-10-18T11:03:36.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.360+0000 7fb38885b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3840d0350 con 0x7fb38415c330 2023-10-18T11:03:36.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.360+0000 7fb37affd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb38415c710 0x7fb3840b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:36.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.360+0000 7fb383fff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb3840b44f0 0x7fb3840d19e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39618/0 (socket says 172.21.15.89:39618) 2023-10-18T11:03:36.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.360+0000 7fb383fff640 1 -- 172.21.15.89:0/1056580453 learned_addr learned my addr 172.21.15.89:0/1056580453 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:36.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.360+0000 7fb3837fe640 1 --2- 172.21.15.89:0/1056580453 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb38415c330 0x7fb3840d1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:36.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.360+0000 7fb37affd640 1 -- 172.21.15.89:0/1056580453 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb38415c330 msgr2=0x7fb3840d1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:36.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.360+0000 7fb37affd640 1 --2- 172.21.15.89:0/1056580453 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb38415c330 0x7fb3840d1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:36.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.360+0000 7fb37affd640 1 -- 172.21.15.89:0/1056580453 >> v1:172.21.15.89:6789/0 conn(0x7fb3840b44f0 legacy=0x7fb3840d19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:36.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.360+0000 7fb37affd640 1 -- 172.21.15.89:0/1056580453 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3840d0630 con 0x7fb38415c710 2023-10-18T11:03:36.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.364+0000 7fb37affd640 1 --2- 172.21.15.89:0/1056580453 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb38415c710 0x7fb3840b3fb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fb36c0097c0 tx=0x7fb36c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff9be366a8a4513 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:36.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.364+0000 7fb382ffd640 1 -- 172.21.15.89:0/1056580453 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36c0ab600 con 0x7fb38415c710 2023-10-18T11:03:36.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.364+0000 7fb382ffd640 1 -- 172.21.15.89:0/1056580453 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb36c0a7050 con 0x7fb38415c710 2023-10-18T11:03:36.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.364+0000 7fb382ffd640 1 -- 172.21.15.89:0/1056580453 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36c0ab980 con 0x7fb38415c710 2023-10-18T11:03:36.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.364+0000 7fb38885b640 1 -- 172.21.15.89:0/1056580453 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb38415c710 msgr2=0x7fb3840b3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:36.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.364+0000 7fb38885b640 1 --2- 172.21.15.89:0/1056580453 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb38415c710 0x7fb3840b3fb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fb36c0097c0 tx=0x7fb36c01c3f0 comp rx=0 tx=0).stop 2023-10-18T11:03:36.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.364+0000 7fb38885b640 1 -- 172.21.15.89:0/1056580453 shutdown_connections 2023-10-18T11:03:36.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.364+0000 7fb38885b640 1 --2- 172.21.15.89:0/1056580453 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb38415c710 0x7fb3840b3fb0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:36.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.364+0000 7fb38885b640 1 --2- 172.21.15.89:0/1056580453 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb38415c330 0x7fb3840d1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:36.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.364+0000 7fb38885b640 1 -- 172.21.15.89:0/1056580453 >> 172.21.15.89:0/1056580453 conn(0x7fb384070320 msgr2=0x7fb384070730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:36.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.364+0000 7fb38885b640 1 -- 172.21.15.89:0/1056580453 shutdown_connections 2023-10-18T11:03:36.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.364+0000 7fb38885b640 1 -- 172.21.15.89:0/1056580453 wait complete. 2023-10-18T11:03:36.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.368+0000 7fb38885b640 1 Processor -- start 2023-10-18T11:03:36.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.368+0000 7fb38885b640 1 -- start start 2023-10-18T11:03:36.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.368+0000 7fb38885b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3840b44f0 0x7fb38415f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:36.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.368+0000 7fb38885b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb38415c330 0x7fb38415cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:36.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.368+0000 7fb38885b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb38415c710 0x7fb3840d2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:36.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.368+0000 7fb38885b640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38415faf0 con 0x7fb3840b44f0 2023-10-18T11:03:36.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.368+0000 7fb38885b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38415f8b0 con 0x7fb38415c710 2023-10-18T11:03:36.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.368+0000 7fb38885b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3840cad70 con 0x7fb38415c330 2023-10-18T11:03:36.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.368+0000 7fb3837fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3840b44f0 0x7fb38415f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:36.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.368+0000 7fb3837fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3840b44f0 0x7fb38415f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51316/0 (socket says 172.21.15.89:51316) 2023-10-18T11:03:36.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.368+0000 7fb3837fe640 1 -- 172.21.15.89:0/172673221 learned_addr learned my addr 172.21.15.89:0/172673221 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:36.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.368+0000 7fb37affd640 1 --2- 172.21.15.89:0/172673221 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb38415c330 0x7fb38415cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:36.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.368+0000 7fb383fff640 1 --2- 172.21.15.89:0/172673221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb38415c710 0x7fb3840d2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:36.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.368+0000 7fb3837fe640 1 -- 172.21.15.89:0/172673221 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb38415c330 msgr2=0x7fb38415cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:36.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.368+0000 7fb3837fe640 1 --2- 172.21.15.89:0/172673221 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb38415c330 0x7fb38415cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:36.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.368+0000 7fb3837fe640 1 -- 172.21.15.89:0/172673221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb38415c710 msgr2=0x7fb3840d2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:36.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.368+0000 7fb3837fe640 1 --2- 172.21.15.89:0/172673221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb38415c710 0x7fb3840d2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:36.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.368+0000 7fb3837fe640 1 -- 172.21.15.89:0/172673221 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb36c009420 con 0x7fb3840b44f0 2023-10-18T11:03:36.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.368+0000 7fb383fff640 1 --2- 172.21.15.89:0/172673221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb38415c710 0x7fb3840d2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:03:36.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.368+0000 7fb3837fe640 1 --2- 172.21.15.89:0/172673221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3840b44f0 0x7fb38415f370 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fb37400d830 tx=0x7fb37400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:36.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.368+0000 7fb380ff9640 1 -- 172.21.15.89:0/172673221 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37400b660 con 0x7fb3840b44f0 2023-10-18T11:03:36.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.368+0000 7fb380ff9640 1 -- 172.21.15.89:0/172673221 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37400b7c0 con 0x7fb3840b44f0 2023-10-18T11:03:36.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.368+0000 7fb380ff9640 1 -- 172.21.15.89:0/172673221 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37400bb30 con 0x7fb3840b44f0 2023-10-18T11:03:36.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.368+0000 7fb38885b640 1 -- 172.21.15.89:0/172673221 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb384155070 con 0x7fb3840b44f0 2023-10-18T11:03:36.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.368+0000 7fb38885b640 1 -- 172.21.15.89:0/172673221 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb384155630 con 0x7fb3840b44f0 2023-10-18T11:03:36.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.372+0000 7fb38885b640 1 -- 172.21.15.89:0/172673221 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb384066970 con 0x7fb3840b44f0 2023-10-18T11:03:36.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.372+0000 7fb380ff9640 1 -- 172.21.15.89:0/172673221 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fb37401e460 con 0x7fb3840b44f0 2023-10-18T11:03:36.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.372+0000 7fb380ff9640 1 --2- 172.21.15.89:0/172673221 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb364065680 0x7fb364067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:36.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.372+0000 7fb380ff9640 1 -- 172.21.15.89:0/172673221 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fb3740836c0 con 0x7fb3840b44f0 2023-10-18T11:03:36.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.372+0000 7fb37affd640 1 --2- 172.21.15.89:0/172673221 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb364065680 0x7fb364067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:36.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.372+0000 7fb37affd640 1 --2- 172.21.15.89:0/172673221 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb364065680 0x7fb364067b40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb36c0ad000 tx=0x7fb36c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:36.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.376+0000 7fb380ff9640 1 -- 172.21.15.89:0/172673221 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fb374050fc0 con 0x7fb3840b44f0 2023-10-18T11:03:36.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.520+0000 7fb38885b640 1 -- 172.21.15.89:0/172673221 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb3840cc850 con 0x7fb3840b44f0 2023-10-18T11:03:36.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.520+0000 7fb380ff9640 1 -- 172.21.15.89:0/172673221 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb3740545e0 con 0x7fb3840b44f0 2023-10-18T11:03:36.520 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-18T11:03:36.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.524+0000 7fb38885b640 1 -- 172.21.15.89:0/172673221 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb364065680 msgr2=0x7fb364067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:36.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.524+0000 7fb38885b640 1 --2- 172.21.15.89:0/172673221 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb364065680 0x7fb364067b40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb36c0ad000 tx=0x7fb36c008040 comp rx=0 tx=0).stop 2023-10-18T11:03:36.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.524+0000 7fb38885b640 1 -- 172.21.15.89:0/172673221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3840b44f0 msgr2=0x7fb38415f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:36.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.524+0000 7fb38885b640 1 --2- 172.21.15.89:0/172673221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3840b44f0 0x7fb38415f370 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fb37400d830 tx=0x7fb37400dcf0 comp rx=0 tx=0).stop 2023-10-18T11:03:36.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.524+0000 7fb38885b640 1 -- 172.21.15.89:0/172673221 shutdown_connections 2023-10-18T11:03:36.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.524+0000 7fb38885b640 1 --2- 172.21.15.89:0/172673221 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fb364065680 0x7fb364067b40 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:36.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.524+0000 7fb38885b640 1 --2- 172.21.15.89:0/172673221 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb38415c330 0x7fb38415cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:36.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.524+0000 7fb38885b640 1 --2- 172.21.15.89:0/172673221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb38415c710 0x7fb3840d2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:36.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.524+0000 7fb38885b640 1 --2- 172.21.15.89:0/172673221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3840b44f0 0x7fb38415f370 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:36.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.524+0000 7fb38885b640 1 -- 172.21.15.89:0/172673221 >> 172.21.15.89:0/172673221 conn(0x7fb384070320 msgr2=0x7fb3841501c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:36.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.524+0000 7fb38885b640 1 -- 172.21.15.89:0/172673221 shutdown_connections 2023-10-18T11:03:36.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.524+0000 7fb38885b640 1 -- 172.21.15.89:0/172673221 wait complete. 2023-10-18T11:03:36.541 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T11:03:36.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.680+0000 7f2e50a5f640 1 Processor -- start 2023-10-18T11:03:36.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.680+0000 7f2e50a5f640 1 -- start start 2023-10-18T11:03:36.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.684+0000 7f2e50a5f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e4c159490 0x7f2e4c15d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:36.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.684+0000 7f2e50a5f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2e4c06bee0 0x7f2e4c0c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:36.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.684+0000 7f2e50a5f640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e4c15ddf0 con 0x7f2e4c070600 2023-10-18T11:03:36.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.684+0000 7f2e50a5f640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e4c070b90 con 0x7f2e4c06bee0 2023-10-18T11:03:36.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.684+0000 7f2e50a5f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c070e70 con 0x7f2e4c159490 2023-10-18T11:03:36.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.684+0000 7f2e4ad9d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2e4c06bee0 0x7f2e4c0c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:36.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.684+0000 7f2e4ad9d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2e4c06bee0 0x7f2e4c0c8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51320/0 (socket says 172.21.15.89:51320) 2023-10-18T11:03:36.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.684+0000 7f2e4ad9d640 1 -- 172.21.15.89:0/280306526 learned_addr learned my addr 172.21.15.89:0/280306526 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:36.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.684+0000 7f2e4a59c640 1 --2- 172.21.15.89:0/280306526 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e4c159490 0x7f2e4c15d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:36.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.684+0000 7f2e4ad9d640 1 -- 172.21.15.89:0/280306526 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e4c159490 msgr2=0x7f2e4c15d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:36.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.684+0000 7f2e4ad9d640 1 --2- 172.21.15.89:0/280306526 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e4c159490 0x7f2e4c15d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:36.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.684+0000 7f2e4ad9d640 1 -- 172.21.15.89:0/280306526 >> v1:172.21.15.154:6789/0 conn(0x7f2e4c070600 legacy=0x7f2e4c06b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:36.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.684+0000 7f2e4ad9d640 1 -- 172.21.15.89:0/280306526 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e4c071150 con 0x7f2e4c06bee0 2023-10-18T11:03:36.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.688+0000 7f2e4ad9d640 1 --2- 172.21.15.89:0/280306526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e4c06bee0 0x7f2e4c0c8960 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2e44009e50 tx=0x7f2e4401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e876cee9fe531a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:36.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.688+0000 7f2e49d9b640 1 -- 172.21.15.89:0/280306526 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e440a5810 con 0x7f2e4c06bee0 2023-10-18T11:03:36.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.688+0000 7f2e49d9b640 1 -- 172.21.15.89:0/280306526 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e440a5970 con 0x7f2e4c06bee0 2023-10-18T11:03:36.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.688+0000 7f2e49d9b640 1 -- 172.21.15.89:0/280306526 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e440a5d00 con 0x7f2e4c06bee0 2023-10-18T11:03:36.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.688+0000 7f2e50a5f640 1 -- 172.21.15.89:0/280306526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e4c06bee0 msgr2=0x7f2e4c0c8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:36.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.688+0000 7f2e50a5f640 1 --2- 172.21.15.89:0/280306526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e4c06bee0 0x7f2e4c0c8960 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2e44009e50 tx=0x7f2e4401c950 comp rx=0 tx=0).stop 2023-10-18T11:03:36.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.688+0000 7f2e50a5f640 1 -- 172.21.15.89:0/280306526 shutdown_connections 2023-10-18T11:03:36.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.688+0000 7f2e50a5f640 1 --2- 172.21.15.89:0/280306526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e4c06bee0 0x7f2e4c0c8960 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:36.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.688+0000 7f2e50a5f640 1 --2- 172.21.15.89:0/280306526 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e4c159490 0x7f2e4c15d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:36.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.688+0000 7f2e50a5f640 1 -- 172.21.15.89:0/280306526 >> 172.21.15.89:0/280306526 conn(0x7f2e4c069f00 msgr2=0x7f2e4c0cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:36.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.688+0000 7f2e50a5f640 1 -- 172.21.15.89:0/280306526 shutdown_connections 2023-10-18T11:03:36.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.688+0000 7f2e50a5f640 1 -- 172.21.15.89:0/280306526 wait complete. 2023-10-18T11:03:36.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.688+0000 7f2e50a5f640 1 Processor -- start 2023-10-18T11:03:36.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.688+0000 7f2e50a5f640 1 -- start start 2023-10-18T11:03:36.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.692+0000 7f2e50a5f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e4c06bee0 0x7f2e4c0c2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:36.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.692+0000 7f2e50a5f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e4c070600 0x7f2e4c1529d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:36.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.692+0000 7f2e50a5f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e4c159490 0x7f2e4c157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:36.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.692+0000 7f2e50a5f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c153fb0 con 0x7f2e4c070600 2023-10-18T11:03:36.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.692+0000 7f2e50a5f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c0c2d90 con 0x7f2e4c06bee0 2023-10-18T11:03:36.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.692+0000 7f2e50a5f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c0c3060 con 0x7f2e4c159490 2023-10-18T11:03:36.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.692+0000 7f2e41d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e4c070600 0x7f2e4c1529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:36.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.692+0000 7f2e41d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e4c070600 0x7f2e4c1529d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51328/0 (socket says 172.21.15.89:51328) 2023-10-18T11:03:36.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.692+0000 7f2e41d9b640 1 -- 172.21.15.89:0/292047345 learned_addr learned my addr 172.21.15.89:0/292047345 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:36.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.692+0000 7f2e4a59c640 1 --2- 172.21.15.89:0/292047345 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e4c06bee0 0x7f2e4c0c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:36.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.692+0000 7f2e4ad9d640 1 --2- 172.21.15.89:0/292047345 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e4c159490 0x7f2e4c157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:36.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.692+0000 7f2e41d9b640 1 -- 172.21.15.89:0/292047345 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e4c159490 msgr2=0x7f2e4c157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:36.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.692+0000 7f2e41d9b640 1 --2- 172.21.15.89:0/292047345 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e4c159490 0x7f2e4c157370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:36.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.692+0000 7f2e41d9b640 1 -- 172.21.15.89:0/292047345 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e4c06bee0 msgr2=0x7f2e4c0c2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:36.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.692+0000 7f2e41d9b640 1 --2- 172.21.15.89:0/292047345 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e4c06bee0 0x7f2e4c0c2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:36.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.692+0000 7f2e41d9b640 1 -- 172.21.15.89:0/292047345 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e44009b30 con 0x7f2e4c070600 2023-10-18T11:03:36.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.692+0000 7f2e41d9b640 1 --2- 172.21.15.89:0/292047345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e4c070600 0x7f2e4c1529d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400c500 tx=0x7f2e3400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:36.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.692+0000 7f2e437fe640 1 -- 172.21.15.89:0/292047345 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e34005220 con 0x7f2e4c070600 2023-10-18T11:03:36.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.692+0000 7f2e437fe640 1 -- 172.21.15.89:0/292047345 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e34005380 con 0x7f2e4c070600 2023-10-18T11:03:36.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.692+0000 7f2e437fe640 1 -- 172.21.15.89:0/292047345 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e34007270 con 0x7f2e4c070600 2023-10-18T11:03:36.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.692+0000 7f2e50a5f640 1 -- 172.21.15.89:0/292047345 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e4c0c3340 con 0x7f2e4c070600 2023-10-18T11:03:36.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.692+0000 7f2e50a5f640 1 -- 172.21.15.89:0/292047345 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e4c154200 con 0x7f2e4c070600 2023-10-18T11:03:36.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.692+0000 7f2e50a5f640 1 -- 172.21.15.89:0/292047345 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e4c154420 con 0x7f2e4c070600 2023-10-18T11:03:36.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.692+0000 7f2e437fe640 1 -- 172.21.15.89:0/292047345 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f2e340073d0 con 0x7f2e4c070600 2023-10-18T11:03:36.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.696+0000 7f2e437fe640 1 --2- 172.21.15.89:0/292047345 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2e2c065680 0x7f2e2c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:36.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.696+0000 7f2e437fe640 1 -- 172.21.15.89:0/292047345 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f2e34082870 con 0x7f2e4c070600 2023-10-18T11:03:36.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.696+0000 7f2e4a59c640 1 --2- 172.21.15.89:0/292047345 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2e2c065680 0x7f2e2c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:36.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.696+0000 7f2e4a59c640 1 --2- 172.21.15.89:0/292047345 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2e2c065680 0x7f2e2c067b40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0c3b20 tx=0x7f2e3c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:36.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.700+0000 7f2e437fe640 1 -- 172.21.15.89:0/292047345 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f2e340501a0 con 0x7f2e4c070600 2023-10-18T11:03:36.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.860+0000 7f2e50a5f640 1 -- 172.21.15.89:0/292047345 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2e4c04dbe0 con 0x7f2e4c070600 2023-10-18T11:03:36.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.860+0000 7f2e437fe640 1 -- 172.21.15.89:0/292047345 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 1 entries v56) v1 ==== 80+0+57 (secure 0 0 0) 0x7f2e340537c0 con 0x7f2e4c070600 2023-10-18T11:03:36.860 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/330049992 2023-10-18T11:55:17.981058+0000 2023-10-18T11:03:36.861 INFO:teuthology.orchestra.run.smithi089.stderr:listed 1 entries 2023-10-18T11:03:36.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.864+0000 7f2e50a5f640 1 -- 172.21.15.89:0/292047345 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2e2c065680 msgr2=0x7f2e2c067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:36.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.864+0000 7f2e50a5f640 1 --2- 172.21.15.89:0/292047345 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2e2c065680 0x7f2e2c067b40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0c3b20 tx=0x7f2e3c008040 comp rx=0 tx=0).stop 2023-10-18T11:03:36.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.864+0000 7f2e50a5f640 1 -- 172.21.15.89:0/292047345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e4c070600 msgr2=0x7f2e4c1529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:36.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.864+0000 7f2e50a5f640 1 --2- 172.21.15.89:0/292047345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e4c070600 0x7f2e4c1529d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400c500 tx=0x7f2e3400c9c0 comp rx=0 tx=0).stop 2023-10-18T11:03:36.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.864+0000 7f2e50a5f640 1 -- 172.21.15.89:0/292047345 shutdown_connections 2023-10-18T11:03:36.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.864+0000 7f2e50a5f640 1 --2- 172.21.15.89:0/292047345 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2e2c065680 0x7f2e2c067b40 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:36.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.864+0000 7f2e50a5f640 1 --2- 172.21.15.89:0/292047345 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e4c159490 0x7f2e4c157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:36.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.864+0000 7f2e50a5f640 1 --2- 172.21.15.89:0/292047345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e4c070600 0x7f2e4c1529d0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:36.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.864+0000 7f2e50a5f640 1 --2- 172.21.15.89:0/292047345 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e4c06bee0 0x7f2e4c0c2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:36.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.864+0000 7f2e50a5f640 1 -- 172.21.15.89:0/292047345 >> 172.21.15.89:0/292047345 conn(0x7f2e4c069f00 msgr2=0x7f2e4c0c53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:36.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.864+0000 7f2e50a5f640 1 -- 172.21.15.89:0/292047345 shutdown_connections 2023-10-18T11:03:36.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:36.864+0000 7f2e50a5f640 1 -- 172.21.15.89:0/292047345 wait complete. 2023-10-18T11:03:36.880 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-18T11:03:36.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:03:36.881 DEBUG:teuthology.orchestra.run.smithi154:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-18T11:03:36.912 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2023-10-18T11:03:36.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:03:36.913 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-18T11:03:36.964 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T11:03:36.964 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:03:36.973 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354508k,nr_inodes=4088627,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275352k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9360 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275348k,nr_inodes=818837,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275352k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T11:03:36.974 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T11:03:36.974 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:03:37.015 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354672k,nr_inodes=4088668,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4198 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T11:03:37.015 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T11:03:37.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.152+0000 7f2459ce5640 1 Processor -- start 2023-10-18T11:03:37.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.152+0000 7f2459ce5640 1 -- start start 2023-10-18T11:03:37.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.156+0000 7f2459ce5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2454159490 0x7f245415d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:37.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.156+0000 7f2459ce5640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f245415ddf0 con 0x7f245406bee0 2023-10-18T11:03:37.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.156+0000 7f2459ce5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2454070b90 con 0x7f2454070600 2023-10-18T11:03:37.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.156+0000 7f2459ce5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2454070e70 con 0x7f2454159490 2023-10-18T11:03:37.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.156+0000 7f2453fff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f245406bee0 0x7f24540c8960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39634/0 (socket says 172.21.15.89:39634) 2023-10-18T11:03:37.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.156+0000 7f2453fff640 1 -- 172.21.15.89:0/1849310024 learned_addr learned my addr 172.21.15.89:0/1849310024 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:37.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.156+0000 7f24537fe640 1 --2- 172.21.15.89:0/1849310024 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2454159490 0x7f245415d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:37.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.156+0000 7f2452ffd640 1 -- 172.21.15.89:0/1849310024 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1128663687 0 0) 0x7f245415ddf0 con 0x7f245406bee0 2023-10-18T11:03:37.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.156+0000 7f2452ffd640 1 -- 172.21.15.89:0/1849310024 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2440005740 con 0x7f245406bee0 2023-10-18T11:03:37.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.156+0000 7f2452ffd640 1 -- 172.21.15.89:0/1849310024 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1201006011 0 0) 0x7f2454070b90 con 0x7f2454070600 2023-10-18T11:03:37.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.156+0000 7f2452ffd640 1 -- 172.21.15.89:0/1849310024 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f245415ddf0 con 0x7f2454070600 2023-10-18T11:03:37.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.156+0000 7f24537fe640 1 -- 172.21.15.89:0/1849310024 >> v1:172.21.15.154:6789/0 conn(0x7f2454070600 legacy=0x7f245406b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:37.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.156+0000 7f24537fe640 1 -- 172.21.15.89:0/1849310024 >> v1:172.21.15.89:6789/0 conn(0x7f245406bee0 legacy=0x7f24540c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:37.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.156+0000 7f2452ffd640 1 -- 172.21.15.89:0/1849310024 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3902267466 0 0) 0x7f2440005740 con 0x7f245406bee0 2023-10-18T11:03:37.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.156+0000 7f24537fe640 1 -- 172.21.15.89:0/1849310024 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2454071150 con 0x7f2454159490 2023-10-18T11:03:37.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.160+0000 7f24537fe640 1 --2- 172.21.15.89:0/1849310024 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2454159490 0x7f245415d8b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f24440097f0 tx=0x7f244401c320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65aa96b22e6e55fb server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:37.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.160+0000 7f2452ffd640 1 -- 172.21.15.89:0/1849310024 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c001a00 con 0x7f2454159490 2023-10-18T11:03:37.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.160+0000 7f2452ffd640 1 -- 172.21.15.89:0/1849310024 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24440a7050 con 0x7f2454159490 2023-10-18T11:03:37.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.160+0000 7f2452ffd640 1 -- 172.21.15.89:0/1849310024 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24440ab600 con 0x7f2454159490 2023-10-18T11:03:37.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.160+0000 7f2459ce5640 1 -- 172.21.15.89:0/1849310024 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2454159490 msgr2=0x7f245415d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:37.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.160+0000 7f2459ce5640 1 --2- 172.21.15.89:0/1849310024 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2454159490 0x7f245415d8b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f24440097f0 tx=0x7f244401c320 comp rx=0 tx=0).stop 2023-10-18T11:03:37.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.160+0000 7f2459ce5640 1 -- 172.21.15.89:0/1849310024 shutdown_connections 2023-10-18T11:03:37.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.160+0000 7f2459ce5640 1 --2- 172.21.15.89:0/1849310024 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2454159490 0x7f245415d8b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:37.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.160+0000 7f2459ce5640 1 -- 172.21.15.89:0/1849310024 >> 172.21.15.89:0/1849310024 conn(0x7f2454069f00 msgr2=0x7f24540cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:37.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.160+0000 7f2459ce5640 1 -- 172.21.15.89:0/1849310024 shutdown_connections 2023-10-18T11:03:37.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.160+0000 7f2459ce5640 1 -- 172.21.15.89:0/1849310024 wait complete. 2023-10-18T11:03:37.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.160+0000 7f2459ce5640 1 Processor -- start 2023-10-18T11:03:37.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.160+0000 7f2459ce5640 1 -- start start 2023-10-18T11:03:37.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.164+0000 7f2459ce5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f245406bee0 0x7f2454152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:37.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.164+0000 7f2459ce5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2454070600 0x7f2454151930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:37.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.164+0000 7f2459ce5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2454159490 0x7f24540c5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:37.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.164+0000 7f2459ce5640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24540c47e0 con 0x7f2454070600 2023-10-18T11:03:37.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.164+0000 7f2459ce5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245414a460 con 0x7f2454159490 2023-10-18T11:03:37.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.164+0000 7f2459ce5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24540c3b60 con 0x7f245406bee0 2023-10-18T11:03:37.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.164+0000 7f244affd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2454070600 0x7f2454151930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:37.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.164+0000 7f244affd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2454070600 0x7f2454151930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51336/0 (socket says 172.21.15.89:51336) 2023-10-18T11:03:37.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.164+0000 7f244affd640 1 -- 172.21.15.89:0/3511163444 learned_addr learned my addr 172.21.15.89:0/3511163444 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:37.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.164+0000 7f24537fe640 1 --2- 172.21.15.89:0/3511163444 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f245406bee0 0x7f2454152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:37.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.164+0000 7f2453fff640 1 --2- 172.21.15.89:0/3511163444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2454159490 0x7f24540c5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:37.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.164+0000 7f244affd640 1 -- 172.21.15.89:0/3511163444 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f245406bee0 msgr2=0x7f2454152c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:37.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.164+0000 7f244affd640 1 --2- 172.21.15.89:0/3511163444 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f245406bee0 0x7f2454152c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:37.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.164+0000 7f244affd640 1 -- 172.21.15.89:0/3511163444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2454159490 msgr2=0x7f24540c5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:37.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.164+0000 7f244affd640 1 --2- 172.21.15.89:0/3511163444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2454159490 0x7f24540c5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:37.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.164+0000 7f244affd640 1 -- 172.21.15.89:0/3511163444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2444009420 con 0x7f2454070600 2023-10-18T11:03:37.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.164+0000 7f2453fff640 1 --2- 172.21.15.89:0/3511163444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2454159490 0x7f24540c5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:03:37.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.164+0000 7f244affd640 1 --2- 172.21.15.89:0/3511163444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2454070600 0x7f2454151930 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f243c00c500 tx=0x7f243c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:37.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.164+0000 7f2450ff9640 1 -- 172.21.15.89:0/3511163444 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c0052d0 con 0x7f2454070600 2023-10-18T11:03:37.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.164+0000 7f2450ff9640 1 -- 172.21.15.89:0/3511163444 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243c005470 con 0x7f2454070600 2023-10-18T11:03:37.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.164+0000 7f2459ce5640 1 -- 172.21.15.89:0/3511163444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24540c3e40 con 0x7f2454070600 2023-10-18T11:03:37.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.164+0000 7f2450ff9640 1 -- 172.21.15.89:0/3511163444 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c003840 con 0x7f2454070600 2023-10-18T11:03:37.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.164+0000 7f2459ce5640 1 -- 172.21.15.89:0/3511163444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24540c4400 con 0x7f2454070600 2023-10-18T11:03:37.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.164+0000 7f2459ce5640 1 -- 172.21.15.89:0/3511163444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f245404dbe0 con 0x7f2454070600 2023-10-18T11:03:37.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.168+0000 7f2450ff9640 1 -- 172.21.15.89:0/3511163444 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f243c0039a0 con 0x7f2454070600 2023-10-18T11:03:37.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.168+0000 7f2450ff9640 1 --2- 172.21.15.89:0/3511163444 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2434065680 0x7f2434067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:37.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.168+0000 7f2450ff9640 1 -- 172.21.15.89:0/3511163444 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f243c082490 con 0x7f2454070600 2023-10-18T11:03:37.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.168+0000 7f24537fe640 1 --2- 172.21.15.89:0/3511163444 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2434065680 0x7f2434067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:37.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.168+0000 7f24537fe640 1 --2- 172.21.15.89:0/3511163444 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2434065680 0x7f2434067b40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f24440ad000 tx=0x7f24440a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:37.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.172+0000 7f2450ff9640 1 -- 172.21.15.89:0/3511163444 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f243c04fdc0 con 0x7f2454070600 2023-10-18T11:03:37.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.352+0000 7f2459ce5640 1 -- 172.21.15.89:0/3511163444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24540cbb60 con 0x7f2454070600 2023-10-18T11:03:37.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.352+0000 7f2450ff9640 1 -- 172.21.15.89:0/3511163444 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4877 (secure 0 0 0) 0x7f243c0533e0 con 0x7f2454070600 2023-10-18T11:03:37.352 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T11:03:37.352 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/654729641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":654729641},{"type":"v1","addr":"172.21.15.154:6836","nonce":654729641}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/964566480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":964566480},{"type":"v1","addr":"172.21.15.154:6837","nonce":964566480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2261878538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2261878538},{"type":"v1","addr":"172.21.15.89:6833","nonce":2261878538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:47:15.199575+0000","modified":"2023-10-18T10:55:20.892655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5517":{"gid":5517,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":124,"addr":"172.21.15.89:6835/2092274731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2092274731},{"type":"v1","addr":"172.21.15.89:6835","nonce":2092274731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T11:03:37.353 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-18T11:03:37.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.356+0000 7f2459ce5640 1 -- 172.21.15.89:0/3511163444 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2434065680 msgr2=0x7f2434067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:37.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.356+0000 7f2459ce5640 1 --2- 172.21.15.89:0/3511163444 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2434065680 0x7f2434067b40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f24440ad000 tx=0x7f24440a6000 comp rx=0 tx=0).stop 2023-10-18T11:03:37.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.356+0000 7f2459ce5640 1 -- 172.21.15.89:0/3511163444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2454070600 msgr2=0x7f2454151930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:37.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.356+0000 7f2459ce5640 1 --2- 172.21.15.89:0/3511163444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2454070600 0x7f2454151930 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f243c00c500 tx=0x7f243c00c9c0 comp rx=0 tx=0).stop 2023-10-18T11:03:37.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.356+0000 7f2459ce5640 1 -- 172.21.15.89:0/3511163444 shutdown_connections 2023-10-18T11:03:37.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.356+0000 7f2459ce5640 1 --2- 172.21.15.89:0/3511163444 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2434065680 0x7f2434067b40 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:37.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.356+0000 7f2459ce5640 1 --2- 172.21.15.89:0/3511163444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2454159490 0x7f24540c5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:37.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.356+0000 7f2459ce5640 1 --2- 172.21.15.89:0/3511163444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2454070600 0x7f2454151930 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:37.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.356+0000 7f2459ce5640 1 --2- 172.21.15.89:0/3511163444 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f245406bee0 0x7f2454152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:37.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.356+0000 7f2459ce5640 1 -- 172.21.15.89:0/3511163444 >> 172.21.15.89:0/3511163444 conn(0x7f2454069f00 msgr2=0x7f24541584b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:37.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.356+0000 7f2459ce5640 1 -- 172.21.15.89:0/3511163444 shutdown_connections 2023-10-18T11:03:37.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.356+0000 7f2459ce5640 1 -- 172.21.15.89:0/3511163444 wait complete. 2023-10-18T11:03:37.373 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T11:03:37.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.512+0000 7f0200bd5640 1 Processor -- start 2023-10-18T11:03:37.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.512+0000 7f0200bd5640 1 -- start start 2023-10-18T11:03:37.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.512+0000 7f0200bd5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f01fc15c940 0x7f01fc0d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:37.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.512+0000 7f0200bd5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f01fc0d2f40 0x7f01fc0d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:37.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.512+0000 7f0200bd5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01fc15da50 con 0x7f01fc0d0ca0 2023-10-18T11:03:37.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.512+0000 7f0200bd5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01fc0d1130 con 0x7f01fc15c940 2023-10-18T11:03:37.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.512+0000 7f0200bd5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc15ce40 con 0x7f01fc0d2f40 2023-10-18T11:03:37.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.512+0000 7f01fa59c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f01fc15c940 0x7f01fc0d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:37.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.512+0000 7f01f9d9b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f01fc0d2f40 0x7f01fc0d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:37.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.512+0000 7f01fa59c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f01fc15c940 0x7f01fc0d2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:58532/0 (socket says 172.21.15.89:58532) 2023-10-18T11:03:37.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.512+0000 7f01fa59c640 1 -- 172.21.15.89:0/1309070713 learned_addr learned my addr 172.21.15.89:0/1309070713 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:37.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.512+0000 7f01fad9d640 1 --1- 172.21.15.89:0/1309070713 >> v1:172.21.15.154:6789/0 conn(0x7f01fc0d0ca0 0x7f01fc0cf060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.89:38850/0 (socket says 172.21.15.89:38850) 2023-10-18T11:03:37.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.512+0000 7f01fa59c640 1 -- 172.21.15.89:0/1309070713 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f01fc0d2f40 msgr2=0x7f01fc0d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:37.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.512+0000 7f01fa59c640 1 --2- 172.21.15.89:0/1309070713 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f01fc0d2f40 0x7f01fc0d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:37.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.512+0000 7f01fa59c640 1 -- 172.21.15.89:0/1309070713 >> v1:172.21.15.154:6789/0 conn(0x7f01fc0d0ca0 legacy=0x7f01fc0cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:37.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.512+0000 7f01f959a640 1 -- 172.21.15.89:0/1309070713 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2813731503 0 0) 0x7f01fc15da50 con 0x7f01fc0d0ca0 2023-10-18T11:03:37.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.512+0000 7f01fa59c640 1 -- 172.21.15.89:0/1309070713 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01fc15d120 con 0x7f01fc15c940 2023-10-18T11:03:37.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.512+0000 7f01f9d9b640 1 --2- 172.21.15.89:0/1309070713 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f01fc0d2f40 0x7f01fc0d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T11:03:37.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.516+0000 7f01fa59c640 1 --2- 172.21.15.89:0/1309070713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f01fc15c940 0x7f01fc0d2a00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f01e4009870 tx=0x7f01e401c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7222161f6f2e40ec server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:37.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.516+0000 7f01f959a640 1 -- 172.21.15.89:0/1309070713 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e40ab620 con 0x7f01fc15c940 2023-10-18T11:03:37.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.516+0000 7f01f959a640 1 -- 172.21.15.89:0/1309070713 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e40ab780 con 0x7f01fc15c940 2023-10-18T11:03:37.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.516+0000 7f01f959a640 1 -- 172.21.15.89:0/1309070713 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e40abb10 con 0x7f01fc15c940 2023-10-18T11:03:37.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.516+0000 7f0200bd5640 1 -- 172.21.15.89:0/1309070713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f01fc15c940 msgr2=0x7f01fc0d2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:37.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.516+0000 7f0200bd5640 1 --2- 172.21.15.89:0/1309070713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f01fc15c940 0x7f01fc0d2a00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f01e4009870 tx=0x7f01e401c510 comp rx=0 tx=0).stop 2023-10-18T11:03:37.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.516+0000 7f0200bd5640 1 -- 172.21.15.89:0/1309070713 shutdown_connections 2023-10-18T11:03:37.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.516+0000 7f0200bd5640 1 --2- 172.21.15.89:0/1309070713 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f01fc0d2f40 0x7f01fc0d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:37.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.516+0000 7f0200bd5640 1 --2- 172.21.15.89:0/1309070713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f01fc15c940 0x7f01fc0d2a00 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:37.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.516+0000 7f0200bd5640 1 -- 172.21.15.89:0/1309070713 >> 172.21.15.89:0/1309070713 conn(0x7f01fc089fd0 msgr2=0x7f01fc09b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:37.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.516+0000 7f0200bd5640 1 -- 172.21.15.89:0/1309070713 shutdown_connections 2023-10-18T11:03:37.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.516+0000 7f0200bd5640 1 -- 172.21.15.89:0/1309070713 wait complete. 2023-10-18T11:03:37.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.520+0000 7f0200bd5640 1 Processor -- start 2023-10-18T11:03:37.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.520+0000 7f0200bd5640 1 -- start start 2023-10-18T11:03:37.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.520+0000 7f0200bd5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01fc0d0ca0 0x7f01fc142520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:37.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.520+0000 7f0200bd5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f01fc154940 0x7f01fc0b7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:37.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.520+0000 7f0200bd5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f01fc0b7980 0x7f01fc069c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:37.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.520+0000 7f0200bd5640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc0c8f10 con 0x7f01fc0d0ca0 2023-10-18T11:03:37.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.520+0000 7f0200bd5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc0b7e00 con 0x7f01fc154940 2023-10-18T11:03:37.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.520+0000 7f0200bd5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc142a60 con 0x7f01fc0b7980 2023-10-18T11:03:37.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.520+0000 7f01fa59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01fc0d0ca0 0x7f01fc142520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:37.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.520+0000 7f01fa59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01fc0d0ca0 0x7f01fc142520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51338/0 (socket says 172.21.15.89:51338) 2023-10-18T11:03:37.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.520+0000 7f01fa59c640 1 -- 172.21.15.89:0/3351767417 learned_addr learned my addr 172.21.15.89:0/3351767417 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:37.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.520+0000 7f01f9d9b640 1 --2- 172.21.15.89:0/3351767417 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f01fc154940 0x7f01fc0b7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:37.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.520+0000 7f01fad9d640 1 --2- 172.21.15.89:0/3351767417 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f01fc0b7980 0x7f01fc069c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:37.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.520+0000 7f01fa59c640 1 -- 172.21.15.89:0/3351767417 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f01fc0b7980 msgr2=0x7f01fc069c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:37.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.520+0000 7f01fa59c640 1 --2- 172.21.15.89:0/3351767417 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f01fc0b7980 0x7f01fc069c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:37.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.520+0000 7f01fa59c640 1 -- 172.21.15.89:0/3351767417 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f01fc154940 msgr2=0x7f01fc0b7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:37.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.520+0000 7f01fa59c640 1 --2- 172.21.15.89:0/3351767417 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f01fc154940 0x7f01fc0b7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:37.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.520+0000 7f01fa59c640 1 -- 172.21.15.89:0/3351767417 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ec009420 con 0x7f01fc0d0ca0 2023-10-18T11:03:37.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.520+0000 7f01fa59c640 1 --2- 172.21.15.89:0/3351767417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01fc0d0ca0 0x7f01fc142520 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f01e40ad040 tx=0x7f01e401ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:37.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.520+0000 7f01f2ffd640 1 -- 172.21.15.89:0/3351767417 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e40abd00 con 0x7f01fc0d0ca0 2023-10-18T11:03:37.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.524+0000 7f01f2ffd640 1 -- 172.21.15.89:0/3351767417 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e40b8070 con 0x7f01fc0d0ca0 2023-10-18T11:03:37.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.524+0000 7f0200bd5640 1 -- 172.21.15.89:0/3351767417 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e40094d0 con 0x7f01fc0d0ca0 2023-10-18T11:03:37.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.524+0000 7f01f2ffd640 1 -- 172.21.15.89:0/3351767417 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e4004220 con 0x7f01fc0d0ca0 2023-10-18T11:03:37.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.524+0000 7f0200bd5640 1 -- 172.21.15.89:0/3351767417 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01fc0c8570 con 0x7f01fc0d0ca0 2023-10-18T11:03:37.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.524+0000 7f0200bd5640 1 -- 172.21.15.89:0/3351767417 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01fc15f390 con 0x7f01fc0d0ca0 2023-10-18T11:03:37.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.524+0000 7f01f2ffd640 1 -- 172.21.15.89:0/3351767417 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f01e4004380 con 0x7f01fc0d0ca0 2023-10-18T11:03:37.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.524+0000 7f01f2ffd640 1 --2- 172.21.15.89:0/3351767417 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f01dc065680 0x7f01dc067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:37.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.524+0000 7f01f2ffd640 1 -- 172.21.15.89:0/3351767417 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f01e411ced0 con 0x7f01fc0d0ca0 2023-10-18T11:03:37.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.524+0000 7f01f9d9b640 1 --2- 172.21.15.89:0/3351767417 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f01dc065680 0x7f01dc067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:37.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.528+0000 7f01f9d9b640 1 --2- 172.21.15.89:0/3351767417 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f01dc065680 0x7f01dc067b40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f01fc167680 tx=0x7f01ec009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:37.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.528+0000 7f01f2ffd640 1 -- 172.21.15.89:0/3351767417 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f01e40ea770 con 0x7f01fc0d0ca0 2023-10-18T11:03:37.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.676+0000 7f0200bd5640 1 -- 172.21.15.89:0/3351767417 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f01fc04dbe0 con 0x7f01fc0d0ca0 2023-10-18T11:03:37.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.680+0000 7f01f2ffd640 1 -- 172.21.15.89:0/3351767417 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+18666 (secure 0 0 0) 0x7f01e40ba090 con 0x7f01fc0d0ca0 2023-10-18T11:03:37.677 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T11:03:37.678 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":56,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:55:22.907577+0000","last_up_change":"2023-10-18T10:55:21.905065+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:47:13.188701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-18T10:47:14.198224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":50,"up_from":55,"up_thru":55,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":1133012599},{"type":"v1","addr":"172.21.15.154:6824","nonce":1133012599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1133012599},{"type":"v1","addr":"172.21.15.154:6827","nonce":1133012599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1133012599},{"type":"v1","addr":"172.21.15.154:6831","nonce":1133012599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1133012599},{"type":"v1","addr":"172.21.15.154:6829","nonce":1133012599}]},"public_addr":"172.21.15.154:6824/1133012599","cluster_addr":"172.21.15.154:6827/1133012599","heartbeat_back_addr":"172.21.15.154:6831/1133012599","heartbeat_front_addr":"172.21.15.154:6829/1133012599","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"2023-10-18T10:47:48.955023+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":51},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.9","mappings":[{"from":0,"to":5}]},{"pgid":"7.18","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:0/330049992":"2023-10-18T11:55:17.981058+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T11:03:37.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.680+0000 7f0200bd5640 1 -- 172.21.15.89:0/3351767417 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f01dc065680 msgr2=0x7f01dc067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:37.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.680+0000 7f0200bd5640 1 --2- 172.21.15.89:0/3351767417 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f01dc065680 0x7f01dc067b40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f01fc167680 tx=0x7f01ec009190 comp rx=0 tx=0).stop 2023-10-18T11:03:37.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.680+0000 7f0200bd5640 1 -- 172.21.15.89:0/3351767417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01fc0d0ca0 msgr2=0x7f01fc142520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:37.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.680+0000 7f0200bd5640 1 --2- 172.21.15.89:0/3351767417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01fc0d0ca0 0x7f01fc142520 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f01e40ad040 tx=0x7f01e401ad50 comp rx=0 tx=0).stop 2023-10-18T11:03:37.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.680+0000 7f0200bd5640 1 -- 172.21.15.89:0/3351767417 shutdown_connections 2023-10-18T11:03:37.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.680+0000 7f0200bd5640 1 --2- 172.21.15.89:0/3351767417 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f01dc065680 0x7f01dc067b40 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:37.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.680+0000 7f0200bd5640 1 --2- 172.21.15.89:0/3351767417 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f01fc0b7980 0x7f01fc069c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:37.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.680+0000 7f0200bd5640 1 --2- 172.21.15.89:0/3351767417 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f01fc154940 0x7f01fc0b7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:37.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.680+0000 7f0200bd5640 1 --2- 172.21.15.89:0/3351767417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01fc0d0ca0 0x7f01fc142520 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:37.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.680+0000 7f0200bd5640 1 -- 172.21.15.89:0/3351767417 >> 172.21.15.89:0/3351767417 conn(0x7f01fc089fd0 msgr2=0x7f01fc15ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:37.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.680+0000 7f0200bd5640 1 -- 172.21.15.89:0/3351767417 shutdown_connections 2023-10-18T11:03:37.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.680+0000 7f0200bd5640 1 -- 172.21.15.89:0/3351767417 wait complete. 2023-10-18T11:03:37.698 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:45:43.114184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-18T10:45:48.730968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:47:13.188701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:47:14.198224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T11:03:37.699 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T11:03:37.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.840+0000 7ff6fadfa640 1 Processor -- start 2023-10-18T11:03:37.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.840+0000 7ff6fadfa640 1 -- start start 2023-10-18T11:03:37.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.840+0000 7ff6fadfa640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff6f40cd4f0 0x7ff6f4143470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:37.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.844+0000 7ff6fadfa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6f40ce070 0x7ff6f40ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:37.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.844+0000 7ff6fadfa640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6f40cd8d0 con 0x7ff6f4145b70 2023-10-18T11:03:37.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.844+0000 7ff6fadfa640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6f4069900 con 0x7ff6f40cd4f0 2023-10-18T11:03:37.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.844+0000 7ff6fadfa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f40cea20 con 0x7ff6f40ce070 2023-10-18T11:03:37.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.844+0000 7ff6f9397640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6f40ce070 0x7ff6f40ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:37.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.844+0000 7ff6f8b96640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff6f40cd4f0 0x7ff6f4143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:37.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.844+0000 7ff6f9397640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6f40ce070 0x7ff6f40ce4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:37262/0 (socket says 172.21.15.89:37262) 2023-10-18T11:03:37.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.844+0000 7ff6f8b96640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff6f40cd4f0 0x7ff6f4143470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:58548/0 (socket says 172.21.15.89:58548) 2023-10-18T11:03:37.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.844+0000 7ff6f9397640 1 -- 172.21.15.89:0/676776953 learned_addr learned my addr 172.21.15.89:0/676776953 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:37.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.844+0000 7ff6ebfff640 1 --1- 172.21.15.89:0/676776953 >> v1:172.21.15.154:6789/0 conn(0x7ff6f4145b70 0x7ff6f4160cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.89:38862/0 (socket says 172.21.15.89:38862) 2023-10-18T11:03:37.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.844+0000 7ff6f8b96640 1 -- 172.21.15.89:0/676776953 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6f40ce070 msgr2=0x7ff6f40ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:37.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.844+0000 7ff6f8b96640 1 --2- 172.21.15.89:0/676776953 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6f40ce070 0x7ff6f40ce4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:37.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.844+0000 7ff6f8b96640 1 -- 172.21.15.89:0/676776953 >> v1:172.21.15.154:6789/0 conn(0x7ff6f4145b70 legacy=0x7ff6f4160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:37.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.844+0000 7ff6f8b96640 1 -- 172.21.15.89:0/676776953 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f415cbf0 con 0x7ff6f40cd4f0 2023-10-18T11:03:37.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.844+0000 7ff6eb7fe640 1 -- 172.21.15.89:0/676776953 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2244829380 0 0) 0x7ff6f40cd8d0 con 0x7ff6f4145b70 2023-10-18T11:03:37.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.844+0000 7ff6f8b96640 1 --2- 172.21.15.89:0/676776953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6f40cd4f0 0x7ff6f4143470 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4069a40 tx=0x7ff6f001c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4f09abbecbdb522 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:37.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.848+0000 7ff6eb7fe640 1 -- 172.21.15.89:0/676776953 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f00ab650 con 0x7ff6f40cd4f0 2023-10-18T11:03:37.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.848+0000 7ff6eb7fe640 1 -- 172.21.15.89:0/676776953 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f00ab7b0 con 0x7ff6f40cd4f0 2023-10-18T11:03:37.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.848+0000 7ff6eb7fe640 1 -- 172.21.15.89:0/676776953 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f00abb40 con 0x7ff6f40cd4f0 2023-10-18T11:03:37.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.848+0000 7ff6fadfa640 1 -- 172.21.15.89:0/676776953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6f40cd4f0 msgr2=0x7ff6f4143470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:37.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.848+0000 7ff6fadfa640 1 --2- 172.21.15.89:0/676776953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6f40cd4f0 0x7ff6f4143470 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4069a40 tx=0x7ff6f001c450 comp rx=0 tx=0).stop 2023-10-18T11:03:37.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.848+0000 7ff6fadfa640 1 -- 172.21.15.89:0/676776953 shutdown_connections 2023-10-18T11:03:37.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.848+0000 7ff6fadfa640 1 --2- 172.21.15.89:0/676776953 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6f40ce070 0x7ff6f40ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:37.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.848+0000 7ff6fadfa640 1 --2- 172.21.15.89:0/676776953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6f40cd4f0 0x7ff6f4143470 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:37.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.848+0000 7ff6fadfa640 1 -- 172.21.15.89:0/676776953 >> 172.21.15.89:0/676776953 conn(0x7ff6f406dad0 msgr2=0x7ff6f406dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:37.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.848+0000 7ff6fadfa640 1 -- 172.21.15.89:0/676776953 shutdown_connections 2023-10-18T11:03:37.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.848+0000 7ff6fadfa640 1 -- 172.21.15.89:0/676776953 wait complete. 2023-10-18T11:03:37.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.848+0000 7ff6fadfa640 1 Processor -- start 2023-10-18T11:03:37.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.848+0000 7ff6fadfa640 1 -- start start 2023-10-18T11:03:37.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.848+0000 7ff6fadfa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6f40cd4f0 0x7ff6f40c34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:37.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.848+0000 7ff6fadfa640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6f40ce070 0x7ff6f41570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:37.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.848+0000 7ff6fadfa640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6f4145b70 0x7ff6f40c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:37.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.852+0000 7ff6fadfa640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f40c94b0 con 0x7ff6f40ce070 2023-10-18T11:03:37.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.852+0000 7ff6fadfa640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f4156690 con 0x7ff6f4145b70 2023-10-18T11:03:37.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.852+0000 7ff6fadfa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f40c96e0 con 0x7ff6f40cd4f0 2023-10-18T11:03:37.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.852+0000 7ff6ebfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6f40ce070 0x7ff6f41570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:37.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.852+0000 7ff6ebfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6f40ce070 0x7ff6f41570e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51342/0 (socket says 172.21.15.89:51342) 2023-10-18T11:03:37.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.852+0000 7ff6ebfff640 1 -- 172.21.15.89:0/722629546 learned_addr learned my addr 172.21.15.89:0/722629546 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:37.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.852+0000 7ff6f8b96640 1 --2- 172.21.15.89:0/722629546 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6f40cd4f0 0x7ff6f40c34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:37.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.852+0000 7ff6f9397640 1 --2- 172.21.15.89:0/722629546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6f4145b70 0x7ff6f40c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:37.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.852+0000 7ff6ebfff640 1 -- 172.21.15.89:0/722629546 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6f40cd4f0 msgr2=0x7ff6f40c34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:37.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.852+0000 7ff6ebfff640 1 --2- 172.21.15.89:0/722629546 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6f40cd4f0 0x7ff6f40c34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:37.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.852+0000 7ff6ebfff640 1 -- 172.21.15.89:0/722629546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6f4145b70 msgr2=0x7ff6f40c8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:37.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.852+0000 7ff6ebfff640 1 --2- 172.21.15.89:0/722629546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6f4145b70 0x7ff6f40c8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:37.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.852+0000 7ff6ebfff640 1 -- 172.21.15.89:0/722629546 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f0009420 con 0x7ff6f40ce070 2023-10-18T11:03:37.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.852+0000 7ff6ebfff640 1 --2- 172.21.15.89:0/722629546 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6f40ce070 0x7ff6f41570e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff6e40011f0 tx=0x7ff6e400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:37.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.852+0000 7ff6e97fa640 1 -- 172.21.15.89:0/722629546 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e40074e0 con 0x7ff6f40ce070 2023-10-18T11:03:37.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.852+0000 7ff6e97fa640 1 -- 172.21.15.89:0/722629546 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e4007640 con 0x7ff6f40ce070 2023-10-18T11:03:37.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.852+0000 7ff6fadfa640 1 -- 172.21.15.89:0/722629546 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f40c99c0 con 0x7ff6f40ce070 2023-10-18T11:03:37.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.852+0000 7ff6e97fa640 1 -- 172.21.15.89:0/722629546 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4004a10 con 0x7ff6f40ce070 2023-10-18T11:03:37.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.852+0000 7ff6fadfa640 1 -- 172.21.15.89:0/722629546 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6f4156bc0 con 0x7ff6f40ce070 2023-10-18T11:03:37.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.852+0000 7ff6fadfa640 1 -- 172.21.15.89:0/722629546 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6f4156de0 con 0x7ff6f40ce070 2023-10-18T11:03:37.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.852+0000 7ff6e97fa640 1 -- 172.21.15.89:0/722629546 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7ff6e4004b70 con 0x7ff6f40ce070 2023-10-18T11:03:37.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.856+0000 7ff6e97fa640 1 --2- 172.21.15.89:0/722629546 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff6e0065680 0x7ff6e0067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:37.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.856+0000 7ff6e97fa640 1 -- 172.21.15.89:0/722629546 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7ff6e4082040 con 0x7ff6f40ce070 2023-10-18T11:03:37.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.856+0000 7ff6f8b96640 1 --2- 172.21.15.89:0/722629546 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff6e0065680 0x7ff6e0067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:37.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.856+0000 7ff6f8b96640 1 --2- 172.21.15.89:0/722629546 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff6e0065680 0x7ff6e0067b40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff6f00af000 tx=0x7ff6f00a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:37.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:37.860+0000 7ff6e97fa640 1 -- 172.21.15.89:0/722629546 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7ff6e404fa90 con 0x7ff6f40ce070 2023-10-18T11:03:38.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.008+0000 7ff6fadfa640 1 -- 172.21.15.89:0/722629546 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff6f404dbe0 con 0x7ff6f40ce070 2023-10-18T11:03:38.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.008+0000 7ff6e97fa640 1 -- 172.21.15.89:0/722629546 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff6e40530b0 con 0x7ff6f40ce070 2023-10-18T11:03:38.008 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-18T11:03:38.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.012+0000 7ff6fadfa640 1 -- 172.21.15.89:0/722629546 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff6e0065680 msgr2=0x7ff6e0067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:38.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.012+0000 7ff6fadfa640 1 --2- 172.21.15.89:0/722629546 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff6e0065680 0x7ff6e0067b40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff6f00af000 tx=0x7ff6f00a7000 comp rx=0 tx=0).stop 2023-10-18T11:03:38.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.012+0000 7ff6fadfa640 1 -- 172.21.15.89:0/722629546 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6f40ce070 msgr2=0x7ff6f41570e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:38.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.012+0000 7ff6fadfa640 1 --2- 172.21.15.89:0/722629546 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6f40ce070 0x7ff6f41570e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff6e40011f0 tx=0x7ff6e400c9b0 comp rx=0 tx=0).stop 2023-10-18T11:03:38.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.012+0000 7ff6fadfa640 1 -- 172.21.15.89:0/722629546 shutdown_connections 2023-10-18T11:03:38.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.012+0000 7ff6fadfa640 1 --2- 172.21.15.89:0/722629546 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff6e0065680 0x7ff6e0067b40 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:38.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.012+0000 7ff6fadfa640 1 --2- 172.21.15.89:0/722629546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6f4145b70 0x7ff6f40c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:38.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.012+0000 7ff6fadfa640 1 --2- 172.21.15.89:0/722629546 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6f40ce070 0x7ff6f41570e0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:38.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.012+0000 7ff6fadfa640 1 --2- 172.21.15.89:0/722629546 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6f40cd4f0 0x7ff6f40c34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:38.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.012+0000 7ff6fadfa640 1 -- 172.21.15.89:0/722629546 >> 172.21.15.89:0/722629546 conn(0x7ff6f406dad0 msgr2=0x7ff6f415cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:38.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.012+0000 7ff6fadfa640 1 -- 172.21.15.89:0/722629546 shutdown_connections 2023-10-18T11:03:38.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.012+0000 7ff6fadfa640 1 -- 172.21.15.89:0/722629546 wait complete. 2023-10-18T11:03:38.028 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-18T11:03:38.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.172+0000 7f0c9f89c640 1 Processor -- start 2023-10-18T11:03:38.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.172+0000 7f0c9f89c640 1 -- start start 2023-10-18T11:03:38.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.172+0000 7f0c9f89c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0c980b1860 0x7f0c980cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:38.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.172+0000 7f0c9f89c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0c980b1cc0 0x7f0c980cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:38.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.172+0000 7f0c9f89c640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c980d1e50 con 0x7f0c9815c550 2023-10-18T11:03:38.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.172+0000 7f0c9f89c640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c980d2130 con 0x7f0c980b1cc0 2023-10-18T11:03:38.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.172+0000 7f0c9f89c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c980d2410 con 0x7f0c980b1860 2023-10-18T11:03:38.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.172+0000 7f0c9d638640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0c9815c550 0x7f0c980d2ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39642/0 (socket says 172.21.15.89:39642) 2023-10-18T11:03:38.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.172+0000 7f0c9d638640 1 -- 172.21.15.89:0/1907857011 learned_addr learned my addr 172.21.15.89:0/1907857011 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:38.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.172+0000 7f0c9de39640 1 --2- 172.21.15.89:0/1907857011 >> v2:172.21.15.89:3300/0 conn(0x7f0c980b1cc0 0x7f0c980cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:38.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.172+0000 7f0c9ce37640 1 --2- 172.21.15.89:0/1907857011 >> v2:172.21.15.154:3300/0 conn(0x7f0c980b1860 0x7f0c980cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:38.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.172+0000 7f0c9de39640 1 -- 172.21.15.89:0/1907857011 >> v2:172.21.15.154:3300/0 conn(0x7f0c980b1860 msgr2=0x7f0c980cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:38.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.172+0000 7f0c9de39640 1 --2- 172.21.15.89:0/1907857011 >> v2:172.21.15.154:3300/0 conn(0x7f0c980b1860 0x7f0c980cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:38.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.172+0000 7f0c9de39640 1 -- 172.21.15.89:0/1907857011 >> v1:172.21.15.89:6789/0 conn(0x7f0c9815c550 legacy=0x7f0c980d2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:38.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.172+0000 7f0c9de39640 1 -- 172.21.15.89:0/1907857011 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c980d26f0 con 0x7f0c980b1cc0 2023-10-18T11:03:38.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.172+0000 7f0c9ce37640 1 --2- 172.21.15.89:0/1907857011 >> v2:172.21.15.154:3300/0 conn(0x7f0c980b1860 0x7f0c980cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:03:38.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.176+0000 7f0c9de39640 1 --2- 172.21.15.89:0/1907857011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c980b1cc0 0x7f0c980cfb50 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f0c980b4e60 tx=0x7f0c9001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff57d770c2a6f1b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:38.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.176+0000 7f0c8ffff640 1 -- 172.21.15.89:0/1907857011 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c900a59c0 con 0x7f0c980b1cc0 2023-10-18T11:03:38.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.176+0000 7f0c8ffff640 1 -- 172.21.15.89:0/1907857011 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c900a5b20 con 0x7f0c980b1cc0 2023-10-18T11:03:38.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.176+0000 7f0c8ffff640 1 -- 172.21.15.89:0/1907857011 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c900ad440 con 0x7f0c980b1cc0 2023-10-18T11:03:38.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.176+0000 7f0c9f89c640 1 -- 172.21.15.89:0/1907857011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c980b1cc0 msgr2=0x7f0c980cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:38.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.176+0000 7f0c9f89c640 1 --2- 172.21.15.89:0/1907857011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c980b1cc0 0x7f0c980cfb50 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f0c980b4e60 tx=0x7f0c9001c8e0 comp rx=0 tx=0).stop 2023-10-18T11:03:38.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.176+0000 7f0c9f89c640 1 -- 172.21.15.89:0/1907857011 shutdown_connections 2023-10-18T11:03:38.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.176+0000 7f0c9f89c640 1 --2- 172.21.15.89:0/1907857011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c980b1cc0 0x7f0c980cfb50 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:38.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.176+0000 7f0c9f89c640 1 --2- 172.21.15.89:0/1907857011 >> v2:172.21.15.154:3300/0 conn(0x7f0c980b1860 0x7f0c980cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:38.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.176+0000 7f0c9f89c640 1 -- 172.21.15.89:0/1907857011 >> 172.21.15.89:0/1907857011 conn(0x7f0c98069d00 msgr2=0x7f0c9806a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:38.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.176+0000 7f0c9f89c640 1 -- 172.21.15.89:0/1907857011 shutdown_connections 2023-10-18T11:03:38.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.176+0000 7f0c9f89c640 1 -- 172.21.15.89:0/1907857011 wait complete. 2023-10-18T11:03:38.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.180+0000 7f0c9f89c640 1 Processor -- start 2023-10-18T11:03:38.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.180+0000 7f0c9f89c640 1 -- start start 2023-10-18T11:03:38.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.180+0000 7f0c9f89c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c980b1860 0x7f0c98158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:38.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.180+0000 7f0c9f89c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c980b1cc0 0x7f0c98160190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:38.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.180+0000 7f0c9f89c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c9815c550 0x7f0c980ca180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:38.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.180+0000 7f0c9f89c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c981624b0 con 0x7f0c980b1cc0 2023-10-18T11:03:38.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.180+0000 7f0c9f89c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c980ca6c0 con 0x7f0c980b1860 2023-10-18T11:03:38.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.180+0000 7f0c9f89c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c980ca8f0 con 0x7f0c9815c550 2023-10-18T11:03:38.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.180+0000 7f0c9ce37640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c980b1cc0 0x7f0c98160190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:38.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.180+0000 7f0c9ce37640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c980b1cc0 0x7f0c98160190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51362/0 (socket says 172.21.15.89:51362) 2023-10-18T11:03:38.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.180+0000 7f0c9ce37640 1 -- 172.21.15.89:0/4123670132 learned_addr learned my addr 172.21.15.89:0/4123670132 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:38.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.180+0000 7f0c9d638640 1 --2- 172.21.15.89:0/4123670132 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c980b1860 0x7f0c98158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:38.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.180+0000 7f0c9de39640 1 --2- 172.21.15.89:0/4123670132 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c9815c550 0x7f0c980ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:38.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.180+0000 7f0c9ce37640 1 -- 172.21.15.89:0/4123670132 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c9815c550 msgr2=0x7f0c980ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:38.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.180+0000 7f0c9ce37640 1 --2- 172.21.15.89:0/4123670132 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c9815c550 0x7f0c980ca180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:38.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.180+0000 7f0c9ce37640 1 -- 172.21.15.89:0/4123670132 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c980b1860 msgr2=0x7f0c98158050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:38.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.180+0000 7f0c9ce37640 1 --2- 172.21.15.89:0/4123670132 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c980b1860 0x7f0c98158050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:38.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.180+0000 7f0c9ce37640 1 -- 172.21.15.89:0/4123670132 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c90009a70 con 0x7f0c980b1cc0 2023-10-18T11:03:38.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.180+0000 7f0c9ce37640 1 --2- 172.21.15.89:0/4123670132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c980b1cc0 0x7f0c98160190 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f0c8800e810 tx=0x7f0c8800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:38.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.180+0000 7f0c8dffb640 1 -- 172.21.15.89:0/4123670132 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8800c8d0 con 0x7f0c980b1cc0 2023-10-18T11:03:38.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.180+0000 7f0c8dffb640 1 -- 172.21.15.89:0/4123670132 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c8800ca30 con 0x7f0c980b1cc0 2023-10-18T11:03:38.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.180+0000 7f0c8dffb640 1 -- 172.21.15.89:0/4123670132 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8800cde0 con 0x7f0c980b1cc0 2023-10-18T11:03:38.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.180+0000 7f0c9f89c640 1 -- 172.21.15.89:0/4123670132 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c980cabd0 con 0x7f0c980b1cc0 2023-10-18T11:03:38.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.184+0000 7f0c9f89c640 1 -- 172.21.15.89:0/4123670132 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c980cb190 con 0x7f0c980b1cc0 2023-10-18T11:03:38.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.184+0000 7f0c9f89c640 1 -- 172.21.15.89:0/4123670132 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c980cb350 con 0x7f0c980b1cc0 2023-10-18T11:03:38.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.184+0000 7f0c8dffb640 1 -- 172.21.15.89:0/4123670132 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f0c88013410 con 0x7f0c980b1cc0 2023-10-18T11:03:38.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.184+0000 7f0c8dffb640 1 --2- 172.21.15.89:0/4123670132 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0c7c065680 0x7f0c7c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:38.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.184+0000 7f0c8dffb640 1 -- 172.21.15.89:0/4123670132 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f0c88082db0 con 0x7f0c980b1cc0 2023-10-18T11:03:38.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.184+0000 7f0c9d638640 1 --2- 172.21.15.89:0/4123670132 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0c7c065680 0x7f0c7c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:38.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.184+0000 7f0c9d638640 1 --2- 172.21.15.89:0/4123670132 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0c7c065680 0x7f0c7c067b40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0c94005890 tx=0x7f0c94009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:38.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.188+0000 7f0c8dffb640 1 -- 172.21.15.89:0/4123670132 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f0c88087070 con 0x7f0c980b1cc0 2023-10-18T11:03:38.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.340+0000 7f0c9f89c640 1 -- 172.21.15.89:0/4123670132 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0c980cb6b0 con 0x7f0c980b1cc0 2023-10-18T11:03:38.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.340+0000 7f0c8dffb640 1 -- 172.21.15.89:0/4123670132 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v56) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0c980cb6b0 con 0x7f0c980b1cc0 2023-10-18T11:03:38.338 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2023-10-18T11:03:38.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.340+0000 7f0c9f89c640 1 -- 172.21.15.89:0/4123670132 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0c7c065680 msgr2=0x7f0c7c067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:38.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.340+0000 7f0c9f89c640 1 --2- 172.21.15.89:0/4123670132 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0c7c065680 0x7f0c7c067b40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0c94005890 tx=0x7f0c94009040 comp rx=0 tx=0).stop 2023-10-18T11:03:38.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.340+0000 7f0c9f89c640 1 -- 172.21.15.89:0/4123670132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c980b1cc0 msgr2=0x7f0c98160190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:38.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.340+0000 7f0c9f89c640 1 --2- 172.21.15.89:0/4123670132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c980b1cc0 0x7f0c98160190 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f0c8800e810 tx=0x7f0c8800ecd0 comp rx=0 tx=0).stop 2023-10-18T11:03:38.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.340+0000 7f0c9f89c640 1 -- 172.21.15.89:0/4123670132 shutdown_connections 2023-10-18T11:03:38.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.340+0000 7f0c9f89c640 1 --2- 172.21.15.89:0/4123670132 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0c7c065680 0x7f0c7c067b40 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:38.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.340+0000 7f0c9f89c640 1 --2- 172.21.15.89:0/4123670132 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0c9815c550 0x7f0c980ca180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:38.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.340+0000 7f0c9f89c640 1 --2- 172.21.15.89:0/4123670132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c980b1cc0 0x7f0c98160190 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:38.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.340+0000 7f0c9f89c640 1 --2- 172.21.15.89:0/4123670132 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c980b1860 0x7f0c98158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:38.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.340+0000 7f0c9f89c640 1 -- 172.21.15.89:0/4123670132 >> 172.21.15.89:0/4123670132 conn(0x7f0c98069d00 msgr2=0x7f0c980b2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:38.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.340+0000 7f0c9f89c640 1 -- 172.21.15.89:0/4123670132 shutdown_connections 2023-10-18T11:03:38.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.340+0000 7f0c9f89c640 1 -- 172.21.15.89:0/4123670132 wait complete. 2023-10-18T11:03:38.358 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T11:03:38.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.500+0000 7f4048834640 1 Processor -- start 2023-10-18T11:03:38.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.500+0000 7f4048834640 1 -- start start 2023-10-18T11:03:38.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.500+0000 7f4048834640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f40440d0760 0x7f40440d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:38.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.500+0000 7f4048834640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40440d1100 con 0x7f404415c940 2023-10-18T11:03:38.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.500+0000 7f4048834640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f404415da50 con 0x7f40440cd160 2023-10-18T11:03:38.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.500+0000 7f4048834640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40440d3110 con 0x7f40440d0760 2023-10-18T11:03:38.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.500+0000 7f404259c640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f404415c940 0x7f40440d2a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39648/0 (socket says 172.21.15.89:39648) 2023-10-18T11:03:38.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.500+0000 7f404259c640 1 -- 172.21.15.89:0/3248530204 learned_addr learned my addr 172.21.15.89:0/3248530204 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:38.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.500+0000 7f4039d9b640 1 --2- 172.21.15.89:0/3248530204 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f40440d0760 0x7f40440d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:38.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.500+0000 7f4041d9b640 1 -- 172.21.15.89:0/3248530204 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1546351512 0 0) 0x7f40440d1100 con 0x7f404415c940 2023-10-18T11:03:38.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.500+0000 7f4041d9b640 1 -- 172.21.15.89:0/3248530204 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4030003530 con 0x7f404415c940 2023-10-18T11:03:38.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.500+0000 7f4041d9b640 1 -- 172.21.15.89:0/3248530204 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4164921613 0 0) 0x7f4030003530 con 0x7f404415c940 2023-10-18T11:03:38.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.500+0000 7f4041d9b640 1 -- 172.21.15.89:0/3248530204 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f40440d0760 msgr2=0x7f40440d0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:38.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.500+0000 7f4041d9b640 1 --2- 172.21.15.89:0/3248530204 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f40440d0760 0x7f40440d0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:38.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.500+0000 7f4041d9b640 1 -- 172.21.15.89:0/3248530204 >> v1:172.21.15.154:6789/0 conn(0x7f40440cd160 legacy=0x7f40440cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:38.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.500+0000 7f4041d9b640 1 -- 172.21.15.89:0/3248530204 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f404415ce40 con 0x7f404415c940 2023-10-18T11:03:38.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.500+0000 7f4041d9b640 1 -- 172.21.15.89:0/3248530204 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7f4034001a00 con 0x7f404415c940 2023-10-18T11:03:38.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.500+0000 7f4041d9b640 1 -- 172.21.15.89:0/3248530204 >> v1:172.21.15.89:6789/0 conn(0x7f404415c940 legacy=0x7f40440d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:38.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.500+0000 7f4041d9b640 1 --2- 172.21.15.89:0/3248530204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40300039b0 0x7f4030003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:38.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.500+0000 7f4041d9b640 1 --2- 172.21.15.89:0/3248530204 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f40440d0760 0x7f40300047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:38.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.500+0000 7f4041d9b640 1 --2- 172.21.15.89:0/3248530204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4030004d10 0x7f4030005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:38.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.504+0000 7f4041d9b640 1 -- 172.21.15.89:0/3248530204 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40440d3110 con 0x7f4030004d10 2023-10-18T11:03:38.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.504+0000 7f4041d9b640 1 -- 172.21.15.89:0/3248530204 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403000b990 con 0x7f40300039b0 2023-10-18T11:03:38.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.504+0000 7f4041d9b640 1 -- 172.21.15.89:0/3248530204 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f403000bb50 con 0x7f40440d0760 2023-10-18T11:03:38.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.504+0000 7f4042d9d640 1 --2- 172.21.15.89:0/3248530204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4030004d10 0x7f4030005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:38.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.504+0000 7f4039d9b640 1 --2- 172.21.15.89:0/3248530204 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f40440d0760 0x7f40300047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:38.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.504+0000 7f404259c640 1 --2- 172.21.15.89:0/3248530204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40300039b0 0x7f4030003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:38.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.504+0000 7f4042d9d640 1 -- 172.21.15.89:0/3248530204 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f40440d0760 msgr2=0x7f40300047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:38.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.504+0000 7f4042d9d640 1 --2- 172.21.15.89:0/3248530204 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f40440d0760 0x7f40300047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:38.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.504+0000 7f4042d9d640 1 -- 172.21.15.89:0/3248530204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40300039b0 msgr2=0x7f4030003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:38.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.504+0000 7f4042d9d640 1 --2- 172.21.15.89:0/3248530204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40300039b0 0x7f4030003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:38.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.504+0000 7f4042d9d640 1 -- 172.21.15.89:0/3248530204 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4030003530 con 0x7f4030004d10 2023-10-18T11:03:38.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.504+0000 7f4042d9d640 1 --2- 172.21.15.89:0/3248530204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4030004d10 0x7f4030005190 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f403c001860 tx=0x7f403c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d8852d1cbcde11 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:38.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.504+0000 7f4041d9b640 1 -- 172.21.15.89:0/3248530204 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403c0ac640 con 0x7f4030004d10 2023-10-18T11:03:38.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.504+0000 7f4041d9b640 1 -- 172.21.15.89:0/3248530204 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f403c0ac7a0 con 0x7f4030004d10 2023-10-18T11:03:38.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.504+0000 7f4041d9b640 1 -- 172.21.15.89:0/3248530204 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403c0acb50 con 0x7f4030004d10 2023-10-18T11:03:38.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.508+0000 7f4048834640 1 -- 172.21.15.89:0/3248530204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4030004d10 msgr2=0x7f4030005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:38.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.508+0000 7f4048834640 1 --2- 172.21.15.89:0/3248530204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4030004d10 0x7f4030005190 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f403c001860 tx=0x7f403c01c4e0 comp rx=0 tx=0).stop 2023-10-18T11:03:38.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.508+0000 7f4042d9d640 1 -- 172.21.15.89:0/3248530204 reap_dead start 2023-10-18T11:03:38.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.508+0000 7f4048834640 1 -- 172.21.15.89:0/3248530204 shutdown_connections 2023-10-18T11:03:38.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.508+0000 7f4048834640 1 -- 172.21.15.89:0/3248530204 >> 172.21.15.89:0/3248530204 conn(0x7f4044089fd0 msgr2=0x7f404409b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:38.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.508+0000 7f4048834640 1 -- 172.21.15.89:0/3248530204 shutdown_connections 2023-10-18T11:03:38.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.508+0000 7f4048834640 1 -- 172.21.15.89:0/3248530204 wait complete. 2023-10-18T11:03:38.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.508+0000 7f4048834640 1 Processor -- start 2023-10-18T11:03:38.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.508+0000 7f4048834640 1 -- start start 2023-10-18T11:03:38.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.508+0000 7f4048834640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4030004d10 0x7f40441f27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:38.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.508+0000 7f4048834640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f40440cb730 0x7f40440b6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:38.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.508+0000 7f4048834640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40441f1510 0x7f40440b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:38.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.508+0000 7f4048834640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40441f2d30 con 0x7f40441f1510 2023-10-18T11:03:38.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.508+0000 7f4048834640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40440c6420 con 0x7f4030004d10 2023-10-18T11:03:38.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.508+0000 7f4048834640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40440cbc10 con 0x7f40440cb730 2023-10-18T11:03:38.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.508+0000 7f4042d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40441f1510 0x7f40440b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:38.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.508+0000 7f4042d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40441f1510 0x7f40440b3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51368/0 (socket says 172.21.15.89:51368) 2023-10-18T11:03:38.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.508+0000 7f4042d9d640 1 -- 172.21.15.89:0/829901828 learned_addr learned my addr 172.21.15.89:0/829901828 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:38.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.512+0000 7f404259c640 1 --2- 172.21.15.89:0/829901828 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4030004d10 0x7f40441f27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:38.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.512+0000 7f4039d9b640 1 --2- 172.21.15.89:0/829901828 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f40440cb730 0x7f40440b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:38.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.512+0000 7f4042d9d640 1 -- 172.21.15.89:0/829901828 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f40440cb730 msgr2=0x7f40440b6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:38.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.512+0000 7f4042d9d640 1 --2- 172.21.15.89:0/829901828 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f40440cb730 0x7f40440b6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:38.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.512+0000 7f4042d9d640 1 -- 172.21.15.89:0/829901828 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4030004d10 msgr2=0x7f40441f27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:38.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.512+0000 7f4042d9d640 1 --2- 172.21.15.89:0/829901828 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4030004d10 0x7f40441f27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:38.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.512+0000 7f4042d9d640 1 -- 172.21.15.89:0/829901828 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40440cbef0 con 0x7f40441f1510 2023-10-18T11:03:38.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.512+0000 7f404259c640 1 --2- 172.21.15.89:0/829901828 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4030004d10 0x7f40441f27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:03:38.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.512+0000 7f4042d9d640 1 --2- 172.21.15.89:0/829901828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40441f1510 0x7f40440b3e40 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f403c0af040 tx=0x7f4030005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:38.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.512+0000 7f403b7fe640 1 -- 172.21.15.89:0/829901828 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403c005270 con 0x7f40441f1510 2023-10-18T11:03:38.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.512+0000 7f403b7fe640 1 -- 172.21.15.89:0/829901828 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f403c0053d0 con 0x7f40441f1510 2023-10-18T11:03:38.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.512+0000 7f403b7fe640 1 -- 172.21.15.89:0/829901828 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403c0069e0 con 0x7f40441f1510 2023-10-18T11:03:38.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.512+0000 7f4048834640 1 -- 172.21.15.89:0/829901828 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40440cc1d0 con 0x7f40441f1510 2023-10-18T11:03:38.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.512+0000 7f4048834640 1 -- 172.21.15.89:0/829901828 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40440b66d0 con 0x7f40441f1510 2023-10-18T11:03:38.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.512+0000 7f4048834640 1 -- 172.21.15.89:0/829901828 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40440c6590 con 0x7f40441f1510 2023-10-18T11:03:38.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.512+0000 7f403b7fe640 1 -- 172.21.15.89:0/829901828 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f403c0b3050 con 0x7f40441f1510 2023-10-18T11:03:38.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.512+0000 7f403b7fe640 1 --2- 172.21.15.89:0/829901828 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4024065680 0x7f4024067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:38.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.512+0000 7f403b7fe640 1 -- 172.21.15.89:0/829901828 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f403c11cfa0 con 0x7f40441f1510 2023-10-18T11:03:38.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.516+0000 7f404259c640 1 --2- 172.21.15.89:0/829901828 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4024065680 0x7f4024067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:38.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.516+0000 7f404259c640 1 --2- 172.21.15.89:0/829901828 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4024065680 0x7f4024067b40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4034004c00 tx=0x7f4034004b70 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:38.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.516+0000 7f403b7fe640 1 -- 172.21.15.89:0/829901828 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f403c0ea9b0 con 0x7f40441f1510 2023-10-18T11:03:38.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.664+0000 7f4048834640 1 -- 172.21.15.89:0/829901828 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f404404dbe0 con 0x7f40441f1510 2023-10-18T11:03:38.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.668+0000 7f403b7fe640 1 -- 172.21.15.89:0/829901828 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) v1 ==== 104+0+11 (secure 0 0 0) 0x7f403c0b10a0 con 0x7f40441f1510 2023-10-18T11:03:38.665 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-18T11:03:38.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.668+0000 7f4048834640 1 -- 172.21.15.89:0/829901828 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4024065680 msgr2=0x7f4024067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:38.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.668+0000 7f4048834640 1 --2- 172.21.15.89:0/829901828 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4024065680 0x7f4024067b40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4034004c00 tx=0x7f4034004b70 comp rx=0 tx=0).stop 2023-10-18T11:03:38.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.668+0000 7f4048834640 1 -- 172.21.15.89:0/829901828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40441f1510 msgr2=0x7f40440b3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:38.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.668+0000 7f4048834640 1 --2- 172.21.15.89:0/829901828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40441f1510 0x7f40440b3e40 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f403c0af040 tx=0x7f4030005800 comp rx=0 tx=0).stop 2023-10-18T11:03:38.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.668+0000 7f4048834640 1 -- 172.21.15.89:0/829901828 shutdown_connections 2023-10-18T11:03:38.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.668+0000 7f4048834640 1 --2- 172.21.15.89:0/829901828 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4024065680 0x7f4024067b40 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:38.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.668+0000 7f4048834640 1 --2- 172.21.15.89:0/829901828 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f40440cb730 0x7f40440b6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:38.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.668+0000 7f4048834640 1 --2- 172.21.15.89:0/829901828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40441f1510 0x7f40440b3e40 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:38.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.668+0000 7f4048834640 1 --2- 172.21.15.89:0/829901828 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4030004d10 0x7f40441f27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:38.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.668+0000 7f4048834640 1 -- 172.21.15.89:0/829901828 >> 172.21.15.89:0/829901828 conn(0x7f4044089fd0 msgr2=0x7f404415e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:38.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.668+0000 7f4048834640 1 -- 172.21.15.89:0/829901828 shutdown_connections 2023-10-18T11:03:38.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.668+0000 7f4048834640 1 -- 172.21.15.89:0/829901828 wait complete. 2023-10-18T11:03:38.686 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T11:03:38.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.828+0000 7f5e36448640 1 Processor -- start 2023-10-18T11:03:38.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.828+0000 7f5e36448640 1 -- start start 2023-10-18T11:03:38.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.828+0000 7f5e36448640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5e3015c550 0x7f5e300d2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:38.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.828+0000 7f5e36448640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e300d1e50 con 0x7f5e300b1860 2023-10-18T11:03:38.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.828+0000 7f5e36448640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e300d2130 con 0x7f5e300cfd20 2023-10-18T11:03:38.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.828+0000 7f5e36448640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e300d2410 con 0x7f5e3015c550 2023-10-18T11:03:38.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.832+0000 7f5e2ffff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5e3015c550 0x7f5e300d2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:38.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.832+0000 7f5e2f7fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5e300b1860 0x7f5e300cf610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39660/0 (socket says 172.21.15.89:39660) 2023-10-18T11:03:38.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.832+0000 7f5e2f7fe640 1 -- 172.21.15.89:0/3553407787 learned_addr learned my addr 172.21.15.89:0/3553407787 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:38.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.832+0000 7f5e2ffff640 1 -- 172.21.15.89:0/3553407787 >> v1:172.21.15.154:6789/0 conn(0x7f5e300cfd20 legacy=0x7f5e300b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:38.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.832+0000 7f5e2ffff640 1 -- 172.21.15.89:0/3553407787 >> v1:172.21.15.89:6789/0 conn(0x7f5e300b1860 legacy=0x7f5e300cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:38.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.832+0000 7f5e2ffff640 1 -- 172.21.15.89:0/3553407787 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e300d26f0 con 0x7f5e3015c550 2023-10-18T11:03:38.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.832+0000 7f5e2ffff640 1 --2- 172.21.15.89:0/3553407787 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e3015c550 0x7f5e300d2ef0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f5e200097c0 tx=0x7f5e2001ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cba2c56d2831e55 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:38.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.832+0000 7f5e2effd640 1 -- 172.21.15.89:0/3553407787 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e200a7070 con 0x7f5e3015c550 2023-10-18T11:03:38.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.832+0000 7f5e2effd640 1 -- 172.21.15.89:0/3553407787 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e200ab430 con 0x7f5e3015c550 2023-10-18T11:03:38.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.832+0000 7f5e2effd640 1 -- 172.21.15.89:0/3553407787 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e200ab7c0 con 0x7f5e3015c550 2023-10-18T11:03:38.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.836+0000 7f5e36448640 1 -- 172.21.15.89:0/3553407787 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e3015c550 msgr2=0x7f5e300d2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:38.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.836+0000 7f5e36448640 1 --2- 172.21.15.89:0/3553407787 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e3015c550 0x7f5e300d2ef0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f5e200097c0 tx=0x7f5e2001ce70 comp rx=0 tx=0).stop 2023-10-18T11:03:38.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.836+0000 7f5e36448640 1 -- 172.21.15.89:0/3553407787 shutdown_connections 2023-10-18T11:03:38.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.836+0000 7f5e36448640 1 --2- 172.21.15.89:0/3553407787 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e3015c550 0x7f5e300d2ef0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:38.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.836+0000 7f5e36448640 1 -- 172.21.15.89:0/3553407787 >> 172.21.15.89:0/3553407787 conn(0x7f5e30069d00 msgr2=0x7f5e3006a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:38.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.836+0000 7f5e36448640 1 -- 172.21.15.89:0/3553407787 shutdown_connections 2023-10-18T11:03:38.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.836+0000 7f5e36448640 1 -- 172.21.15.89:0/3553407787 wait complete. 2023-10-18T11:03:38.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.836+0000 7f5e36448640 1 Processor -- start 2023-10-18T11:03:38.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.836+0000 7f5e36448640 1 -- start start 2023-10-18T11:03:38.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.836+0000 7f5e36448640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e300b1860 0x7f5e300c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:38.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.836+0000 7f5e36448640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e300cfd20 0x7f5e30153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:38.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.836+0000 7f5e36448640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e3015c550 0x7f5e30153540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:38.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.836+0000 7f5e36448640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e300c92d0 con 0x7f5e300cfd20 2023-10-18T11:03:38.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.836+0000 7f5e36448640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e301503c0 con 0x7f5e3015c550 2023-10-18T11:03:38.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.836+0000 7f5e36448640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e300cafd0 con 0x7f5e300b1860 2023-10-18T11:03:38.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.836+0000 7f5e2f7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e300cfd20 0x7f5e30153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:38.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.836+0000 7f5e2f7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e300cfd20 0x7f5e30153000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51378/0 (socket says 172.21.15.89:51378) 2023-10-18T11:03:38.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.836+0000 7f5e2f7fe640 1 -- 172.21.15.89:0/1019660586 learned_addr learned my addr 172.21.15.89:0/1019660586 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:38.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.836+0000 7f5e2ffff640 1 --2- 172.21.15.89:0/1019660586 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e300b1860 0x7f5e300c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:38.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.836+0000 7f5e349e5640 1 --2- 172.21.15.89:0/1019660586 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e3015c550 0x7f5e30153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:38.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.840+0000 7f5e2f7fe640 1 -- 172.21.15.89:0/1019660586 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e300b1860 msgr2=0x7f5e300c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:38.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.840+0000 7f5e2f7fe640 1 --2- 172.21.15.89:0/1019660586 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e300b1860 0x7f5e300c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:38.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.840+0000 7f5e2f7fe640 1 -- 172.21.15.89:0/1019660586 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e3015c550 msgr2=0x7f5e30153540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:38.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.840+0000 7f5e2f7fe640 1 --2- 172.21.15.89:0/1019660586 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e3015c550 0x7f5e30153540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:38.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.840+0000 7f5e2f7fe640 1 -- 172.21.15.89:0/1019660586 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e20009420 con 0x7f5e300cfd20 2023-10-18T11:03:38.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.840+0000 7f5e349e5640 1 --2- 172.21.15.89:0/1019660586 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e3015c550 0x7f5e30153540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:03:38.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.840+0000 7f5e2f7fe640 1 --2- 172.21.15.89:0/1019660586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e300cfd20 0x7f5e30153000 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400c450 tx=0x7f5e2400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:38.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.840+0000 7f5e2cff9640 1 -- 172.21.15.89:0/1019660586 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e24005cf0 con 0x7f5e300cfd20 2023-10-18T11:03:38.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.840+0000 7f5e2cff9640 1 -- 172.21.15.89:0/1019660586 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e24005e50 con 0x7f5e300cfd20 2023-10-18T11:03:38.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.840+0000 7f5e2cff9640 1 -- 172.21.15.89:0/1019660586 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e24007290 con 0x7f5e300cfd20 2023-10-18T11:03:38.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.840+0000 7f5e36448640 1 -- 172.21.15.89:0/1019660586 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e30153a80 con 0x7f5e300cfd20 2023-10-18T11:03:38.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.840+0000 7f5e36448640 1 -- 172.21.15.89:0/1019660586 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e300ca6a0 con 0x7f5e300cfd20 2023-10-18T11:03:38.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.840+0000 7f5e36448640 1 -- 172.21.15.89:0/1019660586 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e3004dbe0 con 0x7f5e300cfd20 2023-10-18T11:03:38.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.840+0000 7f5e2cff9640 1 -- 172.21.15.89:0/1019660586 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f5e240073f0 con 0x7f5e300cfd20 2023-10-18T11:03:38.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.840+0000 7f5e2cff9640 1 --2- 172.21.15.89:0/1019660586 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5e1c065680 0x7f5e1c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:38.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.840+0000 7f5e2cff9640 1 -- 172.21.15.89:0/1019660586 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f5e240824b0 con 0x7f5e300cfd20 2023-10-18T11:03:38.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.840+0000 7f5e2ffff640 1 --2- 172.21.15.89:0/1019660586 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5e1c065680 0x7f5e1c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:38.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.844+0000 7f5e2ffff640 1 --2- 172.21.15.89:0/1019660586 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5e1c065680 0x7f5e1c067b40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5e200af000 tx=0x7f5e20004740 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:38.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.844+0000 7f5e2cff9640 1 -- 172.21.15.89:0/1019660586 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f5e2404fdd0 con 0x7f5e300cfd20 2023-10-18T11:03:38.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.996+0000 7f5e36448640 1 -- 172.21.15.89:0/1019660586 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5e300b48e0 con 0x7f5e300cfd20 2023-10-18T11:03:38.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:38.996+0000 7f5e2cff9640 1 -- 172.21.15.89:0/1019660586 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5e240533f0 con 0x7f5e300cfd20 2023-10-18T11:03:38.996 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-18T11:03:38.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.000+0000 7f5e36448640 1 -- 172.21.15.89:0/1019660586 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5e1c065680 msgr2=0x7f5e1c067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:38.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.000+0000 7f5e36448640 1 --2- 172.21.15.89:0/1019660586 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5e1c065680 0x7f5e1c067b40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5e200af000 tx=0x7f5e20004740 comp rx=0 tx=0).stop 2023-10-18T11:03:38.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.000+0000 7f5e36448640 1 -- 172.21.15.89:0/1019660586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e300cfd20 msgr2=0x7f5e30153000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:38.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.000+0000 7f5e36448640 1 --2- 172.21.15.89:0/1019660586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e300cfd20 0x7f5e30153000 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400c450 tx=0x7f5e2400c910 comp rx=0 tx=0).stop 2023-10-18T11:03:38.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.000+0000 7f5e36448640 1 -- 172.21.15.89:0/1019660586 shutdown_connections 2023-10-18T11:03:38.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.000+0000 7f5e36448640 1 --2- 172.21.15.89:0/1019660586 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f5e1c065680 0x7f5e1c067b40 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:38.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.000+0000 7f5e36448640 1 --2- 172.21.15.89:0/1019660586 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e3015c550 0x7f5e30153540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:38.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.000+0000 7f5e36448640 1 --2- 172.21.15.89:0/1019660586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e300cfd20 0x7f5e30153000 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:38.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.000+0000 7f5e36448640 1 --2- 172.21.15.89:0/1019660586 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5e300b1860 0x7f5e300c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:38.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.000+0000 7f5e36448640 1 -- 172.21.15.89:0/1019660586 >> 172.21.15.89:0/1019660586 conn(0x7f5e30069d00 msgr2=0x7f5e300d0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:38.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.000+0000 7f5e36448640 1 -- 172.21.15.89:0/1019660586 shutdown_connections 2023-10-18T11:03:38.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.000+0000 7f5e36448640 1 -- 172.21.15.89:0/1019660586 wait complete. 2023-10-18T11:03:39.016 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T11:03:39.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b8c52c640 1 Processor -- start 2023-10-18T11:03:39.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b8c52c640 1 -- start start 2023-10-18T11:03:39.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b8c52c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3b8415c940 0x7f3b840d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:39.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b8c52c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b840d2f40 0x7f3b840d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:39.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b8c52c640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b8415da50 con 0x7f3b840d0ca0 2023-10-18T11:03:39.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b8c52c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b840d1130 con 0x7f3b8415c940 2023-10-18T11:03:39.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b8c52c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8415ce40 con 0x7f3b840d2f40 2023-10-18T11:03:39.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b8aac9640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f3b840d0ca0 0x7f3b840cf060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39664/0 (socket says 172.21.15.89:39664) 2023-10-18T11:03:39.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b8aac9640 1 -- 172.21.15.89:0/2556026106 learned_addr learned my addr 172.21.15.89:0/2556026106 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:39.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b89ac7640 1 --2- 172.21.15.89:0/2556026106 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b840d2f40 0x7f3b840d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:39.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b8a2c8640 1 --2- 172.21.15.89:0/2556026106 >> v2:172.21.15.154:3300/0 conn(0x7f3b8415c940 0x7f3b840d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:39.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b892c6640 1 -- 172.21.15.89:0/2556026106 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3911468464 0 0) 0x7f3b8415da50 con 0x7f3b840d0ca0 2023-10-18T11:03:39.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b892c6640 1 -- 172.21.15.89:0/2556026106 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b70003530 con 0x7f3b840d0ca0 2023-10-18T11:03:39.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b892c6640 1 -- 172.21.15.89:0/2556026106 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 776272072 0 0) 0x7f3b70003530 con 0x7f3b840d0ca0 2023-10-18T11:03:39.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b892c6640 1 -- 172.21.15.89:0/2556026106 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b840d2f40 msgr2=0x7f3b840d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:39.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b892c6640 1 --2- 172.21.15.89:0/2556026106 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b840d2f40 0x7f3b840d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:39.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b892c6640 1 -- 172.21.15.89:0/2556026106 >> v2:172.21.15.154:3300/0 conn(0x7f3b8415c940 msgr2=0x7f3b840d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:39.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b892c6640 1 --2- 172.21.15.89:0/2556026106 >> v2:172.21.15.154:3300/0 conn(0x7f3b8415c940 0x7f3b840d2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:39.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b892c6640 1 -- 172.21.15.89:0/2556026106 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b8415d120 con 0x7f3b840d0ca0 2023-10-18T11:03:39.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b892c6640 1 -- 172.21.15.89:0/2556026106 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7f3b7c001a00 con 0x7f3b840d0ca0 2023-10-18T11:03:39.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b892c6640 1 -- 172.21.15.89:0/2556026106 >> v1:172.21.15.89:6789/0 conn(0x7f3b840d0ca0 legacy=0x7f3b840cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:39.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b892c6640 1 --2- 172.21.15.89:0/2556026106 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b70003980 0x7f3b70003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:39.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b892c6640 1 --2- 172.21.15.89:0/2556026106 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b700047d0 0x7f3b70004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:39.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b892c6640 1 --2- 172.21.15.89:0/2556026106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b70005190 0x7f3b7000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:39.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b892c6640 1 -- 172.21.15.89:0/2556026106 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b840d1130 con 0x7f3b70005190 2023-10-18T11:03:39.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b892c6640 1 -- 172.21.15.89:0/2556026106 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8415ce40 con 0x7f3b70003980 2023-10-18T11:03:39.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b892c6640 1 -- 172.21.15.89:0/2556026106 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7000bed0 con 0x7f3b700047d0 2023-10-18T11:03:39.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b8aac9640 1 --2- 172.21.15.89:0/2556026106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b70005190 0x7f3b7000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:39.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b89ac7640 1 --2- 172.21.15.89:0/2556026106 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b700047d0 0x7f3b70004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:39.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b8a2c8640 1 --2- 172.21.15.89:0/2556026106 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b70003980 0x7f3b70003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:39.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b8aac9640 1 -- 172.21.15.89:0/2556026106 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b700047d0 msgr2=0x7f3b70004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:39.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b8aac9640 1 --2- 172.21.15.89:0/2556026106 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b700047d0 0x7f3b70004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:39.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b8aac9640 1 -- 172.21.15.89:0/2556026106 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b70003980 msgr2=0x7f3b70003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:39.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b8aac9640 1 --2- 172.21.15.89:0/2556026106 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b70003980 0x7f3b70003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:39.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b8aac9640 1 -- 172.21.15.89:0/2556026106 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b70003530 con 0x7f3b70005190 2023-10-18T11:03:39.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.156+0000 7f3b8a2c8640 1 --2- 172.21.15.89:0/2556026106 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b70003980 0x7f3b70003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:03:39.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.160+0000 7f3b8aac9640 1 --2- 172.21.15.89:0/2556026106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b70005190 0x7f3b7000b990 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c00a890 tx=0x7f3b7c01d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738c21c99bcd9f83 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:39.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.160+0000 7f3b892c6640 1 -- 172.21.15.89:0/2556026106 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c0ad630 con 0x7f3b70005190 2023-10-18T11:03:39.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.160+0000 7f3b892c6640 1 -- 172.21.15.89:0/2556026106 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b7c0a8070 con 0x7f3b70005190 2023-10-18T11:03:39.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.160+0000 7f3b892c6640 1 -- 172.21.15.89:0/2556026106 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c0ad9f0 con 0x7f3b70005190 2023-10-18T11:03:39.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.160+0000 7f3b8c52c640 1 -- 172.21.15.89:0/2556026106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b70005190 msgr2=0x7f3b7000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:39.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.160+0000 7f3b8c52c640 1 --2- 172.21.15.89:0/2556026106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b70005190 0x7f3b7000b990 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c00a890 tx=0x7f3b7c01d360 comp rx=0 tx=0).stop 2023-10-18T11:03:39.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.160+0000 7f3b8aac9640 1 -- 172.21.15.89:0/2556026106 reap_dead start 2023-10-18T11:03:39.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.160+0000 7f3b8c52c640 1 -- 172.21.15.89:0/2556026106 shutdown_connections 2023-10-18T11:03:39.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.160+0000 7f3b8c52c640 1 -- 172.21.15.89:0/2556026106 >> 172.21.15.89:0/2556026106 conn(0x7f3b84089fd0 msgr2=0x7f3b8409b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:39.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.160+0000 7f3b8c52c640 1 -- 172.21.15.89:0/2556026106 shutdown_connections 2023-10-18T11:03:39.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.160+0000 7f3b8c52c640 1 -- 172.21.15.89:0/2556026106 wait complete. 2023-10-18T11:03:39.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.164+0000 7f3b8c52c640 1 Processor -- start 2023-10-18T11:03:39.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.164+0000 7f3b8c52c640 1 -- start start 2023-10-18T11:03:39.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.164+0000 7f3b8c52c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b70005190 0x7f3b841f1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:39.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.164+0000 7f3b8c52c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b840c6180 0x7f3b841f4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:39.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.164+0000 7f3b8c52c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b840c7e90 0x7f3b840ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:39.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.164+0000 7f3b8c52c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b841f5230 con 0x7f3b840c6180 2023-10-18T11:03:39.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.164+0000 7f3b8c52c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8409c340 con 0x7f3b840c7e90 2023-10-18T11:03:39.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.164+0000 7f3b8c52c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b840c6680 con 0x7f3b70005190 2023-10-18T11:03:39.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.164+0000 7f3b89ac7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b840c6180 0x7f3b841f4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:39.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.164+0000 7f3b89ac7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b840c6180 0x7f3b841f4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51400/0 (socket says 172.21.15.89:51400) 2023-10-18T11:03:39.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.164+0000 7f3b89ac7640 1 -- 172.21.15.89:0/3540562085 learned_addr learned my addr 172.21.15.89:0/3540562085 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:39.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.164+0000 7f3b8aac9640 1 --2- 172.21.15.89:0/3540562085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b840c7e90 0x7f3b840ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:39.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.164+0000 7f3b8a2c8640 1 --2- 172.21.15.89:0/3540562085 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b70005190 0x7f3b841f1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:39.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.164+0000 7f3b89ac7640 1 -- 172.21.15.89:0/3540562085 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b70005190 msgr2=0x7f3b841f1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:39.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.164+0000 7f3b89ac7640 1 --2- 172.21.15.89:0/3540562085 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b70005190 0x7f3b841f1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:39.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.164+0000 7f3b89ac7640 1 -- 172.21.15.89:0/3540562085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b840c7e90 msgr2=0x7f3b840ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:39.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.164+0000 7f3b89ac7640 1 --2- 172.21.15.89:0/3540562085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b840c7e90 0x7f3b840ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:39.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.164+0000 7f3b89ac7640 1 -- 172.21.15.89:0/3540562085 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b840c6960 con 0x7f3b840c6180 2023-10-18T11:03:39.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.164+0000 7f3b8a2c8640 1 --2- 172.21.15.89:0/3540562085 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b70005190 0x7f3b841f1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:03:39.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.164+0000 7f3b8aac9640 1 --2- 172.21.15.89:0/3540562085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b840c7e90 0x7f3b840ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:03:39.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.164+0000 7f3b89ac7640 1 --2- 172.21.15.89:0/3540562085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b840c6180 0x7f3b841f4cf0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f3b74011890 tx=0x7f3b74011d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:39.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.164+0000 7f3b7affd640 1 -- 172.21.15.89:0/3540562085 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b74004240 con 0x7f3b840c6180 2023-10-18T11:03:39.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.164+0000 7f3b7affd640 1 -- 172.21.15.89:0/3540562085 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b740043a0 con 0x7f3b840c6180 2023-10-18T11:03:39.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.164+0000 7f3b7affd640 1 -- 172.21.15.89:0/3540562085 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7400f9e0 con 0x7f3b840c6180 2023-10-18T11:03:39.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.164+0000 7f3b8c52c640 1 -- 172.21.15.89:0/3540562085 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b840c6c40 con 0x7f3b840c6180 2023-10-18T11:03:39.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.164+0000 7f3b8c52c640 1 -- 172.21.15.89:0/3540562085 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b840c8340 con 0x7f3b840c6180 2023-10-18T11:03:39.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.168+0000 7f3b8c52c640 1 -- 172.21.15.89:0/3540562085 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b8404dbe0 con 0x7f3b840c6180 2023-10-18T11:03:39.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.168+0000 7f3b7affd640 1 -- 172.21.15.89:0/3540562085 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f3b7400fb40 con 0x7f3b840c6180 2023-10-18T11:03:39.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.168+0000 7f3b7affd640 1 --2- 172.21.15.89:0/3540562085 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3b68065680 0x7f3b68067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:39.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.168+0000 7f3b7affd640 1 -- 172.21.15.89:0/3540562085 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f3b74083540 con 0x7f3b840c6180 2023-10-18T11:03:39.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.168+0000 7f3b8a2c8640 1 --2- 172.21.15.89:0/3540562085 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3b68065680 0x7f3b68067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:39.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.168+0000 7f3b8a2c8640 1 --2- 172.21.15.89:0/3540562085 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3b68065680 0x7f3b68067b40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3b80009af0 tx=0x7f3b80008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:39.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.172+0000 7f3b7affd640 1 -- 172.21.15.89:0/3540562085 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f3b74050ee0 con 0x7f3b840c6180 2023-10-18T11:03:39.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.352+0000 7f3b8c52c640 1 -- 172.21.15.89:0/3540562085 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b840d3330 con 0x7f3b840c6180 2023-10-18T11:03:39.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.352+0000 7f3b7affd640 1 -- 172.21.15.89:0/3540562085 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4877 (secure 0 0 0) 0x7f3b74054500 con 0x7f3b840c6180 2023-10-18T11:03:39.350 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T11:03:39.350 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/654729641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":654729641},{"type":"v1","addr":"172.21.15.154:6836","nonce":654729641}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/964566480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":964566480},{"type":"v1","addr":"172.21.15.154:6837","nonce":964566480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2261878538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2261878538},{"type":"v1","addr":"172.21.15.89:6833","nonce":2261878538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:47:15.199575+0000","modified":"2023-10-18T10:55:20.892655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5517":{"gid":5517,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":124,"addr":"172.21.15.89:6835/2092274731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2092274731},{"type":"v1","addr":"172.21.15.89:6835","nonce":2092274731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T11:03:39.351 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-18T11:03:39.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.352+0000 7f3b8c52c640 1 -- 172.21.15.89:0/3540562085 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3b68065680 msgr2=0x7f3b68067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:39.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.352+0000 7f3b8c52c640 1 --2- 172.21.15.89:0/3540562085 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3b68065680 0x7f3b68067b40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3b80009af0 tx=0x7f3b80008040 comp rx=0 tx=0).stop 2023-10-18T11:03:39.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.352+0000 7f3b8c52c640 1 -- 172.21.15.89:0/3540562085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b840c6180 msgr2=0x7f3b841f4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:39.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.352+0000 7f3b8c52c640 1 --2- 172.21.15.89:0/3540562085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b840c6180 0x7f3b841f4cf0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f3b74011890 tx=0x7f3b74011d50 comp rx=0 tx=0).stop 2023-10-18T11:03:39.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.352+0000 7f3b8c52c640 1 -- 172.21.15.89:0/3540562085 shutdown_connections 2023-10-18T11:03:39.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.352+0000 7f3b8c52c640 1 --2- 172.21.15.89:0/3540562085 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3b68065680 0x7f3b68067b40 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:39.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.352+0000 7f3b8c52c640 1 --2- 172.21.15.89:0/3540562085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b840c7e90 0x7f3b840ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:39.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.352+0000 7f3b8c52c640 1 --2- 172.21.15.89:0/3540562085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b840c6180 0x7f3b841f4cf0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:39.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.352+0000 7f3b8c52c640 1 --2- 172.21.15.89:0/3540562085 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b70005190 0x7f3b841f1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:39.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.352+0000 7f3b8c52c640 1 -- 172.21.15.89:0/3540562085 >> 172.21.15.89:0/3540562085 conn(0x7f3b84089fd0 msgr2=0x7f3b8415bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:39.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.352+0000 7f3b8c52c640 1 -- 172.21.15.89:0/3540562085 shutdown_connections 2023-10-18T11:03:39.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.352+0000 7f3b8c52c640 1 -- 172.21.15.89:0/3540562085 wait complete. 2023-10-18T11:03:39.370 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T11:03:39.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.508+0000 7fd2cc9ba640 1 Processor -- start 2023-10-18T11:03:39.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.508+0000 7fd2cc9ba640 1 -- start start 2023-10-18T11:03:39.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.508+0000 7fd2cc9ba640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd2c80c9970 0x7fd2c8164230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:39.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.508+0000 7fd2cc9ba640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2c8070b90 con 0x7fd2c8070600 2023-10-18T11:03:39.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.508+0000 7fd2cc9ba640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2c8070e70 con 0x7fd2c8159490 2023-10-18T11:03:39.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.508+0000 7fd2cc9ba640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2c8071150 con 0x7fd2c80c9970 2023-10-18T11:03:39.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.508+0000 7fd2c6d9d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd2c80c9970 0x7fd2c8164230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:39.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.508+0000 7fd2bdd9b640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd2c8070600 0x7fd2c806b7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39674/0 (socket says 172.21.15.89:39674) 2023-10-18T11:03:39.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.508+0000 7fd2bdd9b640 1 -- 172.21.15.89:0/3048076382 learned_addr learned my addr 172.21.15.89:0/3048076382 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:39.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.508+0000 7fd2c6d9d640 1 -- 172.21.15.89:0/3048076382 >> v1:172.21.15.154:6789/0 conn(0x7fd2c8159490 legacy=0x7fd2c815d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:39.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.508+0000 7fd2c6d9d640 1 -- 172.21.15.89:0/3048076382 >> v1:172.21.15.89:6789/0 conn(0x7fd2c8070600 legacy=0x7fd2c806b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:39.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.512+0000 7fd2c6d9d640 1 -- 172.21.15.89:0/3048076382 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c8071430 con 0x7fd2c80c9970 2023-10-18T11:03:39.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.512+0000 7fd2c5d9b640 1 -- 172.21.15.89:0/3048076382 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2966116163 0 0) 0x7fd2c8070b90 con 0x7fd2c8070600 2023-10-18T11:03:39.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.512+0000 7fd2c6d9d640 1 --2- 172.21.15.89:0/3048076382 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2c80c9970 0x7fd2c8164230 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd2c00097c0 tx=0x7fd2c001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2006173ddfa7d46c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:39.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.512+0000 7fd2c5d9b640 1 -- 172.21.15.89:0/3048076382 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c00ac600 con 0x7fd2c80c9970 2023-10-18T11:03:39.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.512+0000 7fd2c5d9b640 1 -- 172.21.15.89:0/3048076382 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c00a8050 con 0x7fd2c80c9970 2023-10-18T11:03:39.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.512+0000 7fd2c5d9b640 1 -- 172.21.15.89:0/3048076382 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c00ac980 con 0x7fd2c80c9970 2023-10-18T11:03:39.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.512+0000 7fd2cc9ba640 1 -- 172.21.15.89:0/3048076382 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2c80c9970 msgr2=0x7fd2c8164230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:39.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.512+0000 7fd2cc9ba640 1 --2- 172.21.15.89:0/3048076382 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2c80c9970 0x7fd2c8164230 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd2c00097c0 tx=0x7fd2c001c3c0 comp rx=0 tx=0).stop 2023-10-18T11:03:39.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.512+0000 7fd2cc9ba640 1 -- 172.21.15.89:0/3048076382 shutdown_connections 2023-10-18T11:03:39.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.512+0000 7fd2cc9ba640 1 --2- 172.21.15.89:0/3048076382 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2c80c9970 0x7fd2c8164230 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:39.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.512+0000 7fd2cc9ba640 1 -- 172.21.15.89:0/3048076382 >> 172.21.15.89:0/3048076382 conn(0x7fd2c8069f00 msgr2=0x7fd2c80cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:39.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.512+0000 7fd2cc9ba640 1 -- 172.21.15.89:0/3048076382 shutdown_connections 2023-10-18T11:03:39.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.512+0000 7fd2cc9ba640 1 -- 172.21.15.89:0/3048076382 wait complete. 2023-10-18T11:03:39.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.516+0000 7fd2cc9ba640 1 Processor -- start 2023-10-18T11:03:39.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.516+0000 7fd2cc9ba640 1 -- start start 2023-10-18T11:03:39.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.516+0000 7fd2cc9ba640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2c8070600 0x7fd2c8147610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:39.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.516+0000 7fd2cc9ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2c80c9970 0x7fd2c80c2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:39.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.516+0000 7fd2cc9ba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2c8159490 0x7fd2c80c3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:39.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.516+0000 7fd2cc9ba640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c81f60c0 con 0x7fd2c8070600 2023-10-18T11:03:39.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.516+0000 7fd2cc9ba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c81f5490 con 0x7fd2c80c9970 2023-10-18T11:03:39.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.516+0000 7fd2cc9ba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c81f5770 con 0x7fd2c8159490 2023-10-18T11:03:39.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.516+0000 7fd2c659c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2c8070600 0x7fd2c8147610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:39.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.516+0000 7fd2c659c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2c8070600 0x7fd2c8147610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51410/0 (socket says 172.21.15.89:51410) 2023-10-18T11:03:39.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.516+0000 7fd2c659c640 1 -- 172.21.15.89:0/3449923940 learned_addr learned my addr 172.21.15.89:0/3449923940 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:39.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.516+0000 7fd2c659c640 1 -- 172.21.15.89:0/3449923940 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2c8159490 msgr2=0x7fd2c80c3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:39.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.516+0000 7fd2c659c640 1 --2- 172.21.15.89:0/3449923940 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2c8159490 0x7fd2c80c3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:39.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.516+0000 7fd2c659c640 1 -- 172.21.15.89:0/3449923940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2c80c9970 msgr2=0x7fd2c80c2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:39.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.516+0000 7fd2c659c640 1 --2- 172.21.15.89:0/3449923940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2c80c9970 0x7fd2c80c2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:39.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.516+0000 7fd2c659c640 1 -- 172.21.15.89:0/3449923940 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c0009420 con 0x7fd2c8070600 2023-10-18T11:03:39.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.516+0000 7fd2c659c640 1 --2- 172.21.15.89:0/3449923940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2c8070600 0x7fd2c8147610 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd2b800c7a0 tx=0x7fd2b800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:39.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.520+0000 7fd2bf7fe640 1 -- 172.21.15.89:0/3449923940 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8003b80 con 0x7fd2c8070600 2023-10-18T11:03:39.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.520+0000 7fd2bf7fe640 1 -- 172.21.15.89:0/3449923940 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b8003ce0 con 0x7fd2c8070600 2023-10-18T11:03:39.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.520+0000 7fd2cc9ba640 1 -- 172.21.15.89:0/3449923940 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c81f5a70 con 0x7fd2c8070600 2023-10-18T11:03:39.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.520+0000 7fd2bf7fe640 1 -- 172.21.15.89:0/3449923940 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b800aa90 con 0x7fd2c8070600 2023-10-18T11:03:39.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.520+0000 7fd2cc9ba640 1 -- 172.21.15.89:0/3449923940 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c80ccc00 con 0x7fd2c8070600 2023-10-18T11:03:39.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.520+0000 7fd2cc9ba640 1 -- 172.21.15.89:0/3449923940 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2c804dbe0 con 0x7fd2c8070600 2023-10-18T11:03:39.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.520+0000 7fd2bf7fe640 1 -- 172.21.15.89:0/3449923940 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fd2b800abf0 con 0x7fd2c8070600 2023-10-18T11:03:39.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.520+0000 7fd2bf7fe640 1 --2- 172.21.15.89:0/3449923940 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd2a80656b0 0x7fd2a8067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:39.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.520+0000 7fd2bf7fe640 1 -- 172.21.15.89:0/3449923940 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fd2b8082e40 con 0x7fd2c8070600 2023-10-18T11:03:39.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.520+0000 7fd2bdd9b640 1 --2- 172.21.15.89:0/3449923940 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd2a80656b0 0x7fd2a8067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:39.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.520+0000 7fd2bdd9b640 1 --2- 172.21.15.89:0/3449923940 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd2a80656b0 0x7fd2a8067b70 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0000c10 tx=0x7fd2b0007910 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:39.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.524+0000 7fd2bf7fe640 1 -- 172.21.15.89:0/3449923940 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fd2b80506c0 con 0x7fd2c8070600 2023-10-18T11:03:39.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.672+0000 7fd2cc9ba640 1 -- 172.21.15.89:0/3449923940 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2c80cbf40 con 0x7fd2c8070600 2023-10-18T11:03:39.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.676+0000 7fd2bf7fe640 1 -- 172.21.15.89:0/3449923940 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+18666 (secure 0 0 0) 0x7fd2b8053ce0 con 0x7fd2c8070600 2023-10-18T11:03:39.673 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T11:03:39.674 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":56,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:55:22.907577+0000","last_up_change":"2023-10-18T10:55:21.905065+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:47:13.188701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-18T10:47:14.198224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":50,"up_from":55,"up_thru":55,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":1133012599},{"type":"v1","addr":"172.21.15.154:6824","nonce":1133012599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1133012599},{"type":"v1","addr":"172.21.15.154:6827","nonce":1133012599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1133012599},{"type":"v1","addr":"172.21.15.154:6831","nonce":1133012599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1133012599},{"type":"v1","addr":"172.21.15.154:6829","nonce":1133012599}]},"public_addr":"172.21.15.154:6824/1133012599","cluster_addr":"172.21.15.154:6827/1133012599","heartbeat_back_addr":"172.21.15.154:6831/1133012599","heartbeat_front_addr":"172.21.15.154:6829/1133012599","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"2023-10-18T10:47:48.955023+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":51},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.9","mappings":[{"from":0,"to":5}]},{"pgid":"7.18","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:0/330049992":"2023-10-18T11:55:17.981058+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T11:03:39.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.676+0000 7fd2cc9ba640 1 -- 172.21.15.89:0/3449923940 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd2a80656b0 msgr2=0x7fd2a8067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:39.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.676+0000 7fd2cc9ba640 1 --2- 172.21.15.89:0/3449923940 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd2a80656b0 0x7fd2a8067b70 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0000c10 tx=0x7fd2b0007910 comp rx=0 tx=0).stop 2023-10-18T11:03:39.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.676+0000 7fd2cc9ba640 1 -- 172.21.15.89:0/3449923940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2c8070600 msgr2=0x7fd2c8147610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:39.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.676+0000 7fd2cc9ba640 1 --2- 172.21.15.89:0/3449923940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2c8070600 0x7fd2c8147610 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd2b800c7a0 tx=0x7fd2b800cc60 comp rx=0 tx=0).stop 2023-10-18T11:03:39.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.676+0000 7fd2cc9ba640 1 -- 172.21.15.89:0/3449923940 shutdown_connections 2023-10-18T11:03:39.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.676+0000 7fd2cc9ba640 1 --2- 172.21.15.89:0/3449923940 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fd2a80656b0 0x7fd2a8067b70 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:39.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.676+0000 7fd2cc9ba640 1 --2- 172.21.15.89:0/3449923940 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2c8159490 0x7fd2c80c3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:39.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.676+0000 7fd2cc9ba640 1 --2- 172.21.15.89:0/3449923940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2c80c9970 0x7fd2c80c2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:39.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.676+0000 7fd2cc9ba640 1 --2- 172.21.15.89:0/3449923940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2c8070600 0x7fd2c8147610 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:39.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.676+0000 7fd2cc9ba640 1 -- 172.21.15.89:0/3449923940 >> 172.21.15.89:0/3449923940 conn(0x7fd2c8069f00 msgr2=0x7fd2c80cc160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:39.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.676+0000 7fd2cc9ba640 1 -- 172.21.15.89:0/3449923940 shutdown_connections 2023-10-18T11:03:39.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.676+0000 7fd2cc9ba640 1 -- 172.21.15.89:0/3449923940 wait complete. 2023-10-18T11:03:39.694 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-18T11:03:39.695 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T11:03:39.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.832+0000 7f32dd7ac640 1 Processor -- start 2023-10-18T11:03:39.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.832+0000 7f32dd7ac640 1 -- start start 2023-10-18T11:03:39.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.836+0000 7f32dd7ac640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f32d80d0bc0 0x7f32d80cd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:39.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.836+0000 7f32dd7ac640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d815da50 con 0x7f32d80d0760 2023-10-18T11:03:39.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.836+0000 7f32dd7ac640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d80d3110 con 0x7f32d815c940 2023-10-18T11:03:39.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.836+0000 7f32dd7ac640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d815ce40 con 0x7f32d80d0bc0 2023-10-18T11:03:39.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.836+0000 7f32ce7fc640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f32d80d0760 0x7f32d80cd160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39682/0 (socket says 172.21.15.89:39682) 2023-10-18T11:03:39.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.836+0000 7f32ce7fc640 1 -- 172.21.15.89:0/844495845 learned_addr learned my addr 172.21.15.89:0/844495845 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:39.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.836+0000 7f32d77fe640 1 --2- 172.21.15.89:0/844495845 >> v2:172.21.15.89:3300/0 conn(0x7f32d80d0bc0 0x7f32d80cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:39.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.836+0000 7f32d77fe640 1 -- 172.21.15.89:0/844495845 >> v1:172.21.15.154:6789/0 conn(0x7f32d815c940 legacy=0x7f32d80d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:39.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.836+0000 7f32d77fe640 1 -- 172.21.15.89:0/844495845 >> v1:172.21.15.89:6789/0 conn(0x7f32d80d0760 legacy=0x7f32d80cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:39.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.836+0000 7f32d67fc640 1 -- 172.21.15.89:0/844495845 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1163395076 0 0) 0x7f32d815da50 con 0x7f32d80d0760 2023-10-18T11:03:39.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.836+0000 7f32d77fe640 1 -- 172.21.15.89:0/844495845 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d815d120 con 0x7f32d80d0bc0 2023-10-18T11:03:39.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.836+0000 7f32d77fe640 1 --2- 172.21.15.89:0/844495845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32d80d0bc0 0x7f32d80cd870 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f32d00097c0 tx=0x7f32d001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d562e9eb50564 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:39.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.836+0000 7f32d67fc640 1 -- 172.21.15.89:0/844495845 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d00ab650 con 0x7f32d80d0bc0 2023-10-18T11:03:39.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.836+0000 7f32d67fc640 1 -- 172.21.15.89:0/844495845 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d80ce5c0 con 0x7f32d80d0bc0 2023-10-18T11:03:39.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.836+0000 7f32d67fc640 1 -- 172.21.15.89:0/844495845 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d00ab9d0 con 0x7f32d80d0bc0 2023-10-18T11:03:39.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.840+0000 7f32dd7ac640 1 -- 172.21.15.89:0/844495845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32d80d0bc0 msgr2=0x7f32d80cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:39.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.840+0000 7f32dd7ac640 1 --2- 172.21.15.89:0/844495845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32d80d0bc0 0x7f32d80cd870 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f32d00097c0 tx=0x7f32d001c450 comp rx=0 tx=0).stop 2023-10-18T11:03:39.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.840+0000 7f32dd7ac640 1 -- 172.21.15.89:0/844495845 shutdown_connections 2023-10-18T11:03:39.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.840+0000 7f32dd7ac640 1 --2- 172.21.15.89:0/844495845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32d80d0bc0 0x7f32d80cd870 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:39.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.840+0000 7f32dd7ac640 1 -- 172.21.15.89:0/844495845 >> 172.21.15.89:0/844495845 conn(0x7f32d8089fd0 msgr2=0x7f32d809b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:39.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.840+0000 7f32dd7ac640 1 -- 172.21.15.89:0/844495845 shutdown_connections 2023-10-18T11:03:39.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.840+0000 7f32dd7ac640 1 -- 172.21.15.89:0/844495845 wait complete. 2023-10-18T11:03:39.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.840+0000 7f32dd7ac640 1 Processor -- start 2023-10-18T11:03:39.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.840+0000 7f32dd7ac640 1 -- start start 2023-10-18T11:03:39.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.840+0000 7f32dd7ac640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32d80d0760 0x7f32d80b7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:39.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.840+0000 7f32dd7ac640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32d80d0bc0 0x7f32d815e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:39.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.840+0000 7f32dd7ac640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32d80ca200 0x7f32d80ca690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:39.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.840+0000 7f32dd7ac640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d80c9d70 con 0x7f32d80d0bc0 2023-10-18T11:03:39.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.840+0000 7f32dd7ac640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d8153dc0 con 0x7f32d80ca200 2023-10-18T11:03:39.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.840+0000 7f32dd7ac640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d80b81d0 con 0x7f32d80d0760 2023-10-18T11:03:39.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.840+0000 7f32ce7fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32d80d0bc0 0x7f32d815e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:39.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.840+0000 7f32ce7fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32d80d0bc0 0x7f32d815e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51420/0 (socket says 172.21.15.89:51420) 2023-10-18T11:03:39.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.840+0000 7f32ce7fc640 1 -- 172.21.15.89:0/2691106134 learned_addr learned my addr 172.21.15.89:0/2691106134 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:39.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.840+0000 7f32d6ffd640 1 --2- 172.21.15.89:0/2691106134 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32d80d0760 0x7f32d80b7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:39.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.840+0000 7f32d77fe640 1 --2- 172.21.15.89:0/2691106134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32d80ca200 0x7f32d80ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:39.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.844+0000 7f32ce7fc640 1 -- 172.21.15.89:0/2691106134 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32d80d0760 msgr2=0x7f32d80b7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:39.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.844+0000 7f32ce7fc640 1 --2- 172.21.15.89:0/2691106134 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32d80d0760 0x7f32d80b7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:39.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.844+0000 7f32ce7fc640 1 -- 172.21.15.89:0/2691106134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32d80ca200 msgr2=0x7f32d80ca690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:39.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.844+0000 7f32ce7fc640 1 --2- 172.21.15.89:0/2691106134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32d80ca200 0x7f32d80ca690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:39.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.844+0000 7f32ce7fc640 1 -- 172.21.15.89:0/2691106134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d0009420 con 0x7f32d80d0bc0 2023-10-18T11:03:39.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.844+0000 7f32ce7fc640 1 --2- 172.21.15.89:0/2691106134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32d80d0bc0 0x7f32d815e8c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f32c000c4d0 tx=0x7f32c000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:39.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.844+0000 7f32cffff640 1 -- 172.21.15.89:0/2691106134 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c00074e0 con 0x7f32d80d0bc0 2023-10-18T11:03:39.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.844+0000 7f32cffff640 1 -- 172.21.15.89:0/2691106134 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c0007640 con 0x7f32d80d0bc0 2023-10-18T11:03:39.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.844+0000 7f32dd7ac640 1 -- 172.21.15.89:0/2691106134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d80cc0b0 con 0x7f32d80d0bc0 2023-10-18T11:03:39.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.844+0000 7f32cffff640 1 -- 172.21.15.89:0/2691106134 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0004a30 con 0x7f32d80d0bc0 2023-10-18T11:03:39.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.844+0000 7f32dd7ac640 1 -- 172.21.15.89:0/2691106134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d80c9720 con 0x7f32d80d0bc0 2023-10-18T11:03:39.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.844+0000 7f32dd7ac640 1 -- 172.21.15.89:0/2691106134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32d80c9f20 con 0x7f32d80d0bc0 2023-10-18T11:03:39.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.844+0000 7f32cffff640 1 -- 172.21.15.89:0/2691106134 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f32c0004b90 con 0x7f32d80d0bc0 2023-10-18T11:03:39.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.844+0000 7f32cffff640 1 --2- 172.21.15.89:0/2691106134 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f32b8065680 0x7f32b8067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:39.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.844+0000 7f32cffff640 1 -- 172.21.15.89:0/2691106134 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f32c0082410 con 0x7f32d80d0bc0 2023-10-18T11:03:39.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.848+0000 7f32d6ffd640 1 --2- 172.21.15.89:0/2691106134 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f32b8065680 0x7f32b8067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:39.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.848+0000 7f32d6ffd640 1 --2- 172.21.15.89:0/2691106134 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f32b8065680 0x7f32b8067b40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f32d81583c0 tx=0x7f32c8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:39.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:39.848+0000 7f32cffff640 1 -- 172.21.15.89:0/2691106134 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f32d80c9f20 con 0x7f32d80d0bc0 2023-10-18T11:03:40.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.032+0000 7f32dd7ac640 1 -- 172.21.15.89:0/2691106134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32d80c9f20 con 0x7f32d80d0bc0 2023-10-18T11:03:40.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.032+0000 7f32cffff640 1 -- 172.21.15.89:0/2691106134 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4877 (secure 0 0 0) 0x7f32d80c9f20 con 0x7f32d80d0bc0 2023-10-18T11:03:40.031 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T11:03:40.031 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/654729641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":654729641},{"type":"v1","addr":"172.21.15.154:6836","nonce":654729641}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/964566480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":964566480},{"type":"v1","addr":"172.21.15.154:6837","nonce":964566480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2261878538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2261878538},{"type":"v1","addr":"172.21.15.89:6833","nonce":2261878538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:47:15.199575+0000","modified":"2023-10-18T10:55:20.892655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5517":{"gid":5517,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":124,"addr":"172.21.15.89:6835/2092274731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2092274731},{"type":"v1","addr":"172.21.15.89:6835","nonce":2092274731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T11:03:40.032 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-18T11:03:40.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.032+0000 7f32dd7ac640 1 -- 172.21.15.89:0/2691106134 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f32b8065680 msgr2=0x7f32b8067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:40.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.032+0000 7f32dd7ac640 1 --2- 172.21.15.89:0/2691106134 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f32b8065680 0x7f32b8067b40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f32d81583c0 tx=0x7f32c8009040 comp rx=0 tx=0).stop 2023-10-18T11:03:40.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.032+0000 7f32dd7ac640 1 -- 172.21.15.89:0/2691106134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32d80d0bc0 msgr2=0x7f32d815e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:40.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.032+0000 7f32dd7ac640 1 --2- 172.21.15.89:0/2691106134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32d80d0bc0 0x7f32d815e8c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f32c000c4d0 tx=0x7f32c000c990 comp rx=0 tx=0).stop 2023-10-18T11:03:40.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.032+0000 7f32dd7ac640 1 -- 172.21.15.89:0/2691106134 shutdown_connections 2023-10-18T11:03:40.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.032+0000 7f32dd7ac640 1 --2- 172.21.15.89:0/2691106134 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f32b8065680 0x7f32b8067b40 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:40.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.032+0000 7f32dd7ac640 1 --2- 172.21.15.89:0/2691106134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32d80ca200 0x7f32d80ca690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:40.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.032+0000 7f32dd7ac640 1 --2- 172.21.15.89:0/2691106134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32d80d0bc0 0x7f32d815e8c0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:40.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.032+0000 7f32dd7ac640 1 --2- 172.21.15.89:0/2691106134 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32d80d0760 0x7f32d80b7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:40.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.032+0000 7f32dd7ac640 1 -- 172.21.15.89:0/2691106134 >> 172.21.15.89:0/2691106134 conn(0x7f32d8089fd0 msgr2=0x7f32d80b4400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:40.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.032+0000 7f32dd7ac640 1 -- 172.21.15.89:0/2691106134 shutdown_connections 2023-10-18T11:03:40.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.032+0000 7f32dd7ac640 1 -- 172.21.15.89:0/2691106134 wait complete. 2023-10-18T11:03:40.052 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T11:03:40.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.192+0000 7f65a5c5f640 1 Processor -- start 2023-10-18T11:03:40.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.192+0000 7f65a5c5f640 1 -- start start 2023-10-18T11:03:40.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.192+0000 7f65a5c5f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65a00d0bc0 0x7f65a00cd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:40.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.192+0000 7f65a5c5f640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65a015da50 con 0x7f65a00d0760 2023-10-18T11:03:40.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.192+0000 7f65a5c5f640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65a00d3110 con 0x7f65a015c940 2023-10-18T11:03:40.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.192+0000 7f65a5c5f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a015ce40 con 0x7f65a00d0bc0 2023-10-18T11:03:40.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.192+0000 7f659effd640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f65a00d0760 0x7f65a00cd160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39690/0 (socket says 172.21.15.89:39690) 2023-10-18T11:03:40.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.192+0000 7f659effd640 1 -- 172.21.15.89:0/430449003 learned_addr learned my addr 172.21.15.89:0/430449003 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:40.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.192+0000 7f659ffff640 1 --2- 172.21.15.89:0/430449003 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65a00d0bc0 0x7f65a00cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:40.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.192+0000 7f659e7fc640 1 -- 172.21.15.89:0/430449003 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1640412797 0 0) 0x7f65a015da50 con 0x7f65a00d0760 2023-10-18T11:03:40.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.192+0000 7f659e7fc640 1 -- 172.21.15.89:0/430449003 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6588003530 con 0x7f65a00d0760 2023-10-18T11:03:40.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.196+0000 7f659ffff640 1 -- 172.21.15.89:0/430449003 >> v1:172.21.15.154:6789/0 conn(0x7f65a015c940 legacy=0x7f65a00d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:40.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.196+0000 7f659ffff640 1 -- 172.21.15.89:0/430449003 >> v1:172.21.15.89:6789/0 conn(0x7f65a00d0760 legacy=0x7f65a00cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:40.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.196+0000 7f659ffff640 1 -- 172.21.15.89:0/430449003 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a015d120 con 0x7f65a00d0bc0 2023-10-18T11:03:40.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.196+0000 7f659ffff640 1 --2- 172.21.15.89:0/430449003 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65a00d0bc0 0x7f65a00cd870 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6598009dc0 tx=0x7f659801c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3015eefc9fa2ff1c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:40.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.196+0000 7f659e7fc640 1 -- 172.21.15.89:0/430449003 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65980a5a00 con 0x7f65a00d0bc0 2023-10-18T11:03:40.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.196+0000 7f659e7fc640 1 -- 172.21.15.89:0/430449003 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a00ce5c0 con 0x7f65a00d0bc0 2023-10-18T11:03:40.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.196+0000 7f659e7fc640 1 -- 172.21.15.89:0/430449003 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65980a5d60 con 0x7f65a00d0bc0 2023-10-18T11:03:40.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.200+0000 7f65a5c5f640 1 -- 172.21.15.89:0/430449003 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65a00d0bc0 msgr2=0x7f65a00cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:40.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.200+0000 7f65a5c5f640 1 --2- 172.21.15.89:0/430449003 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65a00d0bc0 0x7f65a00cd870 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6598009dc0 tx=0x7f659801c940 comp rx=0 tx=0).stop 2023-10-18T11:03:40.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.200+0000 7f65a5c5f640 1 -- 172.21.15.89:0/430449003 shutdown_connections 2023-10-18T11:03:40.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.200+0000 7f65a5c5f640 1 --2- 172.21.15.89:0/430449003 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65a00d0bc0 0x7f65a00cd870 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:40.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.200+0000 7f65a5c5f640 1 -- 172.21.15.89:0/430449003 >> 172.21.15.89:0/430449003 conn(0x7f65a0089fd0 msgr2=0x7f65a009b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:40.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.200+0000 7f65a5c5f640 1 -- 172.21.15.89:0/430449003 shutdown_connections 2023-10-18T11:03:40.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.200+0000 7f65a5c5f640 1 -- 172.21.15.89:0/430449003 wait complete. 2023-10-18T11:03:40.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.200+0000 7f65a5c5f640 1 Processor -- start 2023-10-18T11:03:40.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.200+0000 7f65a5c5f640 1 -- start start 2023-10-18T11:03:40.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.200+0000 7f65a5c5f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65a00d0760 0x7f65a0151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:40.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.200+0000 7f65a5c5f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65a00d0bc0 0x7f65a01556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:40.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.200+0000 7f65a5c5f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65a0155c10 0x7f65a01560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:40.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.200+0000 7f65a5c5f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a0151ab0 con 0x7f65a00d0bc0 2023-10-18T11:03:40.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.200+0000 7f65a5c5f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a0148570 con 0x7f65a0155c10 2023-10-18T11:03:40.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.200+0000 7f65a5c5f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a01565e0 con 0x7f65a00d0760 2023-10-18T11:03:40.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.200+0000 7f659effd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65a00d0bc0 0x7f65a01556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:40.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.200+0000 7f659effd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65a00d0bc0 0x7f65a01556d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51432/0 (socket says 172.21.15.89:51432) 2023-10-18T11:03:40.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.200+0000 7f659effd640 1 -- 172.21.15.89:0/3913400713 learned_addr learned my addr 172.21.15.89:0/3913400713 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:40.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.200+0000 7f659f7fe640 1 --2- 172.21.15.89:0/3913400713 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65a00d0760 0x7f65a0151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:40.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.200+0000 7f659ffff640 1 --2- 172.21.15.89:0/3913400713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65a0155c10 0x7f65a01560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:40.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.204+0000 7f659effd640 1 -- 172.21.15.89:0/3913400713 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65a00d0760 msgr2=0x7f65a0151570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:40.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.204+0000 7f659effd640 1 --2- 172.21.15.89:0/3913400713 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65a00d0760 0x7f65a0151570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:40.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.204+0000 7f659effd640 1 -- 172.21.15.89:0/3913400713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65a0155c10 msgr2=0x7f65a01560a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:40.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.204+0000 7f659effd640 1 --2- 172.21.15.89:0/3913400713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65a0155c10 0x7f65a01560a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:40.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.204+0000 7f659effd640 1 -- 172.21.15.89:0/3913400713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6598009a70 con 0x7f65a00d0bc0 2023-10-18T11:03:40.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.204+0000 7f659ffff640 1 --2- 172.21.15.89:0/3913400713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65a0155c10 0x7f65a01560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:03:40.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.204+0000 7f659effd640 1 --2- 172.21.15.89:0/3913400713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65a00d0bc0 0x7f65a01556d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f659400c4d0 tx=0x7f659400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:40.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.204+0000 7f6587fff640 1 -- 172.21.15.89:0/3913400713 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6594005cf0 con 0x7f65a00d0bc0 2023-10-18T11:03:40.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.204+0000 7f6587fff640 1 -- 172.21.15.89:0/3913400713 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6594005e50 con 0x7f65a00d0bc0 2023-10-18T11:03:40.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.204+0000 7f6587fff640 1 -- 172.21.15.89:0/3913400713 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6594007270 con 0x7f65a00d0bc0 2023-10-18T11:03:40.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.204+0000 7f65a5c5f640 1 -- 172.21.15.89:0/3913400713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a0156880 con 0x7f65a00d0bc0 2023-10-18T11:03:40.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.204+0000 7f65a5c5f640 1 -- 172.21.15.89:0/3913400713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a0156e40 con 0x7f65a00d0bc0 2023-10-18T11:03:40.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.204+0000 7f65a5c5f640 1 -- 172.21.15.89:0/3913400713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65a0157060 con 0x7f65a00d0bc0 2023-10-18T11:03:40.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.204+0000 7f6587fff640 1 -- 172.21.15.89:0/3913400713 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f65940073d0 con 0x7f65a00d0bc0 2023-10-18T11:03:40.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.204+0000 7f6587fff640 1 --2- 172.21.15.89:0/3913400713 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f658c065680 0x7f658c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:40.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.204+0000 7f6587fff640 1 -- 172.21.15.89:0/3913400713 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f6594082580 con 0x7f65a00d0bc0 2023-10-18T11:03:40.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.204+0000 7f659f7fe640 1 --2- 172.21.15.89:0/3913400713 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f658c065680 0x7f658c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:40.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.208+0000 7f659f7fe640 1 --2- 172.21.15.89:0/3913400713 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f658c065680 0x7f658c067b40 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6590007fd0 tx=0x7f6590009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:40.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.208+0000 7f6587fff640 1 -- 172.21.15.89:0/3913400713 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f659404fea0 con 0x7f65a00d0bc0 2023-10-18T11:03:40.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.388+0000 7f65a5c5f640 1 -- 172.21.15.89:0/3913400713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65a004dbe0 con 0x7f65a00d0bc0 2023-10-18T11:03:40.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.388+0000 7f6587fff640 1 -- 172.21.15.89:0/3913400713 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4877 (secure 0 0 0) 0x7f65940534c0 con 0x7f65a00d0bc0 2023-10-18T11:03:40.387 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T11:03:40.387 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/654729641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":654729641},{"type":"v1","addr":"172.21.15.154:6836","nonce":654729641}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/964566480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":964566480},{"type":"v1","addr":"172.21.15.154:6837","nonce":964566480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2261878538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2261878538},{"type":"v1","addr":"172.21.15.89:6833","nonce":2261878538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:47:15.199575+0000","modified":"2023-10-18T10:55:20.892655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5517":{"gid":5517,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":124,"addr":"172.21.15.89:6835/2092274731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2092274731},{"type":"v1","addr":"172.21.15.89:6835","nonce":2092274731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T11:03:40.388 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-18T11:03:40.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.388+0000 7f65a5c5f640 1 -- 172.21.15.89:0/3913400713 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f658c065680 msgr2=0x7f658c067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:40.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.388+0000 7f65a5c5f640 1 --2- 172.21.15.89:0/3913400713 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f658c065680 0x7f658c067b40 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6590007fd0 tx=0x7f6590009040 comp rx=0 tx=0).stop 2023-10-18T11:03:40.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.388+0000 7f65a5c5f640 1 -- 172.21.15.89:0/3913400713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65a00d0bc0 msgr2=0x7f65a01556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:40.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.388+0000 7f65a5c5f640 1 --2- 172.21.15.89:0/3913400713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65a00d0bc0 0x7f65a01556d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f659400c4d0 tx=0x7f659400c990 comp rx=0 tx=0).stop 2023-10-18T11:03:40.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.392+0000 7f65a5c5f640 1 -- 172.21.15.89:0/3913400713 shutdown_connections 2023-10-18T11:03:40.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.392+0000 7f65a5c5f640 1 --2- 172.21.15.89:0/3913400713 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f658c065680 0x7f658c067b40 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:40.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.392+0000 7f65a5c5f640 1 --2- 172.21.15.89:0/3913400713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65a0155c10 0x7f65a01560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:40.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.392+0000 7f65a5c5f640 1 --2- 172.21.15.89:0/3913400713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65a00d0bc0 0x7f65a01556d0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:40.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.392+0000 7f65a5c5f640 1 --2- 172.21.15.89:0/3913400713 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65a00d0760 0x7f65a0151570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:40.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.392+0000 7f65a5c5f640 1 -- 172.21.15.89:0/3913400713 >> 172.21.15.89:0/3913400713 conn(0x7f65a0089fd0 msgr2=0x7f65a015e760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:40.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.392+0000 7f65a5c5f640 1 -- 172.21.15.89:0/3913400713 shutdown_connections 2023-10-18T11:03:40.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.392+0000 7f65a5c5f640 1 -- 172.21.15.89:0/3913400713 wait complete. 2023-10-18T11:03:40.408 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T11:03:40.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.548+0000 7f89f78e5640 1 Processor -- start 2023-10-18T11:03:40.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.548+0000 7f89f78e5640 1 -- start start 2023-10-18T11:03:40.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.548+0000 7f89f78e5640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f89f015c940 0x7f89f00d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:40.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.548+0000 7f89f78e5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89f00d0e70 0x7f89f00cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:40.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.548+0000 7f89f78e5640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89f015da50 con 0x7f89f00d2f40 2023-10-18T11:03:40.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.548+0000 7f89f78e5640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89f015ce40 con 0x7f89f015c940 2023-10-18T11:03:40.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.548+0000 7f89f78e5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f015d120 con 0x7f89f00d0e70 2023-10-18T11:03:40.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.548+0000 7f89f5681640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f89f015c940 0x7f89f00d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:40.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.548+0000 7f89f4e80640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f89f00d2f40 0x7f89f00d0760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39696/0 (socket says 172.21.15.89:39696) 2023-10-18T11:03:40.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.548+0000 7f89f4e80640 1 -- 172.21.15.89:0/3331664872 learned_addr learned my addr 172.21.15.89:0/3331664872 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:40.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.548+0000 7f89f5e82640 1 --2- 172.21.15.89:0/3331664872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89f00d0e70 0x7f89f00cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:40.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.548+0000 7f89f5681640 1 -- 172.21.15.89:0/3331664872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89f00d0e70 msgr2=0x7f89f00cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:40.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.548+0000 7f89f5681640 1 --2- 172.21.15.89:0/3331664872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89f00d0e70 0x7f89f00cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:40.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.548+0000 7f89f5681640 1 -- 172.21.15.89:0/3331664872 >> v1:172.21.15.89:6789/0 conn(0x7f89f00d2f40 legacy=0x7f89f00d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:40.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.548+0000 7f89f5681640 1 -- 172.21.15.89:0/3331664872 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f015d400 con 0x7f89f015c940 2023-10-18T11:03:40.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.552+0000 7f89f5681640 1 --2- 172.21.15.89:0/3331664872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89f015c940 0x7f89f00d2a00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f89e40097c0 tx=0x7f89e401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f593811977b7af4 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:40.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.552+0000 7f89e3fff640 1 -- 172.21.15.89:0/3331664872 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e40ab5f0 con 0x7f89f015c940 2023-10-18T11:03:40.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.552+0000 7f89e3fff640 1 -- 172.21.15.89:0/3331664872 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e40ab750 con 0x7f89f015c940 2023-10-18T11:03:40.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.552+0000 7f89e3fff640 1 -- 172.21.15.89:0/3331664872 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e40abae0 con 0x7f89f015c940 2023-10-18T11:03:40.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.552+0000 7f89f78e5640 1 -- 172.21.15.89:0/3331664872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89f015c940 msgr2=0x7f89f00d2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:40.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.552+0000 7f89f78e5640 1 --2- 172.21.15.89:0/3331664872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89f015c940 0x7f89f00d2a00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f89e40097c0 tx=0x7f89e401c450 comp rx=0 tx=0).stop 2023-10-18T11:03:40.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.552+0000 7f89f78e5640 1 -- 172.21.15.89:0/3331664872 shutdown_connections 2023-10-18T11:03:40.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.552+0000 7f89f78e5640 1 --2- 172.21.15.89:0/3331664872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89f00d0e70 0x7f89f00cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:40.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.552+0000 7f89f78e5640 1 --2- 172.21.15.89:0/3331664872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89f015c940 0x7f89f00d2a00 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:40.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.552+0000 7f89f78e5640 1 -- 172.21.15.89:0/3331664872 >> 172.21.15.89:0/3331664872 conn(0x7f89f0089fd0 msgr2=0x7f89f009b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:40.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.552+0000 7f89f78e5640 1 -- 172.21.15.89:0/3331664872 shutdown_connections 2023-10-18T11:03:40.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.552+0000 7f89f78e5640 1 -- 172.21.15.89:0/3331664872 wait complete. 2023-10-18T11:03:40.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.556+0000 7f89f78e5640 1 Processor -- start 2023-10-18T11:03:40.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.556+0000 7f89f78e5640 1 -- start start 2023-10-18T11:03:40.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.556+0000 7f89f78e5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89f00d0e70 0x7f89f00b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:40.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.556+0000 7f89f78e5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89f00c8430 0x7f89f0154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:40.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.556+0000 7f89f78e5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89f00b5270 0x7f89f0154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:40.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.556+0000 7f89f78e5640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f015a030 con 0x7f89f00d0e70 2023-10-18T11:03:40.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.556+0000 7f89f78e5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f01552e0 con 0x7f89f00c8430 2023-10-18T11:03:40.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.556+0000 7f89f78e5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f00c89a0 con 0x7f89f00b5270 2023-10-18T11:03:40.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.556+0000 7f89f5681640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89f00d0e70 0x7f89f00b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:40.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.556+0000 7f89f5681640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89f00d0e70 0x7f89f00b77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51452/0 (socket says 172.21.15.89:51452) 2023-10-18T11:03:40.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.556+0000 7f89f5681640 1 -- 172.21.15.89:0/1164416094 learned_addr learned my addr 172.21.15.89:0/1164416094 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:40.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.556+0000 7f89f5e82640 1 --2- 172.21.15.89:0/1164416094 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89f00b5270 0x7f89f0154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:40.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.556+0000 7f89f4e80640 1 --2- 172.21.15.89:0/1164416094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89f00c8430 0x7f89f0154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:40.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.556+0000 7f89f5681640 1 -- 172.21.15.89:0/1164416094 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89f00b5270 msgr2=0x7f89f0154da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:40.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.556+0000 7f89f5681640 1 --2- 172.21.15.89:0/1164416094 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89f00b5270 0x7f89f0154da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:40.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.556+0000 7f89f5681640 1 -- 172.21.15.89:0/1164416094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89f00c8430 msgr2=0x7f89f0154860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:40.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.556+0000 7f89f5681640 1 --2- 172.21.15.89:0/1164416094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89f00c8430 0x7f89f0154860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:40.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.556+0000 7f89f5681640 1 -- 172.21.15.89:0/1164416094 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e4009420 con 0x7f89f00d0e70 2023-10-18T11:03:40.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.556+0000 7f89f4e80640 1 --2- 172.21.15.89:0/1164416094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89f00c8430 0x7f89f0154860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:03:40.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.556+0000 7f89f5681640 1 --2- 172.21.15.89:0/1164416094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89f00d0e70 0x7f89f00b77e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f89e40ad040 tx=0x7f89e401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:40.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.556+0000 7f89e1ffb640 1 -- 172.21.15.89:0/1164416094 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e40bf610 con 0x7f89f00d0e70 2023-10-18T11:03:40.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.556+0000 7f89e1ffb640 1 -- 172.21.15.89:0/1164416094 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e40bf770 con 0x7f89f00d0e70 2023-10-18T11:03:40.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.556+0000 7f89e1ffb640 1 -- 172.21.15.89:0/1164416094 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e40bfb20 con 0x7f89f00d0e70 2023-10-18T11:03:40.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.556+0000 7f89f78e5640 1 -- 172.21.15.89:0/1164416094 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f00c8c80 con 0x7f89f00d0e70 2023-10-18T11:03:40.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.556+0000 7f89f78e5640 1 -- 172.21.15.89:0/1164416094 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f00c9240 con 0x7f89f00d0e70 2023-10-18T11:03:40.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.556+0000 7f89f78e5640 1 -- 172.21.15.89:0/1164416094 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89f015a170 con 0x7f89f00d0e70 2023-10-18T11:03:40.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.560+0000 7f89e1ffb640 1 -- 172.21.15.89:0/1164416094 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f89e40bfc80 con 0x7f89f00d0e70 2023-10-18T11:03:40.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.560+0000 7f89e1ffb640 1 --2- 172.21.15.89:0/1164416094 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f89d40656b0 0x7f89d4067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:40.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.560+0000 7f89e1ffb640 1 -- 172.21.15.89:0/1164416094 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f89e40a8050 con 0x7f89f00d0e70 2023-10-18T11:03:40.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.560+0000 7f89f4e80640 1 --2- 172.21.15.89:0/1164416094 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f89d40656b0 0x7f89d4067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:40.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.560+0000 7f89f4e80640 1 --2- 172.21.15.89:0/1164416094 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f89d40656b0 0x7f89d4067b70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f89ec007fd0 tx=0x7f89ec009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:40.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.564+0000 7f89e1ffb640 1 -- 172.21.15.89:0/1164416094 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f89f015a170 con 0x7f89f00d0e70 2023-10-18T11:03:40.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.712+0000 7f89f78e5640 1 -- 172.21.15.89:0/1164416094 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89f015a170 con 0x7f89f00d0e70 2023-10-18T11:03:40.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.712+0000 7f89e1ffb640 1 -- 172.21.15.89:0/1164416094 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+18666 (secure 0 0 0) 0x7f89f015a170 con 0x7f89f00d0e70 2023-10-18T11:03:40.711 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T11:03:40.711 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":56,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T10:55:22.907577+0000","last_up_change":"2023-10-18T10:55:21.905065+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:47:13.188701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-18T10:47:14.198224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":50,"up_from":55,"up_thru":55,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":1133012599},{"type":"v1","addr":"172.21.15.154:6824","nonce":1133012599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1133012599},{"type":"v1","addr":"172.21.15.154:6827","nonce":1133012599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1133012599},{"type":"v1","addr":"172.21.15.154:6831","nonce":1133012599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1133012599},{"type":"v1","addr":"172.21.15.154:6829","nonce":1133012599}]},"public_addr":"172.21.15.154:6824/1133012599","cluster_addr":"172.21.15.154:6827/1133012599","heartbeat_back_addr":"172.21.15.154:6831/1133012599","heartbeat_front_addr":"172.21.15.154:6829/1133012599","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"2023-10-18T10:47:48.955023+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":51},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.9","mappings":[{"from":0,"to":5}]},{"pgid":"7.18","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:0/330049992":"2023-10-18T11:55:17.981058+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T11:03:40.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.716+0000 7f89f78e5640 1 -- 172.21.15.89:0/1164416094 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f89d40656b0 msgr2=0x7f89d4067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:40.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.716+0000 7f89f78e5640 1 --2- 172.21.15.89:0/1164416094 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f89d40656b0 0x7f89d4067b70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f89ec007fd0 tx=0x7f89ec009040 comp rx=0 tx=0).stop 2023-10-18T11:03:40.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.716+0000 7f89f78e5640 1 -- 172.21.15.89:0/1164416094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89f00d0e70 msgr2=0x7f89f00b77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:40.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.716+0000 7f89f78e5640 1 --2- 172.21.15.89:0/1164416094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89f00d0e70 0x7f89f00b77e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f89e40ad040 tx=0x7f89e401ac90 comp rx=0 tx=0).stop 2023-10-18T11:03:40.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.716+0000 7f89f78e5640 1 -- 172.21.15.89:0/1164416094 shutdown_connections 2023-10-18T11:03:40.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.716+0000 7f89f78e5640 1 --2- 172.21.15.89:0/1164416094 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f89d40656b0 0x7f89d4067b70 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:40.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.716+0000 7f89f78e5640 1 --2- 172.21.15.89:0/1164416094 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89f00b5270 0x7f89f0154da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:40.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.716+0000 7f89f78e5640 1 --2- 172.21.15.89:0/1164416094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89f00c8430 0x7f89f0154860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:40.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.716+0000 7f89f78e5640 1 --2- 172.21.15.89:0/1164416094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89f00d0e70 0x7f89f00b77e0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:40.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.716+0000 7f89f78e5640 1 -- 172.21.15.89:0/1164416094 >> 172.21.15.89:0/1164416094 conn(0x7f89f0089fd0 msgr2=0x7f89f015efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:40.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.716+0000 7f89f78e5640 1 -- 172.21.15.89:0/1164416094 shutdown_connections 2023-10-18T11:03:40.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.716+0000 7f89f78e5640 1 -- 172.21.15.89:0/1164416094 wait complete. 2023-10-18T11:03:40.733 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-18T11:03:40.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.872+0000 7f3a983ba640 1 Processor -- start 2023-10-18T11:03:40.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.872+0000 7f3a983ba640 1 -- start start 2023-10-18T11:03:40.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.872+0000 7f3a983ba640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3a9015c670 0x7f3a900713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:40.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.872+0000 7f3a983ba640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a901602a0 con 0x7f3a90060a30 2023-10-18T11:03:40.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.872+0000 7f3a983ba640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a90071900 con 0x7f3a90160a10 2023-10-18T11:03:40.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.872+0000 7f3a983ba640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a900cd2b0 con 0x7f3a9015c670 2023-10-18T11:03:40.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.876+0000 7f3a96156640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3a9015c670 0x7f3a900713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:40.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.876+0000 7f3a96957640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f3a90060a30 0x7f3a900c9990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39710/0 (socket says 172.21.15.89:39710) 2023-10-18T11:03:40.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.876+0000 7f3a96957640 1 -- 172.21.15.89:0/3791108460 learned_addr learned my addr 172.21.15.89:0/3791108460 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:40.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.876+0000 7f3a96156640 1 -- 172.21.15.89:0/3791108460 >> v1:172.21.15.154:6789/0 conn(0x7f3a90160a10 legacy=0x7f3a90060320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:40.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.876+0000 7f3a96156640 1 -- 172.21.15.89:0/3791108460 >> v1:172.21.15.89:6789/0 conn(0x7f3a90060a30 legacy=0x7f3a900c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:40.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.876+0000 7f3a96156640 1 -- 172.21.15.89:0/3791108460 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a900cd590 con 0x7f3a9015c670 2023-10-18T11:03:40.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.876+0000 7f3a95154640 1 -- 172.21.15.89:0/3791108460 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1851847178 0 0) 0x7f3a901602a0 con 0x7f3a90060a30 2023-10-18T11:03:40.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.876+0000 7f3a96156640 1 --2- 172.21.15.89:0/3791108460 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a9015c670 0x7f3a900713c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c0097c0 tx=0x7f3a8c01ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b1a20eb7466c67f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:40.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.876+0000 7f3a95154640 1 -- 172.21.15.89:0/3791108460 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c0a8970 con 0x7f3a9015c670 2023-10-18T11:03:40.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.876+0000 7f3a95154640 1 -- 172.21.15.89:0/3791108460 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8c0a8ad0 con 0x7f3a9015c670 2023-10-18T11:03:40.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.876+0000 7f3a95154640 1 -- 172.21.15.89:0/3791108460 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c0a8e60 con 0x7f3a9015c670 2023-10-18T11:03:40.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.880+0000 7f3a983ba640 1 -- 172.21.15.89:0/3791108460 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a9015c670 msgr2=0x7f3a900713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:40.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.880+0000 7f3a983ba640 1 --2- 172.21.15.89:0/3791108460 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a9015c670 0x7f3a900713c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c0097c0 tx=0x7f3a8c01ec70 comp rx=0 tx=0).stop 2023-10-18T11:03:40.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.880+0000 7f3a983ba640 1 -- 172.21.15.89:0/3791108460 shutdown_connections 2023-10-18T11:03:40.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.880+0000 7f3a983ba640 1 --2- 172.21.15.89:0/3791108460 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a9015c670 0x7f3a900713c0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:40.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.880+0000 7f3a983ba640 1 -- 172.21.15.89:0/3791108460 >> 172.21.15.89:0/3791108460 conn(0x7f3a900cbdc0 msgr2=0x7f3a900cc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:40.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.880+0000 7f3a983ba640 1 -- 172.21.15.89:0/3791108460 shutdown_connections 2023-10-18T11:03:40.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.880+0000 7f3a983ba640 1 -- 172.21.15.89:0/3791108460 wait complete. 2023-10-18T11:03:40.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.880+0000 7f3a983ba640 1 Processor -- start 2023-10-18T11:03:40.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.880+0000 7f3a983ba640 1 -- start start 2023-10-18T11:03:40.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.880+0000 7f3a983ba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3a90060a30 0x7f3a90155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:40.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.880+0000 7f3a983ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3a9015c670 0x7f3a9015f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:40.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.880+0000 7f3a983ba640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a90160a10 0x7f3a901e5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:40.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.880+0000 7f3a983ba640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9015ef80 con 0x7f3a90160a10 2023-10-18T11:03:40.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.880+0000 7f3a983ba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9015e0d0 con 0x7f3a9015c670 2023-10-18T11:03:40.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.880+0000 7f3a983ba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9015e300 con 0x7f3a90060a30 2023-10-18T11:03:40.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.880+0000 7f3a96957640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a90160a10 0x7f3a901e5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:40.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.880+0000 7f3a96957640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a90160a10 0x7f3a901e5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51464/0 (socket says 172.21.15.89:51464) 2023-10-18T11:03:40.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.880+0000 7f3a96957640 1 -- 172.21.15.89:0/130000941 learned_addr learned my addr 172.21.15.89:0/130000941 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:40.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.880+0000 7f3a95955640 1 --2- 172.21.15.89:0/130000941 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3a9015c670 0x7f3a9015f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:40.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.880+0000 7f3a96156640 1 --2- 172.21.15.89:0/130000941 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3a90060a30 0x7f3a90155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:40.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.880+0000 7f3a96957640 1 -- 172.21.15.89:0/130000941 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3a90060a30 msgr2=0x7f3a90155690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:40.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.880+0000 7f3a96957640 1 --2- 172.21.15.89:0/130000941 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3a90060a30 0x7f3a90155690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:40.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.880+0000 7f3a96957640 1 -- 172.21.15.89:0/130000941 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3a9015c670 msgr2=0x7f3a9015f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:40.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.880+0000 7f3a96957640 1 --2- 172.21.15.89:0/130000941 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3a9015c670 0x7f3a9015f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:40.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.880+0000 7f3a96957640 1 -- 172.21.15.89:0/130000941 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a8c009420 con 0x7f3a90160a10 2023-10-18T11:03:40.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.880+0000 7f3a96957640 1 --2- 172.21.15.89:0/130000941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a90160a10 0x7f3a901e5b70 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f3a88001570 tx=0x7f3a8800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:40.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.884+0000 7f3a86ffd640 1 -- 172.21.15.89:0/130000941 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8800a640 con 0x7f3a90160a10 2023-10-18T11:03:40.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.884+0000 7f3a86ffd640 1 -- 172.21.15.89:0/130000941 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8800a7a0 con 0x7f3a90160a10 2023-10-18T11:03:40.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.884+0000 7f3a86ffd640 1 -- 172.21.15.89:0/130000941 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8800ab90 con 0x7f3a90160a10 2023-10-18T11:03:40.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.884+0000 7f3a983ba640 1 -- 172.21.15.89:0/130000941 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a9015e5e0 con 0x7f3a90160a10 2023-10-18T11:03:40.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.884+0000 7f3a983ba640 1 -- 172.21.15.89:0/130000941 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a9015eba0 con 0x7f3a90160a10 2023-10-18T11:03:40.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.884+0000 7f3a983ba640 1 -- 172.21.15.89:0/130000941 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a9004dbe0 con 0x7f3a90160a10 2023-10-18T11:03:40.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.884+0000 7f3a86ffd640 1 -- 172.21.15.89:0/130000941 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f3a8800acf0 con 0x7f3a90160a10 2023-10-18T11:03:40.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.884+0000 7f3a86ffd640 1 --2- 172.21.15.89:0/130000941 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3a74065680 0x7f3a74067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:40.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.884+0000 7f3a86ffd640 1 -- 172.21.15.89:0/130000941 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f3a88082440 con 0x7f3a90160a10 2023-10-18T11:03:40.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.884+0000 7f3a96156640 1 --2- 172.21.15.89:0/130000941 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3a74065680 0x7f3a74067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:40.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.888+0000 7f3a96156640 1 --2- 172.21.15.89:0/130000941 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3a74065680 0x7f3a74067b40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c0b2000 tx=0x7f3a8c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:40.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:40.888+0000 7f3a86ffd640 1 -- 172.21.15.89:0/130000941 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f3a8804fdf0 con 0x7f3a90160a10 2023-10-18T11:03:41.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.068+0000 7f3a983ba640 1 -- 172.21.15.89:0/130000941 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3a90071c60 con 0x7f3a90160a10 2023-10-18T11:03:41.067 INFO:tasks.ceph.mon.a.smithi089.stderr:2023-10-18T11:03:41.068+0000 7f921e576640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T11:03:41.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.075+0000 7f3a86ffd640 1 -- 172.21.15.89:0/130000941 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v21) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3a88053410 con 0x7f3a90160a10 2023-10-18T11:03:41.073 INFO:teuthology.orchestra.run.smithi089.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-18T11:03:41.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.075+0000 7f3a983ba640 1 -- 172.21.15.89:0/130000941 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3a74065680 msgr2=0x7f3a74067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:41.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.075+0000 7f3a983ba640 1 --2- 172.21.15.89:0/130000941 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3a74065680 0x7f3a74067b40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c0b2000 tx=0x7f3a8c008040 comp rx=0 tx=0).stop 2023-10-18T11:03:41.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.075+0000 7f3a983ba640 1 -- 172.21.15.89:0/130000941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a90160a10 msgr2=0x7f3a901e5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:41.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.075+0000 7f3a983ba640 1 --2- 172.21.15.89:0/130000941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a90160a10 0x7f3a901e5b70 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f3a88001570 tx=0x7f3a8800cc70 comp rx=0 tx=0).stop 2023-10-18T11:03:41.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.075+0000 7f3a983ba640 1 -- 172.21.15.89:0/130000941 shutdown_connections 2023-10-18T11:03:41.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.075+0000 7f3a983ba640 1 --2- 172.21.15.89:0/130000941 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f3a74065680 0x7f3a74067b40 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:41.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.075+0000 7f3a983ba640 1 --2- 172.21.15.89:0/130000941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a90160a10 0x7f3a901e5b70 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:41.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.075+0000 7f3a983ba640 1 --2- 172.21.15.89:0/130000941 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3a9015c670 0x7f3a9015f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:41.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.075+0000 7f3a983ba640 1 --2- 172.21.15.89:0/130000941 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3a90060a30 0x7f3a90155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:41.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.075+0000 7f3a983ba640 1 -- 172.21.15.89:0/130000941 >> 172.21.15.89:0/130000941 conn(0x7f3a900cbdc0 msgr2=0x7f3a90069900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:41.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.075+0000 7f3a983ba640 1 -- 172.21.15.89:0/130000941 shutdown_connections 2023-10-18T11:03:41.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.075+0000 7f3a983ba640 1 -- 172.21.15.89:0/130000941 wait complete. 2023-10-18T11:03:41.095 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-18T11:03:41.133 INFO:tasks.ceph.mds.c.smithi089.stdout:starting mds.c at 2023-10-18T11:03:41.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.239+0000 7f8f2333a640 1 Processor -- start 2023-10-18T11:03:41.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.239+0000 7f8f2333a640 1 -- start start 2023-10-18T11:03:41.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.243+0000 7f8f2333a640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8f1c0ce780 0x7f8f1c143470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:41.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.243+0000 7f8f2333a640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f1c0cd8d0 con 0x7f8f1c0cd4f0 2023-10-18T11:03:41.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.243+0000 7f8f2333a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f1c15cbf0 con 0x7f8f1c145b70 2023-10-18T11:03:41.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.243+0000 7f8f2333a640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f1c069900 con 0x7f8f1c0ce780 2023-10-18T11:03:41.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.243+0000 7f8f218d7640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8f1c0ce780 0x7f8f1c143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:41.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.243+0000 7f8f210d6640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f8f1c0cd4f0 0x7f8f1c160cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39728/0 (socket says 172.21.15.89:39728) 2023-10-18T11:03:41.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.243+0000 7f8f210d6640 1 -- 172.21.15.89:0/2040484210 learned_addr learned my addr 172.21.15.89:0/2040484210 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:41.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.243+0000 7f8f218d7640 1 -- 172.21.15.89:0/2040484210 >> v1:172.21.15.154:6789/0 conn(0x7f8f1c145b70 legacy=0x7f8f1c0ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:41.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.243+0000 7f8f218d7640 1 -- 172.21.15.89:0/2040484210 >> v1:172.21.15.89:6789/0 conn(0x7f8f1c0cd4f0 legacy=0x7f8f1c160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:41.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.243+0000 7f8f218d7640 1 -- 172.21.15.89:0/2040484210 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f1c069b10 con 0x7f8f1c0ce780 2023-10-18T11:03:41.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.243+0000 7f8f218d7640 1 --2- 172.21.15.89:0/2040484210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f1c0ce780 0x7f8f1c143470 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8f140097c0 tx=0x7f8f1401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b78b1625ba8ca272 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:41.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.243+0000 7f8f1bfff640 1 -- 172.21.15.89:0/2040484210 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f140ab650 con 0x7f8f1c0ce780 2023-10-18T11:03:41.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.243+0000 7f8f1bfff640 1 -- 172.21.15.89:0/2040484210 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f140ab7b0 con 0x7f8f1c0ce780 2023-10-18T11:03:41.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.243+0000 7f8f1bfff640 1 -- 172.21.15.89:0/2040484210 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f140abb40 con 0x7f8f1c0ce780 2023-10-18T11:03:41.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.247+0000 7f8f2333a640 1 -- 172.21.15.89:0/2040484210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f1c0ce780 msgr2=0x7f8f1c143470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:41.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.247+0000 7f8f2333a640 1 --2- 172.21.15.89:0/2040484210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f1c0ce780 0x7f8f1c143470 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8f140097c0 tx=0x7f8f1401c450 comp rx=0 tx=0).stop 2023-10-18T11:03:41.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.247+0000 7f8f2333a640 1 -- 172.21.15.89:0/2040484210 shutdown_connections 2023-10-18T11:03:41.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.247+0000 7f8f2333a640 1 --2- 172.21.15.89:0/2040484210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f1c0ce780 0x7f8f1c143470 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:41.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.247+0000 7f8f2333a640 1 -- 172.21.15.89:0/2040484210 >> 172.21.15.89:0/2040484210 conn(0x7f8f1c06dad0 msgr2=0x7f8f1c06dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:41.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.247+0000 7f8f2333a640 1 -- 172.21.15.89:0/2040484210 shutdown_connections 2023-10-18T11:03:41.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.247+0000 7f8f2333a640 1 -- 172.21.15.89:0/2040484210 wait complete. 2023-10-18T11:03:41.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.247+0000 7f8f2333a640 1 Processor -- start 2023-10-18T11:03:41.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.247+0000 7f8f2333a640 1 -- start start 2023-10-18T11:03:41.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.247+0000 7f8f2333a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f1c0cd4f0 0x7f8f1c0c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:41.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.247+0000 7f8f2333a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f1c0ce780 0x7f8f1c0c5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:41.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.247+0000 7f8f2333a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f1c145b70 0x7f8f1c0c5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:41.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.247+0000 7f8f2333a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c0c7010 con 0x7f8f1c0cd4f0 2023-10-18T11:03:41.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.247+0000 7f8f2333a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c0b57e0 con 0x7f8f1c0ce780 2023-10-18T11:03:41.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.247+0000 7f8f2333a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c0b5ac0 con 0x7f8f1c145b70 2023-10-18T11:03:41.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.247+0000 7f8f210d6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f1c0cd4f0 0x7f8f1c0c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:41.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.247+0000 7f8f210d6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f1c0cd4f0 0x7f8f1c0c6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51490/0 (socket says 172.21.15.89:51490) 2023-10-18T11:03:41.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.247+0000 7f8f210d6640 1 -- 172.21.15.89:0/1329870701 learned_addr learned my addr 172.21.15.89:0/1329870701 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:41.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.247+0000 7f8f218d7640 1 --2- 172.21.15.89:0/1329870701 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f1c145b70 0x7f8f1c0c5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:41.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.247+0000 7f8f208d5640 1 --2- 172.21.15.89:0/1329870701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f1c0ce780 0x7f8f1c0c5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:41.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.251+0000 7f8f210d6640 1 -- 172.21.15.89:0/1329870701 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f1c145b70 msgr2=0x7f8f1c0c5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:41.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.251+0000 7f8f210d6640 1 --2- 172.21.15.89:0/1329870701 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f1c145b70 0x7f8f1c0c5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:41.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.251+0000 7f8f210d6640 1 -- 172.21.15.89:0/1329870701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f1c0ce780 msgr2=0x7f8f1c0c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:41.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.251+0000 7f8f210d6640 1 --2- 172.21.15.89:0/1329870701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f1c0ce780 0x7f8f1c0c5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:41.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.251+0000 7f8f210d6640 1 -- 172.21.15.89:0/1329870701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f14009420 con 0x7f8f1c0cd4f0 2023-10-18T11:03:41.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.251+0000 7f8f218d7640 1 --2- 172.21.15.89:0/1329870701 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f1c145b70 0x7f8f1c0c5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:03:41.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.251+0000 7f8f208d5640 1 --2- 172.21.15.89:0/1329870701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f1c0ce780 0x7f8f1c0c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:03:41.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.251+0000 7f8f210d6640 1 --2- 172.21.15.89:0/1329870701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f1c0cd4f0 0x7f8f1c0c6ad0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f8f10002240 tx=0x7f8f1000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:41.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.251+0000 7f8f19ffb640 1 -- 172.21.15.89:0/1329870701 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1000ad40 con 0x7f8f1c0cd4f0 2023-10-18T11:03:41.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.251+0000 7f8f19ffb640 1 -- 172.21.15.89:0/1329870701 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f10012400 con 0x7f8f1c0cd4f0 2023-10-18T11:03:41.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.251+0000 7f8f19ffb640 1 -- 172.21.15.89:0/1329870701 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1000aea0 con 0x7f8f1c0cd4f0 2023-10-18T11:03:41.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.251+0000 7f8f2333a640 1 -- 172.21.15.89:0/1329870701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f1c0b5da0 con 0x7f8f1c0cd4f0 2023-10-18T11:03:41.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.251+0000 7f8f2333a640 1 -- 172.21.15.89:0/1329870701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f1c157e20 con 0x7f8f1c0cd4f0 2023-10-18T11:03:41.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.251+0000 7f8f2333a640 1 -- 172.21.15.89:0/1329870701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f1c0b5f90 con 0x7f8f1c0cd4f0 2023-10-18T11:03:41.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.251+0000 7f8f19ffb640 1 -- 172.21.15.89:0/1329870701 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f8f10007070 con 0x7f8f1c0cd4f0 2023-10-18T11:03:41.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.251+0000 7f8f19ffb640 1 --2- 172.21.15.89:0/1329870701 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8f00065680 0x7f8f00067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:41.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.251+0000 7f8f19ffb640 1 -- 172.21.15.89:0/1329870701 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6569+0+0 (secure 0 0 0) 0x7f8f10083070 con 0x7f8f1c0cd4f0 2023-10-18T11:03:41.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.255+0000 7f8f208d5640 1 --2- 172.21.15.89:0/1329870701 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8f00065680 0x7f8f00067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:41.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.255+0000 7f8f208d5640 1 --2- 172.21.15.89:0/1329870701 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8f00065680 0x7f8f00067b40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c0ca390 tx=0x7f8f08009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:41.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.255+0000 7f8f19ffb640 1 -- 172.21.15.89:0/1329870701 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f8f100b3d40 con 0x7f8f1c0cd4f0 2023-10-18T11:03:41.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:41.435+0000 7f8f2333a640 1 -- 172.21.15.89:0/1329870701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8f1c04dbe0 con 0x7f8f1c0cd4f0 2023-10-18T11:03:42.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.075+0000 7f8f19ffb640 1 -- 172.21.15.89:0/1329870701 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v22) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8f1c0b5f90 con 0x7f8f1c0cd4f0 2023-10-18T11:03:42.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.079+0000 7f8f2333a640 1 -- 172.21.15.89:0/1329870701 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8f00065680 msgr2=0x7f8f00067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:42.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.079+0000 7f8f2333a640 1 --2- 172.21.15.89:0/1329870701 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8f00065680 0x7f8f00067b40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c0ca390 tx=0x7f8f08009040 comp rx=0 tx=0).stop 2023-10-18T11:03:42.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.079+0000 7f8f2333a640 1 -- 172.21.15.89:0/1329870701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f1c0cd4f0 msgr2=0x7f8f1c0c6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:42.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.079+0000 7f8f2333a640 1 --2- 172.21.15.89:0/1329870701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f1c0cd4f0 0x7f8f1c0c6ad0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f8f10002240 tx=0x7f8f1000a430 comp rx=0 tx=0).stop 2023-10-18T11:03:42.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.079+0000 7f8f2333a640 1 -- 172.21.15.89:0/1329870701 shutdown_connections 2023-10-18T11:03:42.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.079+0000 7f8f2333a640 1 --2- 172.21.15.89:0/1329870701 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f8f00065680 0x7f8f00067b40 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:42.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.079+0000 7f8f2333a640 1 --2- 172.21.15.89:0/1329870701 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f1c145b70 0x7f8f1c0c5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:42.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.079+0000 7f8f2333a640 1 --2- 172.21.15.89:0/1329870701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f1c0ce780 0x7f8f1c0c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:42.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.079+0000 7f8f2333a640 1 --2- 172.21.15.89:0/1329870701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f1c0cd4f0 0x7f8f1c0c6ad0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:42.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.079+0000 7f8f2333a640 1 -- 172.21.15.89:0/1329870701 >> 172.21.15.89:0/1329870701 conn(0x7f8f1c06dad0 msgr2=0x7f8f1c15ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:42.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.079+0000 7f8f2333a640 1 -- 172.21.15.89:0/1329870701 shutdown_connections 2023-10-18T11:03:42.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.079+0000 7f8f2333a640 1 -- 172.21.15.89:0/1329870701 wait complete. 2023-10-18T11:03:42.099 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-18T11:03:42.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.291+0000 7f7705488640 1 Processor -- start 2023-10-18T11:03:42.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.291+0000 7f7705488640 1 -- start start 2023-10-18T11:03:42.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.295+0000 7f7705488640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7700161020 0x7f77000d3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:42.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.295+0000 7f7705488640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f77000b0690 0x7f77000b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:42.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.295+0000 7f7705488640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77000d3690 con 0x7f77000b2020 2023-10-18T11:03:42.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.295+0000 7f7705488640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77000b3a20 con 0x7f77000b0690 2023-10-18T11:03:42.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.295+0000 7f7705488640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77000b2ea0 con 0x7f7700161020 2023-10-18T11:03:42.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.295+0000 7f76feffd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7700161020 0x7f77000d3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:42.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.295+0000 7f76ff7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f77000b0690 0x7f77000b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:42.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.295+0000 7f76feffd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7700161020 0x7f77000d3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:46504/0 (socket says 172.21.15.89:46504) 2023-10-18T11:03:42.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.295+0000 7f76feffd640 1 -- 172.21.15.89:0/3398595186 learned_addr learned my addr 172.21.15.89:0/3398595186 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:42.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.295+0000 7f76f67fc640 1 --1- 172.21.15.89:0/3398595186 >> v1:172.21.15.154:6789/0 conn(0x7f77000b2020 0x7f77000b2440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.89:48418/0 (socket says 172.21.15.89:48418) 2023-10-18T11:03:42.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.295+0000 7f76fe7fc640 1 -- 172.21.15.89:0/3398595186 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2575071616 0 0) 0x7f77000d3690 con 0x7f77000b2020 2023-10-18T11:03:42.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.295+0000 7f76fe7fc640 1 -- 172.21.15.89:0/3398595186 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76ec003530 con 0x7f77000b2020 2023-10-18T11:03:42.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.295+0000 7f76feffd640 1 -- 172.21.15.89:0/3398595186 >> v2:172.21.15.154:3300/0 conn(0x7f77000b0690 msgr2=0x7f77000b0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:42.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.295+0000 7f76feffd640 1 --2- 172.21.15.89:0/3398595186 >> v2:172.21.15.154:3300/0 conn(0x7f77000b0690 0x7f77000b0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:42.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.295+0000 7f76feffd640 1 -- 172.21.15.89:0/3398595186 >> v1:172.21.15.154:6789/0 conn(0x7f77000b2020 legacy=0x7f77000b2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:42.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.295+0000 7f76feffd640 1 -- 172.21.15.89:0/3398595186 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77000b3180 con 0x7f7700161020 2023-10-18T11:03:42.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.295+0000 7f76ff7fe640 1 --2- 172.21.15.89:0/3398595186 >> v2:172.21.15.154:3300/0 conn(0x7f77000b0690 0x7f77000b0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T11:03:42.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.299+0000 7f76feffd640 1 --2- 172.21.15.89:0/3398595186 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7700161020 0x7f77000d3150 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f76f0009e50 tx=0x7f76f001ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54ea6530fd8ab0dd server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:42.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.299+0000 7f76fe7fc640 1 -- 172.21.15.89:0/3398595186 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f00a57d0 con 0x7f7700161020 2023-10-18T11:03:42.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.299+0000 7f76fe7fc640 1 -- 172.21.15.89:0/3398595186 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f00a5930 con 0x7f7700161020 2023-10-18T11:03:42.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.299+0000 7f76fe7fc640 1 -- 172.21.15.89:0/3398595186 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f00a5cc0 con 0x7f7700161020 2023-10-18T11:03:42.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.299+0000 7f7705488640 1 -- 172.21.15.89:0/3398595186 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7700161020 msgr2=0x7f77000d3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:42.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.299+0000 7f7705488640 1 --2- 172.21.15.89:0/3398595186 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7700161020 0x7f77000d3150 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f76f0009e50 tx=0x7f76f001ca10 comp rx=0 tx=0).stop 2023-10-18T11:03:42.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.299+0000 7f7705488640 1 -- 172.21.15.89:0/3398595186 shutdown_connections 2023-10-18T11:03:42.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.299+0000 7f7705488640 1 --2- 172.21.15.89:0/3398595186 >> v2:172.21.15.154:3300/0 conn(0x7f77000b0690 0x7f77000b0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:42.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.299+0000 7f7705488640 1 --2- 172.21.15.89:0/3398595186 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7700161020 0x7f77000d3150 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:42.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.299+0000 7f7705488640 1 -- 172.21.15.89:0/3398595186 >> 172.21.15.89:0/3398595186 conn(0x7f770009b9b0 msgr2=0x7f77000d27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:42.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.299+0000 7f7705488640 1 -- 172.21.15.89:0/3398595186 shutdown_connections 2023-10-18T11:03:42.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.299+0000 7f7705488640 1 -- 172.21.15.89:0/3398595186 wait complete. 2023-10-18T11:03:42.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.299+0000 7f7705488640 1 Processor -- start 2023-10-18T11:03:42.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.303+0000 7f7705488640 1 -- start start 2023-10-18T11:03:42.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.303+0000 7f7705488640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77000b0690 0x7f770015f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:42.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.303+0000 7f7705488640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77000b2020 0x7f77000ca250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:42.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.303+0000 7f7705488640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7700161020 0x7f77000ca790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:42.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.303+0000 7f7705488640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77001606f0 con 0x7f77000b2020 2023-10-18T11:03:42.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.303+0000 7f7705488640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770015fa70 con 0x7f77000b0690 2023-10-18T11:03:42.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.303+0000 7f7705488640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f770015fd50 con 0x7f7700161020 2023-10-18T11:03:42.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.303+0000 7f76f67fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77000b2020 0x7f77000ca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:42.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.303+0000 7f76f67fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77000b2020 0x7f77000ca250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55588/0 (socket says 172.21.15.89:55588) 2023-10-18T11:03:42.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.303+0000 7f76f67fc640 1 -- 172.21.15.89:0/4013166089 learned_addr learned my addr 172.21.15.89:0/4013166089 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:42.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.303+0000 7f76ff7fe640 1 --2- 172.21.15.89:0/4013166089 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7700161020 0x7f77000ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:42.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.303+0000 7f76feffd640 1 --2- 172.21.15.89:0/4013166089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77000b0690 0x7f770015f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:42.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.303+0000 7f76f67fc640 1 -- 172.21.15.89:0/4013166089 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7700161020 msgr2=0x7f77000ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:42.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.303+0000 7f76f67fc640 1 --2- 172.21.15.89:0/4013166089 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7700161020 0x7f77000ca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:42.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.303+0000 7f76f67fc640 1 -- 172.21.15.89:0/4013166089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77000b0690 msgr2=0x7f770015f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:42.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.303+0000 7f76f67fc640 1 --2- 172.21.15.89:0/4013166089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77000b0690 0x7f770015f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:42.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.303+0000 7f76f67fc640 1 -- 172.21.15.89:0/4013166089 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f8009420 con 0x7f77000b2020 2023-10-18T11:03:42.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.303+0000 7f76feffd640 1 --2- 172.21.15.89:0/4013166089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77000b0690 0x7f770015f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:03:42.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.303+0000 7f76f67fc640 1 --2- 172.21.15.89:0/4013166089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77000b2020 0x7f77000ca250 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f76e800c430 tx=0x7f76e800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:42.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.303+0000 7f76f7fff640 1 -- 172.21.15.89:0/4013166089 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e80074e0 con 0x7f77000b2020 2023-10-18T11:03:42.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.303+0000 7f76f7fff640 1 -- 172.21.15.89:0/4013166089 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e8007660 con 0x7f77000b2020 2023-10-18T11:03:42.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.303+0000 7f7705488640 1 -- 172.21.15.89:0/4013166089 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f0009b30 con 0x7f77000b2020 2023-10-18T11:03:42.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.303+0000 7f76f7fff640 1 -- 172.21.15.89:0/4013166089 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e8004a30 con 0x7f77000b2020 2023-10-18T11:03:42.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.303+0000 7f7705488640 1 -- 172.21.15.89:0/4013166089 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7700160310 con 0x7f77000b2020 2023-10-18T11:03:42.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.303+0000 7f7705488640 1 -- 172.21.15.89:0/4013166089 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7700160830 con 0x7f77000b2020 2023-10-18T11:03:42.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.307+0000 7f76f7fff640 1 -- 172.21.15.89:0/4013166089 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f76e8004b90 con 0x7f77000b2020 2023-10-18T11:03:42.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.307+0000 7f76f7fff640 1 --2- 172.21.15.89:0/4013166089 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f76e0065680 0x7f76e0067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:42.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.307+0000 7f76f7fff640 1 -- 172.21.15.89:0/4013166089 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6569+0+0 (secure 0 0 0) 0x7f76e8082c30 con 0x7f77000b2020 2023-10-18T11:03:42.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.307+0000 7f76feffd640 1 --2- 172.21.15.89:0/4013166089 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f76e0065680 0x7f76e0067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:42.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.307+0000 7f76feffd640 1 --2- 172.21.15.89:0/4013166089 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f76e0065680 0x7f76e0067b40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f76f00ae000 tx=0x7f76f00a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:42.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.311+0000 7f76f7fff640 1 -- 172.21.15.89:0/4013166089 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f76e80503c0 con 0x7f77000b2020 2023-10-18T11:03:42.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:42.459+0000 7f7705488640 1 -- 172.21.15.89:0/4013166089 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7700066ca0 con 0x7f77000b2020 2023-10-18T11:03:43.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.211+0000 7f76f7fff640 1 -- 172.21.15.89:0/4013166089 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v58) v1 ==== 181+0+0 (secure 0 0 0) 0x7f76e80539e0 con 0x7f77000b2020 2023-10-18T11:03:43.212 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' removed 2023-10-18T11:03:43.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.215+0000 7f7705488640 1 -- 172.21.15.89:0/4013166089 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f76e0065680 msgr2=0x7f76e0067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:43.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.215+0000 7f7705488640 1 --2- 172.21.15.89:0/4013166089 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f76e0065680 0x7f76e0067b40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f76f00ae000 tx=0x7f76f00a7070 comp rx=0 tx=0).stop 2023-10-18T11:03:43.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.215+0000 7f7705488640 1 -- 172.21.15.89:0/4013166089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77000b2020 msgr2=0x7f77000ca250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:43.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.215+0000 7f7705488640 1 --2- 172.21.15.89:0/4013166089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77000b2020 0x7f77000ca250 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f76e800c430 tx=0x7f76e800c8f0 comp rx=0 tx=0).stop 2023-10-18T11:03:43.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.215+0000 7f7705488640 1 -- 172.21.15.89:0/4013166089 shutdown_connections 2023-10-18T11:03:43.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.215+0000 7f7705488640 1 --2- 172.21.15.89:0/4013166089 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f76e0065680 0x7f76e0067b40 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:43.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.215+0000 7f7705488640 1 --2- 172.21.15.89:0/4013166089 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7700161020 0x7f77000ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:43.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.215+0000 7f7705488640 1 --2- 172.21.15.89:0/4013166089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77000b2020 0x7f77000ca250 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:43.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.215+0000 7f7705488640 1 --2- 172.21.15.89:0/4013166089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77000b0690 0x7f770015f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:43.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.215+0000 7f7705488640 1 -- 172.21.15.89:0/4013166089 >> 172.21.15.89:0/4013166089 conn(0x7f770009b9b0 msgr2=0x7f7700159d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:43.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.215+0000 7f7705488640 1 -- 172.21.15.89:0/4013166089 shutdown_connections 2023-10-18T11:03:43.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.215+0000 7f7705488640 1 -- 172.21.15.89:0/4013166089 wait complete. 2023-10-18T11:03:43.236 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-18T11:03:43.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.379+0000 7fe9b2d8f640 1 Processor -- start 2023-10-18T11:03:43.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.379+0000 7fe9b2d8f640 1 -- start start 2023-10-18T11:03:43.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.379+0000 7fe9b2d8f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe9ac15c7f0 0x7fe9ac160ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:43.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.379+0000 7fe9b2d8f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9ac0ce740 0x7fe9ac06a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:43.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.379+0000 7fe9b2d8f640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9ac15cbd0 con 0x7fe9ac0caab0 2023-10-18T11:03:43.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.379+0000 7fe9b2d8f640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9ac069cd0 con 0x7fe9ac15c7f0 2023-10-18T11:03:43.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.379+0000 7fe9b2d8f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac0b2a20 con 0x7fe9ac0ce740 2023-10-18T11:03:43.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.379+0000 7fe9a3fff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe9ac0caab0 0x7fe9ac0ce030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48090/0 (socket says 172.21.15.89:48090) 2023-10-18T11:03:43.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.379+0000 7fe9a3fff640 1 -- 172.21.15.89:0/167367596 learned_addr learned my addr 172.21.15.89:0/167367596 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:43.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.383+0000 7fe9b132c640 1 --2- 172.21.15.89:0/167367596 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9ac0ce740 0x7fe9ac06a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:43.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.383+0000 7fe9b0b2b640 1 --2- 172.21.15.89:0/167367596 >> v2:172.21.15.154:3300/0 conn(0x7fe9ac15c7f0 0x7fe9ac160ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:43.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.383+0000 7fe9a37fe640 1 -- 172.21.15.89:0/167367596 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3774414654 0 0) 0x7fe9ac15cbd0 con 0x7fe9ac0caab0 2023-10-18T11:03:43.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.383+0000 7fe9a37fe640 1 -- 172.21.15.89:0/167367596 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe994003530 con 0x7fe9ac0caab0 2023-10-18T11:03:43.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.383+0000 7fe9a37fe640 1 -- 172.21.15.89:0/167367596 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1673794207 0 0) 0x7fe994003530 con 0x7fe9ac0caab0 2023-10-18T11:03:43.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.383+0000 7fe9a37fe640 1 -- 172.21.15.89:0/167367596 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9ac0ce740 msgr2=0x7fe9ac06a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:43.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.383+0000 7fe9a37fe640 1 --2- 172.21.15.89:0/167367596 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9ac0ce740 0x7fe9ac06a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:43.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.383+0000 7fe9a37fe640 1 -- 172.21.15.89:0/167367596 >> v2:172.21.15.154:3300/0 conn(0x7fe9ac15c7f0 msgr2=0x7fe9ac160ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:43.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.383+0000 7fe9a37fe640 1 --2- 172.21.15.89:0/167367596 >> v2:172.21.15.154:3300/0 conn(0x7fe9ac15c7f0 0x7fe9ac160ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:43.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.383+0000 7fe9a37fe640 1 -- 172.21.15.89:0/167367596 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ac0b1ee0 con 0x7fe9ac0caab0 2023-10-18T11:03:43.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.383+0000 7fe9a37fe640 1 -- 172.21.15.89:0/167367596 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7fe99c0016a0 con 0x7fe9ac0caab0 2023-10-18T11:03:43.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.383+0000 7fe9a37fe640 1 -- 172.21.15.89:0/167367596 >> v1:172.21.15.89:6789/0 conn(0x7fe9ac0caab0 legacy=0x7fe9ac0ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:43.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.383+0000 7fe9a37fe640 1 --2- 172.21.15.89:0/167367596 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9ac0ce740 0x7fe994003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:43.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.383+0000 7fe9a37fe640 1 --2- 172.21.15.89:0/167367596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe994003f00 0x7fe9940047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:43.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.383+0000 7fe9a37fe640 1 --2- 172.21.15.89:0/167367596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe994004d10 0x7fe994005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:43.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.383+0000 7fe9a37fe640 1 -- 172.21.15.89:0/167367596 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac069cd0 con 0x7fe994004d10 2023-10-18T11:03:43.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.383+0000 7fe9a37fe640 1 -- 172.21.15.89:0/167367596 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac0b2a20 con 0x7fe994003f00 2023-10-18T11:03:43.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.383+0000 7fe9a37fe640 1 -- 172.21.15.89:0/167367596 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe99400b9e0 con 0x7fe9ac0ce740 2023-10-18T11:03:43.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.383+0000 7fe9b132c640 1 --2- 172.21.15.89:0/167367596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe994004d10 0x7fe994005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:43.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.383+0000 7fe9a3fff640 1 --2- 172.21.15.89:0/167367596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe994003f00 0x7fe9940047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:43.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.383+0000 7fe9b0b2b640 1 --2- 172.21.15.89:0/167367596 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9ac0ce740 0x7fe994003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:43.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.383+0000 7fe9b132c640 1 -- 172.21.15.89:0/167367596 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9ac0ce740 msgr2=0x7fe994003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:43.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.383+0000 7fe9b132c640 1 --2- 172.21.15.89:0/167367596 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9ac0ce740 0x7fe994003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:43.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.383+0000 7fe9b132c640 1 -- 172.21.15.89:0/167367596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe994003f00 msgr2=0x7fe9940047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:43.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.383+0000 7fe9b132c640 1 --2- 172.21.15.89:0/167367596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe994003f00 0x7fe9940047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:43.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.383+0000 7fe9b132c640 1 -- 172.21.15.89:0/167367596 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe994003530 con 0x7fe994004d10 2023-10-18T11:03:43.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.383+0000 7fe9a3fff640 1 --2- 172.21.15.89:0/167367596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe994003f00 0x7fe9940047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:03:43.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.387+0000 7fe9b132c640 1 --2- 172.21.15.89:0/167367596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe994004d10 0x7fe994005190 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400e170 tx=0x7fe9a401cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d6346698bd5831 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:43.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.387+0000 7fe9a37fe640 1 -- 172.21.15.89:0/167367596 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a40a5c50 con 0x7fe994004d10 2023-10-18T11:03:43.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.387+0000 7fe9a37fe640 1 -- 172.21.15.89:0/167367596 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a400f040 con 0x7fe994004d10 2023-10-18T11:03:43.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.387+0000 7fe9a37fe640 1 -- 172.21.15.89:0/167367596 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a40ad440 con 0x7fe994004d10 2023-10-18T11:03:43.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.387+0000 7fe9b2d8f640 1 -- 172.21.15.89:0/167367596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe994004d10 msgr2=0x7fe994005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:43.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.387+0000 7fe9b2d8f640 1 --2- 172.21.15.89:0/167367596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe994004d10 0x7fe994005190 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400e170 tx=0x7fe9a401cc40 comp rx=0 tx=0).stop 2023-10-18T11:03:43.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.387+0000 7fe9b132c640 1 -- 172.21.15.89:0/167367596 reap_dead start 2023-10-18T11:03:43.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.387+0000 7fe9b2d8f640 1 -- 172.21.15.89:0/167367596 shutdown_connections 2023-10-18T11:03:43.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.387+0000 7fe9b2d8f640 1 -- 172.21.15.89:0/167367596 >> 172.21.15.89:0/167367596 conn(0x7fe9ac0ccee0 msgr2=0x7fe9ac0cd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:43.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.387+0000 7fe9b2d8f640 1 -- 172.21.15.89:0/167367596 shutdown_connections 2023-10-18T11:03:43.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.387+0000 7fe9b2d8f640 1 -- 172.21.15.89:0/167367596 wait complete. 2023-10-18T11:03:43.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.387+0000 7fe9b2d8f640 1 Processor -- start 2023-10-18T11:03:43.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.387+0000 7fe9b2d8f640 1 -- start start 2023-10-18T11:03:43.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.391+0000 7fe9b2d8f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe994004d10 0x7fe9ac1f5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:43.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.391+0000 7fe9b2d8f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9ac0c3ef0 0x7fe9ac0c4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:43.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.391+0000 7fe9b2d8f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9ac1f13e0 0x7fe9ac1f1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:43.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.391+0000 7fe9b2d8f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac0c4370 con 0x7fe9ac1f13e0 2023-10-18T11:03:43.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.391+0000 7fe9b2d8f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac0c4eb0 con 0x7fe9ac0c3ef0 2023-10-18T11:03:43.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.391+0000 7fe9b2d8f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac0c5110 con 0x7fe994004d10 2023-10-18T11:03:43.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.391+0000 7fe9b132c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9ac1f13e0 0x7fe9ac1f1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:43.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.391+0000 7fe9b132c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9ac1f13e0 0x7fe9ac1f1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55606/0 (socket says 172.21.15.89:55606) 2023-10-18T11:03:43.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.391+0000 7fe9b132c640 1 -- 172.21.15.89:0/3269676028 learned_addr learned my addr 172.21.15.89:0/3269676028 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:43.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.391+0000 7fe9b0b2b640 1 --2- 172.21.15.89:0/3269676028 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe994004d10 0x7fe9ac1f5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:43.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.391+0000 7fe9a3fff640 1 --2- 172.21.15.89:0/3269676028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9ac0c3ef0 0x7fe9ac0c4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:43.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.391+0000 7fe9b132c640 1 -- 172.21.15.89:0/3269676028 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe994004d10 msgr2=0x7fe9ac1f5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:43.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.391+0000 7fe9b132c640 1 --2- 172.21.15.89:0/3269676028 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe994004d10 0x7fe9ac1f5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:43.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.391+0000 7fe9b132c640 1 -- 172.21.15.89:0/3269676028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9ac0c3ef0 msgr2=0x7fe9ac0c4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:43.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.391+0000 7fe9b132c640 1 --2- 172.21.15.89:0/3269676028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9ac0c3ef0 0x7fe9ac0c4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:43.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.391+0000 7fe9b132c640 1 -- 172.21.15.89:0/3269676028 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ac0c53f0 con 0x7fe9ac1f13e0 2023-10-18T11:03:43.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.391+0000 7fe9a3fff640 1 --2- 172.21.15.89:0/3269676028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9ac0c3ef0 0x7fe9ac0c4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:03:43.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.391+0000 7fe9b132c640 1 --2- 172.21.15.89:0/3269676028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9ac1f13e0 0x7fe9ac1f1860 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400e170 tx=0x7fe9a4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:43.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.391+0000 7fe9a17fa640 1 -- 172.21.15.89:0/3269676028 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a40bc070 con 0x7fe9ac1f13e0 2023-10-18T11:03:43.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.391+0000 7fe9a17fa640 1 -- 172.21.15.89:0/3269676028 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a40039a0 con 0x7fe9ac1f13e0 2023-10-18T11:03:43.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.391+0000 7fe9a17fa640 1 -- 172.21.15.89:0/3269676028 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4003b50 con 0x7fe9ac1f13e0 2023-10-18T11:03:43.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.391+0000 7fe9b2d8f640 1 -- 172.21.15.89:0/3269676028 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9ac1f5460 con 0x7fe9ac1f13e0 2023-10-18T11:03:43.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.391+0000 7fe9b2d8f640 1 -- 172.21.15.89:0/3269676028 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9ac1f5a20 con 0x7fe9ac1f13e0 2023-10-18T11:03:43.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.391+0000 7fe9b2d8f640 1 -- 172.21.15.89:0/3269676028 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9ac1f5c10 con 0x7fe9ac1f13e0 2023-10-18T11:03:43.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.391+0000 7fe9a17fa640 1 -- 172.21.15.89:0/3269676028 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fe9a40b4040 con 0x7fe9ac1f13e0 2023-10-18T11:03:43.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.395+0000 7fe9a17fa640 1 --2- 172.21.15.89:0/3269676028 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe998065680 0x7fe998067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:43.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.395+0000 7fe9a17fa640 1 -- 172.21.15.89:0/3269676028 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6131+0+0 (secure 0 0 0) 0x7fe9a411d060 con 0x7fe9ac1f13e0 2023-10-18T11:03:43.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.395+0000 7fe9b0b2b640 1 --2- 172.21.15.89:0/3269676028 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe998065680 0x7fe998067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:43.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.395+0000 7fe9b0b2b640 1 --2- 172.21.15.89:0/3269676028 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe998065680 0x7fe998067b40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8009b60 tx=0x7fe9a8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:43.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.399+0000 7fe9a17fa640 1 -- 172.21.15.89:0/3269676028 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fe9a40eaae0 con 0x7fe9ac1f13e0 2023-10-18T11:03:43.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:43.559+0000 7fe9b2d8f640 1 -- 172.21.15.89:0/3269676028 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe9ac04dbe0 con 0x7fe9ac1f13e0 2023-10-18T11:03:44.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.223+0000 7fe9a17fa640 1 -- 172.21.15.89:0/3269676028 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v59) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe9a40b20d0 con 0x7fe9ac1f13e0 2023-10-18T11:03:44.223 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' removed 2023-10-18T11:03:44.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.227+0000 7fe9b2d8f640 1 -- 172.21.15.89:0/3269676028 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe998065680 msgr2=0x7fe998067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:44.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.227+0000 7fe9b2d8f640 1 --2- 172.21.15.89:0/3269676028 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe998065680 0x7fe998067b40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8009b60 tx=0x7fe9a8008040 comp rx=0 tx=0).stop 2023-10-18T11:03:44.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.227+0000 7fe9b2d8f640 1 -- 172.21.15.89:0/3269676028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9ac1f13e0 msgr2=0x7fe9ac1f1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:44.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.227+0000 7fe9b2d8f640 1 --2- 172.21.15.89:0/3269676028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9ac1f13e0 0x7fe9ac1f1860 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400e170 tx=0x7fe9a4009750 comp rx=0 tx=0).stop 2023-10-18T11:03:44.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.227+0000 7fe9b2d8f640 1 -- 172.21.15.89:0/3269676028 shutdown_connections 2023-10-18T11:03:44.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.227+0000 7fe9b2d8f640 1 --2- 172.21.15.89:0/3269676028 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fe998065680 0x7fe998067b40 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:44.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.227+0000 7fe9b2d8f640 1 --2- 172.21.15.89:0/3269676028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9ac1f13e0 0x7fe9ac1f1860 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:44.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.227+0000 7fe9b2d8f640 1 --2- 172.21.15.89:0/3269676028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9ac0c3ef0 0x7fe9ac0c4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:44.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.227+0000 7fe9b2d8f640 1 --2- 172.21.15.89:0/3269676028 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe994004d10 0x7fe9ac1f5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:44.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.227+0000 7fe9b2d8f640 1 -- 172.21.15.89:0/3269676028 >> 172.21.15.89:0/3269676028 conn(0x7fe9ac0ccee0 msgr2=0x7fe9ac15b730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:44.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.227+0000 7fe9b2d8f640 1 -- 172.21.15.89:0/3269676028 shutdown_connections 2023-10-18T11:03:44.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.227+0000 7fe9b2d8f640 1 -- 172.21.15.89:0/3269676028 wait complete. 2023-10-18T11:03:44.256 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds defer_client_eviction_on_laggy_osds 2023-10-18T11:03:44.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.399+0000 7f938c7fa640 1 Processor -- start 2023-10-18T11:03:44.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.399+0000 7f938c7fa640 1 -- start start 2023-10-18T11:03:44.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.399+0000 7f938c7fa640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9384159490 0x7f938415d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:44.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.399+0000 7f938c7fa640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f938415ddf0 con 0x7f938406bee0 2023-10-18T11:03:44.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.399+0000 7f938c7fa640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9384070b90 con 0x7f9384070600 2023-10-18T11:03:44.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.399+0000 7f938c7fa640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9384070e70 con 0x7f9384159490 2023-10-18T11:03:44.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.399+0000 7f938a596640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9384159490 0x7f938415d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:44.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.399+0000 7f938ad97640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f938406bee0 0x7f93840c8960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48094/0 (socket says 172.21.15.89:48094) 2023-10-18T11:03:44.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.399+0000 7f938ad97640 1 -- 172.21.15.89:0/1137747380 learned_addr learned my addr 172.21.15.89:0/1137747380 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:44.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.399+0000 7f938a596640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9384159490 0x7f938415d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55614/0 (socket says 172.21.15.89:55614) 2023-10-18T11:03:44.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.399+0000 7f938a596640 1 -- 172.21.15.89:0/1137747380 >> v1:172.21.15.154:6789/0 conn(0x7f9384070600 legacy=0x7f938406b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:44.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.399+0000 7f938a596640 1 -- 172.21.15.89:0/1137747380 >> v1:172.21.15.89:6789/0 conn(0x7f938406bee0 legacy=0x7f93840c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:44.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.399+0000 7f938a596640 1 -- 172.21.15.89:0/1137747380 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9384071150 con 0x7f9384159490 2023-10-18T11:03:44.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.403+0000 7f938a596640 1 --2- 172.21.15.89:0/1137747380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9384159490 0x7f938415d8b0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f93800097c0 tx=0x7f938001c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9de7e0654cdd3b5b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:44.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.403+0000 7f9389594640 1 -- 172.21.15.89:0/1137747380 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93800ad680 con 0x7f9384159490 2023-10-18T11:03:44.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.403+0000 7f9389594640 1 -- 172.21.15.89:0/1137747380 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93800ad7e0 con 0x7f9384159490 2023-10-18T11:03:44.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.403+0000 7f9389594640 1 -- 172.21.15.89:0/1137747380 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93800adb70 con 0x7f9384159490 2023-10-18T11:03:44.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.403+0000 7f938c7fa640 1 -- 172.21.15.89:0/1137747380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9384159490 msgr2=0x7f938415d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:44.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.403+0000 7f938c7fa640 1 --2- 172.21.15.89:0/1137747380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9384159490 0x7f938415d8b0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f93800097c0 tx=0x7f938001c380 comp rx=0 tx=0).stop 2023-10-18T11:03:44.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.403+0000 7f938c7fa640 1 -- 172.21.15.89:0/1137747380 shutdown_connections 2023-10-18T11:03:44.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.403+0000 7f938c7fa640 1 --2- 172.21.15.89:0/1137747380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9384159490 0x7f938415d8b0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:44.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.403+0000 7f938c7fa640 1 -- 172.21.15.89:0/1137747380 >> 172.21.15.89:0/1137747380 conn(0x7f9384069f00 msgr2=0x7f93840cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:44.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.403+0000 7f938c7fa640 1 -- 172.21.15.89:0/1137747380 shutdown_connections 2023-10-18T11:03:44.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.403+0000 7f938c7fa640 1 -- 172.21.15.89:0/1137747380 wait complete. 2023-10-18T11:03:44.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.407+0000 7f938c7fa640 1 Processor -- start 2023-10-18T11:03:44.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.407+0000 7f938c7fa640 1 -- start start 2023-10-18T11:03:44.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.407+0000 7f938c7fa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f938406bee0 0x7f93840b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:44.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.407+0000 7f938c7fa640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9384070600 0x7f9384151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:44.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.407+0000 7f938c7fa640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9384159490 0x7f93840c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:44.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.407+0000 7f938c7fa640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93840b3c00 con 0x7f9384159490 2023-10-18T11:03:44.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.407+0000 7f938c7fa640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9384151da0 con 0x7f9384070600 2023-10-18T11:03:44.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.407+0000 7f938c7fa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93840abcd0 con 0x7f938406bee0 2023-10-18T11:03:44.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.407+0000 7f938ad97640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9384159490 0x7f93840c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:44.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.407+0000 7f938ad97640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9384159490 0x7f93840c5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55620/0 (socket says 172.21.15.89:55620) 2023-10-18T11:03:44.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.407+0000 7f938ad97640 1 -- 172.21.15.89:0/231548079 learned_addr learned my addr 172.21.15.89:0/231548079 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:44.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.407+0000 7f9389d95640 1 --2- 172.21.15.89:0/231548079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9384070600 0x7f9384151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:44.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.407+0000 7f938a596640 1 --2- 172.21.15.89:0/231548079 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f938406bee0 0x7f93840b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:44.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.407+0000 7f938ad97640 1 -- 172.21.15.89:0/231548079 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f938406bee0 msgr2=0x7f93840b36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:44.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.407+0000 7f938ad97640 1 --2- 172.21.15.89:0/231548079 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f938406bee0 0x7f93840b36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:44.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.407+0000 7f938ad97640 1 -- 172.21.15.89:0/231548079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9384070600 msgr2=0x7f9384151860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:44.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.407+0000 7f938ad97640 1 --2- 172.21.15.89:0/231548079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9384070600 0x7f9384151860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:44.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.407+0000 7f938ad97640 1 -- 172.21.15.89:0/231548079 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9380009420 con 0x7f9384159490 2023-10-18T11:03:44.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.407+0000 7f938ad97640 1 --2- 172.21.15.89:0/231548079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9384159490 0x7f93840c5ae0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f937c00caa0 tx=0x7f937c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:44.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.407+0000 7f937affd640 1 -- 172.21.15.89:0/231548079 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937c00aba0 con 0x7f9384159490 2023-10-18T11:03:44.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.407+0000 7f937affd640 1 -- 172.21.15.89:0/231548079 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f937c00ad00 con 0x7f9384159490 2023-10-18T11:03:44.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.407+0000 7f938c7fa640 1 -- 172.21.15.89:0/231548079 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93840b1c70 con 0x7f9384159490 2023-10-18T11:03:44.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.407+0000 7f937affd640 1 -- 172.21.15.89:0/231548079 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937c012600 con 0x7f9384159490 2023-10-18T11:03:44.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.407+0000 7f938c7fa640 1 -- 172.21.15.89:0/231548079 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93840ab960 con 0x7f9384159490 2023-10-18T11:03:44.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.411+0000 7f938c7fa640 1 -- 172.21.15.89:0/231548079 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93840abe60 con 0x7f9384159490 2023-10-18T11:03:44.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.411+0000 7f937affd640 1 -- 172.21.15.89:0/231548079 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f937c012760 con 0x7f9384159490 2023-10-18T11:03:44.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.411+0000 7f937affd640 1 --2- 172.21.15.89:0/231548079 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f93680656b0 0x7f9368067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:44.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.411+0000 7f937affd640 1 -- 172.21.15.89:0/231548079 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f937c0829b0 con 0x7f9384159490 2023-10-18T11:03:44.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.411+0000 7f938a596640 1 --2- 172.21.15.89:0/231548079 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f93680656b0 0x7f9368067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:44.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.411+0000 7f938a596640 1 --2- 172.21.15.89:0/231548079 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f93680656b0 0x7f9368067b70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f93800af000 tx=0x7f9380008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:44.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.415+0000 7f937affd640 1 -- 172.21.15.89:0/231548079 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f937c0506a0 con 0x7f9384159490 2023-10-18T11:03:44.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.563+0000 7f938c7fa640 1 -- 172.21.15.89:0/231548079 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "defer_client_eviction_on_laggy_osds"} v 0) v1 -- 0x7f938404dbe0 con 0x7f9384159490 2023-10-18T11:03:44.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.567+0000 7f937affd640 1 -- 172.21.15.89:0/231548079 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "defer_client_eviction_on_laggy_osds"}]=0 v14) v1 ==== 118+0+0 (secure 0 0 0) 0x7f937c0170b0 con 0x7f9384159490 2023-10-18T11:03:44.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.571+0000 7f938c7fa640 1 -- 172.21.15.89:0/231548079 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f93680656b0 msgr2=0x7f9368067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:44.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.571+0000 7f938c7fa640 1 --2- 172.21.15.89:0/231548079 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f93680656b0 0x7f9368067b70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f93800af000 tx=0x7f9380008040 comp rx=0 tx=0).stop 2023-10-18T11:03:44.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.571+0000 7f938c7fa640 1 -- 172.21.15.89:0/231548079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9384159490 msgr2=0x7f93840c5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:44.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.571+0000 7f938c7fa640 1 --2- 172.21.15.89:0/231548079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9384159490 0x7f93840c5ae0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f937c00caa0 tx=0x7f937c00cf60 comp rx=0 tx=0).stop 2023-10-18T11:03:44.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.571+0000 7f938c7fa640 1 -- 172.21.15.89:0/231548079 shutdown_connections 2023-10-18T11:03:44.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.571+0000 7f938c7fa640 1 --2- 172.21.15.89:0/231548079 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f93680656b0 0x7f9368067b70 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:44.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.571+0000 7f938c7fa640 1 --2- 172.21.15.89:0/231548079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9384159490 0x7f93840c5ae0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:44.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.571+0000 7f938c7fa640 1 --2- 172.21.15.89:0/231548079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9384070600 0x7f9384151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:44.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.571+0000 7f938c7fa640 1 --2- 172.21.15.89:0/231548079 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f938406bee0 0x7f93840b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:44.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.571+0000 7f938c7fa640 1 -- 172.21.15.89:0/231548079 >> 172.21.15.89:0/231548079 conn(0x7f9384069f00 msgr2=0x7f938406b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:44.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.571+0000 7f938c7fa640 1 -- 172.21.15.89:0/231548079 shutdown_connections 2023-10-18T11:03:44.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.571+0000 7f938c7fa640 1 -- 172.21.15.89:0/231548079 wait complete. 2023-10-18T11:03:44.589 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_osd_laggy_weight 2023-10-18T11:03:44.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.739+0000 7f4b6e4cb640 1 Processor -- start 2023-10-18T11:03:44.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.739+0000 7f4b6e4cb640 1 -- start start 2023-10-18T11:03:44.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.739+0000 7f4b6e4cb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b680afd90 0x7f4b680b0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:44.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.739+0000 7f4b6e4cb640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b68161160 con 0x7f4b6815c7f0 2023-10-18T11:03:44.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.739+0000 7f4b6e4cb640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b681605c0 con 0x7f4b680ad5b0 2023-10-18T11:03:44.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.739+0000 7f4b6e4cb640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b680b06b0 con 0x7f4b680afd90 2023-10-18T11:03:44.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.739+0000 7f4b677fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f4b6815c7f0 0x7f4b6806b7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48108/0 (socket says 172.21.15.89:48108) 2023-10-18T11:03:44.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.739+0000 7f4b677fe640 1 -- 172.21.15.89:0/259334711 learned_addr learned my addr 172.21.15.89:0/259334711 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:44.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.739+0000 7f4b67fff640 1 --2- 172.21.15.89:0/259334711 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b680afd90 0x7f4b680b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:44.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.739+0000 7f4b66ffd640 1 -- 172.21.15.89:0/259334711 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3024884301 0 0) 0x7f4b5c001120 con 0x7f4b6815c7f0 2023-10-18T11:03:44.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.739+0000 7f4b66ffd640 1 -- 172.21.15.89:0/259334711 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b50003530 con 0x7f4b6815c7f0 2023-10-18T11:03:44.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.739+0000 7f4b66ffd640 1 -- 172.21.15.89:0/259334711 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 749512900 0 0) 0x7f4b50003530 con 0x7f4b6815c7f0 2023-10-18T11:03:44.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.739+0000 7f4b66ffd640 1 -- 172.21.15.89:0/259334711 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b680afd90 msgr2=0x7f4b680b0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:44.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.739+0000 7f4b66ffd640 1 --2- 172.21.15.89:0/259334711 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b680afd90 0x7f4b680b0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:44.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.739+0000 7f4b66ffd640 1 -- 172.21.15.89:0/259334711 >> v1:172.21.15.154:6789/0 conn(0x7f4b680ad5b0 legacy=0x7f4b68160a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:44.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.739+0000 7f4b66ffd640 1 -- 172.21.15.89:0/259334711 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b6806c000 con 0x7f4b6815c7f0 2023-10-18T11:03:44.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.739+0000 7f4b66ffd640 1 -- 172.21.15.89:0/259334711 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7f4b5c001810 con 0x7f4b6815c7f0 2023-10-18T11:03:44.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.739+0000 7f4b66ffd640 1 -- 172.21.15.89:0/259334711 >> v1:172.21.15.89:6789/0 conn(0x7f4b6815c7f0 legacy=0x7f4b6806b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:44.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.739+0000 7f4b66ffd640 1 --2- 172.21.15.89:0/259334711 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b500039b0 0x7f4b50003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:44.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.739+0000 7f4b66ffd640 1 --2- 172.21.15.89:0/259334711 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b680afd90 0x7f4b500047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:44.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.739+0000 7f4b66ffd640 1 --2- 172.21.15.89:0/259334711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b50004d10 0x7f4b50005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:44.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.739+0000 7f4b66ffd640 1 -- 172.21.15.89:0/259334711 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b680b06b0 con 0x7f4b50004d10 2023-10-18T11:03:44.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.739+0000 7f4b66ffd640 1 -- 172.21.15.89:0/259334711 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5000bac0 con 0x7f4b500039b0 2023-10-18T11:03:44.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.739+0000 7f4b66ffd640 1 -- 172.21.15.89:0/259334711 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5000bc80 con 0x7f4b680afd90 2023-10-18T11:03:44.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.739+0000 7f4b6ca68640 1 --2- 172.21.15.89:0/259334711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b50004d10 0x7f4b50005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:44.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.739+0000 7f4b677fe640 1 --2- 172.21.15.89:0/259334711 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b680afd90 0x7f4b500047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:44.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.739+0000 7f4b67fff640 1 --2- 172.21.15.89:0/259334711 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b500039b0 0x7f4b50003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:44.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.743+0000 7f4b6ca68640 1 -- 172.21.15.89:0/259334711 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b680afd90 msgr2=0x7f4b500047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:44.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.743+0000 7f4b6ca68640 1 --2- 172.21.15.89:0/259334711 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b680afd90 0x7f4b500047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:44.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.743+0000 7f4b6ca68640 1 -- 172.21.15.89:0/259334711 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b500039b0 msgr2=0x7f4b50003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:44.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.743+0000 7f4b6ca68640 1 --2- 172.21.15.89:0/259334711 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b500039b0 0x7f4b50003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:44.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.743+0000 7f4b6ca68640 1 -- 172.21.15.89:0/259334711 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b50003530 con 0x7f4b50004d10 2023-10-18T11:03:44.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.743+0000 7f4b6ca68640 1 --2- 172.21.15.89:0/259334711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b50004d10 0x7f4b50005190 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f4b60001e40 tx=0x7f4b6001c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc580195fa5cec4 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:44.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.743+0000 7f4b66ffd640 1 -- 172.21.15.89:0/259334711 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b60003b80 con 0x7f4b50004d10 2023-10-18T11:03:44.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.743+0000 7f4b66ffd640 1 -- 172.21.15.89:0/259334711 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b60003ce0 con 0x7f4b50004d10 2023-10-18T11:03:44.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.743+0000 7f4b66ffd640 1 -- 172.21.15.89:0/259334711 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b600052c0 con 0x7f4b50004d10 2023-10-18T11:03:44.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.743+0000 7f4b6e4cb640 1 -- 172.21.15.89:0/259334711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b50004d10 msgr2=0x7f4b50005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:44.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.743+0000 7f4b6e4cb640 1 --2- 172.21.15.89:0/259334711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b50004d10 0x7f4b50005190 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f4b60001e40 tx=0x7f4b6001c740 comp rx=0 tx=0).stop 2023-10-18T11:03:44.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.743+0000 7f4b6ca68640 1 -- 172.21.15.89:0/259334711 reap_dead start 2023-10-18T11:03:44.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.743+0000 7f4b6e4cb640 1 -- 172.21.15.89:0/259334711 shutdown_connections 2023-10-18T11:03:44.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.743+0000 7f4b6e4cb640 1 -- 172.21.15.89:0/259334711 >> 172.21.15.89:0/259334711 conn(0x7f4b68069f00 msgr2=0x7f4b680cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:44.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.743+0000 7f4b6e4cb640 1 -- 172.21.15.89:0/259334711 shutdown_connections 2023-10-18T11:03:44.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.743+0000 7f4b6e4cb640 1 -- 172.21.15.89:0/259334711 wait complete. 2023-10-18T11:03:44.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.747+0000 7f4b6e4cb640 1 Processor -- start 2023-10-18T11:03:44.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.747+0000 7f4b6e4cb640 1 -- start start 2023-10-18T11:03:44.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.747+0000 7f4b6e4cb640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b50004d10 0x7f4b680b3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:44.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.747+0000 7f4b6e4cb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b681f6020 0x7f4b6815cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:44.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.747+0000 7f4b6e4cb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b6815d150 0x7f4b680b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:44.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.747+0000 7f4b6e4cb640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6815d5d0 con 0x7f4b50004d10 2023-10-18T11:03:44.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.747+0000 7f4b6e4cb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b680b3d40 con 0x7f4b681f6020 2023-10-18T11:03:44.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.747+0000 7f4b6e4cb640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6815e5b0 con 0x7f4b6815d150 2023-10-18T11:03:44.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.747+0000 7f4b67fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b50004d10 0x7f4b680b3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:44.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.747+0000 7f4b67fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b50004d10 0x7f4b680b3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55632/0 (socket says 172.21.15.89:55632) 2023-10-18T11:03:44.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.747+0000 7f4b67fff640 1 -- 172.21.15.89:0/2803448872 learned_addr learned my addr 172.21.15.89:0/2803448872 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:44.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.747+0000 7f4b677fe640 1 --2- 172.21.15.89:0/2803448872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b681f6020 0x7f4b6815cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:44.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.747+0000 7f4b6ca68640 1 --2- 172.21.15.89:0/2803448872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b6815d150 0x7f4b680b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:44.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.747+0000 7f4b67fff640 1 -- 172.21.15.89:0/2803448872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b6815d150 msgr2=0x7f4b680b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:44.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.747+0000 7f4b67fff640 1 --2- 172.21.15.89:0/2803448872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b6815d150 0x7f4b680b0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:44.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.747+0000 7f4b67fff640 1 -- 172.21.15.89:0/2803448872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b681f6020 msgr2=0x7f4b6815cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:44.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.747+0000 7f4b67fff640 1 --2- 172.21.15.89:0/2803448872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b681f6020 0x7f4b6815cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:44.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.747+0000 7f4b67fff640 1 -- 172.21.15.89:0/2803448872 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b6815e890 con 0x7f4b50004d10 2023-10-18T11:03:44.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.747+0000 7f4b677fe640 1 --2- 172.21.15.89:0/2803448872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b681f6020 0x7f4b6815cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:03:44.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.747+0000 7f4b6ca68640 1 --2- 172.21.15.89:0/2803448872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b6815d150 0x7f4b680b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:03:44.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.747+0000 7f4b67fff640 1 --2- 172.21.15.89:0/2803448872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b50004d10 0x7f4b680b3510 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f4b580119d0 tx=0x7f4b58011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:44.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.751+0000 7f4b64ff9640 1 -- 172.21.15.89:0/2803448872 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5800fb50 con 0x7f4b50004d10 2023-10-18T11:03:44.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.751+0000 7f4b64ff9640 1 -- 172.21.15.89:0/2803448872 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5800fcb0 con 0x7f4b50004d10 2023-10-18T11:03:44.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.751+0000 7f4b64ff9640 1 -- 172.21.15.89:0/2803448872 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b58014440 con 0x7f4b50004d10 2023-10-18T11:03:44.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.751+0000 7f4b6e4cb640 1 -- 172.21.15.89:0/2803448872 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b6815eb70 con 0x7f4b50004d10 2023-10-18T11:03:44.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.751+0000 7f4b6e4cb640 1 -- 172.21.15.89:0/2803448872 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b680b3a50 con 0x7f4b50004d10 2023-10-18T11:03:44.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.751+0000 7f4b6e4cb640 1 -- 172.21.15.89:0/2803448872 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b6804dbe0 con 0x7f4b50004d10 2023-10-18T11:03:44.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.751+0000 7f4b64ff9640 1 -- 172.21.15.89:0/2803448872 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f4b580145a0 con 0x7f4b50004d10 2023-10-18T11:03:44.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.751+0000 7f4b64ff9640 1 --2- 172.21.15.89:0/2803448872 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4b540656b0 0x7f4b54067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:44.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.751+0000 7f4b64ff9640 1 -- 172.21.15.89:0/2803448872 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f4b58083500 con 0x7f4b50004d10 2023-10-18T11:03:44.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.751+0000 7f4b677fe640 1 --2- 172.21.15.89:0/2803448872 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4b540656b0 0x7f4b54067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:44.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.755+0000 7f4b677fe640 1 --2- 172.21.15.89:0/2803448872 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4b540656b0 0x7f4b54067b70 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c001280 tx=0x7f4b5c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:44.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.755+0000 7f4b64ff9640 1 -- 172.21.15.89:0/2803448872 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f4b58054c90 con 0x7f4b50004d10 2023-10-18T11:03:44.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.903+0000 7f4b6e4cb640 1 -- 172.21.15.89:0/2803448872 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_osd_laggy_weight"} v 0) v1 -- 0x7f4b680cba40 con 0x7f4b50004d10 2023-10-18T11:03:44.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.911+0000 7f4b64ff9640 1 -- 172.21.15.89:0/2803448872 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_osd_laggy_weight"}]=0 v15) v1 ==== 103+0+0 (secure 0 0 0) 0x7f4b58054820 con 0x7f4b50004d10 2023-10-18T11:03:44.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.911+0000 7f4b6e4cb640 1 -- 172.21.15.89:0/2803448872 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4b540656b0 msgr2=0x7f4b54067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:44.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.911+0000 7f4b6e4cb640 1 --2- 172.21.15.89:0/2803448872 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4b540656b0 0x7f4b54067b70 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c001280 tx=0x7f4b5c009040 comp rx=0 tx=0).stop 2023-10-18T11:03:44.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.911+0000 7f4b6e4cb640 1 -- 172.21.15.89:0/2803448872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b50004d10 msgr2=0x7f4b680b3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:44.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.911+0000 7f4b6e4cb640 1 --2- 172.21.15.89:0/2803448872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b50004d10 0x7f4b680b3510 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f4b580119d0 tx=0x7f4b58011e90 comp rx=0 tx=0).stop 2023-10-18T11:03:44.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.911+0000 7f4b6e4cb640 1 -- 172.21.15.89:0/2803448872 shutdown_connections 2023-10-18T11:03:44.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.911+0000 7f4b6e4cb640 1 --2- 172.21.15.89:0/2803448872 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4b540656b0 0x7f4b54067b70 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:44.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.911+0000 7f4b6e4cb640 1 --2- 172.21.15.89:0/2803448872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4b6815d150 0x7f4b680b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:44.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.911+0000 7f4b6e4cb640 1 --2- 172.21.15.89:0/2803448872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b681f6020 0x7f4b6815cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:44.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.911+0000 7f4b6e4cb640 1 --2- 172.21.15.89:0/2803448872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b50004d10 0x7f4b680b3510 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:44.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.911+0000 7f4b6e4cb640 1 -- 172.21.15.89:0/2803448872 >> 172.21.15.89:0/2803448872 conn(0x7f4b68069f00 msgr2=0x7f4b680ccbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:44.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.911+0000 7f4b6e4cb640 1 -- 172.21.15.89:0/2803448872 shutdown_connections 2023-10-18T11:03:44.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:44.911+0000 7f4b6e4cb640 1 -- 172.21.15.89:0/2803448872 wait complete. 2023-10-18T11:03:44.930 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-18T11:03:45.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.071+0000 7f8066fba640 1 Processor -- start 2023-10-18T11:03:45.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.071+0000 7f8066fba640 1 -- start start 2023-10-18T11:03:45.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.071+0000 7f8066fba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80600d0bc0 0x7f80600cd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:45.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.071+0000 7f8066fba640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f806015da50 con 0x7f806015c940 2023-10-18T11:03:45.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.071+0000 7f8066fba640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80600d3110 con 0x7f80600d0760 2023-10-18T11:03:45.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.071+0000 7f8066fba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f806015ce40 con 0x7f80600d0bc0 2023-10-18T11:03:45.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.071+0000 7f8064d56640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f806015c940 0x7f80600d2a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48120/0 (socket says 172.21.15.89:48120) 2023-10-18T11:03:45.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.071+0000 7f8064d56640 1 -- 172.21.15.89:0/1769908087 learned_addr learned my addr 172.21.15.89:0/1769908087 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:45.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.071+0000 7f8065557640 1 --2- 172.21.15.89:0/1769908087 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80600d0bc0 0x7f80600cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:45.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.071+0000 7f80577fe640 1 -- 172.21.15.89:0/1769908087 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 410874836 0 0) 0x7f806015da50 con 0x7f806015c940 2023-10-18T11:03:45.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.071+0000 7f80577fe640 1 -- 172.21.15.89:0/1769908087 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8048003530 con 0x7f806015c940 2023-10-18T11:03:45.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.071+0000 7f80577fe640 1 -- 172.21.15.89:0/1769908087 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4081304829 0 0) 0x7f8048003530 con 0x7f806015c940 2023-10-18T11:03:45.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.071+0000 7f80577fe640 1 -- 172.21.15.89:0/1769908087 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80600d0bc0 msgr2=0x7f80600cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:45.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.071+0000 7f80577fe640 1 --2- 172.21.15.89:0/1769908087 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80600d0bc0 0x7f80600cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:45.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.071+0000 7f80577fe640 1 -- 172.21.15.89:0/1769908087 >> v1:172.21.15.154:6789/0 conn(0x7f80600d0760 legacy=0x7f80600cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:45.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.071+0000 7f80577fe640 1 -- 172.21.15.89:0/1769908087 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f806015d120 con 0x7f806015c940 2023-10-18T11:03:45.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.071+0000 7f80577fe640 1 -- 172.21.15.89:0/1769908087 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7f805c001680 con 0x7f806015c940 2023-10-18T11:03:45.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.071+0000 7f80577fe640 1 -- 172.21.15.89:0/1769908087 >> v1:172.21.15.89:6789/0 conn(0x7f806015c940 legacy=0x7f80600d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:45.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.071+0000 7f80577fe640 1 --2- 172.21.15.89:0/1769908087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80480039b0 0x7f8048003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:45.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.071+0000 7f80577fe640 1 --2- 172.21.15.89:0/1769908087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8048004360 0x7f8048004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:45.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.071+0000 7f80577fe640 1 --2- 172.21.15.89:0/1769908087 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80600d0bc0 0x7f8048005090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:45.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.075+0000 7f80577fe640 1 -- 172.21.15.89:0/1769908087 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806015ce40 con 0x7f8048004360 2023-10-18T11:03:45.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.075+0000 7f80577fe640 1 -- 172.21.15.89:0/1769908087 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804800b9c0 con 0x7f80480039b0 2023-10-18T11:03:45.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.075+0000 7f80577fe640 1 -- 172.21.15.89:0/1769908087 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804800bb80 con 0x7f80600d0bc0 2023-10-18T11:03:45.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.075+0000 7f8057fff640 1 --2- 172.21.15.89:0/1769908087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8048004360 0x7f8048004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:45.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.075+0000 7f8064d56640 1 --2- 172.21.15.89:0/1769908087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80480039b0 0x7f8048003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:45.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.075+0000 7f8065557640 1 --2- 172.21.15.89:0/1769908087 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80600d0bc0 0x7f8048005090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:45.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.075+0000 7f8057fff640 1 -- 172.21.15.89:0/1769908087 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80600d0bc0 msgr2=0x7f8048005090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:45.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.075+0000 7f8057fff640 1 --2- 172.21.15.89:0/1769908087 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f80600d0bc0 0x7f8048005090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:45.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.075+0000 7f8057fff640 1 -- 172.21.15.89:0/1769908087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80480039b0 msgr2=0x7f8048003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:45.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.075+0000 7f8057fff640 1 --2- 172.21.15.89:0/1769908087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80480039b0 0x7f8048003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:45.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.075+0000 7f8057fff640 1 -- 172.21.15.89:0/1769908087 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8048003530 con 0x7f8048004360 2023-10-18T11:03:45.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.075+0000 7f8064d56640 1 --2- 172.21.15.89:0/1769908087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80480039b0 0x7f8048003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T11:03:45.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.075+0000 7f8057fff640 1 --2- 172.21.15.89:0/1769908087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8048004360 0x7f8048004830 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f8050001910 tx=0x7f805001ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d594dd1cec70a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:45.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.075+0000 7f80577fe640 1 -- 172.21.15.89:0/1769908087 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80500a88d0 con 0x7f8048004360 2023-10-18T11:03:45.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.075+0000 7f80577fe640 1 -- 172.21.15.89:0/1769908087 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80500a8a50 con 0x7f8048004360 2023-10-18T11:03:45.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.075+0000 7f80577fe640 1 -- 172.21.15.89:0/1769908087 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80500a8e00 con 0x7f8048004360 2023-10-18T11:03:45.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.079+0000 7f8066fba640 1 -- 172.21.15.89:0/1769908087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8048004360 msgr2=0x7f8048004830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:45.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.079+0000 7f8066fba640 1 --2- 172.21.15.89:0/1769908087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8048004360 0x7f8048004830 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f8050001910 tx=0x7f805001ecb0 comp rx=0 tx=0).stop 2023-10-18T11:03:45.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.079+0000 7f8065557640 1 -- 172.21.15.89:0/1769908087 reap_dead start 2023-10-18T11:03:45.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.079+0000 7f8066fba640 1 -- 172.21.15.89:0/1769908087 shutdown_connections 2023-10-18T11:03:45.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.079+0000 7f8066fba640 1 -- 172.21.15.89:0/1769908087 >> 172.21.15.89:0/1769908087 conn(0x7f8060089fd0 msgr2=0x7f806009b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:45.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.079+0000 7f8066fba640 1 -- 172.21.15.89:0/1769908087 shutdown_connections 2023-10-18T11:03:45.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.079+0000 7f8066fba640 1 -- 172.21.15.89:0/1769908087 wait complete. 2023-10-18T11:03:45.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.079+0000 7f8066fba640 1 Processor -- start 2023-10-18T11:03:45.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.079+0000 7f8066fba640 1 -- start start 2023-10-18T11:03:45.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.079+0000 7f8066fba640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8048004360 0x7f80600c6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:45.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.079+0000 7f8066fba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80600b7f10 0x7f8060150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:45.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.079+0000 7f8066fba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f806015e010 0x7f806015e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:45.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.079+0000 7f8066fba640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80600c7500 con 0x7f8048004360 2023-10-18T11:03:45.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.079+0000 7f8066fba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806015e9d0 con 0x7f80600b7f10 2023-10-18T11:03:45.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.079+0000 7f8066fba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8060151360 con 0x7f806015e010 2023-10-18T11:03:45.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.079+0000 7f8064d56640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8048004360 0x7f80600c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:45.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.079+0000 7f8064d56640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8048004360 0x7f80600c6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55656/0 (socket says 172.21.15.89:55656) 2023-10-18T11:03:45.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.079+0000 7f8064d56640 1 -- 172.21.15.89:0/1620824862 learned_addr learned my addr 172.21.15.89:0/1620824862 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:45.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.079+0000 7f8065557640 1 --2- 172.21.15.89:0/1620824862 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f806015e010 0x7f806015e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:45.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.079+0000 7f8057fff640 1 --2- 172.21.15.89:0/1620824862 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80600b7f10 0x7f8060150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:45.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.083+0000 7f8064d56640 1 -- 172.21.15.89:0/1620824862 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f806015e010 msgr2=0x7f806015e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:45.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.083+0000 7f8064d56640 1 --2- 172.21.15.89:0/1620824862 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f806015e010 0x7f806015e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:45.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.083+0000 7f8064d56640 1 -- 172.21.15.89:0/1620824862 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80600b7f10 msgr2=0x7f8060150e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:45.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.083+0000 7f8064d56640 1 --2- 172.21.15.89:0/1620824862 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80600b7f10 0x7f8060150e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:45.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.083+0000 7f8064d56640 1 -- 172.21.15.89:0/1620824862 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80601582c0 con 0x7f8048004360 2023-10-18T11:03:45.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.083+0000 7f8057fff640 1 --2- 172.21.15.89:0/1620824862 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80600b7f10 0x7f8060150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:03:45.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.083+0000 7f8064d56640 1 --2- 172.21.15.89:0/1620824862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8048004360 0x7f80600c6180 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f805c007290 tx=0x7f805c007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:45.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.083+0000 7f80557fa640 1 -- 172.21.15.89:0/1620824862 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805c00f9a0 con 0x7f8048004360 2023-10-18T11:03:45.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.083+0000 7f80557fa640 1 -- 172.21.15.89:0/1620824862 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f805c00fb00 con 0x7f8048004360 2023-10-18T11:03:45.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.083+0000 7f80557fa640 1 -- 172.21.15.89:0/1620824862 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805c00fe70 con 0x7f8048004360 2023-10-18T11:03:45.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.083+0000 7f8066fba640 1 -- 172.21.15.89:0/1620824862 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80600c6880 con 0x7f8048004360 2023-10-18T11:03:45.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.083+0000 7f8066fba640 1 -- 172.21.15.89:0/1620824862 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80600c6e40 con 0x7f8048004360 2023-10-18T11:03:45.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.083+0000 7f8066fba640 1 -- 172.21.15.89:0/1620824862 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f806004dbe0 con 0x7f8048004360 2023-10-18T11:03:45.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.083+0000 7f80557fa640 1 -- 172.21.15.89:0/1620824862 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f805c01d460 con 0x7f8048004360 2023-10-18T11:03:45.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.083+0000 7f80557fa640 1 --2- 172.21.15.89:0/1620824862 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f804c065680 0x7f804c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:45.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.083+0000 7f80557fa640 1 -- 172.21.15.89:0/1620824862 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f805c082ef0 con 0x7f8048004360 2023-10-18T11:03:45.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.083+0000 7f8057fff640 1 --2- 172.21.15.89:0/1620824862 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f804c065680 0x7f804c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:45.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.087+0000 7f8057fff640 1 --2- 172.21.15.89:0/1620824862 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f804c065680 0x7f804c067b40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f80500b2000 tx=0x7f8050001330 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:45.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.087+0000 7f80557fa640 1 -- 172.21.15.89:0/1620824862 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f805c050b40 con 0x7f8048004360 2023-10-18T11:03:45.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.239+0000 7f8066fba640 1 -- 172.21.15.89:0/1620824862 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f80600c98e0 con 0x7f8048004360 2023-10-18T11:03:45.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.243+0000 7f80557fa640 1 -- 172.21.15.89:0/1620824862 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v16) v1 ==== 104+0+0 (secure 0 0 0) 0x7f805c054160 con 0x7f8048004360 2023-10-18T11:03:45.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.247+0000 7f8066fba640 1 -- 172.21.15.89:0/1620824862 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f804c065680 msgr2=0x7f804c067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:45.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.247+0000 7f8066fba640 1 --2- 172.21.15.89:0/1620824862 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f804c065680 0x7f804c067b40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f80500b2000 tx=0x7f8050001330 comp rx=0 tx=0).stop 2023-10-18T11:03:45.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.247+0000 7f8066fba640 1 -- 172.21.15.89:0/1620824862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8048004360 msgr2=0x7f80600c6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:45.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.247+0000 7f8066fba640 1 --2- 172.21.15.89:0/1620824862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8048004360 0x7f80600c6180 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f805c007290 tx=0x7f805c007750 comp rx=0 tx=0).stop 2023-10-18T11:03:45.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.247+0000 7f8066fba640 1 -- 172.21.15.89:0/1620824862 shutdown_connections 2023-10-18T11:03:45.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.247+0000 7f8066fba640 1 --2- 172.21.15.89:0/1620824862 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f804c065680 0x7f804c067b40 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:45.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.247+0000 7f8066fba640 1 --2- 172.21.15.89:0/1620824862 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f806015e010 0x7f806015e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:45.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.247+0000 7f8066fba640 1 --2- 172.21.15.89:0/1620824862 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80600b7f10 0x7f8060150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:45.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.247+0000 7f8066fba640 1 --2- 172.21.15.89:0/1620824862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8048004360 0x7f80600c6180 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:45.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.247+0000 7f8066fba640 1 -- 172.21.15.89:0/1620824862 >> 172.21.15.89:0/1620824862 conn(0x7f8060089fd0 msgr2=0x7f806015a950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:45.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.247+0000 7f8066fba640 1 -- 172.21.15.89:0/1620824862 shutdown_connections 2023-10-18T11:03:45.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.247+0000 7f8066fba640 1 -- 172.21.15.89:0/1620824862 wait complete. 2023-10-18T11:03:45.265 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_recovery.TestClientOnLaggyOSD.test_client_eviction_if_config_is_set' 2023-10-18T11:03:45.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.407+0000 7f0f693fa640 1 Processor -- start 2023-10-18T11:03:45.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.407+0000 7f0f693fa640 1 -- start start 2023-10-18T11:03:45.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.407+0000 7f0f693fa640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0f6415c570 0x7f0f640d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:45.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.407+0000 7f0f693fa640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0f640cf560 0x7f0f640b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:45.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.407+0000 7f0f693fa640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f640d0e60 con 0x7f0f640d1a60 2023-10-18T11:03:45.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.407+0000 7f0f693fa640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f640d1140 con 0x7f0f640cf560 2023-10-18T11:03:45.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.407+0000 7f0f693fa640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f640d1420 con 0x7f0f6415c570 2023-10-18T11:03:45.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.407+0000 7f0f637fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0f640cf560 0x7f0f640b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:45.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.407+0000 7f0f627fc640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0f640d1a60 0x7f0f640cee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48130/0 (socket says 172.21.15.89:48130) 2023-10-18T11:03:45.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.407+0000 7f0f627fc640 1 -- 172.21.15.89:0/3973952843 learned_addr learned my addr 172.21.15.89:0/3973952843 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:45.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.407+0000 7f0f62ffd640 1 --2- 172.21.15.89:0/3973952843 >> v2:172.21.15.154:3300/0 conn(0x7f0f6415c570 0x7f0f640d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:45.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.407+0000 7f0f61ffb640 1 -- 172.21.15.89:0/3973952843 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2913386289 0 0) 0x7f0f640d0e60 con 0x7f0f640d1a60 2023-10-18T11:03:45.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.407+0000 7f0f61ffb640 1 -- 172.21.15.89:0/3973952843 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f4c003530 con 0x7f0f640d1a60 2023-10-18T11:03:45.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.407+0000 7f0f637fe640 1 -- 172.21.15.89:0/3973952843 >> v2:172.21.15.154:3300/0 conn(0x7f0f6415c570 msgr2=0x7f0f640d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:45.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.407+0000 7f0f637fe640 1 --2- 172.21.15.89:0/3973952843 >> v2:172.21.15.154:3300/0 conn(0x7f0f6415c570 0x7f0f640d0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:45.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.407+0000 7f0f637fe640 1 -- 172.21.15.89:0/3973952843 >> v1:172.21.15.89:6789/0 conn(0x7f0f640d1a60 legacy=0x7f0f640cee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:45.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.407+0000 7f0f637fe640 1 -- 172.21.15.89:0/3973952843 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f640b57e0 con 0x7f0f640cf560 2023-10-18T11:03:45.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.411+0000 7f0f637fe640 1 --2- 172.21.15.89:0/3973952843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f640cf560 0x7f0f640b5d00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f0f640d3590 tx=0x7f0f5c01c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8879751f9a0d5123 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:45.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.411+0000 7f0f61ffb640 1 -- 172.21.15.89:0/3973952843 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c0a58f0 con 0x7f0f640cf560 2023-10-18T11:03:45.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.411+0000 7f0f61ffb640 1 -- 172.21.15.89:0/3973952843 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f5c0a5a50 con 0x7f0f640cf560 2023-10-18T11:03:45.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.411+0000 7f0f61ffb640 1 -- 172.21.15.89:0/3973952843 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c0a5de0 con 0x7f0f640cf560 2023-10-18T11:03:45.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.411+0000 7f0f693fa640 1 -- 172.21.15.89:0/3973952843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f640cf560 msgr2=0x7f0f640b5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:45.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.411+0000 7f0f693fa640 1 --2- 172.21.15.89:0/3973952843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f640cf560 0x7f0f640b5d00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f0f640d3590 tx=0x7f0f5c01c910 comp rx=0 tx=0).stop 2023-10-18T11:03:45.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.411+0000 7f0f693fa640 1 -- 172.21.15.89:0/3973952843 shutdown_connections 2023-10-18T11:03:45.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.411+0000 7f0f693fa640 1 --2- 172.21.15.89:0/3973952843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f640cf560 0x7f0f640b5d00 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:45.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.411+0000 7f0f693fa640 1 --2- 172.21.15.89:0/3973952843 >> v2:172.21.15.154:3300/0 conn(0x7f0f6415c570 0x7f0f640d0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:45.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.411+0000 7f0f693fa640 1 -- 172.21.15.89:0/3973952843 >> 172.21.15.89:0/3973952843 conn(0x7f0f64070320 msgr2=0x7f0f64070730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:45.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.411+0000 7f0f693fa640 1 -- 172.21.15.89:0/3973952843 shutdown_connections 2023-10-18T11:03:45.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.411+0000 7f0f693fa640 1 -- 172.21.15.89:0/3973952843 wait complete. 2023-10-18T11:03:45.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.415+0000 7f0f693fa640 1 Processor -- start 2023-10-18T11:03:45.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.415+0000 7f0f693fa640 1 -- start start 2023-10-18T11:03:45.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.415+0000 7f0f693fa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f640cf560 0x7f0f640cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:45.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.415+0000 7f0f693fa640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f640d1a60 0x7f0f640c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:45.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.415+0000 7f0f693fa640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f6415c570 0x7f0f640c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:45.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.415+0000 7f0f693fa640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f640cc4d0 con 0x7f0f6415c570 2023-10-18T11:03:45.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.415+0000 7f0f693fa640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f640c70e0 con 0x7f0f640d1a60 2023-10-18T11:03:45.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.415+0000 7f0f693fa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f640c73c0 con 0x7f0f640cf560 2023-10-18T11:03:45.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.415+0000 7f0f637fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f6415c570 0x7f0f640c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:45.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.415+0000 7f0f637fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f6415c570 0x7f0f640c64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55666/0 (socket says 172.21.15.89:55666) 2023-10-18T11:03:45.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.415+0000 7f0f637fe640 1 -- 172.21.15.89:0/2123439269 learned_addr learned my addr 172.21.15.89:0/2123439269 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:45.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.415+0000 7f0f62ffd640 1 --2- 172.21.15.89:0/2123439269 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f640cf560 0x7f0f640cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:45.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.415+0000 7f0f627fc640 1 --2- 172.21.15.89:0/2123439269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f640d1a60 0x7f0f640c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:45.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.415+0000 7f0f637fe640 1 -- 172.21.15.89:0/2123439269 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f640cf560 msgr2=0x7f0f640cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:45.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.415+0000 7f0f637fe640 1 --2- 172.21.15.89:0/2123439269 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f640cf560 0x7f0f640cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:45.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.415+0000 7f0f637fe640 1 -- 172.21.15.89:0/2123439269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f640d1a60 msgr2=0x7f0f640c5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:45.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.415+0000 7f0f637fe640 1 --2- 172.21.15.89:0/2123439269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f640d1a60 0x7f0f640c5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:45.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.415+0000 7f0f637fe640 1 -- 172.21.15.89:0/2123439269 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5c009a70 con 0x7f0f6415c570 2023-10-18T11:03:45.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.415+0000 7f0f627fc640 1 --2- 172.21.15.89:0/2123439269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f640d1a60 0x7f0f640c5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:03:45.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.415+0000 7f0f637fe640 1 --2- 172.21.15.89:0/2123439269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f6415c570 0x7f0f640c64e0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c019520 tx=0x7f0f5c0196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:45.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.415+0000 7f0f4b7fe640 1 -- 172.21.15.89:0/2123439269 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c0b9860 con 0x7f0f6415c570 2023-10-18T11:03:45.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.415+0000 7f0f4b7fe640 1 -- 172.21.15.89:0/2123439269 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f5c0b99c0 con 0x7f0f6415c570 2023-10-18T11:03:45.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.415+0000 7f0f693fa640 1 -- 172.21.15.89:0/2123439269 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f640c76a0 con 0x7f0f6415c570 2023-10-18T11:03:45.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.415+0000 7f0f4b7fe640 1 -- 172.21.15.89:0/2123439269 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c0b9d70 con 0x7f0f6415c570 2023-10-18T11:03:45.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.415+0000 7f0f693fa640 1 -- 172.21.15.89:0/2123439269 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f6415fb00 con 0x7f0f6415c570 2023-10-18T11:03:45.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.415+0000 7f0f693fa640 1 -- 172.21.15.89:0/2123439269 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f640c7bc0 con 0x7f0f6415c570 2023-10-18T11:03:45.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.419+0000 7f0f4b7fe640 1 -- 172.21.15.89:0/2123439269 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f0f5c0c4460 con 0x7f0f6415c570 2023-10-18T11:03:45.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.419+0000 7f0f4b7fe640 1 --2- 172.21.15.89:0/2123439269 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0f50065680 0x7f0f50067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:45.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.419+0000 7f0f4b7fe640 1 -- 172.21.15.89:0/2123439269 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f0f5c11e0f0 con 0x7f0f6415c570 2023-10-18T11:03:45.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.419+0000 7f0f62ffd640 1 --2- 172.21.15.89:0/2123439269 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0f50065680 0x7f0f50067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:45.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.419+0000 7f0f62ffd640 1 --2- 172.21.15.89:0/2123439269 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0f50065680 0x7f0f50067b40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0f54009ac0 tx=0x7f0f54009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:45.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.423+0000 7f0f4b7fe640 1 -- 172.21.15.89:0/2123439269 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f0f5c0ebdc0 con 0x7f0f6415c570 2023-10-18T11:03:45.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:45.623+0000 7f0f693fa640 1 -- 172.21.15.89:0/2123439269 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientOnLaggyOSD.test_client_eviction_if_config_is_set"]} v 0) v1 -- 0x7f0f64070c70 con 0x7f0f6415c570 2023-10-18T11:03:46.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:46.223+0000 7f0f4b7fe640 1 -- 172.21.15.89:0/2123439269 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_recovery.TestClientOnLaggyOSD.test_client_eviction_if_config_is_set"]}]=0 v634) v1 ==== 171+0+0 (secure 0 0 0) 0x7f0f5c0ef3e0 con 0x7f0f6415c570 2023-10-18T11:03:46.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:46.227+0000 7f0f693fa640 1 -- 172.21.15.89:0/2123439269 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0f50065680 msgr2=0x7f0f50067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:46.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:46.227+0000 7f0f693fa640 1 --2- 172.21.15.89:0/2123439269 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0f50065680 0x7f0f50067b40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0f54009ac0 tx=0x7f0f54009190 comp rx=0 tx=0).stop 2023-10-18T11:03:46.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:46.227+0000 7f0f693fa640 1 -- 172.21.15.89:0/2123439269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f6415c570 msgr2=0x7f0f640c64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:46.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:46.227+0000 7f0f693fa640 1 --2- 172.21.15.89:0/2123439269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f6415c570 0x7f0f640c64e0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c019520 tx=0x7f0f5c0196b0 comp rx=0 tx=0).stop 2023-10-18T11:03:46.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:46.227+0000 7f0f693fa640 1 -- 172.21.15.89:0/2123439269 shutdown_connections 2023-10-18T11:03:46.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:46.227+0000 7f0f693fa640 1 --2- 172.21.15.89:0/2123439269 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0f50065680 0x7f0f50067b40 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:46.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:46.227+0000 7f0f693fa640 1 --2- 172.21.15.89:0/2123439269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f6415c570 0x7f0f640c64e0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:46.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:46.227+0000 7f0f693fa640 1 --2- 172.21.15.89:0/2123439269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f640d1a60 0x7f0f640c5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:46.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:46.227+0000 7f0f693fa640 1 --2- 172.21.15.89:0/2123439269 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f640cf560 0x7f0f640cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:46.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:46.227+0000 7f0f693fa640 1 -- 172.21.15.89:0/2123439269 >> 172.21.15.89:0/2123439269 conn(0x7f0f64070320 msgr2=0x7f0f641583d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:46.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:46.227+0000 7f0f693fa640 1 -- 172.21.15.89:0/2123439269 shutdown_connections 2023-10-18T11:03:46.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:46.227+0000 7f0f693fa640 1 -- 172.21.15.89:0/2123439269 wait complete. 2023-10-18T11:03:46.245 INFO:tasks.cephfs_test_runner:If any client gets unresponsive/it's session get idle due to lagginess ... ERROR 2023-10-18T11:03:46.246 INFO:tasks.cephfs_test_runner: 2023-10-18T11:03:46.246 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-18T11:03:46.246 INFO:tasks.cephfs_test_runner:ERROR: test_client_eviction_if_config_is_set (tasks.cephfs.test_client_recovery.TestClientOnLaggyOSD) 2023-10-18T11:03:46.247 INFO:tasks.cephfs_test_runner:If any client gets unresponsive/it's session get idle due to lagginess 2023-10-18T11:03:46.247 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-18T11:03:46.247 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-18T11:03:46.247 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_dparmar18_ceph_22c76d63886ee29a8c38dcd6634331fbd621f64a/qa/tasks/cephfs/test_client_recovery.py", line 827, in test_client_eviction_if_config_is_set 2023-10-18T11:03:46.248 INFO:tasks.cephfs_test_runner: while proceed(): 2023-10-18T11:03:46.248 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/contextutil.py", line 134, in __call__ 2023-10-18T11:03:46.248 INFO:tasks.cephfs_test_runner: raise MaxWhileTries(error_msg) 2023-10-18T11:03:46.248 INFO:tasks.cephfs_test_runner:teuthology.exceptions.MaxWhileTries: reached maximum tries (13) after waiting for 60 seconds 2023-10-18T11:03:46.249 INFO:tasks.cephfs_test_runner: 2023-10-18T11:03:46.249 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-18T11:03:46.249 INFO:tasks.cephfs_test_runner:Ran 2 tests in 1024.543s 2023-10-18T11:03:46.250 INFO:tasks.cephfs_test_runner: 2023-10-18T11:03:46.250 INFO:tasks.cephfs_test_runner:FAILED (errors=1, skipped=1) 2023-10-18T11:03:46.250 INFO:tasks.cephfs_test_runner: 2023-10-18T11:03:46.250 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-18T11:03:46.251 INFO:tasks.cephfs_test_runner:ERROR: test_client_eviction_if_config_is_set (tasks.cephfs.test_client_recovery.TestClientOnLaggyOSD) 2023-10-18T11:03:46.251 INFO:tasks.cephfs_test_runner:If any client gets unresponsive/it's session get idle due to lagginess 2023-10-18T11:03:46.251 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-18T11:03:46.252 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-18T11:03:46.252 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_dparmar18_ceph_22c76d63886ee29a8c38dcd6634331fbd621f64a/qa/tasks/cephfs/test_client_recovery.py", line 827, in test_client_eviction_if_config_is_set 2023-10-18T11:03:46.252 INFO:tasks.cephfs_test_runner: while proceed(): 2023-10-18T11:03:46.252 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/contextutil.py", line 134, in __call__ 2023-10-18T11:03:46.253 INFO:tasks.cephfs_test_runner: raise MaxWhileTries(error_msg) 2023-10-18T11:03:46.253 INFO:tasks.cephfs_test_runner:teuthology.exceptions.MaxWhileTries: reached maximum tries (13) after waiting for 60 seconds 2023-10-18T11:03:46.253 INFO:tasks.cephfs_test_runner: 2023-10-18T11:03:46.254 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_dparmar18_ceph_22c76d63886ee29a8c38dcd6634331fbd621f64a/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_client_eviction_if_config_is_set (tasks.cephfs.test_client_recovery.TestClientOnLaggyOSD) 2023-10-18T11:03:47.405 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=2373597022de49a9bdc383235bb1fedd Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_dparmar18_ceph_22c76d63886ee29a8c38dcd6634331fbd621f64a/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_client_eviction_if_config_is_set (tasks.cephfs.test_client_recovery.TestClientOnLaggyOSD) 2023-10-18T11:03:47.430 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-18T11:03:47.439 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-18T11:03:47.448 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-18T11:03:47.449 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T11:03:47.449 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:03:47.471 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354508k,nr_inodes=4088627,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275352k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9360 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275348k,nr_inodes=818837,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275352k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T11:03:47.471 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-18T11:03:47.472 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:03:47.521 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354508k,nr_inodes=4088627,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275352k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9360 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275348k,nr_inodes=818837,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275352k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T11:03:47.522 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T11:03:47.522 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:03:47.532 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354672k,nr_inodes=4088668,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4198 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T11:03:47.533 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-18T11:03:47.533 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:03:47.587 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354672k,nr_inodes=4088668,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4198 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T11:03:47.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:03:47.588 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-10-18T11:03:47.598 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-18T11:03:47.599 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-18T11:03:47.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:03:47.600 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T11:03:47.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:03:47.663 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T11:03:47.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:03:47.740 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link delete ceph-brx 2023-10-18T11:03:47.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:03:47.915 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-10-18T11:03:47.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:03:47.926 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link delete ceph-brx 2023-10-18T11:03:47.995 INFO:teuthology.orchestra.run.smithi089.stderr:Cannot find device "ceph-brx" 2023-10-18T11:03:47.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T11:03:47.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:03:47.998 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2023-10-18T11:03:48.008 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-18T11:03:48.009 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-18T11:03:48.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:03:48.010 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-18T11:03:48.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:03:48.078 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-18T11:03:48.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:03:48.169 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link delete ceph-brx 2023-10-18T11:03:48.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:03:48.367 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2023-10-18T11:03:48.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:03:48.377 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link delete ceph-brx 2023-10-18T11:03:48.442 INFO:teuthology.orchestra.run.smithi154.stderr:Cannot find device "ceph-brx" 2023-10-18T11:03:48.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T11:03:48.445 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-18T11:03:48.455 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-18T11:03:48.456 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T11:03:48.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.595+0000 7f2dff293640 1 Processor -- start 2023-10-18T11:03:48.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.595+0000 7f2dff293640 1 -- start start 2023-10-18T11:03:48.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.599+0000 7f2dff293640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2df8069900 0x7f2df8069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:48.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.599+0000 7f2dff293640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2df806cf30 0x7f2df80700d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:48.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.599+0000 7f2dff293640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2df806a220 con 0x7f2df815c8f0 2023-10-18T11:03:48.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.599+0000 7f2dff293640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2df815cdf0 con 0x7f2df8069900 2023-10-18T11:03:48.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.599+0000 7f2dff293640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df815d0d0 con 0x7f2df806cf30 2023-10-18T11:03:48.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.599+0000 7f2dfd02f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2df8069900 0x7f2df8069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:48.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.599+0000 7f2dfd02f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2df8069900 0x7f2df8069ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55674/0 (socket says 172.21.15.89:55674) 2023-10-18T11:03:48.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.599+0000 7f2dfd02f640 1 -- 172.21.15.89:0/3212461647 learned_addr learned my addr 172.21.15.89:0/3212461647 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:48.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.599+0000 7f2dfd02f640 1 -- 172.21.15.89:0/3212461647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2df806cf30 msgr2=0x7f2df80700d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:48.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.599+0000 7f2dfd02f640 1 --2- 172.21.15.89:0/3212461647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2df806cf30 0x7f2df80700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:48.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.599+0000 7f2dfd02f640 1 -- 172.21.15.89:0/3212461647 >> v1:172.21.15.154:6789/0 conn(0x7f2df815c8f0 legacy=0x7f2df806c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:48.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.599+0000 7f2dfd02f640 1 -- 172.21.15.89:0/3212461647 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df815d3b0 con 0x7f2df8069900 2023-10-18T11:03:48.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.599+0000 7f2dfd02f640 1 --2- 172.21.15.89:0/3212461647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df8069900 0x7f2df8069ce0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f2df4009870 tx=0x7f2df401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d28b1be40b7f245c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:48.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.603+0000 7f2deffff640 1 -- 172.21.15.89:0/3212461647 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df40aa5f0 con 0x7f2df8069900 2023-10-18T11:03:48.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.603+0000 7f2deffff640 1 -- 172.21.15.89:0/3212461647 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df40aa750 con 0x7f2df8069900 2023-10-18T11:03:48.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.603+0000 7f2deffff640 1 -- 172.21.15.89:0/3212461647 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df40aaae0 con 0x7f2df8069900 2023-10-18T11:03:48.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.603+0000 7f2dff293640 1 -- 172.21.15.89:0/3212461647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df8069900 msgr2=0x7f2df8069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:48.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.603+0000 7f2dff293640 1 --2- 172.21.15.89:0/3212461647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df8069900 0x7f2df8069ce0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f2df4009870 tx=0x7f2df401c4d0 comp rx=0 tx=0).stop 2023-10-18T11:03:48.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.603+0000 7f2dff293640 1 -- 172.21.15.89:0/3212461647 shutdown_connections 2023-10-18T11:03:48.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.603+0000 7f2dff293640 1 --2- 172.21.15.89:0/3212461647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2df806cf30 0x7f2df80700d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:48.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.603+0000 7f2dff293640 1 --2- 172.21.15.89:0/3212461647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df8069900 0x7f2df8069ce0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:48.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.603+0000 7f2dff293640 1 -- 172.21.15.89:0/3212461647 >> 172.21.15.89:0/3212461647 conn(0x7f2df80cb5a0 msgr2=0x7f2df80cb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:48.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.603+0000 7f2dff293640 1 -- 172.21.15.89:0/3212461647 shutdown_connections 2023-10-18T11:03:48.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.603+0000 7f2dff293640 1 -- 172.21.15.89:0/3212461647 wait complete. 2023-10-18T11:03:48.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.603+0000 7f2dff293640 1 Processor -- start 2023-10-18T11:03:48.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.603+0000 7f2dff293640 1 -- start start 2023-10-18T11:03:48.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.603+0000 7f2dff293640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2df8069900 0x7f2df8156030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:48.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.603+0000 7f2dff293640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df806cf30 0x7f2df8156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:48.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.603+0000 7f2dff293640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2df815c8f0 0x7f2df80c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:48.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.603+0000 7f2dff293640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df80ae500 con 0x7f2df806cf30 2023-10-18T11:03:48.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.603+0000 7f2dff293640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df80ae710 con 0x7f2df815c8f0 2023-10-18T11:03:48.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.603+0000 7f2dff293640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df80ae9f0 con 0x7f2df8069900 2023-10-18T11:03:48.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.607+0000 7f2dfc82e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df806cf30 0x7f2df8156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:48.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.607+0000 7f2dfd02f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2df8069900 0x7f2df8156030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:48.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.607+0000 7f2dfc82e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df806cf30 0x7f2df8156570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55684/0 (socket says 172.21.15.89:55684) 2023-10-18T11:03:48.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.607+0000 7f2dfc82e640 1 -- 172.21.15.89:0/1092990911 learned_addr learned my addr 172.21.15.89:0/1092990911 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:48.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.607+0000 7f2dfd830640 1 --2- 172.21.15.89:0/1092990911 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2df815c8f0 0x7f2df80c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:48.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.607+0000 7f2dfc82e640 1 -- 172.21.15.89:0/1092990911 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2df8069900 msgr2=0x7f2df8156030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:48.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.607+0000 7f2dfc82e640 1 --2- 172.21.15.89:0/1092990911 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2df8069900 0x7f2df8156030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:48.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.607+0000 7f2dfc82e640 1 -- 172.21.15.89:0/1092990911 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2df815c8f0 msgr2=0x7f2df80c93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:48.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.607+0000 7f2dfc82e640 1 --2- 172.21.15.89:0/1092990911 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2df815c8f0 0x7f2df80c93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:48.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.607+0000 7f2dfc82e640 1 -- 172.21.15.89:0/1092990911 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df40094d0 con 0x7f2df806cf30 2023-10-18T11:03:48.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.607+0000 7f2dfc82e640 1 --2- 172.21.15.89:0/1092990911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df806cf30 0x7f2df8156570 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2de800b3e0 tx=0x7f2de800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:48.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.607+0000 7f2dedffb640 1 -- 172.21.15.89:0/1092990911 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de8011830 con 0x7f2df806cf30 2023-10-18T11:03:48.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.607+0000 7f2dedffb640 1 -- 172.21.15.89:0/1092990911 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de80119b0 con 0x7f2df806cf30 2023-10-18T11:03:48.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.607+0000 7f2dff293640 1 -- 172.21.15.89:0/1092990911 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2df80aecd0 con 0x7f2df806cf30 2023-10-18T11:03:48.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.607+0000 7f2dedffb640 1 -- 172.21.15.89:0/1092990911 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de8011d60 con 0x7f2df806cf30 2023-10-18T11:03:48.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.607+0000 7f2dff293640 1 -- 172.21.15.89:0/1092990911 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df80af290 con 0x7f2df806cf30 2023-10-18T11:03:48.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.607+0000 7f2dff293640 1 -- 172.21.15.89:0/1092990911 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2df804dbe0 con 0x7f2df806cf30 2023-10-18T11:03:48.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.607+0000 7f2dedffb640 1 -- 172.21.15.89:0/1092990911 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f2de801a400 con 0x7f2df806cf30 2023-10-18T11:03:48.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.611+0000 7f2dedffb640 1 --2- 172.21.15.89:0/1092990911 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2ddc065680 0x7f2ddc067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:48.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.611+0000 7f2dedffb640 1 -- 172.21.15.89:0/1092990911 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f2de8082d20 con 0x7f2df806cf30 2023-10-18T11:03:48.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.611+0000 7f2dfd02f640 1 --2- 172.21.15.89:0/1092990911 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2ddc065680 0x7f2ddc067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:48.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.611+0000 7f2dfd02f640 1 --2- 172.21.15.89:0/1092990911 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2ddc065680 0x7f2ddc067b40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2df40ae000 tx=0x7f2df40b5040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:48.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.615+0000 7f2dedffb640 1 -- 172.21.15.89:0/1092990911 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f2de8050980 con 0x7f2df806cf30 2023-10-18T11:03:48.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.763+0000 7f2dff293640 1 -- 172.21.15.89:0/1092990911 --> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2df8073390 con 0x7f2ddc065680 2023-10-18T11:03:48.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.767+0000 7f2dedffb640 1 -- 172.21.15.89:0/1092990911 <== mgr.4099 v2:172.21.15.154:6832/1086608968 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72743 (secure 0 0 0) 0x7f2df8073390 con 0x7f2ddc065680 2023-10-18T11:03:48.766 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T11:03:48.767 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-18T11:03:48.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.767+0000 7f2dff293640 1 -- 172.21.15.89:0/1092990911 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2ddc065680 msgr2=0x7f2ddc067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:48.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.767+0000 7f2dff293640 1 --2- 172.21.15.89:0/1092990911 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2ddc065680 0x7f2ddc067b40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2df40ae000 tx=0x7f2df40b5040 comp rx=0 tx=0).stop 2023-10-18T11:03:48.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.767+0000 7f2dff293640 1 -- 172.21.15.89:0/1092990911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df806cf30 msgr2=0x7f2df8156570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:48.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.767+0000 7f2dff293640 1 --2- 172.21.15.89:0/1092990911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df806cf30 0x7f2df8156570 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2de800b3e0 tx=0x7f2de800b8a0 comp rx=0 tx=0).stop 2023-10-18T11:03:48.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.767+0000 7f2dff293640 1 -- 172.21.15.89:0/1092990911 shutdown_connections 2023-10-18T11:03:48.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.767+0000 7f2dff293640 1 --2- 172.21.15.89:0/1092990911 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2ddc065680 0x7f2ddc067b40 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:48.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.767+0000 7f2dff293640 1 --2- 172.21.15.89:0/1092990911 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2df815c8f0 0x7f2df80c93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:48.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.767+0000 7f2dff293640 1 --2- 172.21.15.89:0/1092990911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df806cf30 0x7f2df8156570 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:48.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.767+0000 7f2dff293640 1 --2- 172.21.15.89:0/1092990911 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2df8069900 0x7f2df8156030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:48.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.767+0000 7f2dff293640 1 -- 172.21.15.89:0/1092990911 >> 172.21.15.89:0/1092990911 conn(0x7f2df80cb5a0 msgr2=0x7f2df81531b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:48.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.767+0000 7f2dff293640 1 -- 172.21.15.89:0/1092990911 shutdown_connections 2023-10-18T11:03:48.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.767+0000 7f2dff293640 1 -- 172.21.15.89:0/1092990911 wait complete. 2023-10-18T11:03:48.786 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":595,"stamp":"2023-10-18T11:03:47.357608+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223192,"kb_used_data":6452,"kb_used_omap":18,"kb_used_meta":216621,"kb_avail":754751528,"statfs":{"total":773094113280,"available":772865564672,"internally_reserved":0,"allocated":6606848,"data_stored":3792169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19166,"internal_metadata":221820194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11768,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-63,"num_write_kb":-35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.275686"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":90,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-18T11:03:44.227989+0000","last_change":"2023-10-18T10:55:22.931642+0000","last_active":"2023-10-18T11:03:44.227989+0000","last_peered":"2023-10-18T11:03:44.227989+0000","last_clean":"2023-10-18T11:03:44.227989+0000","last_became_active":"2023-10-18T10:55:22.931232+0000","last_became_peered":"2023-10-18T10:55:22.931232+0000","last_unstale":"2023-10-18T11:03:44.227989+0000","last_undegraded":"2023-10-18T11:03:44.227989+0000","last_fullsized":"2023-10-18T11:03:44.227989+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:49:50.637795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-18T11:03:41.076191+0000","last_change":"2023-10-18T10:55:22.928478+0000","last_active":"2023-10-18T11:03:41.076191+0000","last_peered":"2023-10-18T11:03:41.076191+0000","last_clean":"2023-10-18T11:03:41.076191+0000","last_became_active":"2023-10-18T10:55:22.928049+0000","last_became_peered":"2023-10-18T10:55:22.928049+0000","last_unstale":"2023-10-18T11:03:41.076191+0000","last_undegraded":"2023-10-18T11:03:41.076191+0000","last_fullsized":"2023-10-18T11:03:41.076191+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:47:17.049053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":87,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-18T11:00:27.847340+0000","last_change":"2023-10-18T10:45:50.585224+0000","last_active":"2023-10-18T11:00:27.847340+0000","last_peered":"2023-10-18T11:00:27.847340+0000","last_clean":"2023-10-18T11:00:27.847340+0000","last_became_active":"2023-10-18T10:45:50.584872+0000","last_became_peered":"2023-10-18T10:45:50.584872+0000","last_unstale":"2023-10-18T11:00:27.847340+0000","last_undegraded":"2023-10-18T11:00:27.847340+0000","last_fullsized":"2023-10-18T11:00:27.847340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:54:40.531243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-18T11:03:41.076005+0000","last_change":"2023-10-18T10:45:50.584946+0000","last_active":"2023-10-18T11:03:41.076005+0000","last_peered":"2023-10-18T11:03:41.076005+0000","last_clean":"2023-10-18T11:03:41.076005+0000","last_became_active":"2023-10-18T10:45:50.584638+0000","last_became_peered":"2023-10-18T10:45:50.584638+0000","last_unstale":"2023-10-18T11:03:41.076005+0000","last_undegraded":"2023-10-18T11:03:41.076005+0000","last_fullsized":"2023-10-18T11:03:41.076005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:32:53.692894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":85,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-18T11:03:42.459961+0000","last_change":"2023-10-18T10:45:50.251505+0000","last_active":"2023-10-18T11:03:42.459961+0000","last_peered":"2023-10-18T11:03:42.459961+0000","last_clean":"2023-10-18T11:03:42.459961+0000","last_became_active":"2023-10-18T10:45:50.251111+0000","last_became_peered":"2023-10-18T10:45:50.251111+0000","last_unstale":"2023-10-18T11:03:42.459961+0000","last_undegraded":"2023-10-18T11:03:42.459961+0000","last_fullsized":"2023-10-18T11:03:42.459961+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:17:25.885756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":87,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-18T11:00:27.843486+0000","last_change":"2023-10-18T10:45:50.259019+0000","last_active":"2023-10-18T11:00:27.843486+0000","last_peered":"2023-10-18T11:00:27.843486+0000","last_clean":"2023-10-18T11:00:27.843486+0000","last_became_active":"2023-10-18T10:45:50.258607+0000","last_became_peered":"2023-10-18T10:45:50.258607+0000","last_unstale":"2023-10-18T11:00:27.843486+0000","last_undegraded":"2023-10-18T11:00:27.843486+0000","last_fullsized":"2023-10-18T11:00:27.843486+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:10:23.424148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":87,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-18T10:59:52.842325+0000","last_change":"2023-10-18T10:45:50.251375+0000","last_active":"2023-10-18T10:59:52.842325+0000","last_peered":"2023-10-18T10:59:52.842325+0000","last_clean":"2023-10-18T10:59:52.842325+0000","last_became_active":"2023-10-18T10:45:50.251055+0000","last_became_peered":"2023-10-18T10:45:50.251055+0000","last_unstale":"2023-10-18T10:59:52.842325+0000","last_undegraded":"2023-10-18T10:59:52.842325+0000","last_fullsized":"2023-10-18T10:59:52.842325+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:40:02.661776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":85,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-18T11:03:42.460309+0000","last_change":"2023-10-18T10:45:50.251310+0000","last_active":"2023-10-18T11:03:42.460309+0000","last_peered":"2023-10-18T11:03:42.460309+0000","last_clean":"2023-10-18T11:03:42.460309+0000","last_became_active":"2023-10-18T10:45:50.250823+0000","last_became_peered":"2023-10-18T10:45:50.250823+0000","last_unstale":"2023-10-18T11:03:42.460309+0000","last_undegraded":"2023-10-18T11:03:42.460309+0000","last_fullsized":"2023-10-18T11:03:42.460309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:58:21.895793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":243,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-18T10:59:47.841610+0000","last_change":"2023-10-18T10:45:44.563531+0000","last_active":"2023-10-18T10:59:47.841610+0000","last_peered":"2023-10-18T10:59:47.841610+0000","last_clean":"2023-10-18T10:59:47.841610+0000","last_became_active":"2023-10-18T10:45:44.562504+0000","last_became_peered":"2023-10-18T10:45:44.562504+0000","last_unstale":"2023-10-18T10:59:47.841610+0000","last_undegraded":"2023-10-18T10:59:47.841610+0000","last_fullsized":"2023-10-18T10:59:47.841610+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:43.181461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:43.181461+0000","last_clean_scrub_stamp":"2023-10-18T10:45:43.181461+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:06:40.551586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575067,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28240,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343600,"statfs":{"total":96636764160,"available":96607846400,"internally_reserved":0,"allocated":1253376,"data_stored":907493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:03:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70399999999999996,"15min":0.70499999999999996},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.40699999999999997},"max":{"1min":0.85599999999999998,"5min":0.93799999999999994,"15min":1.097},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.74399999999999999,"15min":0.72099999999999997},"min":{"1min":0.59599999999999997,"5min":0.434,"15min":0.432},"max":{"1min":0.89300000000000002,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.65100000000000002}]},{"osd":1,"last update":"Wed Oct 18 11:03:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72299999999999998,"15min":0.71699999999999997},"min":{"1min":0.57199999999999995,"5min":0.503,"15min":0.44400000000000001},"max":{"1min":0.88800000000000001,"5min":0.93300000000000005,"15min":0.95599999999999996},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.72499999999999998,"15min":0.73199999999999998},"min":{"1min":0.54600000000000004,"5min":0.52100000000000002,"15min":0.38200000000000001},"max":{"1min":0.94699999999999995,"5min":0.95499999999999996,"15min":0.99199999999999999},"last":0.60699999999999998}]},{"osd":2,"last update":"Wed Oct 18 11:03:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.73999999999999999,"15min":0.73599999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.39700000000000002},"max":{"1min":0.85499999999999998,"5min":0.97699999999999998,"15min":1.0760000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.74299999999999999,"15min":0.73699999999999999},"min":{"1min":0.57999999999999996,"5min":0.505,"15min":0.42099999999999999},"max":{"1min":0.95799999999999996,"5min":0.97699999999999998,"15min":1.0800000000000001},"last":0.48399999999999999}]},{"osd":3,"last update":"Wed Oct 18 11:03:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.76600000000000001,"15min":0.76200000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.92400000000000004,"5min":1.004,"15min":1.032},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.79900000000000004,"15min":0.79400000000000004},"min":{"1min":0.66200000000000003,"5min":0.50700000000000001,"15min":0.5},"max":{"1min":0.998,"5min":0.998,"15min":1.1060000000000001},"last":0.55200000000000005}]},{"osd":4,"last update":"Wed Oct 18 11:03:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58099999999999996,"15min":0.59599999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.34000000000000002},"max":{"1min":0.80600000000000005,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65900000000000003,"15min":0.66800000000000004},"min":{"1min":0.42699999999999999,"5min":0.38300000000000001,"15min":0.35799999999999998},"max":{"1min":0.83799999999999997,"5min":0.92800000000000005,"15min":1.046},"last":0.67800000000000005}]},{"osd":5,"last update":"Wed Oct 18 11:03:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58599999999999997,"15min":0.59699999999999998},"min":{"1min":0.42799999999999999,"5min":0.41399999999999998,"15min":0.375},"max":{"1min":0.79300000000000004,"5min":0.86599999999999999,"15min":0.95099999999999996},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60999999999999999,"15min":0.625},"min":{"1min":0.45900000000000002,"5min":0.44,"15min":0.33100000000000002},"max":{"1min":0.76100000000000001,"5min":0.94799999999999995,"15min":0.95699999999999996},"last":0.69799999999999995}]},{"osd":6,"last update":"Wed Oct 18 11:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70199999999999996,"15min":0.69499999999999995},"min":{"1min":0.53100000000000003,"5min":0.504,"15min":0.44400000000000001},"max":{"1min":0.94699999999999995,"5min":0.99299999999999999,"15min":1.004},"last":0.624},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.73899999999999999,"15min":0.753},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.85399999999999998,"5min":1.002,"15min":1.002},"last":0.76500000000000001}]}]},{"osd":6,"up_from":55,"seq":236223201382,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28288,"kb_used_data":688,"kb_used_omap":6,"kb_used_meta":27577,"kb_avail":94343552,"statfs":{"total":96636764160,"available":96607797248,"internally_reserved":0,"allocated":704512,"data_stored":338429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6699,"internal_metadata":28239317},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.754,"15min":0.71499999999999997},"min":{"1min":0.53900000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.96899999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73299999999999998,"15min":0.70199999999999996},"min":{"1min":0.56100000000000005,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.93899999999999995,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.71599999999999997}]},{"osd":1,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.76100000000000001,"15min":0.72499999999999998},"min":{"1min":0.61299999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96999999999999997,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.78200000000000003,"15min":0.76400000000000001},"min":{"1min":0.63900000000000001,"5min":0.54600000000000004,"15min":0.51300000000000001},"max":{"1min":1.0049999999999999,"5min":1.04,"15min":1.04},"last":0.92000000000000004}]},{"osd":2,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.73499999999999999,"15min":0.72499999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.48099999999999998},"max":{"1min":0.90100000000000002,"5min":1.024,"15min":1.024},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.77600000000000002,"15min":0.76300000000000001},"min":{"1min":0.56399999999999995,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.01,"5min":1.077,"15min":1.077},"last":0.94399999999999995}]},{"osd":3,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.75900000000000001,"15min":0.73599999999999999},"min":{"1min":0.58099999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.91100000000000003,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80100000000000005,"15min":0.76900000000000002},"min":{"1min":0.66000000000000003,"5min":0.57899999999999996,"15min":0.495},"max":{"1min":0.98799999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.73299999999999998}]},{"osd":4,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58899999999999997,"15min":0.63600000000000001},"min":{"1min":0.40200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78500000000000003,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.44},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61799999999999999,"15min":0.64500000000000002},"min":{"1min":0.45700000000000002,"5min":0.42999999999999999,"15min":0.40899999999999997},"max":{"1min":0.79600000000000004,"5min":0.91300000000000003,"15min":0.95099999999999996},"last":0.51900000000000002}]},{"osd":5,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64800000000000002,"15min":0.66900000000000004},"min":{"1min":0.48499999999999999,"5min":0.443,"15min":0.436},"max":{"1min":0.84499999999999997,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69499999999999995,"15min":0.72499999999999998},"min":{"1min":0.51000000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.90800000000000003,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.58699999999999997}]},{"osd":7,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71399999999999997,"15min":0.749},"min":{"1min":0.50700000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.89200000000000002,"5min":0.98799999999999999,"15min":1.0189999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73899999999999999,"15min":0.78100000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.871,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.66900000000000004}]}]},{"osd":5,"up_from":13,"seq":55834575067,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":663552,"data_stored":318436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71099999999999997,"15min":0.69699999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.40200000000000002},"max":{"1min":0.89000000000000001,"5min":0.93400000000000005,"15min":0.94499999999999995},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.76700000000000002,"15min":0.746},"min":{"1min":0.61299999999999999,"5min":0.42799999999999999,"15min":0.372},"max":{"1min":1.014,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.67400000000000004}]},{"osd":1,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.746,"15min":0.74199999999999999},"min":{"1min":0.57799999999999996,"5min":0.502,"15min":0.35599999999999998},"max":{"1min":0.95199999999999996,"5min":1.115,"15min":1.115},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72099999999999997,"15min":0.71399999999999997},"min":{"1min":0.53900000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.89600000000000002,"5min":1.038,"15min":1.038},"last":0.65600000000000003}]},{"osd":2,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.77800000000000002,"15min":0.752},"min":{"1min":0.57299999999999995,"5min":0.47999999999999998,"15min":0.40799999999999997},"max":{"1min":0.96899999999999997,"5min":1.133,"15min":1.133},"last":0.502},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.76300000000000001,"15min":0.76200000000000001},"min":{"1min":0.53100000000000003,"5min":0.52500000000000002,"15min":0.48099999999999998},"max":{"1min":0.98199999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.54900000000000004}]},{"osd":3,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.78500000000000003,"15min":0.78600000000000003},"min":{"1min":0.621,"5min":0.53200000000000003,"15min":0.52900000000000003},"max":{"1min":0.98999999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76300000000000001},"min":{"1min":0.63700000000000001,"5min":0.55600000000000005,"15min":0.44800000000000001},"max":{"1min":0.92600000000000005,"5min":1.083,"15min":1.083},"last":0.624}]},{"osd":4,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55500000000000005,"15min":0.54600000000000004},"min":{"1min":0.36199999999999999,"5min":0.35199999999999998,"15min":0.316},"max":{"1min":0.81299999999999994,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.69099999999999995,"15min":0.68200000000000005},"min":{"1min":0.47099999999999997,"5min":0.434,"15min":0.376},"max":{"1min":0.95399999999999996,"5min":1.099,"15min":1.099},"last":0.63900000000000001}]},{"osd":6,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72499999999999998,"15min":0.71999999999999997},"min":{"1min":0.49199999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.875,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.65100000000000002,"15min":0.65600000000000003},"min":{"1min":0.497,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88200000000000001,"5min":1.004,"15min":1.004},"last":0.41899999999999998}]},{"osd":7,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.753,"15min":0.73299999999999998},"min":{"1min":0.58699999999999997,"5min":0.46600000000000003,"15min":0.41899999999999998},"max":{"1min":0.98299999999999998,"5min":1.018,"15min":1.046},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.71199999999999997,"15min":0.69599999999999995},"min":{"1min":0.51500000000000001,"5min":0.45500000000000002,"15min":0.317},"max":{"1min":0.997,"5min":1.046,"15min":1.046},"last":0.47999999999999998}]}]},{"osd":4,"up_from":13,"seq":55834575067,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27692,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":688128,"data_stored":336468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.70199999999999996,"15min":0.69899999999999995},"min":{"1min":0.496,"5min":0.47999999999999998,"15min":0.39800000000000002},"max":{"1min":0.80900000000000005,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74099999999999999,"15min":0.72099999999999997},"min":{"1min":0.57399999999999995,"5min":0.376,"15min":0.376},"max":{"1min":0.88600000000000001,"5min":0.97299999999999998,"15min":1.018},"last":0.441}]},{"osd":1,"last update":"Wed Oct 18 11:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73399999999999999,"15min":0.754},"min":{"1min":0.52800000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.91900000000000004,"5min":0.96299999999999997,"15min":1.0449999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.80300000000000005,"15min":0.78100000000000003},"min":{"1min":0.57599999999999996,"5min":0.52900000000000003,"15min":0.40200000000000002},"max":{"1min":0.995,"5min":1.02,"15min":1.03},"last":0.77000000000000002}]},{"osd":2,"last update":"Wed Oct 18 11:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.746,"15min":0.76300000000000001},"min":{"1min":0.46200000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92900000000000005,"5min":1.022,"15min":1.022},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80900000000000005,"15min":0.79000000000000004},"min":{"1min":0.624,"5min":0.57299999999999995,"15min":0.52500000000000002},"max":{"1min":0.95399999999999996,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.72999999999999998}]},{"osd":3,"last update":"Wed Oct 18 11:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.78000000000000003,"15min":0.76100000000000001},"min":{"1min":0.52800000000000002,"5min":0.48099999999999998,"15min":0.439},"max":{"1min":0.92100000000000004,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.78400000000000003,"15min":0.79800000000000004},"min":{"1min":0.61699999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.97499999999999998,"5min":1.036,"15min":1.0940000000000001},"last":0.69399999999999995}]},{"osd":5,"last update":"Wed Oct 18 11:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.63,"15min":0.626},"min":{"1min":0.38900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82299999999999995,"5min":0.96499999999999997,"15min":1.048},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.66200000000000003,"15min":0.67700000000000005},"min":{"1min":0.39700000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.0620000000000001},"last":0.66200000000000003}]},{"osd":6,"last update":"Wed Oct 18 11:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63100000000000001,"15min":0.65600000000000003},"min":{"1min":0.441,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96199999999999997,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61799999999999999,"15min":0.65500000000000003},"min":{"1min":0.46999999999999997,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94299999999999995,"5min":1.0209999999999999,"15min":1.109},"last":0.71199999999999997}]},{"osd":7,"last update":"Wed Oct 18 11:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.68999999999999995,"15min":0.67400000000000004},"min":{"1min":0.502,"5min":0.45000000000000001,"15min":0.41599999999999998},"max":{"1min":1.0449999999999999,"5min":1.0549999999999999,"15min":1.1080000000000001},"last":0.625},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.628,"15min":0.66000000000000003},"min":{"1min":0.44800000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.0209999999999999,"5min":1.036,"15min":1.129},"last":0.75}]}]},{"osd":3,"up_from":13,"seq":55834575066,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":679936,"data_stored":324607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.46700000000000003,"15min":0.46000000000000002},"min":{"1min":0.29099999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.52500000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52700000000000002,"15min":0.52000000000000002},"min":{"1min":0.33700000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.65100000000000002,"5min":0.78000000000000003,"15min":0.78900000000000003},"last":0.41799999999999998}]},{"osd":1,"last update":"Wed Oct 18 11:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.48799999999999999,"15min":0.48999999999999999},"min":{"1min":0.37,"5min":0.33200000000000002,"15min":0.316},"max":{"1min":0.59999999999999998,"5min":0.76700000000000002,"15min":0.80000000000000004},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.42699999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.60299999999999998,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.51500000000000001}]},{"osd":2,"last update":"Wed Oct 18 11:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.44900000000000001,"5min":0.376,"15min":0.371},"max":{"1min":0.71099999999999997,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61399999999999999,"15min":0.60999999999999999},"min":{"1min":0.40300000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.86599999999999999,"5min":0.874,"15min":0.874},"last":0.44400000000000001}]},{"osd":4,"last update":"Wed Oct 18 11:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.754,"15min":0.77200000000000002},"min":{"1min":0.45200000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.88600000000000001,"5min":1.111,"15min":1.1919999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.78700000000000003,"15min":0.79900000000000004},"min":{"1min":0.59099999999999997,"5min":0.495,"15min":0.495},"max":{"1min":0.86599999999999999,"5min":1.137,"15min":1.246},"last":0.88700000000000001}]},{"osd":5,"last update":"Wed Oct 18 11:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.80900000000000005,"15min":0.81799999999999995},"min":{"1min":0.60299999999999998,"5min":0.59099999999999997,"15min":0.55100000000000005},"max":{"1min":0.90800000000000003,"5min":1.119,"15min":1.1859999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.78900000000000003,"15min":0.79000000000000004},"min":{"1min":0.59599999999999997,"5min":0.58499999999999996,"15min":0.49099999999999999},"max":{"1min":0.93600000000000005,"5min":1.137,"15min":1.155},"last":0.86199999999999999}]},{"osd":6,"last update":"Wed Oct 18 11:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.77100000000000002,"15min":0.83099999999999996},"min":{"1min":0.53800000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.97699999999999998,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.81699999999999995,"15min":0.85199999999999998},"min":{"1min":0.65500000000000003,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.96599999999999997,"5min":1.25,"15min":1.25},"last":0.78300000000000003}]},{"osd":7,"last update":"Wed Oct 18 11:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.78800000000000003,"15min":0.80800000000000005},"min":{"1min":0.61199999999999999,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.98599999999999999,"5min":1.1599999999999999,"15min":1.2529999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.83999999999999997,"15min":0.84599999999999997},"min":{"1min":0.65500000000000003,"5min":0.63100000000000001,"15min":0.62},"max":{"1min":0.93500000000000005,"5min":1.171,"15min":1.2410000000000001},"last":0.745}]}]},{"osd":2,"up_from":13,"seq":55834575068,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":675840,"data_stored":328267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.49399999999999999,"15min":0.48399999999999999},"min":{"1min":0.35399999999999998,"5min":0.313,"15min":0.27700000000000002},"max":{"1min":0.67300000000000004,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56299999999999994,"15min":0.56399999999999995},"min":{"1min":0.42499999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.73699999999999999,"5min":0.78400000000000003,"15min":0.80400000000000005},"last":0.46300000000000002}]},{"osd":1,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53500000000000003,"15min":0.53800000000000003},"min":{"1min":0.379,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.75,"5min":0.75700000000000001,"15min":0.82999999999999996},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.51600000000000001,"15min":0.51500000000000001},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.34599999999999997},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.83699999999999997},"last":0.377}]},{"osd":3,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57999999999999996,"15min":0.58099999999999996},"min":{"1min":0.46400000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79800000000000004,"5min":0.81599999999999995,"15min":0.91400000000000003},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.45900000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.73699999999999999,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.48599999999999999}]},{"osd":4,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81399999999999995,"15min":0.80600000000000005},"min":{"1min":0.60799999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0700000000000001,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.78200000000000003,"15min":0.77600000000000002},"min":{"1min":0.61799999999999999,"5min":0.42799999999999999,"15min":0.36699999999999999},"max":{"1min":0.98899999999999999,"5min":1.29,"15min":1.29},"last":0.78700000000000003}]},{"osd":5,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.80900000000000005,"15min":0.79300000000000004},"min":{"1min":0.57399999999999995,"5min":0.56999999999999995,"15min":0.51700000000000002},"max":{"1min":0.90500000000000003,"5min":1.264,"15min":1.264},"last":0.63},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.63300000000000001,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.99099999999999999,"5min":1.163,"15min":1.3120000000000001},"last":0.75}]},{"osd":6,"last update":"Wed Oct 18 11:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.80700000000000005,"15min":0.80300000000000005},"min":{"1min":0.60699999999999998,"5min":0.48599999999999999,"15min":0.42899999999999999},"max":{"1min":0.92600000000000005,"5min":1.107,"15min":1.1339999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.82299999999999995,"15min":0.81599999999999995},"min":{"1min":0.627,"5min":0.58499999999999996,"15min":0.57699999999999996},"max":{"1min":1.054,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.65400000000000003}]},{"osd":7,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81200000000000006,"15min":0.81999999999999995},"min":{"1min":0.65100000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0660000000000001,"5min":1.1299999999999999,"15min":1.2609999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.81999999999999995,"15min":0.80200000000000005},"min":{"1min":0.60099999999999998,"5min":0.58499999999999996,"15min":0.47499999999999998},"max":{"1min":1.026,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.73299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575067,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":675840,"data_stored":324119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.443,"15min":0.45300000000000001},"min":{"1min":0.311,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.63800000000000001,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.495,"15min":0.51000000000000001},"min":{"1min":0.34899999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.752,"5min":0.752,"15min":0.92500000000000004},"last":0.33300000000000002}]},{"osd":2,"last update":"Wed Oct 18 11:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46500000000000002,"15min":0.47999999999999998},"min":{"1min":0.39500000000000002,"5min":0.32600000000000001,"15min":0.32200000000000001},"max":{"1min":0.64700000000000002,"5min":0.74199999999999999,"15min":0.85599999999999998},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.52800000000000002,"15min":0.54900000000000004},"min":{"1min":0.40799999999999997,"5min":0.314,"15min":0.314},"max":{"1min":0.753,"5min":0.83499999999999996,"15min":0.97199999999999998},"last":0.46800000000000003}]},{"osd":3,"last update":"Wed Oct 18 11:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.54600000000000004,"15min":0.56599999999999995},"min":{"1min":0.44500000000000001,"5min":0.40899999999999997,"15min":0.377},"max":{"1min":0.71099999999999997,"5min":0.82899999999999996,"15min":0.90800000000000003},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.57399999999999995,"15min":0.58999999999999997},"min":{"1min":0.48299999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80500000000000005,"5min":0.91500000000000004,"15min":0.93899999999999995},"last":0.54900000000000004}]},{"osd":4,"last update":"Wed Oct 18 11:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.80600000000000005,"15min":0.79400000000000004},"min":{"1min":0.68000000000000005,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0700000000000001,"5min":1.389,"15min":1.389},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.75900000000000001,"15min":0.77500000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.50900000000000001},"max":{"1min":1.145,"5min":1.349,"15min":1.349},"last":0.61899999999999999}]},{"osd":5,"last update":"Wed Oct 18 11:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.78900000000000003,"15min":0.79900000000000004},"min":{"1min":0.65600000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.998,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.68500000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1830000000000001},"last":0.70599999999999996}]},{"osd":6,"last update":"Wed Oct 18 11:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.77700000000000002,"15min":0.83799999999999997},"min":{"1min":0.63900000000000001,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.99199999999999999,"5min":1.1339999999999999,"15min":1.157},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.81599999999999995,"15min":0.84599999999999997},"min":{"1min":0.68500000000000005,"5min":0.60699999999999998,"15min":0.56999999999999995},"max":{"1min":1.0329999999999999,"5min":1.1060000000000001,"15min":1.278},"last":0.68500000000000005}]},{"osd":7,"last update":"Wed Oct 18 11:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.81100000000000005,"15min":0.81899999999999995},"min":{"1min":0.63600000000000001,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0549999999999999,"5min":1.1899999999999999,"15min":1.2410000000000001},"last":0.746},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.81200000000000006,"15min":0.82099999999999995},"min":{"1min":0.70599999999999996,"5min":0.58999999999999997,"15min":0.57199999999999995},"max":{"1min":1.127,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.72199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575067,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28256,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343584,"statfs":{"total":96636764160,"available":96607830016,"internally_reserved":0,"allocated":1265664,"data_stored":914350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 11:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.45200000000000001,"15min":0.45100000000000001},"min":{"1min":0.34200000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.63800000000000001,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.504,"15min":0.51100000000000001},"min":{"1min":0.33500000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.753,"5min":0.753,"15min":0.86899999999999999},"last":0.44700000000000001}]},{"osd":2,"last update":"Wed Oct 18 11:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.50800000000000001,"15min":0.51200000000000001},"min":{"1min":0.378,"5min":0.309,"15min":0.309},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.80500000000000005},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54900000000000004,"15min":0.55100000000000005},"min":{"1min":0.42699999999999999,"5min":0.40100000000000002,"15min":0.38100000000000001},"max":{"1min":0.68100000000000005,"5min":0.80400000000000005,"15min":0.86699999999999999},"last":0.52900000000000003}]},{"osd":3,"last update":"Wed Oct 18 11:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.71199999999999997,"5min":0.77400000000000002,"15min":0.81899999999999995},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.45000000000000001,"5min":0.436,"15min":0.40100000000000002},"max":{"1min":0.75,"5min":0.75,"15min":0.85599999999999998},"last":0.56799999999999995}]},{"osd":4,"last update":"Wed Oct 18 11:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.81200000000000006,"15min":0.80600000000000005},"min":{"1min":0.58499999999999996,"5min":0.53000000000000003,"15min":0.48999999999999999},"max":{"1min":1.0649999999999999,"5min":1.2390000000000001,"15min":1.2470000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.81399999999999995,"15min":0.80600000000000005},"min":{"1min":0.52900000000000003,"5min":0.50900000000000001,"15min":0.45700000000000002},"max":{"1min":1.0549999999999999,"5min":1.389,"15min":1.389},"last":0.65300000000000002}]},{"osd":5,"last update":"Wed Oct 18 11:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81799999999999995,"15min":0.80700000000000005},"min":{"1min":0.61299999999999999,"5min":0.56799999999999995,"15min":0.53200000000000003},"max":{"1min":1.0920000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.82599999999999996,"15min":0.82099999999999995},"min":{"1min":0.65500000000000003,"5min":0.56299999999999994,"15min":0.50600000000000001},"max":{"1min":1.0349999999999999,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.76000000000000001}]},{"osd":6,"last update":"Wed Oct 18 11:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.81599999999999995,"15min":0.82199999999999995},"min":{"1min":0.56399999999999995,"5min":0.54900000000000004,"15min":0.54800000000000004},"max":{"1min":0.95099999999999996,"5min":1.24,"15min":1.2629999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.58499999999999996},"max":{"1min":1.008,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.66900000000000004}]},{"osd":7,"last update":"Wed Oct 18 11:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.84499999999999997,"15min":0.83799999999999997},"min":{"1min":0.65500000000000003,"5min":0.59699999999999998,"15min":0.56899999999999995},"max":{"1min":1.1259999999999999,"5min":1.409,"15min":1.409},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.82199999999999995,"15min":0.81200000000000006},"min":{"1min":0.60099999999999998,"5min":0.53700000000000003,"15min":0.503},"max":{"1min":0.98999999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.58499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T11:03:48.789 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T11:03:48.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.927+0000 7f70d638f640 1 Processor -- start 2023-10-18T11:03:48.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.927+0000 7f70d638f640 1 -- start start 2023-10-18T11:03:48.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.927+0000 7f70d638f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f70d00cd4f0 0x7f70d0143470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:48.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.927+0000 7f70d638f640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70d00cd8d0 con 0x7f70d0145b70 2023-10-18T11:03:48.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.927+0000 7f70d638f640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70d015cbf0 con 0x7f70d00ce070 2023-10-18T11:03:48.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.927+0000 7f70d638f640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d0069900 con 0x7f70d00cd4f0 2023-10-18T11:03:48.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.927+0000 7f70cf7fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f70d0145b70 0x7f70d0160cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48146/0 (socket says 172.21.15.89:48146) 2023-10-18T11:03:48.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.927+0000 7f70cf7fe640 1 -- 172.21.15.89:0/411880848 learned_addr learned my addr 172.21.15.89:0/411880848 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:48.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.927+0000 7f70cffff640 1 --2- 172.21.15.89:0/411880848 >> v2:172.21.15.89:3300/0 conn(0x7f70d00cd4f0 0x7f70d0143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:48.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.927+0000 7f70ceffd640 1 -- 172.21.15.89:0/411880848 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2376177177 0 0) 0x7f70d00cd8d0 con 0x7f70d0145b70 2023-10-18T11:03:48.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.931+0000 7f70ceffd640 1 -- 172.21.15.89:0/411880848 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70b8003530 con 0x7f70d0145b70 2023-10-18T11:03:48.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.931+0000 7f70cffff640 1 -- 172.21.15.89:0/411880848 >> v1:172.21.15.154:6789/0 conn(0x7f70d00ce070 legacy=0x7f70d006abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:48.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.931+0000 7f70cffff640 1 -- 172.21.15.89:0/411880848 >> v1:172.21.15.89:6789/0 conn(0x7f70d0145b70 legacy=0x7f70d0160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:48.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.931+0000 7f70cffff640 1 -- 172.21.15.89:0/411880848 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d0069b10 con 0x7f70d00cd4f0 2023-10-18T11:03:48.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.931+0000 7f70cffff640 1 --2- 172.21.15.89:0/411880848 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70d00cd4f0 0x7f70d0143470 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f70c00097c0 tx=0x7f70c001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54462713c247e4d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:48.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.931+0000 7f70ceffd640 1 -- 172.21.15.89:0/411880848 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c00ab650 con 0x7f70d00cd4f0 2023-10-18T11:03:48.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.931+0000 7f70ceffd640 1 -- 172.21.15.89:0/411880848 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c00ab7b0 con 0x7f70d00cd4f0 2023-10-18T11:03:48.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.931+0000 7f70ceffd640 1 -- 172.21.15.89:0/411880848 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c00abb40 con 0x7f70d00cd4f0 2023-10-18T11:03:48.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.931+0000 7f70d638f640 1 -- 172.21.15.89:0/411880848 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70d00cd4f0 msgr2=0x7f70d0143470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:48.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.931+0000 7f70d638f640 1 --2- 172.21.15.89:0/411880848 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70d00cd4f0 0x7f70d0143470 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f70c00097c0 tx=0x7f70c001c450 comp rx=0 tx=0).stop 2023-10-18T11:03:48.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.931+0000 7f70d638f640 1 -- 172.21.15.89:0/411880848 shutdown_connections 2023-10-18T11:03:48.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.931+0000 7f70d638f640 1 --2- 172.21.15.89:0/411880848 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70d00cd4f0 0x7f70d0143470 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:48.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.931+0000 7f70d638f640 1 -- 172.21.15.89:0/411880848 >> 172.21.15.89:0/411880848 conn(0x7f70d006dad0 msgr2=0x7f70d006dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:48.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.931+0000 7f70d638f640 1 -- 172.21.15.89:0/411880848 shutdown_connections 2023-10-18T11:03:48.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.931+0000 7f70d638f640 1 -- 172.21.15.89:0/411880848 wait complete. 2023-10-18T11:03:48.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.935+0000 7f70d638f640 1 Processor -- start 2023-10-18T11:03:48.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.935+0000 7f70d638f640 1 -- start start 2023-10-18T11:03:48.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.935+0000 7f70d638f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70d00cd4f0 0x7f70d00c2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:48.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.935+0000 7f70d638f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70d00ce070 0x7f70d00b1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:48.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.935+0000 7f70d638f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70d0145b70 0x7f70d00b1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:48.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.935+0000 7f70d638f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d015e740 con 0x7f70d00cd4f0 2023-10-18T11:03:48.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.935+0000 7f70d638f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d015ea20 con 0x7f70d00ce070 2023-10-18T11:03:48.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.935+0000 7f70d638f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d015ed00 con 0x7f70d0145b70 2023-10-18T11:03:48.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.935+0000 7f70cffff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70d00cd4f0 0x7f70d00c2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:48.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.935+0000 7f70cffff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70d00cd4f0 0x7f70d00c2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55700/0 (socket says 172.21.15.89:55700) 2023-10-18T11:03:48.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.935+0000 7f70cffff640 1 -- 172.21.15.89:0/746879495 learned_addr learned my addr 172.21.15.89:0/746879495 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:48.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.935+0000 7f70cf7fe640 1 --2- 172.21.15.89:0/746879495 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70d00ce070 0x7f70d00b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:48.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.935+0000 7f70d492c640 1 --2- 172.21.15.89:0/746879495 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70d0145b70 0x7f70d00b1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:48.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.935+0000 7f70cffff640 1 -- 172.21.15.89:0/746879495 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70d0145b70 msgr2=0x7f70d00b1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:48.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.935+0000 7f70cffff640 1 --2- 172.21.15.89:0/746879495 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70d0145b70 0x7f70d00b1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:48.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.935+0000 7f70cffff640 1 -- 172.21.15.89:0/746879495 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70d00ce070 msgr2=0x7f70d00b1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:48.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.935+0000 7f70cffff640 1 --2- 172.21.15.89:0/746879495 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70d00ce070 0x7f70d00b1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:48.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.935+0000 7f70cffff640 1 -- 172.21.15.89:0/746879495 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c0009420 con 0x7f70d00cd4f0 2023-10-18T11:03:48.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.935+0000 7f70cffff640 1 --2- 172.21.15.89:0/746879495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70d00cd4f0 0x7f70d00c2b90 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f70c00ad040 tx=0x7f70c001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:48.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.935+0000 7f70ccff9640 1 -- 172.21.15.89:0/746879495 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c00b5810 con 0x7f70d00cd4f0 2023-10-18T11:03:48.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.939+0000 7f70ccff9640 1 -- 172.21.15.89:0/746879495 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c00b5990 con 0x7f70d00cd4f0 2023-10-18T11:03:48.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.939+0000 7f70d638f640 1 -- 172.21.15.89:0/746879495 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d015efe0 con 0x7f70d00cd4f0 2023-10-18T11:03:48.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.939+0000 7f70ccff9640 1 -- 172.21.15.89:0/746879495 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c00b5d40 con 0x7f70d00cd4f0 2023-10-18T11:03:48.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.939+0000 7f70d638f640 1 -- 172.21.15.89:0/746879495 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d00c3410 con 0x7f70d00cd4f0 2023-10-18T11:03:48.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.939+0000 7f70d638f640 1 -- 172.21.15.89:0/746879495 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70d00b2e10 con 0x7f70d00cd4f0 2023-10-18T11:03:48.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.939+0000 7f70ccff9640 1 -- 172.21.15.89:0/746879495 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f70c00bf410 con 0x7f70d00cd4f0 2023-10-18T11:03:48.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.939+0000 7f70ccff9640 1 --2- 172.21.15.89:0/746879495 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f70bc0656b0 0x7f70bc067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:48.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.939+0000 7f70ccff9640 1 -- 172.21.15.89:0/746879495 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f70c011dd10 con 0x7f70d00cd4f0 2023-10-18T11:03:48.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.939+0000 7f70cf7fe640 1 --2- 172.21.15.89:0/746879495 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f70bc0656b0 0x7f70bc067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:48.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.939+0000 7f70cf7fe640 1 --2- 172.21.15.89:0/746879495 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f70bc0656b0 0x7f70bc067b70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f70d00cc940 tx=0x7f70c4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:48.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:48.943+0000 7f70ccff9640 1 -- 172.21.15.89:0/746879495 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f70c00eb9e0 con 0x7f70d00cd4f0 2023-10-18T11:03:49.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.091+0000 7f70d638f640 1 -- 172.21.15.89:0/746879495 --> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f70d006a690 con 0x7f70bc0656b0 2023-10-18T11:03:49.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.091+0000 7f70ccff9640 1 -- 172.21.15.89:0/746879495 <== mgr.4099 v2:172.21.15.154:6832/1086608968 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72743 (secure 0 0 0) 0x7f70c4001d80 con 0x7f70bc0656b0 2023-10-18T11:03:49.093 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T11:03:49.094 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-18T11:03:49.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.095+0000 7f70d638f640 1 -- 172.21.15.89:0/746879495 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f70bc0656b0 msgr2=0x7f70bc067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:49.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.095+0000 7f70d638f640 1 --2- 172.21.15.89:0/746879495 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f70bc0656b0 0x7f70bc067b70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f70d00cc940 tx=0x7f70c4009040 comp rx=0 tx=0).stop 2023-10-18T11:03:49.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.095+0000 7f70d638f640 1 -- 172.21.15.89:0/746879495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70d00cd4f0 msgr2=0x7f70d00c2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:49.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.095+0000 7f70d638f640 1 --2- 172.21.15.89:0/746879495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70d00cd4f0 0x7f70d00c2b90 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f70c00ad040 tx=0x7f70c001ac90 comp rx=0 tx=0).stop 2023-10-18T11:03:49.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.095+0000 7f70d638f640 1 -- 172.21.15.89:0/746879495 shutdown_connections 2023-10-18T11:03:49.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.095+0000 7f70d638f640 1 --2- 172.21.15.89:0/746879495 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f70bc0656b0 0x7f70bc067b70 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:49.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.095+0000 7f70d638f640 1 --2- 172.21.15.89:0/746879495 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70d0145b70 0x7f70d00b1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:49.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.095+0000 7f70d638f640 1 --2- 172.21.15.89:0/746879495 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70d00ce070 0x7f70d00b1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:49.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.095+0000 7f70d638f640 1 --2- 172.21.15.89:0/746879495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70d00cd4f0 0x7f70d00c2b90 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:49.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.095+0000 7f70d638f640 1 -- 172.21.15.89:0/746879495 >> 172.21.15.89:0/746879495 conn(0x7f70d006dad0 msgr2=0x7f70d00c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:49.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.095+0000 7f70d638f640 1 -- 172.21.15.89:0/746879495 shutdown_connections 2023-10-18T11:03:49.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.095+0000 7f70d638f640 1 -- 172.21.15.89:0/746879495 wait complete. 2023-10-18T11:03:49.113 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":595,"stamp":"2023-10-18T11:03:47.357608+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223192,"kb_used_data":6452,"kb_used_omap":18,"kb_used_meta":216621,"kb_avail":754751528,"statfs":{"total":773094113280,"available":772865564672,"internally_reserved":0,"allocated":6606848,"data_stored":3792169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19166,"internal_metadata":221820194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11768,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-63,"num_write_kb":-35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.275686"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":90,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-18T11:03:44.227989+0000","last_change":"2023-10-18T10:55:22.931642+0000","last_active":"2023-10-18T11:03:44.227989+0000","last_peered":"2023-10-18T11:03:44.227989+0000","last_clean":"2023-10-18T11:03:44.227989+0000","last_became_active":"2023-10-18T10:55:22.931232+0000","last_became_peered":"2023-10-18T10:55:22.931232+0000","last_unstale":"2023-10-18T11:03:44.227989+0000","last_undegraded":"2023-10-18T11:03:44.227989+0000","last_fullsized":"2023-10-18T11:03:44.227989+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:49:50.637795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-18T11:03:41.076191+0000","last_change":"2023-10-18T10:55:22.928478+0000","last_active":"2023-10-18T11:03:41.076191+0000","last_peered":"2023-10-18T11:03:41.076191+0000","last_clean":"2023-10-18T11:03:41.076191+0000","last_became_active":"2023-10-18T10:55:22.928049+0000","last_became_peered":"2023-10-18T10:55:22.928049+0000","last_unstale":"2023-10-18T11:03:41.076191+0000","last_undegraded":"2023-10-18T11:03:41.076191+0000","last_fullsized":"2023-10-18T11:03:41.076191+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:47:17.049053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":87,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-18T11:00:27.847340+0000","last_change":"2023-10-18T10:45:50.585224+0000","last_active":"2023-10-18T11:00:27.847340+0000","last_peered":"2023-10-18T11:00:27.847340+0000","last_clean":"2023-10-18T11:00:27.847340+0000","last_became_active":"2023-10-18T10:45:50.584872+0000","last_became_peered":"2023-10-18T10:45:50.584872+0000","last_unstale":"2023-10-18T11:00:27.847340+0000","last_undegraded":"2023-10-18T11:00:27.847340+0000","last_fullsized":"2023-10-18T11:00:27.847340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:54:40.531243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-18T11:03:41.076005+0000","last_change":"2023-10-18T10:45:50.584946+0000","last_active":"2023-10-18T11:03:41.076005+0000","last_peered":"2023-10-18T11:03:41.076005+0000","last_clean":"2023-10-18T11:03:41.076005+0000","last_became_active":"2023-10-18T10:45:50.584638+0000","last_became_peered":"2023-10-18T10:45:50.584638+0000","last_unstale":"2023-10-18T11:03:41.076005+0000","last_undegraded":"2023-10-18T11:03:41.076005+0000","last_fullsized":"2023-10-18T11:03:41.076005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:32:53.692894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":85,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-18T11:03:42.459961+0000","last_change":"2023-10-18T10:45:50.251505+0000","last_active":"2023-10-18T11:03:42.459961+0000","last_peered":"2023-10-18T11:03:42.459961+0000","last_clean":"2023-10-18T11:03:42.459961+0000","last_became_active":"2023-10-18T10:45:50.251111+0000","last_became_peered":"2023-10-18T10:45:50.251111+0000","last_unstale":"2023-10-18T11:03:42.459961+0000","last_undegraded":"2023-10-18T11:03:42.459961+0000","last_fullsized":"2023-10-18T11:03:42.459961+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:17:25.885756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":87,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-18T11:00:27.843486+0000","last_change":"2023-10-18T10:45:50.259019+0000","last_active":"2023-10-18T11:00:27.843486+0000","last_peered":"2023-10-18T11:00:27.843486+0000","last_clean":"2023-10-18T11:00:27.843486+0000","last_became_active":"2023-10-18T10:45:50.258607+0000","last_became_peered":"2023-10-18T10:45:50.258607+0000","last_unstale":"2023-10-18T11:00:27.843486+0000","last_undegraded":"2023-10-18T11:00:27.843486+0000","last_fullsized":"2023-10-18T11:00:27.843486+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:10:23.424148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":87,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-18T10:59:52.842325+0000","last_change":"2023-10-18T10:45:50.251375+0000","last_active":"2023-10-18T10:59:52.842325+0000","last_peered":"2023-10-18T10:59:52.842325+0000","last_clean":"2023-10-18T10:59:52.842325+0000","last_became_active":"2023-10-18T10:45:50.251055+0000","last_became_peered":"2023-10-18T10:45:50.251055+0000","last_unstale":"2023-10-18T10:59:52.842325+0000","last_undegraded":"2023-10-18T10:59:52.842325+0000","last_fullsized":"2023-10-18T10:59:52.842325+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:40:02.661776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":85,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-18T11:03:42.460309+0000","last_change":"2023-10-18T10:45:50.251310+0000","last_active":"2023-10-18T11:03:42.460309+0000","last_peered":"2023-10-18T11:03:42.460309+0000","last_clean":"2023-10-18T11:03:42.460309+0000","last_became_active":"2023-10-18T10:45:50.250823+0000","last_became_peered":"2023-10-18T10:45:50.250823+0000","last_unstale":"2023-10-18T11:03:42.460309+0000","last_undegraded":"2023-10-18T11:03:42.460309+0000","last_fullsized":"2023-10-18T11:03:42.460309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:58:21.895793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":243,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-18T10:59:47.841610+0000","last_change":"2023-10-18T10:45:44.563531+0000","last_active":"2023-10-18T10:59:47.841610+0000","last_peered":"2023-10-18T10:59:47.841610+0000","last_clean":"2023-10-18T10:59:47.841610+0000","last_became_active":"2023-10-18T10:45:44.562504+0000","last_became_peered":"2023-10-18T10:45:44.562504+0000","last_unstale":"2023-10-18T10:59:47.841610+0000","last_undegraded":"2023-10-18T10:59:47.841610+0000","last_fullsized":"2023-10-18T10:59:47.841610+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:43.181461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:43.181461+0000","last_clean_scrub_stamp":"2023-10-18T10:45:43.181461+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:06:40.551586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575067,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28240,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343600,"statfs":{"total":96636764160,"available":96607846400,"internally_reserved":0,"allocated":1253376,"data_stored":907493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:03:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70399999999999996,"15min":0.70499999999999996},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.40699999999999997},"max":{"1min":0.85599999999999998,"5min":0.93799999999999994,"15min":1.097},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.74399999999999999,"15min":0.72099999999999997},"min":{"1min":0.59599999999999997,"5min":0.434,"15min":0.432},"max":{"1min":0.89300000000000002,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.65100000000000002}]},{"osd":1,"last update":"Wed Oct 18 11:03:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72299999999999998,"15min":0.71699999999999997},"min":{"1min":0.57199999999999995,"5min":0.503,"15min":0.44400000000000001},"max":{"1min":0.88800000000000001,"5min":0.93300000000000005,"15min":0.95599999999999996},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.72499999999999998,"15min":0.73199999999999998},"min":{"1min":0.54600000000000004,"5min":0.52100000000000002,"15min":0.38200000000000001},"max":{"1min":0.94699999999999995,"5min":0.95499999999999996,"15min":0.99199999999999999},"last":0.60699999999999998}]},{"osd":2,"last update":"Wed Oct 18 11:03:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.73999999999999999,"15min":0.73599999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.39700000000000002},"max":{"1min":0.85499999999999998,"5min":0.97699999999999998,"15min":1.0760000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.74299999999999999,"15min":0.73699999999999999},"min":{"1min":0.57999999999999996,"5min":0.505,"15min":0.42099999999999999},"max":{"1min":0.95799999999999996,"5min":0.97699999999999998,"15min":1.0800000000000001},"last":0.48399999999999999}]},{"osd":3,"last update":"Wed Oct 18 11:03:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.76600000000000001,"15min":0.76200000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.92400000000000004,"5min":1.004,"15min":1.032},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.79900000000000004,"15min":0.79400000000000004},"min":{"1min":0.66200000000000003,"5min":0.50700000000000001,"15min":0.5},"max":{"1min":0.998,"5min":0.998,"15min":1.1060000000000001},"last":0.55200000000000005}]},{"osd":4,"last update":"Wed Oct 18 11:03:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58099999999999996,"15min":0.59599999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.34000000000000002},"max":{"1min":0.80600000000000005,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65900000000000003,"15min":0.66800000000000004},"min":{"1min":0.42699999999999999,"5min":0.38300000000000001,"15min":0.35799999999999998},"max":{"1min":0.83799999999999997,"5min":0.92800000000000005,"15min":1.046},"last":0.67800000000000005}]},{"osd":5,"last update":"Wed Oct 18 11:03:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58599999999999997,"15min":0.59699999999999998},"min":{"1min":0.42799999999999999,"5min":0.41399999999999998,"15min":0.375},"max":{"1min":0.79300000000000004,"5min":0.86599999999999999,"15min":0.95099999999999996},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60999999999999999,"15min":0.625},"min":{"1min":0.45900000000000002,"5min":0.44,"15min":0.33100000000000002},"max":{"1min":0.76100000000000001,"5min":0.94799999999999995,"15min":0.95699999999999996},"last":0.69799999999999995}]},{"osd":6,"last update":"Wed Oct 18 11:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70199999999999996,"15min":0.69499999999999995},"min":{"1min":0.53100000000000003,"5min":0.504,"15min":0.44400000000000001},"max":{"1min":0.94699999999999995,"5min":0.99299999999999999,"15min":1.004},"last":0.624},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.73899999999999999,"15min":0.753},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.85399999999999998,"5min":1.002,"15min":1.002},"last":0.76500000000000001}]}]},{"osd":6,"up_from":55,"seq":236223201382,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28288,"kb_used_data":688,"kb_used_omap":6,"kb_used_meta":27577,"kb_avail":94343552,"statfs":{"total":96636764160,"available":96607797248,"internally_reserved":0,"allocated":704512,"data_stored":338429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6699,"internal_metadata":28239317},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.754,"15min":0.71499999999999997},"min":{"1min":0.53900000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.96899999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73299999999999998,"15min":0.70199999999999996},"min":{"1min":0.56100000000000005,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.93899999999999995,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.71599999999999997}]},{"osd":1,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.76100000000000001,"15min":0.72499999999999998},"min":{"1min":0.61299999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96999999999999997,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.78200000000000003,"15min":0.76400000000000001},"min":{"1min":0.63900000000000001,"5min":0.54600000000000004,"15min":0.51300000000000001},"max":{"1min":1.0049999999999999,"5min":1.04,"15min":1.04},"last":0.92000000000000004}]},{"osd":2,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.73499999999999999,"15min":0.72499999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.48099999999999998},"max":{"1min":0.90100000000000002,"5min":1.024,"15min":1.024},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.77600000000000002,"15min":0.76300000000000001},"min":{"1min":0.56399999999999995,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.01,"5min":1.077,"15min":1.077},"last":0.94399999999999995}]},{"osd":3,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.75900000000000001,"15min":0.73599999999999999},"min":{"1min":0.58099999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.91100000000000003,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80100000000000005,"15min":0.76900000000000002},"min":{"1min":0.66000000000000003,"5min":0.57899999999999996,"15min":0.495},"max":{"1min":0.98799999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.73299999999999998}]},{"osd":4,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58899999999999997,"15min":0.63600000000000001},"min":{"1min":0.40200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78500000000000003,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.44},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61799999999999999,"15min":0.64500000000000002},"min":{"1min":0.45700000000000002,"5min":0.42999999999999999,"15min":0.40899999999999997},"max":{"1min":0.79600000000000004,"5min":0.91300000000000003,"15min":0.95099999999999996},"last":0.51900000000000002}]},{"osd":5,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64800000000000002,"15min":0.66900000000000004},"min":{"1min":0.48499999999999999,"5min":0.443,"15min":0.436},"max":{"1min":0.84499999999999997,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69499999999999995,"15min":0.72499999999999998},"min":{"1min":0.51000000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.90800000000000003,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.58699999999999997}]},{"osd":7,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71399999999999997,"15min":0.749},"min":{"1min":0.50700000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.89200000000000002,"5min":0.98799999999999999,"15min":1.0189999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73899999999999999,"15min":0.78100000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.871,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.66900000000000004}]}]},{"osd":5,"up_from":13,"seq":55834575067,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":663552,"data_stored":318436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71099999999999997,"15min":0.69699999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.40200000000000002},"max":{"1min":0.89000000000000001,"5min":0.93400000000000005,"15min":0.94499999999999995},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.76700000000000002,"15min":0.746},"min":{"1min":0.61299999999999999,"5min":0.42799999999999999,"15min":0.372},"max":{"1min":1.014,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.67400000000000004}]},{"osd":1,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.746,"15min":0.74199999999999999},"min":{"1min":0.57799999999999996,"5min":0.502,"15min":0.35599999999999998},"max":{"1min":0.95199999999999996,"5min":1.115,"15min":1.115},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72099999999999997,"15min":0.71399999999999997},"min":{"1min":0.53900000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.89600000000000002,"5min":1.038,"15min":1.038},"last":0.65600000000000003}]},{"osd":2,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.77800000000000002,"15min":0.752},"min":{"1min":0.57299999999999995,"5min":0.47999999999999998,"15min":0.40799999999999997},"max":{"1min":0.96899999999999997,"5min":1.133,"15min":1.133},"last":0.502},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.76300000000000001,"15min":0.76200000000000001},"min":{"1min":0.53100000000000003,"5min":0.52500000000000002,"15min":0.48099999999999998},"max":{"1min":0.98199999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.54900000000000004}]},{"osd":3,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.78500000000000003,"15min":0.78600000000000003},"min":{"1min":0.621,"5min":0.53200000000000003,"15min":0.52900000000000003},"max":{"1min":0.98999999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76300000000000001},"min":{"1min":0.63700000000000001,"5min":0.55600000000000005,"15min":0.44800000000000001},"max":{"1min":0.92600000000000005,"5min":1.083,"15min":1.083},"last":0.624}]},{"osd":4,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55500000000000005,"15min":0.54600000000000004},"min":{"1min":0.36199999999999999,"5min":0.35199999999999998,"15min":0.316},"max":{"1min":0.81299999999999994,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.69099999999999995,"15min":0.68200000000000005},"min":{"1min":0.47099999999999997,"5min":0.434,"15min":0.376},"max":{"1min":0.95399999999999996,"5min":1.099,"15min":1.099},"last":0.63900000000000001}]},{"osd":6,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72499999999999998,"15min":0.71999999999999997},"min":{"1min":0.49199999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.875,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.65100000000000002,"15min":0.65600000000000003},"min":{"1min":0.497,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88200000000000001,"5min":1.004,"15min":1.004},"last":0.41899999999999998}]},{"osd":7,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.753,"15min":0.73299999999999998},"min":{"1min":0.58699999999999997,"5min":0.46600000000000003,"15min":0.41899999999999998},"max":{"1min":0.98299999999999998,"5min":1.018,"15min":1.046},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.71199999999999997,"15min":0.69599999999999995},"min":{"1min":0.51500000000000001,"5min":0.45500000000000002,"15min":0.317},"max":{"1min":0.997,"5min":1.046,"15min":1.046},"last":0.47999999999999998}]}]},{"osd":4,"up_from":13,"seq":55834575067,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27692,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":688128,"data_stored":336468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.70199999999999996,"15min":0.69899999999999995},"min":{"1min":0.496,"5min":0.47999999999999998,"15min":0.39800000000000002},"max":{"1min":0.80900000000000005,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74099999999999999,"15min":0.72099999999999997},"min":{"1min":0.57399999999999995,"5min":0.376,"15min":0.376},"max":{"1min":0.88600000000000001,"5min":0.97299999999999998,"15min":1.018},"last":0.441}]},{"osd":1,"last update":"Wed Oct 18 11:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73399999999999999,"15min":0.754},"min":{"1min":0.52800000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.91900000000000004,"5min":0.96299999999999997,"15min":1.0449999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.80300000000000005,"15min":0.78100000000000003},"min":{"1min":0.57599999999999996,"5min":0.52900000000000003,"15min":0.40200000000000002},"max":{"1min":0.995,"5min":1.02,"15min":1.03},"last":0.77000000000000002}]},{"osd":2,"last update":"Wed Oct 18 11:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.746,"15min":0.76300000000000001},"min":{"1min":0.46200000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92900000000000005,"5min":1.022,"15min":1.022},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80900000000000005,"15min":0.79000000000000004},"min":{"1min":0.624,"5min":0.57299999999999995,"15min":0.52500000000000002},"max":{"1min":0.95399999999999996,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.72999999999999998}]},{"osd":3,"last update":"Wed Oct 18 11:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.78000000000000003,"15min":0.76100000000000001},"min":{"1min":0.52800000000000002,"5min":0.48099999999999998,"15min":0.439},"max":{"1min":0.92100000000000004,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.78400000000000003,"15min":0.79800000000000004},"min":{"1min":0.61699999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.97499999999999998,"5min":1.036,"15min":1.0940000000000001},"last":0.69399999999999995}]},{"osd":5,"last update":"Wed Oct 18 11:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.63,"15min":0.626},"min":{"1min":0.38900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82299999999999995,"5min":0.96499999999999997,"15min":1.048},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.66200000000000003,"15min":0.67700000000000005},"min":{"1min":0.39700000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.0620000000000001},"last":0.66200000000000003}]},{"osd":6,"last update":"Wed Oct 18 11:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63100000000000001,"15min":0.65600000000000003},"min":{"1min":0.441,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96199999999999997,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61799999999999999,"15min":0.65500000000000003},"min":{"1min":0.46999999999999997,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94299999999999995,"5min":1.0209999999999999,"15min":1.109},"last":0.71199999999999997}]},{"osd":7,"last update":"Wed Oct 18 11:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.68999999999999995,"15min":0.67400000000000004},"min":{"1min":0.502,"5min":0.45000000000000001,"15min":0.41599999999999998},"max":{"1min":1.0449999999999999,"5min":1.0549999999999999,"15min":1.1080000000000001},"last":0.625},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.628,"15min":0.66000000000000003},"min":{"1min":0.44800000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.0209999999999999,"5min":1.036,"15min":1.129},"last":0.75}]}]},{"osd":3,"up_from":13,"seq":55834575066,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":679936,"data_stored":324607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.46700000000000003,"15min":0.46000000000000002},"min":{"1min":0.29099999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.52500000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52700000000000002,"15min":0.52000000000000002},"min":{"1min":0.33700000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.65100000000000002,"5min":0.78000000000000003,"15min":0.78900000000000003},"last":0.41799999999999998}]},{"osd":1,"last update":"Wed Oct 18 11:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.48799999999999999,"15min":0.48999999999999999},"min":{"1min":0.37,"5min":0.33200000000000002,"15min":0.316},"max":{"1min":0.59999999999999998,"5min":0.76700000000000002,"15min":0.80000000000000004},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.42699999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.60299999999999998,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.51500000000000001}]},{"osd":2,"last update":"Wed Oct 18 11:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.44900000000000001,"5min":0.376,"15min":0.371},"max":{"1min":0.71099999999999997,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61399999999999999,"15min":0.60999999999999999},"min":{"1min":0.40300000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.86599999999999999,"5min":0.874,"15min":0.874},"last":0.44400000000000001}]},{"osd":4,"last update":"Wed Oct 18 11:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.754,"15min":0.77200000000000002},"min":{"1min":0.45200000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.88600000000000001,"5min":1.111,"15min":1.1919999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.78700000000000003,"15min":0.79900000000000004},"min":{"1min":0.59099999999999997,"5min":0.495,"15min":0.495},"max":{"1min":0.86599999999999999,"5min":1.137,"15min":1.246},"last":0.88700000000000001}]},{"osd":5,"last update":"Wed Oct 18 11:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.80900000000000005,"15min":0.81799999999999995},"min":{"1min":0.60299999999999998,"5min":0.59099999999999997,"15min":0.55100000000000005},"max":{"1min":0.90800000000000003,"5min":1.119,"15min":1.1859999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.78900000000000003,"15min":0.79000000000000004},"min":{"1min":0.59599999999999997,"5min":0.58499999999999996,"15min":0.49099999999999999},"max":{"1min":0.93600000000000005,"5min":1.137,"15min":1.155},"last":0.86199999999999999}]},{"osd":6,"last update":"Wed Oct 18 11:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.77100000000000002,"15min":0.83099999999999996},"min":{"1min":0.53800000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.97699999999999998,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.81699999999999995,"15min":0.85199999999999998},"min":{"1min":0.65500000000000003,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.96599999999999997,"5min":1.25,"15min":1.25},"last":0.78300000000000003}]},{"osd":7,"last update":"Wed Oct 18 11:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.78800000000000003,"15min":0.80800000000000005},"min":{"1min":0.61199999999999999,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.98599999999999999,"5min":1.1599999999999999,"15min":1.2529999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.83999999999999997,"15min":0.84599999999999997},"min":{"1min":0.65500000000000003,"5min":0.63100000000000001,"15min":0.62},"max":{"1min":0.93500000000000005,"5min":1.171,"15min":1.2410000000000001},"last":0.745}]}]},{"osd":2,"up_from":13,"seq":55834575068,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":675840,"data_stored":328267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.49399999999999999,"15min":0.48399999999999999},"min":{"1min":0.35399999999999998,"5min":0.313,"15min":0.27700000000000002},"max":{"1min":0.67300000000000004,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56299999999999994,"15min":0.56399999999999995},"min":{"1min":0.42499999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.73699999999999999,"5min":0.78400000000000003,"15min":0.80400000000000005},"last":0.46300000000000002}]},{"osd":1,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53500000000000003,"15min":0.53800000000000003},"min":{"1min":0.379,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.75,"5min":0.75700000000000001,"15min":0.82999999999999996},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.51600000000000001,"15min":0.51500000000000001},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.34599999999999997},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.83699999999999997},"last":0.377}]},{"osd":3,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57999999999999996,"15min":0.58099999999999996},"min":{"1min":0.46400000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79800000000000004,"5min":0.81599999999999995,"15min":0.91400000000000003},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.45900000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.73699999999999999,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.48599999999999999}]},{"osd":4,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81399999999999995,"15min":0.80600000000000005},"min":{"1min":0.60799999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0700000000000001,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.78200000000000003,"15min":0.77600000000000002},"min":{"1min":0.61799999999999999,"5min":0.42799999999999999,"15min":0.36699999999999999},"max":{"1min":0.98899999999999999,"5min":1.29,"15min":1.29},"last":0.78700000000000003}]},{"osd":5,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.80900000000000005,"15min":0.79300000000000004},"min":{"1min":0.57399999999999995,"5min":0.56999999999999995,"15min":0.51700000000000002},"max":{"1min":0.90500000000000003,"5min":1.264,"15min":1.264},"last":0.63},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.63300000000000001,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.99099999999999999,"5min":1.163,"15min":1.3120000000000001},"last":0.75}]},{"osd":6,"last update":"Wed Oct 18 11:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.80700000000000005,"15min":0.80300000000000005},"min":{"1min":0.60699999999999998,"5min":0.48599999999999999,"15min":0.42899999999999999},"max":{"1min":0.92600000000000005,"5min":1.107,"15min":1.1339999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.82299999999999995,"15min":0.81599999999999995},"min":{"1min":0.627,"5min":0.58499999999999996,"15min":0.57699999999999996},"max":{"1min":1.054,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.65400000000000003}]},{"osd":7,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81200000000000006,"15min":0.81999999999999995},"min":{"1min":0.65100000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0660000000000001,"5min":1.1299999999999999,"15min":1.2609999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.81999999999999995,"15min":0.80200000000000005},"min":{"1min":0.60099999999999998,"5min":0.58499999999999996,"15min":0.47499999999999998},"max":{"1min":1.026,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.73299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575067,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":675840,"data_stored":324119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.443,"15min":0.45300000000000001},"min":{"1min":0.311,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.63800000000000001,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.495,"15min":0.51000000000000001},"min":{"1min":0.34899999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.752,"5min":0.752,"15min":0.92500000000000004},"last":0.33300000000000002}]},{"osd":2,"last update":"Wed Oct 18 11:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46500000000000002,"15min":0.47999999999999998},"min":{"1min":0.39500000000000002,"5min":0.32600000000000001,"15min":0.32200000000000001},"max":{"1min":0.64700000000000002,"5min":0.74199999999999999,"15min":0.85599999999999998},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.52800000000000002,"15min":0.54900000000000004},"min":{"1min":0.40799999999999997,"5min":0.314,"15min":0.314},"max":{"1min":0.753,"5min":0.83499999999999996,"15min":0.97199999999999998},"last":0.46800000000000003}]},{"osd":3,"last update":"Wed Oct 18 11:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.54600000000000004,"15min":0.56599999999999995},"min":{"1min":0.44500000000000001,"5min":0.40899999999999997,"15min":0.377},"max":{"1min":0.71099999999999997,"5min":0.82899999999999996,"15min":0.90800000000000003},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.57399999999999995,"15min":0.58999999999999997},"min":{"1min":0.48299999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80500000000000005,"5min":0.91500000000000004,"15min":0.93899999999999995},"last":0.54900000000000004}]},{"osd":4,"last update":"Wed Oct 18 11:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.80600000000000005,"15min":0.79400000000000004},"min":{"1min":0.68000000000000005,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0700000000000001,"5min":1.389,"15min":1.389},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.75900000000000001,"15min":0.77500000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.50900000000000001},"max":{"1min":1.145,"5min":1.349,"15min":1.349},"last":0.61899999999999999}]},{"osd":5,"last update":"Wed Oct 18 11:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.78900000000000003,"15min":0.79900000000000004},"min":{"1min":0.65600000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.998,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.68500000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1830000000000001},"last":0.70599999999999996}]},{"osd":6,"last update":"Wed Oct 18 11:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.77700000000000002,"15min":0.83799999999999997},"min":{"1min":0.63900000000000001,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.99199999999999999,"5min":1.1339999999999999,"15min":1.157},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.81599999999999995,"15min":0.84599999999999997},"min":{"1min":0.68500000000000005,"5min":0.60699999999999998,"15min":0.56999999999999995},"max":{"1min":1.0329999999999999,"5min":1.1060000000000001,"15min":1.278},"last":0.68500000000000005}]},{"osd":7,"last update":"Wed Oct 18 11:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.81100000000000005,"15min":0.81899999999999995},"min":{"1min":0.63600000000000001,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0549999999999999,"5min":1.1899999999999999,"15min":1.2410000000000001},"last":0.746},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.81200000000000006,"15min":0.82099999999999995},"min":{"1min":0.70599999999999996,"5min":0.58999999999999997,"15min":0.57199999999999995},"max":{"1min":1.127,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.72199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575067,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28256,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343584,"statfs":{"total":96636764160,"available":96607830016,"internally_reserved":0,"allocated":1265664,"data_stored":914350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 11:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.45200000000000001,"15min":0.45100000000000001},"min":{"1min":0.34200000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.63800000000000001,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.504,"15min":0.51100000000000001},"min":{"1min":0.33500000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.753,"5min":0.753,"15min":0.86899999999999999},"last":0.44700000000000001}]},{"osd":2,"last update":"Wed Oct 18 11:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.50800000000000001,"15min":0.51200000000000001},"min":{"1min":0.378,"5min":0.309,"15min":0.309},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.80500000000000005},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54900000000000004,"15min":0.55100000000000005},"min":{"1min":0.42699999999999999,"5min":0.40100000000000002,"15min":0.38100000000000001},"max":{"1min":0.68100000000000005,"5min":0.80400000000000005,"15min":0.86699999999999999},"last":0.52900000000000003}]},{"osd":3,"last update":"Wed Oct 18 11:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.71199999999999997,"5min":0.77400000000000002,"15min":0.81899999999999995},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.45000000000000001,"5min":0.436,"15min":0.40100000000000002},"max":{"1min":0.75,"5min":0.75,"15min":0.85599999999999998},"last":0.56799999999999995}]},{"osd":4,"last update":"Wed Oct 18 11:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.81200000000000006,"15min":0.80600000000000005},"min":{"1min":0.58499999999999996,"5min":0.53000000000000003,"15min":0.48999999999999999},"max":{"1min":1.0649999999999999,"5min":1.2390000000000001,"15min":1.2470000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.81399999999999995,"15min":0.80600000000000005},"min":{"1min":0.52900000000000003,"5min":0.50900000000000001,"15min":0.45700000000000002},"max":{"1min":1.0549999999999999,"5min":1.389,"15min":1.389},"last":0.65300000000000002}]},{"osd":5,"last update":"Wed Oct 18 11:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81799999999999995,"15min":0.80700000000000005},"min":{"1min":0.61299999999999999,"5min":0.56799999999999995,"15min":0.53200000000000003},"max":{"1min":1.0920000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.82599999999999996,"15min":0.82099999999999995},"min":{"1min":0.65500000000000003,"5min":0.56299999999999994,"15min":0.50600000000000001},"max":{"1min":1.0349999999999999,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.76000000000000001}]},{"osd":6,"last update":"Wed Oct 18 11:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.81599999999999995,"15min":0.82199999999999995},"min":{"1min":0.56399999999999995,"5min":0.54900000000000004,"15min":0.54800000000000004},"max":{"1min":0.95099999999999996,"5min":1.24,"15min":1.2629999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.58499999999999996},"max":{"1min":1.008,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.66900000000000004}]},{"osd":7,"last update":"Wed Oct 18 11:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.84499999999999997,"15min":0.83799999999999997},"min":{"1min":0.65500000000000003,"5min":0.59699999999999998,"15min":0.56899999999999995},"max":{"1min":1.1259999999999999,"5min":1.409,"15min":1.409},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.82199999999999995,"15min":0.81200000000000006},"min":{"1min":0.60099999999999998,"5min":0.53700000000000003,"15min":0.503},"max":{"1min":0.98999999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.58499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T11:03:49.115 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-18T11:03:49.116 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T11:03:49.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.255+0000 7f9060f03640 1 Processor -- start 2023-10-18T11:03:49.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.255+0000 7f9060f03640 1 -- start start 2023-10-18T11:03:49.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.255+0000 7f9060f03640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f905c15c550 0x7f905c0d2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:49.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.255+0000 7f9060f03640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f905c0b1860 0x7f905c0cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:49.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.255+0000 7f9060f03640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f905c0d1e50 con 0x7f905c0b1cc0 2023-10-18T11:03:49.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.255+0000 7f9060f03640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f905c0d2130 con 0x7f905c0b1860 2023-10-18T11:03:49.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.255+0000 7f9060f03640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f905c0d2410 con 0x7f905c15c550 2023-10-18T11:03:49.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.255+0000 7f9059d9b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f905c0b1860 0x7f905c0cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:49.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.255+0000 7f905ad9d640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f905c0b1cc0 0x7f905c0b55f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48150/0 (socket says 172.21.15.89:48150) 2023-10-18T11:03:49.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.255+0000 7f905ad9d640 1 -- 172.21.15.89:0/2087609431 learned_addr learned my addr 172.21.15.89:0/2087609431 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:49.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.255+0000 7f905a59c640 1 --2- 172.21.15.89:0/2087609431 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f905c15c550 0x7f905c0d2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:49.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.255+0000 7f905959a640 1 -- 172.21.15.89:0/2087609431 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2775239830 0 0) 0x7f905c0d1e50 con 0x7f905c0b1cc0 2023-10-18T11:03:49.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.255+0000 7f905959a640 1 -- 172.21.15.89:0/2087609431 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9044003530 con 0x7f905c0b1cc0 2023-10-18T11:03:49.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.255+0000 7f9059d9b640 1 -- 172.21.15.89:0/2087609431 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f905c15c550 msgr2=0x7f905c0d2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:49.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.255+0000 7f9059d9b640 1 --2- 172.21.15.89:0/2087609431 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f905c15c550 0x7f905c0d2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:49.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.255+0000 7f9059d9b640 1 -- 172.21.15.89:0/2087609431 >> v1:172.21.15.89:6789/0 conn(0x7f905c0b1cc0 legacy=0x7f905c0b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:49.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.255+0000 7f9059d9b640 1 -- 172.21.15.89:0/2087609431 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f905c0d26f0 con 0x7f905c0b1860 2023-10-18T11:03:49.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.255+0000 7f905a59c640 1 --2- 172.21.15.89:0/2087609431 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f905c15c550 0x7f905c0d2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T11:03:49.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.259+0000 7f9059d9b640 1 --2- 172.21.15.89:0/2087609431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f905c0b1860 0x7f905c0cf610 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f90500097c0 tx=0x7f905001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a9746d659005f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:49.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.259+0000 7f905959a640 1 -- 172.21.15.89:0/2087609431 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90500ad660 con 0x7f905c0b1860 2023-10-18T11:03:49.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.259+0000 7f905959a640 1 -- 172.21.15.89:0/2087609431 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90500ad7c0 con 0x7f905c0b1860 2023-10-18T11:03:49.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.259+0000 7f905959a640 1 -- 172.21.15.89:0/2087609431 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90500adb30 con 0x7f905c0b1860 2023-10-18T11:03:49.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.259+0000 7f9060f03640 1 -- 172.21.15.89:0/2087609431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f905c0b1860 msgr2=0x7f905c0cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:49.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.259+0000 7f9060f03640 1 --2- 172.21.15.89:0/2087609431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f905c0b1860 0x7f905c0cf610 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f90500097c0 tx=0x7f905001c450 comp rx=0 tx=0).stop 2023-10-18T11:03:49.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.259+0000 7f9060f03640 1 -- 172.21.15.89:0/2087609431 shutdown_connections 2023-10-18T11:03:49.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.259+0000 7f9060f03640 1 --2- 172.21.15.89:0/2087609431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f905c0b1860 0x7f905c0cf610 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:49.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.259+0000 7f9060f03640 1 --2- 172.21.15.89:0/2087609431 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f905c15c550 0x7f905c0d2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:49.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.259+0000 7f9060f03640 1 -- 172.21.15.89:0/2087609431 >> 172.21.15.89:0/2087609431 conn(0x7f905c069d00 msgr2=0x7f905c06a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:49.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.259+0000 7f9060f03640 1 -- 172.21.15.89:0/2087609431 shutdown_connections 2023-10-18T11:03:49.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.259+0000 7f9060f03640 1 -- 172.21.15.89:0/2087609431 wait complete. 2023-10-18T11:03:49.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.263+0000 7f9060f03640 1 Processor -- start 2023-10-18T11:03:49.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.263+0000 7f9060f03640 1 -- start start 2023-10-18T11:03:49.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.263+0000 7f9060f03640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f905c0b1860 0x7f905c1579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:49.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.263+0000 7f9060f03640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f905c0b1cc0 0x7f905c0ca390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:49.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.263+0000 7f9060f03640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f905c15c550 0x7f905c0ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:49.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.263+0000 7f9060f03640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905c157ef0 con 0x7f905c15c550 2023-10-18T11:03:49.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.263+0000 7f9060f03640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905c0cc910 con 0x7f905c0b1860 2023-10-18T11:03:49.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.263+0000 7f9060f03640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f905c0ccbf0 con 0x7f905c0b1cc0 2023-10-18T11:03:49.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.263+0000 7f905ad9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f905c15c550 0x7f905c0ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:49.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.263+0000 7f905ad9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f905c15c550 0x7f905c0ca8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55712/0 (socket says 172.21.15.89:55712) 2023-10-18T11:03:49.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.263+0000 7f905ad9d640 1 -- 172.21.15.89:0/321414908 learned_addr learned my addr 172.21.15.89:0/321414908 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:49.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.263+0000 7f9059d9b640 1 --2- 172.21.15.89:0/321414908 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f905c0b1cc0 0x7f905c0ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:49.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.263+0000 7f905a59c640 1 --2- 172.21.15.89:0/321414908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f905c0b1860 0x7f905c1579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:49.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.263+0000 7f905ad9d640 1 -- 172.21.15.89:0/321414908 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f905c0b1cc0 msgr2=0x7f905c0ca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:49.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.263+0000 7f905ad9d640 1 --2- 172.21.15.89:0/321414908 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f905c0b1cc0 0x7f905c0ca390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:49.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.263+0000 7f905ad9d640 1 -- 172.21.15.89:0/321414908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f905c0b1860 msgr2=0x7f905c1579b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:49.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.263+0000 7f905ad9d640 1 --2- 172.21.15.89:0/321414908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f905c0b1860 0x7f905c1579b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:49.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.263+0000 7f905ad9d640 1 -- 172.21.15.89:0/321414908 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f904c009420 con 0x7f905c15c550 2023-10-18T11:03:49.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.263+0000 7f905ad9d640 1 --2- 172.21.15.89:0/321414908 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f905c15c550 0x7f905c0ca8d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f905400cae0 tx=0x7f905400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:49.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.263+0000 7f904affd640 1 -- 172.21.15.89:0/321414908 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905400ac40 con 0x7f905c15c550 2023-10-18T11:03:49.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.263+0000 7f904affd640 1 -- 172.21.15.89:0/321414908 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f905400ada0 con 0x7f905c15c550 2023-10-18T11:03:49.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.263+0000 7f904affd640 1 -- 172.21.15.89:0/321414908 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9054012640 con 0x7f905c15c550 2023-10-18T11:03:49.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.263+0000 7f9060f03640 1 -- 172.21.15.89:0/321414908 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9050009420 con 0x7f905c15c550 2023-10-18T11:03:49.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.263+0000 7f9060f03640 1 -- 172.21.15.89:0/321414908 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f905c0c7060 con 0x7f905c15c550 2023-10-18T11:03:49.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.267+0000 7f9060f03640 1 -- 172.21.15.89:0/321414908 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f905c0c6be0 con 0x7f905c15c550 2023-10-18T11:03:49.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.267+0000 7f904affd640 1 -- 172.21.15.89:0/321414908 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f9054007070 con 0x7f905c15c550 2023-10-18T11:03:49.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.267+0000 7f904affd640 1 --2- 172.21.15.89:0/321414908 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f903c065680 0x7f903c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:49.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.267+0000 7f904affd640 1 -- 172.21.15.89:0/321414908 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f9054082bc0 con 0x7f905c15c550 2023-10-18T11:03:49.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.267+0000 7f905a59c640 1 --2- 172.21.15.89:0/321414908 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f903c065680 0x7f903c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:49.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.267+0000 7f905a59c640 1 --2- 172.21.15.89:0/321414908 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f903c065680 0x7f903c067b40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f904c009c50 tx=0x7f904c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:49.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.271+0000 7f904affd640 1 -- 172.21.15.89:0/321414908 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f905c0c6be0 con 0x7f905c15c550 2023-10-18T11:03:49.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.419+0000 7f9060f03640 1 -- 172.21.15.89:0/321414908 --> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f905c0b6110 con 0x7f903c065680 2023-10-18T11:03:49.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.419+0000 7f904affd640 1 -- 172.21.15.89:0/321414908 <== mgr.4099 v2:172.21.15.154:6832/1086608968 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72748 (secure 0 0 0) 0x7f904c0098a0 con 0x7f903c065680 2023-10-18T11:03:49.421 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T11:03:49.422 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-18T11:03:49.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.423+0000 7f9060f03640 1 -- 172.21.15.89:0/321414908 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f903c065680 msgr2=0x7f903c067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:49.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.423+0000 7f9060f03640 1 --2- 172.21.15.89:0/321414908 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f903c065680 0x7f903c067b40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f904c009c50 tx=0x7f904c009190 comp rx=0 tx=0).stop 2023-10-18T11:03:49.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.423+0000 7f9060f03640 1 -- 172.21.15.89:0/321414908 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f905c15c550 msgr2=0x7f905c0ca8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:49.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.423+0000 7f9060f03640 1 --2- 172.21.15.89:0/321414908 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f905c15c550 0x7f905c0ca8d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f905400cae0 tx=0x7f905400a060 comp rx=0 tx=0).stop 2023-10-18T11:03:49.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.423+0000 7f9060f03640 1 -- 172.21.15.89:0/321414908 shutdown_connections 2023-10-18T11:03:49.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.423+0000 7f9060f03640 1 --2- 172.21.15.89:0/321414908 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f903c065680 0x7f903c067b40 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:49.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.423+0000 7f9060f03640 1 --2- 172.21.15.89:0/321414908 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f905c0b1cc0 0x7f905c0ca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:49.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.423+0000 7f9060f03640 1 --2- 172.21.15.89:0/321414908 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f905c15c550 0x7f905c0ca8d0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:49.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.423+0000 7f9060f03640 1 --2- 172.21.15.89:0/321414908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f905c0b1860 0x7f905c1579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:49.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.423+0000 7f9060f03640 1 -- 172.21.15.89:0/321414908 >> 172.21.15.89:0/321414908 conn(0x7f905c069d00 msgr2=0x7f905c0b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:49.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.423+0000 7f9060f03640 1 -- 172.21.15.89:0/321414908 shutdown_connections 2023-10-18T11:03:49.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.423+0000 7f9060f03640 1 -- 172.21.15.89:0/321414908 wait complete. 2023-10-18T11:03:49.441 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":596,"stamp":"2023-10-18T11:03:49.358068+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":105,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":223404,"kb_used_data":6636,"kb_used_omap":18,"kb_used_meta":216621,"kb_avail":754751316,"statfs":{"total":773094113280,"available":772865347584,"internally_reserved":0,"allocated":6795264,"data_stored":3878893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19166,"internal_metadata":221820194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11768,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-63,"num_write_kb":-35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001715"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":90,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-18T11:03:44.227989+0000","last_change":"2023-10-18T10:55:22.931642+0000","last_active":"2023-10-18T11:03:44.227989+0000","last_peered":"2023-10-18T11:03:44.227989+0000","last_clean":"2023-10-18T11:03:44.227989+0000","last_became_active":"2023-10-18T10:55:22.931232+0000","last_became_peered":"2023-10-18T10:55:22.931232+0000","last_unstale":"2023-10-18T11:03:44.227989+0000","last_undegraded":"2023-10-18T11:03:44.227989+0000","last_fullsized":"2023-10-18T11:03:44.227989+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:49:50.637795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":105,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-18T11:03:44.242643+0000","last_change":"2023-10-18T10:55:22.928478+0000","last_active":"2023-10-18T11:03:44.242643+0000","last_peered":"2023-10-18T11:03:44.242643+0000","last_clean":"2023-10-18T11:03:44.242643+0000","last_became_active":"2023-10-18T10:55:22.928049+0000","last_became_peered":"2023-10-18T10:55:22.928049+0000","last_unstale":"2023-10-18T11:03:44.242643+0000","last_undegraded":"2023-10-18T11:03:44.242643+0000","last_fullsized":"2023-10-18T11:03:44.242643+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:13:57.698492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-18T11:03:44.566074+0000","last_change":"2023-10-18T10:45:50.585224+0000","last_active":"2023-10-18T11:03:44.566074+0000","last_peered":"2023-10-18T11:03:44.566074+0000","last_clean":"2023-10-18T11:03:44.566074+0000","last_became_active":"2023-10-18T10:45:50.584872+0000","last_became_peered":"2023-10-18T10:45:50.584872+0000","last_unstale":"2023-10-18T11:03:44.566074+0000","last_undegraded":"2023-10-18T11:03:44.566074+0000","last_fullsized":"2023-10-18T11:03:44.566074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:21:09.948010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-18T11:03:44.241791+0000","last_change":"2023-10-18T10:45:50.584946+0000","last_active":"2023-10-18T11:03:44.241791+0000","last_peered":"2023-10-18T11:03:44.241791+0000","last_clean":"2023-10-18T11:03:44.241791+0000","last_became_active":"2023-10-18T10:45:50.584638+0000","last_became_peered":"2023-10-18T10:45:50.584638+0000","last_unstale":"2023-10-18T11:03:44.241791+0000","last_undegraded":"2023-10-18T11:03:44.241791+0000","last_fullsized":"2023-10-18T11:03:44.241791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:17:38.852860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":89,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-18T11:03:44.423992+0000","last_change":"2023-10-18T10:45:50.251505+0000","last_active":"2023-10-18T11:03:44.423992+0000","last_peered":"2023-10-18T11:03:44.423992+0000","last_clean":"2023-10-18T11:03:44.423992+0000","last_became_active":"2023-10-18T10:45:50.251111+0000","last_became_peered":"2023-10-18T10:45:50.251111+0000","last_unstale":"2023-10-18T11:03:44.423992+0000","last_undegraded":"2023-10-18T11:03:44.423992+0000","last_fullsized":"2023-10-18T11:03:44.423992+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:19:33.625865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-18T11:03:44.646343+0000","last_change":"2023-10-18T10:45:50.259019+0000","last_active":"2023-10-18T11:03:44.646343+0000","last_peered":"2023-10-18T11:03:44.646343+0000","last_clean":"2023-10-18T11:03:44.646343+0000","last_became_active":"2023-10-18T10:45:50.258607+0000","last_became_peered":"2023-10-18T10:45:50.258607+0000","last_unstale":"2023-10-18T11:03:44.646343+0000","last_undegraded":"2023-10-18T11:03:44.646343+0000","last_fullsized":"2023-10-18T11:03:44.646343+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:31:02.665368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-18T11:03:44.565394+0000","last_change":"2023-10-18T10:45:50.251375+0000","last_active":"2023-10-18T11:03:44.565394+0000","last_peered":"2023-10-18T11:03:44.565394+0000","last_clean":"2023-10-18T11:03:44.565394+0000","last_became_active":"2023-10-18T10:45:50.251055+0000","last_became_peered":"2023-10-18T10:45:50.251055+0000","last_unstale":"2023-10-18T11:03:44.565394+0000","last_undegraded":"2023-10-18T11:03:44.565394+0000","last_fullsized":"2023-10-18T11:03:44.565394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:52:57.411141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":89,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-18T11:03:44.424751+0000","last_change":"2023-10-18T10:45:50.251310+0000","last_active":"2023-10-18T11:03:44.424751+0000","last_peered":"2023-10-18T11:03:44.424751+0000","last_clean":"2023-10-18T11:03:44.424751+0000","last_became_active":"2023-10-18T10:45:50.250823+0000","last_became_peered":"2023-10-18T10:45:50.250823+0000","last_unstale":"2023-10-18T11:03:44.424751+0000","last_undegraded":"2023-10-18T11:03:44.424751+0000","last_fullsized":"2023-10-18T11:03:44.424751+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:55:28.550749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":250,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-18T11:03:44.565461+0000","last_change":"2023-10-18T10:45:44.563531+0000","last_active":"2023-10-18T11:03:44.565461+0000","last_peered":"2023-10-18T11:03:44.565461+0000","last_clean":"2023-10-18T11:03:44.565461+0000","last_became_active":"2023-10-18T10:45:44.562504+0000","last_became_peered":"2023-10-18T10:45:44.562504+0000","last_unstale":"2023-10-18T11:03:44.565461+0000","last_undegraded":"2023-10-18T11:03:44.565461+0000","last_fullsized":"2023-10-18T11:03:44.565461+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:43.181461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:43.181461+0000","last_clean_scrub_stamp":"2023-10-18T10:45:43.181461+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:03:56.712280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575068,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28292,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343548,"statfs":{"total":96636764160,"available":96607793152,"internally_reserved":0,"allocated":1294336,"data_stored":927466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:03:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70399999999999996,"15min":0.70499999999999996},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.40699999999999997},"max":{"1min":0.85599999999999998,"5min":0.93799999999999994,"15min":1.097},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.74399999999999999,"15min":0.72099999999999997},"min":{"1min":0.59599999999999997,"5min":0.434,"15min":0.432},"max":{"1min":0.89300000000000002,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.68700000000000006}]},{"osd":1,"last update":"Wed Oct 18 11:03:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72299999999999998,"15min":0.71699999999999997},"min":{"1min":0.57199999999999995,"5min":0.503,"15min":0.44400000000000001},"max":{"1min":0.88800000000000001,"5min":0.93300000000000005,"15min":0.95599999999999996},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.72499999999999998,"15min":0.73199999999999998},"min":{"1min":0.54600000000000004,"5min":0.52100000000000002,"15min":0.38200000000000001},"max":{"1min":0.94699999999999995,"5min":0.95499999999999996,"15min":0.99199999999999999},"last":0.58099999999999996}]},{"osd":2,"last update":"Wed Oct 18 11:03:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.73999999999999999,"15min":0.73599999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.39700000000000002},"max":{"1min":0.85499999999999998,"5min":0.97699999999999998,"15min":1.0760000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.751,"5min":0.74299999999999999,"15min":0.73699999999999999},"min":{"1min":0.57999999999999996,"5min":0.505,"15min":0.42099999999999999},"max":{"1min":0.95799999999999996,"5min":0.97699999999999998,"15min":1.0800000000000001},"last":0.76100000000000001}]},{"osd":3,"last update":"Wed Oct 18 11:03:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.76600000000000001,"15min":0.76200000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.92400000000000004,"5min":1.004,"15min":1.032},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.79900000000000004,"15min":0.79400000000000004},"min":{"1min":0.66200000000000003,"5min":0.50700000000000001,"15min":0.5},"max":{"1min":0.998,"5min":0.998,"15min":1.1060000000000001},"last":0.876}]},{"osd":4,"last update":"Wed Oct 18 11:03:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58099999999999996,"15min":0.59599999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.34000000000000002},"max":{"1min":0.80600000000000005,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65900000000000003,"15min":0.66800000000000004},"min":{"1min":0.42699999999999999,"5min":0.38300000000000001,"15min":0.35799999999999998},"max":{"1min":0.83799999999999997,"5min":0.92800000000000005,"15min":1.046},"last":0.53000000000000003}]},{"osd":5,"last update":"Wed Oct 18 11:03:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58599999999999997,"15min":0.59699999999999998},"min":{"1min":0.42799999999999999,"5min":0.41399999999999998,"15min":0.375},"max":{"1min":0.79300000000000004,"5min":0.86599999999999999,"15min":0.95099999999999996},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60999999999999999,"15min":0.625},"min":{"1min":0.45900000000000002,"5min":0.44,"15min":0.33100000000000002},"max":{"1min":0.76100000000000001,"5min":0.94799999999999995,"15min":0.95699999999999996},"last":0.85599999999999998}]},{"osd":6,"last update":"Wed Oct 18 11:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70199999999999996,"15min":0.69499999999999995},"min":{"1min":0.53100000000000003,"5min":0.504,"15min":0.44400000000000001},"max":{"1min":0.94699999999999995,"5min":0.99299999999999999,"15min":1.004},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.73899999999999999,"15min":0.753},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.85399999999999998,"5min":1.002,"15min":1.002},"last":0.74199999999999999}]}]},{"osd":6,"up_from":55,"seq":236223201382,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28288,"kb_used_data":688,"kb_used_omap":6,"kb_used_meta":27577,"kb_avail":94343552,"statfs":{"total":96636764160,"available":96607797248,"internally_reserved":0,"allocated":704512,"data_stored":338429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6699,"internal_metadata":28239317},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.754,"15min":0.71499999999999997},"min":{"1min":0.53900000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.96899999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73299999999999998,"15min":0.70199999999999996},"min":{"1min":0.56100000000000005,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.93899999999999995,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.71599999999999997}]},{"osd":1,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.76100000000000001,"15min":0.72499999999999998},"min":{"1min":0.61299999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96999999999999997,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.78200000000000003,"15min":0.76400000000000001},"min":{"1min":0.63900000000000001,"5min":0.54600000000000004,"15min":0.51300000000000001},"max":{"1min":1.0049999999999999,"5min":1.04,"15min":1.04},"last":0.92000000000000004}]},{"osd":2,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.73499999999999999,"15min":0.72499999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.48099999999999998},"max":{"1min":0.90100000000000002,"5min":1.024,"15min":1.024},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.77600000000000002,"15min":0.76300000000000001},"min":{"1min":0.56399999999999995,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.01,"5min":1.077,"15min":1.077},"last":0.94399999999999995}]},{"osd":3,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.75900000000000001,"15min":0.73599999999999999},"min":{"1min":0.58099999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.91100000000000003,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80100000000000005,"15min":0.76900000000000002},"min":{"1min":0.66000000000000003,"5min":0.57899999999999996,"15min":0.495},"max":{"1min":0.98799999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.73299999999999998}]},{"osd":4,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58899999999999997,"15min":0.63600000000000001},"min":{"1min":0.40200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78500000000000003,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.44},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61799999999999999,"15min":0.64500000000000002},"min":{"1min":0.45700000000000002,"5min":0.42999999999999999,"15min":0.40899999999999997},"max":{"1min":0.79600000000000004,"5min":0.91300000000000003,"15min":0.95099999999999996},"last":0.51900000000000002}]},{"osd":5,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64800000000000002,"15min":0.66900000000000004},"min":{"1min":0.48499999999999999,"5min":0.443,"15min":0.436},"max":{"1min":0.84499999999999997,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69499999999999995,"15min":0.72499999999999998},"min":{"1min":0.51000000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.90800000000000003,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.58699999999999997}]},{"osd":7,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71399999999999997,"15min":0.749},"min":{"1min":0.50700000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.89200000000000002,"5min":0.98799999999999999,"15min":1.0189999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73899999999999999,"15min":0.78100000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.871,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.66900000000000004}]}]},{"osd":5,"up_from":13,"seq":55834575068,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":338429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71099999999999997,"15min":0.69699999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.40200000000000002},"max":{"1min":0.89000000000000001,"5min":0.93400000000000005,"15min":0.94499999999999995},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.76700000000000002,"15min":0.746},"min":{"1min":0.61299999999999999,"5min":0.42799999999999999,"15min":0.372},"max":{"1min":1.014,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.63}]},{"osd":1,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.746,"15min":0.74199999999999999},"min":{"1min":0.57799999999999996,"5min":0.502,"15min":0.35599999999999998},"max":{"1min":0.95199999999999996,"5min":1.115,"15min":1.115},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72099999999999997,"15min":0.71399999999999997},"min":{"1min":0.53900000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.89600000000000002,"5min":1.038,"15min":1.038},"last":0.746}]},{"osd":2,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.77800000000000002,"15min":0.752},"min":{"1min":0.57299999999999995,"5min":0.47999999999999998,"15min":0.40799999999999997},"max":{"1min":0.96899999999999997,"5min":1.133,"15min":1.133},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.76300000000000001,"15min":0.76200000000000001},"min":{"1min":0.53100000000000003,"5min":0.52500000000000002,"15min":0.48099999999999998},"max":{"1min":0.98199999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.65900000000000003}]},{"osd":3,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.78500000000000003,"15min":0.78600000000000003},"min":{"1min":0.621,"5min":0.53200000000000003,"15min":0.52900000000000003},"max":{"1min":0.98999999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76300000000000001},"min":{"1min":0.63700000000000001,"5min":0.55600000000000005,"15min":0.44800000000000001},"max":{"1min":0.92600000000000005,"5min":1.083,"15min":1.083},"last":0.86299999999999999}]},{"osd":4,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55500000000000005,"15min":0.54600000000000004},"min":{"1min":0.36199999999999999,"5min":0.35199999999999998,"15min":0.316},"max":{"1min":0.81299999999999994,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.69099999999999995,"15min":0.68200000000000005},"min":{"1min":0.47099999999999997,"5min":0.434,"15min":0.376},"max":{"1min":0.95399999999999996,"5min":1.099,"15min":1.099},"last":1.0529999999999999}]},{"osd":6,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72499999999999998,"15min":0.71999999999999997},"min":{"1min":0.49199999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.875,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":1.1599999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.65100000000000002,"15min":0.65600000000000003},"min":{"1min":0.497,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88200000000000001,"5min":1.004,"15min":1.004},"last":1.1850000000000001}]},{"osd":7,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.753,"15min":0.73299999999999998},"min":{"1min":0.58699999999999997,"5min":0.46600000000000003,"15min":0.41899999999999998},"max":{"1min":0.98299999999999998,"5min":1.018,"15min":1.046},"last":1.097},{"interface":"front","average":{"1min":0.745,"5min":0.71199999999999997,"15min":0.69599999999999995},"min":{"1min":0.51500000000000001,"5min":0.45500000000000002,"15min":0.317},"max":{"1min":0.997,"5min":1.046,"15min":1.046},"last":1.1379999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575068,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27704,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":700416,"data_stored":337098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.70199999999999996,"15min":0.69899999999999995},"min":{"1min":0.496,"5min":0.47999999999999998,"15min":0.39800000000000002},"max":{"1min":0.80900000000000005,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74099999999999999,"15min":0.72099999999999997},"min":{"1min":0.57399999999999995,"5min":0.376,"15min":0.376},"max":{"1min":0.88600000000000001,"5min":0.97299999999999998,"15min":1.018},"last":0.84199999999999997}]},{"osd":1,"last update":"Wed Oct 18 11:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73399999999999999,"15min":0.754},"min":{"1min":0.52800000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.91900000000000004,"5min":0.96299999999999997,"15min":1.0449999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.80300000000000005,"15min":0.78100000000000003},"min":{"1min":0.57599999999999996,"5min":0.52900000000000003,"15min":0.40200000000000002},"max":{"1min":0.995,"5min":1.02,"15min":1.03},"last":0.51200000000000001}]},{"osd":2,"last update":"Wed Oct 18 11:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.746,"15min":0.76300000000000001},"min":{"1min":0.46200000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92900000000000005,"5min":1.022,"15min":1.022},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80900000000000005,"15min":0.79000000000000004},"min":{"1min":0.624,"5min":0.57299999999999995,"15min":0.52500000000000002},"max":{"1min":0.95399999999999996,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.82499999999999996}]},{"osd":3,"last update":"Wed Oct 18 11:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.78000000000000003,"15min":0.76100000000000001},"min":{"1min":0.52800000000000002,"5min":0.48099999999999998,"15min":0.439},"max":{"1min":0.92100000000000004,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.78400000000000003,"15min":0.79800000000000004},"min":{"1min":0.61699999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.97499999999999998,"5min":1.036,"15min":1.0940000000000001},"last":0.68600000000000005}]},{"osd":5,"last update":"Wed Oct 18 11:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.63,"15min":0.626},"min":{"1min":0.38900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82299999999999995,"5min":0.96499999999999997,"15min":1.048},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.66200000000000003,"15min":0.67700000000000005},"min":{"1min":0.39700000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.0620000000000001},"last":0.58399999999999996}]},{"osd":6,"last update":"Wed Oct 18 11:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63100000000000001,"15min":0.65600000000000003},"min":{"1min":0.441,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96199999999999997,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61799999999999999,"15min":0.65500000000000003},"min":{"1min":0.46999999999999997,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94299999999999995,"5min":1.0209999999999999,"15min":1.109},"last":0.65800000000000003}]},{"osd":7,"last update":"Wed Oct 18 11:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.68999999999999995,"15min":0.67400000000000004},"min":{"1min":0.502,"5min":0.45000000000000001,"15min":0.41599999999999998},"max":{"1min":1.0449999999999999,"5min":1.0549999999999999,"15min":1.1080000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.628,"15min":0.66000000000000003},"min":{"1min":0.44800000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.0209999999999999,"5min":1.036,"15min":1.129},"last":0.44400000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575067,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":700416,"data_stored":337119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.46700000000000003,"15min":0.46000000000000002},"min":{"1min":0.29099999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.52500000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.311},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52700000000000002,"15min":0.52000000000000002},"min":{"1min":0.33700000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.65100000000000002,"5min":0.78000000000000003,"15min":0.78900000000000003},"last":0.38100000000000001}]},{"osd":1,"last update":"Wed Oct 18 11:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.48799999999999999,"15min":0.48999999999999999},"min":{"1min":0.37,"5min":0.33200000000000002,"15min":0.316},"max":{"1min":0.59999999999999998,"5min":0.76700000000000002,"15min":0.80000000000000004},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.42699999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.60299999999999998,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.35999999999999999}]},{"osd":2,"last update":"Wed Oct 18 11:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.44900000000000001,"5min":0.376,"15min":0.371},"max":{"1min":0.71099999999999997,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61399999999999999,"15min":0.60999999999999999},"min":{"1min":0.40300000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.86599999999999999,"5min":0.874,"15min":0.874},"last":0.46700000000000003}]},{"osd":4,"last update":"Wed Oct 18 11:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.754,"15min":0.77200000000000002},"min":{"1min":0.45200000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.88600000000000001,"5min":1.111,"15min":1.1919999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.78700000000000003,"15min":0.79900000000000004},"min":{"1min":0.59099999999999997,"5min":0.495,"15min":0.495},"max":{"1min":0.86599999999999999,"5min":1.137,"15min":1.246},"last":0.622}]},{"osd":5,"last update":"Wed Oct 18 11:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.80900000000000005,"15min":0.81799999999999995},"min":{"1min":0.60299999999999998,"5min":0.59099999999999997,"15min":0.55100000000000005},"max":{"1min":0.90800000000000003,"5min":1.119,"15min":1.1859999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.78900000000000003,"15min":0.79000000000000004},"min":{"1min":0.59599999999999997,"5min":0.58499999999999996,"15min":0.49099999999999999},"max":{"1min":0.93600000000000005,"5min":1.137,"15min":1.155},"last":0.65300000000000002}]},{"osd":6,"last update":"Wed Oct 18 11:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.77100000000000002,"15min":0.83099999999999996},"min":{"1min":0.53800000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.97699999999999998,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.81699999999999995,"15min":0.85199999999999998},"min":{"1min":0.65500000000000003,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.96599999999999997,"5min":1.25,"15min":1.25},"last":0.751}]},{"osd":7,"last update":"Wed Oct 18 11:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.78800000000000003,"15min":0.80800000000000005},"min":{"1min":0.61199999999999999,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.98599999999999999,"5min":1.1599999999999999,"15min":1.2529999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.83999999999999997,"15min":0.84599999999999997},"min":{"1min":0.65500000000000003,"5min":0.63100000000000001,"15min":0.62},"max":{"1min":0.93500000000000005,"5min":1.171,"15min":1.2410000000000001},"last":0.71099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575069,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":696320,"data_stored":335788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.49399999999999999,"15min":0.48399999999999999},"min":{"1min":0.35399999999999998,"5min":0.313,"15min":0.27700000000000002},"max":{"1min":0.67300000000000004,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56299999999999994,"15min":0.56399999999999995},"min":{"1min":0.42499999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.73699999999999999,"5min":0.78400000000000003,"15min":0.80400000000000005},"last":0.63600000000000001}]},{"osd":1,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53500000000000003,"15min":0.53800000000000003},"min":{"1min":0.379,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.75,"5min":0.75700000000000001,"15min":0.82999999999999996},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.51600000000000001,"15min":0.51500000000000001},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.34599999999999997},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.83699999999999997},"last":0.59899999999999998}]},{"osd":3,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57999999999999996,"15min":0.58099999999999996},"min":{"1min":0.46400000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79800000000000004,"5min":0.81599999999999995,"15min":0.91400000000000003},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.45900000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.73699999999999999,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.621}]},{"osd":4,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81399999999999995,"15min":0.80600000000000005},"min":{"1min":0.60799999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0700000000000001,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.78200000000000003,"15min":0.77600000000000002},"min":{"1min":0.61799999999999999,"5min":0.42799999999999999,"15min":0.36699999999999999},"max":{"1min":0.98899999999999999,"5min":1.29,"15min":1.29},"last":0.95999999999999996}]},{"osd":5,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.80900000000000005,"15min":0.79300000000000004},"min":{"1min":0.57399999999999995,"5min":0.56999999999999995,"15min":0.51700000000000002},"max":{"1min":0.90500000000000003,"5min":1.264,"15min":1.264},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.63300000000000001,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.99099999999999999,"5min":1.163,"15min":1.3120000000000001},"last":0.879}]},{"osd":6,"last update":"Wed Oct 18 11:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.80700000000000005,"15min":0.80300000000000005},"min":{"1min":0.60699999999999998,"5min":0.48599999999999999,"15min":0.42899999999999999},"max":{"1min":0.92600000000000005,"5min":1.107,"15min":1.1339999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.82299999999999995,"15min":0.81599999999999995},"min":{"1min":0.627,"5min":0.58499999999999996,"15min":0.57699999999999996},"max":{"1min":1.054,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.028}]},{"osd":7,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81200000000000006,"15min":0.81999999999999995},"min":{"1min":0.65100000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0660000000000001,"5min":1.1299999999999999,"15min":1.2609999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.81999999999999995,"15min":0.80200000000000005},"min":{"1min":0.60099999999999998,"5min":0.58499999999999996,"15min":0.47499999999999998},"max":{"1min":1.026,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":1.05}]}]},{"osd":1,"up_from":13,"seq":55834575068,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":700416,"data_stored":337098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.443,"15min":0.45300000000000001},"min":{"1min":0.311,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.63800000000000001,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.502},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.495,"15min":0.51000000000000001},"min":{"1min":0.34899999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.752,"5min":0.752,"15min":0.92500000000000004},"last":0.47499999999999998}]},{"osd":2,"last update":"Wed Oct 18 11:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46500000000000002,"15min":0.47999999999999998},"min":{"1min":0.39500000000000002,"5min":0.32600000000000001,"15min":0.32200000000000001},"max":{"1min":0.64700000000000002,"5min":0.74199999999999999,"15min":0.85599999999999998},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.52800000000000002,"15min":0.54900000000000004},"min":{"1min":0.40799999999999997,"5min":0.314,"15min":0.314},"max":{"1min":0.753,"5min":0.83499999999999996,"15min":0.97199999999999998},"last":0.53600000000000003}]},{"osd":3,"last update":"Wed Oct 18 11:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.54600000000000004,"15min":0.56599999999999995},"min":{"1min":0.44500000000000001,"5min":0.40899999999999997,"15min":0.377},"max":{"1min":0.71099999999999997,"5min":0.82899999999999996,"15min":0.90800000000000003},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.57399999999999995,"15min":0.58999999999999997},"min":{"1min":0.48299999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80500000000000005,"5min":0.91500000000000004,"15min":0.93899999999999995},"last":0.55900000000000005}]},{"osd":4,"last update":"Wed Oct 18 11:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.80600000000000005,"15min":0.79400000000000004},"min":{"1min":0.68000000000000005,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0700000000000001,"5min":1.389,"15min":1.389},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.75900000000000001,"15min":0.77500000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.50900000000000001},"max":{"1min":1.145,"5min":1.349,"15min":1.349},"last":0.79500000000000004}]},{"osd":5,"last update":"Wed Oct 18 11:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.78900000000000003,"15min":0.79900000000000004},"min":{"1min":0.65600000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.998,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.68500000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1830000000000001},"last":0.74199999999999999}]},{"osd":6,"last update":"Wed Oct 18 11:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.77700000000000002,"15min":0.83799999999999997},"min":{"1min":0.63900000000000001,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.99199999999999999,"5min":1.1339999999999999,"15min":1.157},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.81599999999999995,"15min":0.84599999999999997},"min":{"1min":0.68500000000000005,"5min":0.60699999999999998,"15min":0.56999999999999995},"max":{"1min":1.0329999999999999,"5min":1.1060000000000001,"15min":1.278},"last":0.80500000000000005}]},{"osd":7,"last update":"Wed Oct 18 11:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.81100000000000005,"15min":0.81899999999999995},"min":{"1min":0.63600000000000001,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0549999999999999,"5min":1.1899999999999999,"15min":1.2410000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.81200000000000006,"15min":0.82099999999999995},"min":{"1min":0.70599999999999996,"5min":0.58999999999999997,"15min":0.57199999999999995},"max":{"1min":1.127,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.75800000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575068,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28292,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343548,"statfs":{"total":96636764160,"available":96607793152,"internally_reserved":0,"allocated":1294336,"data_stored":927466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 11:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.45200000000000001,"15min":0.45100000000000001},"min":{"1min":0.34200000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.63800000000000001,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.504,"15min":0.51100000000000001},"min":{"1min":0.33500000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.753,"5min":0.753,"15min":0.86899999999999999},"last":0.40200000000000002}]},{"osd":2,"last update":"Wed Oct 18 11:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.50800000000000001,"15min":0.51200000000000001},"min":{"1min":0.378,"5min":0.309,"15min":0.309},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.80500000000000005},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54900000000000004,"15min":0.55100000000000005},"min":{"1min":0.42699999999999999,"5min":0.40100000000000002,"15min":0.38100000000000001},"max":{"1min":0.68100000000000005,"5min":0.80400000000000005,"15min":0.86699999999999999},"last":0.48199999999999998}]},{"osd":3,"last update":"Wed Oct 18 11:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.71199999999999997,"5min":0.77400000000000002,"15min":0.81899999999999995},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.45000000000000001,"5min":0.436,"15min":0.40100000000000002},"max":{"1min":0.75,"5min":0.75,"15min":0.85599999999999998},"last":0.73599999999999999}]},{"osd":4,"last update":"Wed Oct 18 11:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.81200000000000006,"15min":0.80600000000000005},"min":{"1min":0.58499999999999996,"5min":0.53000000000000003,"15min":0.48999999999999999},"max":{"1min":1.0649999999999999,"5min":1.2390000000000001,"15min":1.2470000000000001},"last":0.751},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.81399999999999995,"15min":0.80600000000000005},"min":{"1min":0.52900000000000003,"5min":0.50900000000000001,"15min":0.45700000000000002},"max":{"1min":1.0549999999999999,"5min":1.389,"15min":1.389},"last":0.71099999999999997}]},{"osd":5,"last update":"Wed Oct 18 11:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81799999999999995,"15min":0.80700000000000005},"min":{"1min":0.61299999999999999,"5min":0.56799999999999995,"15min":0.53200000000000003},"max":{"1min":1.0920000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.82599999999999996,"15min":0.82099999999999995},"min":{"1min":0.65500000000000003,"5min":0.56299999999999994,"15min":0.50600000000000001},"max":{"1min":1.0349999999999999,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.72299999999999998}]},{"osd":6,"last update":"Wed Oct 18 11:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.81599999999999995,"15min":0.82199999999999995},"min":{"1min":0.56399999999999995,"5min":0.54900000000000004,"15min":0.54800000000000004},"max":{"1min":0.95099999999999996,"5min":1.24,"15min":1.2629999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.58499999999999996},"max":{"1min":1.008,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.81799999999999995}]},{"osd":7,"last update":"Wed Oct 18 11:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.84499999999999997,"15min":0.83799999999999997},"min":{"1min":0.65500000000000003,"5min":0.59699999999999998,"15min":0.56899999999999995},"max":{"1min":1.1259999999999999,"5min":1.409,"15min":1.409},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.82199999999999995,"15min":0.81200000000000006},"min":{"1min":0.60099999999999998,"5min":0.53700000000000003,"15min":0.503},"max":{"1min":0.98999999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.76800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T11:03:49.444 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T11:03:49.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.579+0000 7ff9e72cb640 1 Processor -- start 2023-10-18T11:03:49.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.583+0000 7ff9e72cb640 1 -- start start 2023-10-18T11:03:49.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.583+0000 7ff9e72cb640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff9e0159490 0x7ff9e015d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:49.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.583+0000 7ff9e72cb640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff9e0070600 0x7ff9e006b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:49.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.583+0000 7ff9e72cb640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9e015ddf0 con 0x7ff9e006bd10 2023-10-18T11:03:49.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.583+0000 7ff9e72cb640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9e006c1a0 con 0x7ff9e0159490 2023-10-18T11:03:49.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.583+0000 7ff9e72cb640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9e0070b90 con 0x7ff9e0070600 2023-10-18T11:03:49.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.583+0000 7ff9e5868640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff9e006bd10 0x7ff9e00cbee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48160/0 (socket says 172.21.15.89:48160) 2023-10-18T11:03:49.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.583+0000 7ff9e5868640 1 -- 172.21.15.89:0/3977279131 learned_addr learned my addr 172.21.15.89:0/3977279131 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:49.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.583+0000 7ff9e5067640 1 --2- 172.21.15.89:0/3977279131 >> v2:172.21.15.89:3300/0 conn(0x7ff9e0159490 0x7ff9e015d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:49.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.583+0000 7ff9e4866640 1 --2- 172.21.15.89:0/3977279131 >> v2:172.21.15.154:3300/0 conn(0x7ff9e0070600 0x7ff9e006b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:49.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.583+0000 7ff9d7fff640 1 -- 172.21.15.89:0/3977279131 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3189073988 0 0) 0x7ff9e015ddf0 con 0x7ff9e006bd10 2023-10-18T11:03:49.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.583+0000 7ff9d7fff640 1 -- 172.21.15.89:0/3977279131 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9cc003530 con 0x7ff9e006bd10 2023-10-18T11:03:49.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.583+0000 7ff9e5067640 1 -- 172.21.15.89:0/3977279131 >> v2:172.21.15.154:3300/0 conn(0x7ff9e0070600 msgr2=0x7ff9e006b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:49.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.583+0000 7ff9e5067640 1 --2- 172.21.15.89:0/3977279131 >> v2:172.21.15.154:3300/0 conn(0x7ff9e0070600 0x7ff9e006b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:49.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.583+0000 7ff9e5067640 1 -- 172.21.15.89:0/3977279131 >> v1:172.21.15.89:6789/0 conn(0x7ff9e006bd10 legacy=0x7ff9e00cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:49.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.583+0000 7ff9e5067640 1 -- 172.21.15.89:0/3977279131 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e0070e70 con 0x7ff9e0159490 2023-10-18T11:03:49.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.587+0000 7ff9e5067640 1 --2- 172.21.15.89:0/3977279131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9e0159490 0x7ff9e015d8b0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ff9e006dd60 tx=0x7ff9dc01c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7692ad65554eed4 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:49.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.587+0000 7ff9d7fff640 1 -- 172.21.15.89:0/3977279131 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc0ac600 con 0x7ff9e0159490 2023-10-18T11:03:49.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.587+0000 7ff9d7fff640 1 -- 172.21.15.89:0/3977279131 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9dc0a8050 con 0x7ff9e0159490 2023-10-18T11:03:49.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.587+0000 7ff9d7fff640 1 -- 172.21.15.89:0/3977279131 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc0ac980 con 0x7ff9e0159490 2023-10-18T11:03:49.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.587+0000 7ff9e72cb640 1 -- 172.21.15.89:0/3977279131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9e0159490 msgr2=0x7ff9e015d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:49.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.587+0000 7ff9e72cb640 1 --2- 172.21.15.89:0/3977279131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9e0159490 0x7ff9e015d8b0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ff9e006dd60 tx=0x7ff9dc01c3a0 comp rx=0 tx=0).stop 2023-10-18T11:03:49.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.587+0000 7ff9e72cb640 1 -- 172.21.15.89:0/3977279131 shutdown_connections 2023-10-18T11:03:49.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.587+0000 7ff9e72cb640 1 --2- 172.21.15.89:0/3977279131 >> v2:172.21.15.154:3300/0 conn(0x7ff9e0070600 0x7ff9e006b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:49.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.587+0000 7ff9e72cb640 1 --2- 172.21.15.89:0/3977279131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9e0159490 0x7ff9e015d8b0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:49.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.587+0000 7ff9e72cb640 1 -- 172.21.15.89:0/3977279131 >> 172.21.15.89:0/3977279131 conn(0x7ff9e0069f00 msgr2=0x7ff9e00cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:49.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.587+0000 7ff9e72cb640 1 -- 172.21.15.89:0/3977279131 shutdown_connections 2023-10-18T11:03:49.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.587+0000 7ff9e72cb640 1 -- 172.21.15.89:0/3977279131 wait complete. 2023-10-18T11:03:49.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.587+0000 7ff9e72cb640 1 Processor -- start 2023-10-18T11:03:49.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.587+0000 7ff9e72cb640 1 -- start start 2023-10-18T11:03:49.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.591+0000 7ff9e72cb640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9e006bd10 0x7ff9e00c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:49.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.591+0000 7ff9e72cb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9e0070600 0x7ff9e00ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:49.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.591+0000 7ff9e72cb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9e0159490 0x7ff9e0159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:49.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.591+0000 7ff9e72cb640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e00c2070 con 0x7ff9e006bd10 2023-10-18T11:03:49.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.591+0000 7ff9e72cb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e015c0e0 con 0x7ff9e0070600 2023-10-18T11:03:49.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.591+0000 7ff9e72cb640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e015c3c0 con 0x7ff9e0159490 2023-10-18T11:03:49.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.591+0000 7ff9e5067640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9e006bd10 0x7ff9e00c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:49.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.591+0000 7ff9e5067640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9e006bd10 0x7ff9e00c4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55740/0 (socket says 172.21.15.89:55740) 2023-10-18T11:03:49.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.591+0000 7ff9e5067640 1 -- 172.21.15.89:0/4138583728 learned_addr learned my addr 172.21.15.89:0/4138583728 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:49.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.591+0000 7ff9e5067640 1 -- 172.21.15.89:0/4138583728 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9e0159490 msgr2=0x7ff9e0159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:49.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.591+0000 7ff9e5067640 1 --2- 172.21.15.89:0/4138583728 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9e0159490 0x7ff9e0159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:49.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.591+0000 7ff9e5067640 1 -- 172.21.15.89:0/4138583728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9e0070600 msgr2=0x7ff9e00ae930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:49.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.591+0000 7ff9e5067640 1 --2- 172.21.15.89:0/4138583728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9e0070600 0x7ff9e00ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:49.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.591+0000 7ff9e5067640 1 -- 172.21.15.89:0/4138583728 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9dc009420 con 0x7ff9e006bd10 2023-10-18T11:03:49.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.591+0000 7ff9e5067640 1 --2- 172.21.15.89:0/4138583728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9e006bd10 0x7ff9e00c4710 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc018fb0 tx=0x7ff9dc01abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:49.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.591+0000 7ff9d5ffb640 1 -- 172.21.15.89:0/4138583728 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc0b7450 con 0x7ff9e006bd10 2023-10-18T11:03:49.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.591+0000 7ff9d5ffb640 1 -- 172.21.15.89:0/4138583728 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9dc0b75b0 con 0x7ff9e006bd10 2023-10-18T11:03:49.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.591+0000 7ff9d5ffb640 1 -- 172.21.15.89:0/4138583728 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc0b7920 con 0x7ff9e006bd10 2023-10-18T11:03:49.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.591+0000 7ff9e72cb640 1 -- 172.21.15.89:0/4138583728 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9e015c6a0 con 0x7ff9e006bd10 2023-10-18T11:03:49.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.591+0000 7ff9e72cb640 1 -- 172.21.15.89:0/4138583728 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e00aee70 con 0x7ff9e006bd10 2023-10-18T11:03:49.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.591+0000 7ff9e72cb640 1 -- 172.21.15.89:0/4138583728 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9e004dbe0 con 0x7ff9e006bd10 2023-10-18T11:03:49.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.591+0000 7ff9d5ffb640 1 -- 172.21.15.89:0/4138583728 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7ff9dc0b3070 con 0x7ff9e006bd10 2023-10-18T11:03:49.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.595+0000 7ff9d5ffb640 1 --2- 172.21.15.89:0/4138583728 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff9c4065680 0x7ff9c4067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:49.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.595+0000 7ff9d5ffb640 1 -- 172.21.15.89:0/4138583728 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5659+0+0 (secure 0 0 0) 0x7ff9dc11d210 con 0x7ff9e006bd10 2023-10-18T11:03:49.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.595+0000 7ff9e4866640 1 --2- 172.21.15.89:0/4138583728 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff9c4065680 0x7ff9c4067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:49.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.595+0000 7ff9e4866640 1 --2- 172.21.15.89:0/4138583728 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff9c4065680 0x7ff9c4067b40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0006aa0 tx=0x7ff9d0006a30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:49.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.599+0000 7ff9d5ffb640 1 -- 172.21.15.89:0/4138583728 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7ff9dc0eae50 con 0x7ff9e006bd10 2023-10-18T11:03:49.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.747+0000 7ff9e72cb640 1 -- 172.21.15.89:0/4138583728 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9e00c1dc0 con 0x7ff9e006bd10 2023-10-18T11:03:49.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.747+0000 7ff9d5ffb640 1 -- 172.21.15.89:0/4138583728 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+15049 (secure 0 0 0) 0x7ff9dc0ee470 con 0x7ff9e006bd10 2023-10-18T11:03:49.747 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T11:03:49.748 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":59,"fsid":"1352f62a-2330-4e96-afcd-b8e87cd82667","created":"2023-10-18T10:45:29.952211+0000","modified":"2023-10-18T11:03:44.210864+0000","last_up_change":"2023-10-18T10:55:21.905065+0000","last_in_change":"2023-10-18T10:45:32.538820+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:45:43.114184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T10:45:48.730968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"842c9db2-b707-471b-a554-c3d6314ac38b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6801","nonce":308765932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6803","nonce":308765932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6807","nonce":308765932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":308765932},{"type":"v1","addr":"172.21.15.89:6805","nonce":308765932}]},"public_addr":"172.21.15.89:6801/308765932","cluster_addr":"172.21.15.89:6803/308765932","heartbeat_back_addr":"172.21.15.89:6807/308765932","heartbeat_front_addr":"172.21.15.89:6805/308765932","state":["exists","up"]},{"osd":1,"uuid":"0835cc88-e19b-4a4e-8857-d30c7e6804ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6817","nonce":2220030573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6819","nonce":2220030573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6823","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6825","nonce":2220030573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2220030573},{"type":"v1","addr":"172.21.15.89:6821","nonce":2220030573}]},"public_addr":"172.21.15.89:6817/2220030573","cluster_addr":"172.21.15.89:6819/2220030573","heartbeat_back_addr":"172.21.15.89:6825/2220030573","heartbeat_front_addr":"172.21.15.89:6821/2220030573","state":["exists","up"]},{"osd":2,"uuid":"95eb0a15-8b08-4e57-8b37-34096114f81f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6809","nonce":4229386532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6811","nonce":4229386532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6815","nonce":4229386532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":4229386532},{"type":"v1","addr":"172.21.15.89:6813","nonce":4229386532}]},"public_addr":"172.21.15.89:6809/4229386532","cluster_addr":"172.21.15.89:6811/4229386532","heartbeat_back_addr":"172.21.15.89:6815/4229386532","heartbeat_front_addr":"172.21.15.89:6813/4229386532","state":["exists","up"]},{"osd":3,"uuid":"40cd481e-d2af-4210-8d4a-add39171432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6824","nonce":2771730391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6827","nonce":2771730391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6831","nonce":2771730391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2771730391},{"type":"v1","addr":"172.21.15.89:6829","nonce":2771730391}]},"public_addr":"172.21.15.89:6824/2771730391","cluster_addr":"172.21.15.89:6827/2771730391","heartbeat_back_addr":"172.21.15.89:6831/2771730391","heartbeat_front_addr":"172.21.15.89:6829/2771730391","state":["exists","up"]},{"osd":4,"uuid":"2fa04448-d520-4f8c-a9b0-684f526019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6801","nonce":2025560293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6803","nonce":2025560293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6807","nonce":2025560293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2025560293},{"type":"v1","addr":"172.21.15.154:6805","nonce":2025560293}]},"public_addr":"172.21.15.154:6801/2025560293","cluster_addr":"172.21.15.154:6803/2025560293","heartbeat_back_addr":"172.21.15.154:6807/2025560293","heartbeat_front_addr":"172.21.15.154:6805/2025560293","state":["exists","up"]},{"osd":5,"uuid":"93b639fd-01bb-4821-a426-7460cc1abb66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6809","nonce":357281174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6811","nonce":357281174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6817","nonce":357281174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":357281174},{"type":"v1","addr":"172.21.15.154:6813","nonce":357281174}]},"public_addr":"172.21.15.154:6809/357281174","cluster_addr":"172.21.15.154:6811/357281174","heartbeat_back_addr":"172.21.15.154:6817/357281174","heartbeat_front_addr":"172.21.15.154:6813/357281174","state":["exists","up"]},{"osd":6,"uuid":"81f3fc7f-239c-47de-a40b-34bf19289823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":50,"up_from":55,"up_thru":55,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":1133012599},{"type":"v1","addr":"172.21.15.154:6824","nonce":1133012599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1133012599},{"type":"v1","addr":"172.21.15.154:6827","nonce":1133012599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1133012599},{"type":"v1","addr":"172.21.15.154:6831","nonce":1133012599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1133012599},{"type":"v1","addr":"172.21.15.154:6829","nonce":1133012599}]},"public_addr":"172.21.15.154:6824/1133012599","cluster_addr":"172.21.15.154:6827/1133012599","heartbeat_back_addr":"172.21.15.154:6831/1133012599","heartbeat_front_addr":"172.21.15.154:6829/1133012599","state":["exists","up"]},{"osd":7,"uuid":"ceb2d886-a2ee-4974-9079-948986ac3778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6816","nonce":4139186586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6819","nonce":4139186586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6825","nonce":4139186586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":4139186586},{"type":"v1","addr":"172.21.15.154:6821","nonce":4139186586}]},"public_addr":"172.21.15.154:6816/4139186586","cluster_addr":"172.21.15.154:6819/4139186586","heartbeat_back_addr":"172.21.15.154:6825/4139186586","heartbeat_front_addr":"172.21.15.154:6821/4139186586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.062314+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.258072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.039051+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.200217+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.038423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.243654+0000","dead_epoch":0},{"osd":6,"down_stamp":"2023-10-18T10:47:48.955023+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.251482+0000","dead_epoch":51},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:45:38.297937+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6835/2092274731":"2023-10-19T11:03:41.070653+0000","172.21.15.89:6834/2092274731":"2023-10-19T11:03:41.070653+0000","172.21.15.89:0/330049992":"2023-10-18T11:55:17.981058+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T11:03:49.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.751+0000 7ff9e72cb640 1 -- 172.21.15.89:0/4138583728 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff9c4065680 msgr2=0x7ff9c4067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:49.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.751+0000 7ff9e72cb640 1 --2- 172.21.15.89:0/4138583728 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff9c4065680 0x7ff9c4067b40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0006aa0 tx=0x7ff9d0006a30 comp rx=0 tx=0).stop 2023-10-18T11:03:49.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.751+0000 7ff9e72cb640 1 -- 172.21.15.89:0/4138583728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9e006bd10 msgr2=0x7ff9e00c4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:49.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.751+0000 7ff9e72cb640 1 --2- 172.21.15.89:0/4138583728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9e006bd10 0x7ff9e00c4710 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc018fb0 tx=0x7ff9dc01abe0 comp rx=0 tx=0).stop 2023-10-18T11:03:49.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.751+0000 7ff9e72cb640 1 -- 172.21.15.89:0/4138583728 shutdown_connections 2023-10-18T11:03:49.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.751+0000 7ff9e72cb640 1 --2- 172.21.15.89:0/4138583728 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff9c4065680 0x7ff9c4067b40 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:49.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.751+0000 7ff9e72cb640 1 --2- 172.21.15.89:0/4138583728 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9e0159490 0x7ff9e0159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:49.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.751+0000 7ff9e72cb640 1 --2- 172.21.15.89:0/4138583728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9e0070600 0x7ff9e00ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:49.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.751+0000 7ff9e72cb640 1 --2- 172.21.15.89:0/4138583728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9e006bd10 0x7ff9e00c4710 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:49.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.751+0000 7ff9e72cb640 1 -- 172.21.15.89:0/4138583728 >> 172.21.15.89:0/4138583728 conn(0x7ff9e0069f00 msgr2=0x7ff9e0157880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:49.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.751+0000 7ff9e72cb640 1 -- 172.21.15.89:0/4138583728 shutdown_connections 2023-10-18T11:03:49.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:49.751+0000 7ff9e72cb640 1 -- 172.21.15.89:0/4138583728 wait complete. 2023-10-18T11:03:50.769 INFO:tasks.ceph:Scrubbing osd.0 2023-10-18T11:03:50.770 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-18T11:03:50.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.915+0000 7fcd67b67640 1 Processor -- start 2023-10-18T11:03:50.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.915+0000 7fcd67b67640 1 -- start start 2023-10-18T11:03:50.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.915+0000 7fcd67b67640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcd60164b80 0x7fcd60161ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:50.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.915+0000 7fcd67b67640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd60161100 con 0x7fcd600afd40 2023-10-18T11:03:50.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.915+0000 7fcd67b67640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd600c9060 con 0x7fcd600c8910 2023-10-18T11:03:50.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.915+0000 7fcd67b67640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd601617a0 con 0x7fcd60164b80 2023-10-18T11:03:50.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.915+0000 7fcd65903640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcd60164b80 0x7fcd60161ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:50.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.915+0000 7fcd66104640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fcd600afd40 0x7fcd600cbe90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48172/0 (socket says 172.21.15.89:48172) 2023-10-18T11:03:50.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.915+0000 7fcd66104640 1 -- 172.21.15.89:0/53633528 learned_addr learned my addr 172.21.15.89:0/53633528 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:50.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.915+0000 7fcd65903640 1 --2- 172.21.15.89:0/53633528 >> v2:172.21.15.89:3300/0 conn(0x7fcd60164b80 0x7fcd60161ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55756/0 (socket says 172.21.15.89:55756) 2023-10-18T11:03:50.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.915+0000 7fcd65903640 1 -- 172.21.15.89:0/53633528 >> v1:172.21.15.154:6789/0 conn(0x7fcd600c8910 legacy=0x7fcd60164f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:50.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.915+0000 7fcd65903640 1 -- 172.21.15.89:0/53633528 >> v1:172.21.15.89:6789/0 conn(0x7fcd600afd40 legacy=0x7fcd600cbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:50.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.915+0000 7fcd65903640 1 -- 172.21.15.89:0/53633528 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd600c9340 con 0x7fcd60164b80 2023-10-18T11:03:50.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.915+0000 7fcd64901640 1 -- 172.21.15.89:0/53633528 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 418106178 0 0) 0x7fcd60161100 con 0x7fcd600afd40 2023-10-18T11:03:50.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.919+0000 7fcd65903640 1 --2- 172.21.15.89:0/53633528 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd60164b80 0x7fcd60161ba0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c0097c0 tx=0x7fcd5c01c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f51d5edce20c5072 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:50.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.919+0000 7fcd64901640 1 -- 172.21.15.89:0/53633528 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c0ac630 con 0x7fcd60164b80 2023-10-18T11:03:50.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.919+0000 7fcd64901640 1 -- 172.21.15.89:0/53633528 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd5c0ac790 con 0x7fcd60164b80 2023-10-18T11:03:50.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.919+0000 7fcd64901640 1 -- 172.21.15.89:0/53633528 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c0acb20 con 0x7fcd60164b80 2023-10-18T11:03:50.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.919+0000 7fcd67b67640 1 -- 172.21.15.89:0/53633528 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd60164b80 msgr2=0x7fcd60161ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:50.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.919+0000 7fcd67b67640 1 --2- 172.21.15.89:0/53633528 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd60164b80 0x7fcd60161ba0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c0097c0 tx=0x7fcd5c01c3b0 comp rx=0 tx=0).stop 2023-10-18T11:03:50.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.919+0000 7fcd67b67640 1 -- 172.21.15.89:0/53633528 shutdown_connections 2023-10-18T11:03:50.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.919+0000 7fcd67b67640 1 --2- 172.21.15.89:0/53633528 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd60164b80 0x7fcd60161ba0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:50.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.919+0000 7fcd67b67640 1 -- 172.21.15.89:0/53633528 >> 172.21.15.89:0/53633528 conn(0x7fcd600cad40 msgr2=0x7fcd600cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:50.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.919+0000 7fcd67b67640 1 -- 172.21.15.89:0/53633528 shutdown_connections 2023-10-18T11:03:50.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.919+0000 7fcd67b67640 1 -- 172.21.15.89:0/53633528 wait complete. 2023-10-18T11:03:50.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.923+0000 7fcd67b67640 1 Processor -- start 2023-10-18T11:03:50.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.923+0000 7fcd67b67640 1 -- start start 2023-10-18T11:03:50.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.923+0000 7fcd67b67640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd600afd40 0x7fcd600c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:50.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.923+0000 7fcd67b67640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd600c8910 0x7fcd6014fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:50.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.923+0000 7fcd67b67640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd60164b80 0x7fcd6013ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:50.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.923+0000 7fcd67b67640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd60150100 con 0x7fcd60164b80 2023-10-18T11:03:50.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.923+0000 7fcd67b67640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd600c8040 con 0x7fcd600c8910 2023-10-18T11:03:50.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.923+0000 7fcd67b67640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd600c8320 con 0x7fcd600afd40 2023-10-18T11:03:50.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.923+0000 7fcd66104640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd60164b80 0x7fcd6013ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:50.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.923+0000 7fcd66104640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd60164b80 0x7fcd6013ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55770/0 (socket says 172.21.15.89:55770) 2023-10-18T11:03:50.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.923+0000 7fcd66104640 1 -- 172.21.15.89:0/1393379992 learned_addr learned my addr 172.21.15.89:0/1393379992 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:50.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.923+0000 7fcd65903640 1 --2- 172.21.15.89:0/1393379992 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd600afd40 0x7fcd600c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:50.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.923+0000 7fcd65102640 1 --2- 172.21.15.89:0/1393379992 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd600c8910 0x7fcd6014fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:50.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.923+0000 7fcd66104640 1 -- 172.21.15.89:0/1393379992 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd600afd40 msgr2=0x7fcd600c5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:50.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.923+0000 7fcd66104640 1 --2- 172.21.15.89:0/1393379992 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd600afd40 0x7fcd600c5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:50.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.923+0000 7fcd66104640 1 -- 172.21.15.89:0/1393379992 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd600c8910 msgr2=0x7fcd6014fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:50.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.923+0000 7fcd66104640 1 --2- 172.21.15.89:0/1393379992 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd600c8910 0x7fcd6014fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:50.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.923+0000 7fcd66104640 1 -- 172.21.15.89:0/1393379992 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd5c009420 con 0x7fcd60164b80 2023-10-18T11:03:50.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.923+0000 7fcd65102640 1 --2- 172.21.15.89:0/1393379992 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd600c8910 0x7fcd6014fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:03:50.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.923+0000 7fcd66104640 1 --2- 172.21.15.89:0/1393379992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd60164b80 0x7fcd6013ead0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fcd5800c700 tx=0x7fcd5800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:50.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.923+0000 7fcd567fc640 1 -- 172.21.15.89:0/1393379992 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5800ce10 con 0x7fcd60164b80 2023-10-18T11:03:50.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.923+0000 7fcd567fc640 1 -- 172.21.15.89:0/1393379992 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd5800a640 con 0x7fcd60164b80 2023-10-18T11:03:50.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.923+0000 7fcd67b67640 1 -- 172.21.15.89:0/1393379992 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd600c8600 con 0x7fcd60164b80 2023-10-18T11:03:50.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.923+0000 7fcd567fc640 1 -- 172.21.15.89:0/1393379992 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5800aa70 con 0x7fcd60164b80 2023-10-18T11:03:50.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.923+0000 7fcd67b67640 1 -- 172.21.15.89:0/1393379992 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd6013e5b0 con 0x7fcd60164b80 2023-10-18T11:03:50.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.923+0000 7fcd67b67640 1 -- 172.21.15.89:0/1393379992 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcd600b0580 con 0x7fcd60164b80 2023-10-18T11:03:50.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.927+0000 7fcd567fc640 1 -- 172.21.15.89:0/1393379992 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fcd5800abd0 con 0x7fcd60164b80 2023-10-18T11:03:50.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.927+0000 7fcd567fc640 1 --2- 172.21.15.89:0/1393379992 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fcd44065680 0x7fcd44067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:50.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.927+0000 7fcd567fc640 1 -- 172.21.15.89:0/1393379992 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fcd580824e0 con 0x7fcd60164b80 2023-10-18T11:03:50.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.927+0000 7fcd567fc640 1 --2- 172.21.15.89:0/1393379992 >> [v2:172.21.15.89:6800/308765932,v1:172.21.15.89:6801/308765932] conn(0x7fcd4406bff0 0x7fcd4406e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:50.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.927+0000 7fcd567fc640 1 -- 172.21.15.89:0/1393379992 --> [v2:172.21.15.89:6800/308765932,v1:172.21.15.89:6801/308765932] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcd4406eb00 con 0x7fcd4406bff0 2023-10-18T11:03:50.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.927+0000 7fcd567fc640 1 -- 172.21.15.89:0/1393379992 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd58084ac0 con 0x7fcd60164b80 2023-10-18T11:03:50.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.927+0000 7fcd65903640 1 --2- 172.21.15.89:0/1393379992 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fcd44065680 0x7fcd44067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:50.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.927+0000 7fcd65102640 1 --2- 172.21.15.89:0/1393379992 >> [v2:172.21.15.89:6800/308765932,v1:172.21.15.89:6801/308765932] conn(0x7fcd4406bff0 0x7fcd4406e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:50.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.927+0000 7fcd65102640 1 --2- 172.21.15.89:0/1393379992 >> [v2:172.21.15.89:6800/308765932,v1:172.21.15.89:6801/308765932] conn(0x7fcd4406bff0 0x7fcd4406e430 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:50.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.927+0000 7fcd65903640 1 --2- 172.21.15.89:0/1393379992 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fcd44065680 0x7fcd44067b40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c0ae000 tx=0x7fcd5c0a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:50.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.927+0000 7fcd567fc640 1 -- 172.21.15.89:0/1393379992 <== osd.0 v2:172.21.15.89:6800/308765932 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fcd4406eb00 con 0x7fcd4406bff0 2023-10-18T11:03:50.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.939+0000 7fcd67b67640 1 -- 172.21.15.89:0/1393379992 --> [v2:172.21.15.89:6800/308765932,v1:172.21.15.89:6801/308765932] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fcd600c8490 con 0x7fcd4406bff0 2023-10-18T11:03:50.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.939+0000 7fcd567fc640 1 -- 172.21.15.89:0/1393379992 <== osd.0 v2:172.21.15.89:6800/308765932 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fcd600c8490 con 0x7fcd4406bff0 2023-10-18T11:03:50.941 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-18T11:03:50.941 INFO:teuthology.orchestra.run.smithi089.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-18T11:03:50.941 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-18T11:03:50.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.943+0000 7fcd67b67640 1 -- 172.21.15.89:0/1393379992 >> [v2:172.21.15.89:6800/308765932,v1:172.21.15.89:6801/308765932] conn(0x7fcd4406bff0 msgr2=0x7fcd4406e430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:50.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.943+0000 7fcd67b67640 1 --2- 172.21.15.89:0/1393379992 >> [v2:172.21.15.89:6800/308765932,v1:172.21.15.89:6801/308765932] conn(0x7fcd4406bff0 0x7fcd4406e430 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:50.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.943+0000 7fcd67b67640 1 -- 172.21.15.89:0/1393379992 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fcd44065680 msgr2=0x7fcd44067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:50.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.943+0000 7fcd67b67640 1 --2- 172.21.15.89:0/1393379992 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fcd44065680 0x7fcd44067b40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c0ae000 tx=0x7fcd5c0a7000 comp rx=0 tx=0).stop 2023-10-18T11:03:50.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.943+0000 7fcd67b67640 1 -- 172.21.15.89:0/1393379992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd60164b80 msgr2=0x7fcd6013ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:50.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.943+0000 7fcd67b67640 1 --2- 172.21.15.89:0/1393379992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd60164b80 0x7fcd6013ead0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fcd5800c700 tx=0x7fcd5800cbc0 comp rx=0 tx=0).stop 2023-10-18T11:03:50.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.943+0000 7fcd66104640 1 -- 172.21.15.89:0/1393379992 reap_dead start 2023-10-18T11:03:50.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.943+0000 7fcd67b67640 1 -- 172.21.15.89:0/1393379992 shutdown_connections 2023-10-18T11:03:50.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.943+0000 7fcd67b67640 1 -- 172.21.15.89:0/1393379992 >> 172.21.15.89:0/1393379992 conn(0x7fcd600cad40 msgr2=0x7fcd600cbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:50.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.943+0000 7fcd67b67640 1 -- 172.21.15.89:0/1393379992 shutdown_connections 2023-10-18T11:03:50.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:50.943+0000 7fcd67b67640 1 -- 172.21.15.89:0/1393379992 wait complete. 2023-10-18T11:03:50.959 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-18T11:03:51.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.099+0000 7f08a1a6d640 1 Processor -- start 2023-10-18T11:03:51.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.099+0000 7f08a1a6d640 1 -- start start 2023-10-18T11:03:51.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.099+0000 7f08a1a6d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f089c0ca340 0x7f089c15e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:51.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.099+0000 7f08a1a6d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f089c0c8960 0x7f089c0c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:51.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.099+0000 7f08a1a6d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f089c15acb0 con 0x7f089c06ff00 2023-10-18T11:03:51.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.099+0000 7f08a1a6d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f089c15ef30 con 0x7f089c0ca340 2023-10-18T11:03:51.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.099+0000 7f08a1a6d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f089c15aa10 con 0x7f089c0c8960 2023-10-18T11:03:51.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.099+0000 7f0893fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f089c0ca340 0x7f089c15e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:51.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.099+0000 7f089bfff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f089c0c8960 0x7f089c0c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:51.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.099+0000 7f0893fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f089c0ca340 0x7f089c15e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:51138/0 (socket says 172.21.15.89:51138) 2023-10-18T11:03:51.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.099+0000 7f0893fff640 1 -- 172.21.15.89:0/2657158678 learned_addr learned my addr 172.21.15.89:0/2657158678 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:51.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.099+0000 7f089affd640 1 -- 172.21.15.89:0/2657158678 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2724739556 0 0) 0x7f089c15acb0 con 0x7f089c06ff00 2023-10-18T11:03:51.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.099+0000 7f089affd640 1 -- 172.21.15.89:0/2657158678 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0888003530 con 0x7f089c06ff00 2023-10-18T11:03:51.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.099+0000 7f0893fff640 1 -- 172.21.15.89:0/2657158678 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f089c0c8960 msgr2=0x7f089c0c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:51.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.099+0000 7f0893fff640 1 --2- 172.21.15.89:0/2657158678 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f089c0c8960 0x7f089c0c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.099+0000 7f0893fff640 1 -- 172.21.15.89:0/2657158678 >> v1:172.21.15.154:6789/0 conn(0x7f089c06ff00 legacy=0x7f089c15b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:51.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.099+0000 7f0893fff640 1 -- 172.21.15.89:0/2657158678 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f089c0702e0 con 0x7f089c0ca340 2023-10-18T11:03:51.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.099+0000 7f089bfff640 1 --2- 172.21.15.89:0/2657158678 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f089c0c8960 0x7f089c0c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T11:03:51.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.103+0000 7f0893fff640 1 --2- 172.21.15.89:0/2657158678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f089c0ca340 0x7f089c15e000 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0884009870 tx=0x7f088401c4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3509b1ba4ac0891d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:51.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.103+0000 7f089affd640 1 -- 172.21.15.89:0/2657158678 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08840ac600 con 0x7f089c0ca340 2023-10-18T11:03:51.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.103+0000 7f089affd640 1 -- 172.21.15.89:0/2657158678 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08840a8050 con 0x7f089c0ca340 2023-10-18T11:03:51.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.103+0000 7f089affd640 1 -- 172.21.15.89:0/2657158678 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08840ac960 con 0x7f089c0ca340 2023-10-18T11:03:51.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.103+0000 7f08a1a6d640 1 -- 172.21.15.89:0/2657158678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f089c0ca340 msgr2=0x7f089c15e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:51.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.103+0000 7f08a1a6d640 1 --2- 172.21.15.89:0/2657158678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f089c0ca340 0x7f089c15e000 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0884009870 tx=0x7f088401c4b0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.103+0000 7f08a1a6d640 1 -- 172.21.15.89:0/2657158678 shutdown_connections 2023-10-18T11:03:51.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.103+0000 7f08a1a6d640 1 --2- 172.21.15.89:0/2657158678 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f089c0c8960 0x7f089c0c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.103+0000 7f08a1a6d640 1 --2- 172.21.15.89:0/2657158678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f089c0ca340 0x7f089c15e000 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.103+0000 7f08a1a6d640 1 -- 172.21.15.89:0/2657158678 >> 172.21.15.89:0/2657158678 conn(0x7f089c069f00 msgr2=0x7f089c0cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:51.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.103+0000 7f08a1a6d640 1 -- 172.21.15.89:0/2657158678 shutdown_connections 2023-10-18T11:03:51.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.103+0000 7f08a1a6d640 1 -- 172.21.15.89:0/2657158678 wait complete. 2023-10-18T11:03:51.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.107+0000 7f08a1a6d640 1 Processor -- start 2023-10-18T11:03:51.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.107+0000 7f08a1a6d640 1 -- start start 2023-10-18T11:03:51.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.107+0000 7f08a1a6d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f089c06ff00 0x7f089c0b3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:51.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.107+0000 7f08a1a6d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f089c0c8960 0x7f089c150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:51.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.107+0000 7f08a1a6d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f089c0ca340 0x7f089c0c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:51.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.107+0000 7f08a1a6d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089c0b24c0 con 0x7f089c0ca340 2023-10-18T11:03:51.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.107+0000 7f08a1a6d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089c0b27a0 con 0x7f089c06ff00 2023-10-18T11:03:51.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.107+0000 7f08a1a6d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f089c0b2a80 con 0x7f089c0c8960 2023-10-18T11:03:51.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.107+0000 7f089bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f089c0ca340 0x7f089c0c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:51.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.107+0000 7f089bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f089c0ca340 0x7f089c0c8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55772/0 (socket says 172.21.15.89:55772) 2023-10-18T11:03:51.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.107+0000 7f089bfff640 1 -- 172.21.15.89:0/4227699461 learned_addr learned my addr 172.21.15.89:0/4227699461 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:51.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.107+0000 7f0893fff640 1 --2- 172.21.15.89:0/4227699461 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f089c0c8960 0x7f089c150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:51.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.107+0000 7f089b7fe640 1 --2- 172.21.15.89:0/4227699461 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f089c06ff00 0x7f089c0b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:51.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.107+0000 7f089bfff640 1 -- 172.21.15.89:0/4227699461 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f089c0c8960 msgr2=0x7f089c150150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:51.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.107+0000 7f089bfff640 1 --2- 172.21.15.89:0/4227699461 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f089c0c8960 0x7f089c150150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.107+0000 7f089bfff640 1 -- 172.21.15.89:0/4227699461 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f089c06ff00 msgr2=0x7f089c0b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:51.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.107+0000 7f089bfff640 1 --2- 172.21.15.89:0/4227699461 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f089c06ff00 0x7f089c0b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.107+0000 7f089bfff640 1 -- 172.21.15.89:0/4227699461 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0894009a70 con 0x7f089c0ca340 2023-10-18T11:03:51.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.107+0000 7f089bfff640 1 --2- 172.21.15.89:0/4227699461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f089c0ca340 0x7f089c0c8080 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f089400efc0 tx=0x7f089400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:51.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.107+0000 7f0898ff9640 1 -- 172.21.15.89:0/4227699461 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0894009cc0 con 0x7f089c0ca340 2023-10-18T11:03:51.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.107+0000 7f0898ff9640 1 -- 172.21.15.89:0/4227699461 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f089400f040 con 0x7f089c0ca340 2023-10-18T11:03:51.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.107+0000 7f0898ff9640 1 -- 172.21.15.89:0/4227699461 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08940039a0 con 0x7f089c0ca340 2023-10-18T11:03:51.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.107+0000 7f08a1a6d640 1 -- 172.21.15.89:0/4227699461 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08840094d0 con 0x7f089c0ca340 2023-10-18T11:03:51.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.107+0000 7f08a1a6d640 1 -- 172.21.15.89:0/4227699461 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f089c0b3040 con 0x7f089c0ca340 2023-10-18T11:03:51.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.107+0000 7f08a1a6d640 1 -- 172.21.15.89:0/4227699461 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f089c0b3230 con 0x7f089c0ca340 2023-10-18T11:03:51.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.111+0000 7f0898ff9640 1 -- 172.21.15.89:0/4227699461 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f0894014480 con 0x7f089c0ca340 2023-10-18T11:03:51.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.111+0000 7f0898ff9640 1 --2- 172.21.15.89:0/4227699461 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f087c065680 0x7f087c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:51.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.111+0000 7f0898ff9640 1 -- 172.21.15.89:0/4227699461 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f08940837e0 con 0x7f089c0ca340 2023-10-18T11:03:51.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.111+0000 7f089b7fe640 1 --2- 172.21.15.89:0/4227699461 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f087c065680 0x7f087c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:51.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.111+0000 7f089b7fe640 1 --2- 172.21.15.89:0/4227699461 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f087c065680 0x7f087c067b40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f088c0011f0 tx=0x7f088c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:51.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.115+0000 7f0898ff9640 1 -- 172.21.15.89:0/4227699461 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f0894054ed0 con 0x7f089c0ca340 2023-10-18T11:03:51.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.263+0000 7f08a1a6d640 1 -- 172.21.15.89:0/4227699461 --> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f089c15ec50 con 0x7f087c065680 2023-10-18T11:03:51.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.263+0000 7f0898ff9640 1 -- 172.21.15.89:0/4227699461 <== mgr.4099 v2:172.21.15.154:6832/1086608968 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f088c001440 con 0x7f087c065680 2023-10-18T11:03:51.265 INFO:teuthology.orchestra.run.smithi089.stderr:instructed osd(s) 0 to deep-scrub 2023-10-18T11:03:51.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.267+0000 7f08a1a6d640 1 -- 172.21.15.89:0/4227699461 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f087c065680 msgr2=0x7f087c067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:51.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.267+0000 7f08a1a6d640 1 --2- 172.21.15.89:0/4227699461 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f087c065680 0x7f087c067b40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f088c0011f0 tx=0x7f088c009040 comp rx=0 tx=0).stop 2023-10-18T11:03:51.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.267+0000 7f08a1a6d640 1 -- 172.21.15.89:0/4227699461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f089c0ca340 msgr2=0x7f089c0c8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:51.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.267+0000 7f08a1a6d640 1 --2- 172.21.15.89:0/4227699461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f089c0ca340 0x7f089c0c8080 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f089400efc0 tx=0x7f089400c490 comp rx=0 tx=0).stop 2023-10-18T11:03:51.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.267+0000 7f08a1a6d640 1 -- 172.21.15.89:0/4227699461 shutdown_connections 2023-10-18T11:03:51.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.267+0000 7f08a1a6d640 1 --2- 172.21.15.89:0/4227699461 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f087c065680 0x7f087c067b40 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.267+0000 7f08a1a6d640 1 --2- 172.21.15.89:0/4227699461 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f089c0c8960 0x7f089c150150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.267+0000 7f08a1a6d640 1 --2- 172.21.15.89:0/4227699461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f089c0ca340 0x7f089c0c8080 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.267+0000 7f08a1a6d640 1 --2- 172.21.15.89:0/4227699461 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f089c06ff00 0x7f089c0b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.267+0000 7f08a1a6d640 1 -- 172.21.15.89:0/4227699461 >> 172.21.15.89:0/4227699461 conn(0x7f089c069f00 msgr2=0x7f089c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:51.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.267+0000 7f08a1a6d640 1 -- 172.21.15.89:0/4227699461 shutdown_connections 2023-10-18T11:03:51.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.267+0000 7f08a1a6d640 1 -- 172.21.15.89:0/4227699461 wait complete. 2023-10-18T11:03:51.285 INFO:tasks.ceph:Scrubbing osd.1 2023-10-18T11:03:51.286 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-18T11:03:51.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.427+0000 7f376db03640 1 Processor -- start 2023-10-18T11:03:51.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.427+0000 7f376db03640 1 -- start start 2023-10-18T11:03:51.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.431+0000 7f376db03640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f376815c880 0x7f37681602a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:51.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.431+0000 7f376db03640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f376806b510 0x7f37680caab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:51.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.431+0000 7f376db03640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37681607e0 con 0x7f37680caff0 2023-10-18T11:03:51.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.431+0000 7f376db03640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f376815cc60 con 0x7f376806b510 2023-10-18T11:03:51.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.431+0000 7f376db03640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3768069c30 con 0x7f376815c880 2023-10-18T11:03:51.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.431+0000 7f375effd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f376806b510 0x7f37680caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:51.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.431+0000 7f375effd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f376806b510 0x7f37680caab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55780/0 (socket says 172.21.15.89:55780) 2023-10-18T11:03:51.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.431+0000 7f375effd640 1 -- 172.21.15.89:0/268458872 learned_addr learned my addr 172.21.15.89:0/268458872 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:51.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.431+0000 7f37677fe640 1 --2- 172.21.15.89:0/268458872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f376815c880 0x7f37681602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:51.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.431+0000 7f375effd640 1 -- 172.21.15.89:0/268458872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f376815c880 msgr2=0x7f37681602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:51.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.431+0000 7f375effd640 1 --2- 172.21.15.89:0/268458872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f376815c880 0x7f37681602a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.431+0000 7f375effd640 1 -- 172.21.15.89:0/268458872 >> v1:172.21.15.154:6789/0 conn(0x7f37680caff0 legacy=0x7f376806a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:51.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.431+0000 7f375effd640 1 -- 172.21.15.89:0/268458872 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3768069f10 con 0x7f376806b510 2023-10-18T11:03:51.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.431+0000 7f375effd640 1 --2- 172.21.15.89:0/268458872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f376806b510 0x7f37680caab0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f3750009870 tx=0x7f375001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=783e4ab8a2400810 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:51.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.431+0000 7f3766ffd640 1 -- 172.21.15.89:0/268458872 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37500a9400 con 0x7f376806b510 2023-10-18T11:03:51.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.435+0000 7f3766ffd640 1 -- 172.21.15.89:0/268458872 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37500a9560 con 0x7f376806b510 2023-10-18T11:03:51.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.435+0000 7f3766ffd640 1 -- 172.21.15.89:0/268458872 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37500a98f0 con 0x7f376806b510 2023-10-18T11:03:51.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.435+0000 7f376db03640 1 -- 172.21.15.89:0/268458872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f376806b510 msgr2=0x7f37680caab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:51.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.435+0000 7f376db03640 1 --2- 172.21.15.89:0/268458872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f376806b510 0x7f37680caab0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f3750009870 tx=0x7f375001c540 comp rx=0 tx=0).stop 2023-10-18T11:03:51.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.435+0000 7f376db03640 1 -- 172.21.15.89:0/268458872 shutdown_connections 2023-10-18T11:03:51.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.435+0000 7f376db03640 1 --2- 172.21.15.89:0/268458872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f376806b510 0x7f37680caab0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.435+0000 7f376db03640 1 --2- 172.21.15.89:0/268458872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f376815c880 0x7f37681602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.435+0000 7f376db03640 1 -- 172.21.15.89:0/268458872 >> 172.21.15.89:0/268458872 conn(0x7f37680ccee0 msgr2=0x7f37680cd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:51.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.435+0000 7f376db03640 1 -- 172.21.15.89:0/268458872 shutdown_connections 2023-10-18T11:03:51.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.435+0000 7f376db03640 1 -- 172.21.15.89:0/268458872 wait complete. 2023-10-18T11:03:51.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.435+0000 7f376db03640 1 Processor -- start 2023-10-18T11:03:51.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.435+0000 7f376db03640 1 -- start start 2023-10-18T11:03:51.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.435+0000 7f376db03640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f376806b510 0x7f376814a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:51.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.435+0000 7f376db03640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37680caff0 0x7f37680cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:51.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.435+0000 7f376db03640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f376815c880 0x7f37680cb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:51.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.435+0000 7f376db03640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f376815b8d0 con 0x7f376815c880 2023-10-18T11:03:51.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.435+0000 7f376db03640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f376815bbb0 con 0x7f376806b510 2023-10-18T11:03:51.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.435+0000 7f376db03640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f376814e950 con 0x7f37680caff0 2023-10-18T11:03:51.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.435+0000 7f3767fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f376815c880 0x7f37680cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:51.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.435+0000 7f3767fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f376815c880 0x7f37680cb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55788/0 (socket says 172.21.15.89:55788) 2023-10-18T11:03:51.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.435+0000 7f375effd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37680caff0 0x7f37680cb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:51.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.435+0000 7f3767fff640 1 -- 172.21.15.89:0/1445744593 learned_addr learned my addr 172.21.15.89:0/1445744593 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:51.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.435+0000 7f37677fe640 1 --2- 172.21.15.89:0/1445744593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f376806b510 0x7f376814a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:51.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.439+0000 7f3767fff640 1 -- 172.21.15.89:0/1445744593 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37680caff0 msgr2=0x7f37680cb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:51.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.439+0000 7f3767fff640 1 --2- 172.21.15.89:0/1445744593 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37680caff0 0x7f37680cb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.439+0000 7f3767fff640 1 -- 172.21.15.89:0/1445744593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f376806b510 msgr2=0x7f376814a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:51.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.439+0000 7f3767fff640 1 --2- 172.21.15.89:0/1445744593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f376806b510 0x7f376814a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.439+0000 7f3767fff640 1 -- 172.21.15.89:0/1445744593 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37500094d0 con 0x7f376815c880 2023-10-18T11:03:51.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.439+0000 7f3767fff640 1 --2- 172.21.15.89:0/1445744593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f376815c880 0x7f37680cb910 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f376000c770 tx=0x7f376000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:51.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.439+0000 7f3764ff9640 1 -- 172.21.15.89:0/1445744593 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3760003b80 con 0x7f376815c880 2023-10-18T11:03:51.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.439+0000 7f3764ff9640 1 -- 172.21.15.89:0/1445744593 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3760003ce0 con 0x7f376815c880 2023-10-18T11:03:51.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.439+0000 7f3764ff9640 1 -- 172.21.15.89:0/1445744593 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376000aa90 con 0x7f376815c880 2023-10-18T11:03:51.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.439+0000 7f376db03640 1 -- 172.21.15.89:0/1445744593 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f376814ec30 con 0x7f376815c880 2023-10-18T11:03:51.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.439+0000 7f376db03640 1 -- 172.21.15.89:0/1445744593 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37681584b0 con 0x7f376815c880 2023-10-18T11:03:51.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.439+0000 7f376db03640 1 -- 172.21.15.89:0/1445744593 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f376814e7d0 con 0x7f376815c880 2023-10-18T11:03:51.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.439+0000 7f3764ff9640 1 -- 172.21.15.89:0/1445744593 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f376000abf0 con 0x7f376815c880 2023-10-18T11:03:51.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.439+0000 7f3764ff9640 1 --2- 172.21.15.89:0/1445744593 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f37480656b0 0x7f3748067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:51.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.439+0000 7f3764ff9640 1 -- 172.21.15.89:0/1445744593 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f37600829b0 con 0x7f376815c880 2023-10-18T11:03:51.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.439+0000 7f3764ff9640 1 --2- 172.21.15.89:0/1445744593 >> [v2:172.21.15.89:6816/2220030573,v1:172.21.15.89:6817/2220030573] conn(0x7f374806c020 0x7f374806e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:51.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.439+0000 7f3764ff9640 1 -- 172.21.15.89:0/1445744593 --> [v2:172.21.15.89:6816/2220030573,v1:172.21.15.89:6817/2220030573] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f374806eb30 con 0x7f374806c020 2023-10-18T11:03:51.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.439+0000 7f3764ff9640 1 -- 172.21.15.89:0/1445744593 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37600872a0 con 0x7f376815c880 2023-10-18T11:03:51.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.443+0000 7f375effd640 1 --2- 172.21.15.89:0/1445744593 >> [v2:172.21.15.89:6816/2220030573,v1:172.21.15.89:6817/2220030573] conn(0x7f374806c020 0x7f374806e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:51.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.443+0000 7f37677fe640 1 --2- 172.21.15.89:0/1445744593 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f37480656b0 0x7f3748067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:51.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.443+0000 7f375effd640 1 --2- 172.21.15.89:0/1445744593 >> [v2:172.21.15.89:6816/2220030573,v1:172.21.15.89:6817/2220030573] conn(0x7f374806c020 0x7f374806e460 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:51.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.443+0000 7f37677fe640 1 --2- 172.21.15.89:0/1445744593 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f37480656b0 0x7f3748067b70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f37580079c0 tx=0x7f3758008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:51.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.443+0000 7f3764ff9640 1 -- 172.21.15.89:0/1445744593 <== osd.1 v2:172.21.15.89:6816/2220030573 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f374806eb30 con 0x7f374806c020 2023-10-18T11:03:51.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.455+0000 7f376db03640 1 -- 172.21.15.89:0/1445744593 --> [v2:172.21.15.89:6816/2220030573,v1:172.21.15.89:6817/2220030573] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3768160fd0 con 0x7f374806c020 2023-10-18T11:03:51.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.455+0000 7f3764ff9640 1 -- 172.21.15.89:0/1445744593 <== osd.1 v2:172.21.15.89:6816/2220030573 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f37500a9850 con 0x7f374806c020 2023-10-18T11:03:51.455 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-18T11:03:51.456 INFO:teuthology.orchestra.run.smithi089.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-18T11:03:51.456 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-18T11:03:51.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.455+0000 7f376db03640 1 -- 172.21.15.89:0/1445744593 >> [v2:172.21.15.89:6816/2220030573,v1:172.21.15.89:6817/2220030573] conn(0x7f374806c020 msgr2=0x7f374806e460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:51.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.455+0000 7f376db03640 1 --2- 172.21.15.89:0/1445744593 >> [v2:172.21.15.89:6816/2220030573,v1:172.21.15.89:6817/2220030573] conn(0x7f374806c020 0x7f374806e460 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.455+0000 7f376db03640 1 -- 172.21.15.89:0/1445744593 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f37480656b0 msgr2=0x7f3748067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:51.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.455+0000 7f376db03640 1 --2- 172.21.15.89:0/1445744593 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f37480656b0 0x7f3748067b70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f37580079c0 tx=0x7f3758008040 comp rx=0 tx=0).stop 2023-10-18T11:03:51.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.455+0000 7f376db03640 1 -- 172.21.15.89:0/1445744593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f376815c880 msgr2=0x7f37680cb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:51.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.455+0000 7f376db03640 1 --2- 172.21.15.89:0/1445744593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f376815c880 0x7f37680cb910 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f376000c770 tx=0x7f376000cc30 comp rx=0 tx=0).stop 2023-10-18T11:03:51.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.455+0000 7f3767fff640 1 -- 172.21.15.89:0/1445744593 reap_dead start 2023-10-18T11:03:51.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.455+0000 7f376db03640 1 -- 172.21.15.89:0/1445744593 shutdown_connections 2023-10-18T11:03:51.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.455+0000 7f376db03640 1 -- 172.21.15.89:0/1445744593 >> 172.21.15.89:0/1445744593 conn(0x7f37680ccee0 msgr2=0x7f376806ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:51.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.455+0000 7f376db03640 1 -- 172.21.15.89:0/1445744593 shutdown_connections 2023-10-18T11:03:51.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.455+0000 7f376db03640 1 -- 172.21.15.89:0/1445744593 wait complete. 2023-10-18T11:03:51.473 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-18T11:03:51.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.611+0000 7f05921fa640 1 Processor -- start 2023-10-18T11:03:51.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.611+0000 7f05921fa640 1 -- start start 2023-10-18T11:03:51.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.611+0000 7f05921fa640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f058c070600 0x7f058c06b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:51.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.611+0000 7f05921fa640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f058c070b90 con 0x7f058c159490 2023-10-18T11:03:51.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.611+0000 7f05921fa640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f058c070e70 con 0x7f058c06bd10 2023-10-18T11:03:51.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.611+0000 7f05921fa640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f058c06c1a0 con 0x7f058c070600 2023-10-18T11:03:51.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.611+0000 7f058b7fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f058c159490 0x7f058c15d8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48186/0 (socket says 172.21.15.89:48186) 2023-10-18T11:03:51.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.611+0000 7f058b7fe640 1 -- 172.21.15.89:0/1648261217 learned_addr learned my addr 172.21.15.89:0/1648261217 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:51.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.611+0000 7f058affd640 1 --2- 172.21.15.89:0/1648261217 >> v2:172.21.15.154:3300/0 conn(0x7f058c070600 0x7f058c06b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:51.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.611+0000 7f058a7fc640 1 -- 172.21.15.89:0/1648261217 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3108767552 0 0) 0x7f058c070b90 con 0x7f058c159490 2023-10-18T11:03:51.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.611+0000 7f058a7fc640 1 -- 172.21.15.89:0/1648261217 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0574003530 con 0x7f058c159490 2023-10-18T11:03:51.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.615+0000 7f058a7fc640 1 -- 172.21.15.89:0/1648261217 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1636872139 0 0) 0x7f0574003530 con 0x7f058c159490 2023-10-18T11:03:51.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.615+0000 7f058a7fc640 1 -- 172.21.15.89:0/1648261217 >> v2:172.21.15.154:3300/0 conn(0x7f058c070600 msgr2=0x7f058c06b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:51.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.615+0000 7f058a7fc640 1 --2- 172.21.15.89:0/1648261217 >> v2:172.21.15.154:3300/0 conn(0x7f058c070600 0x7f058c06b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.615+0000 7f058a7fc640 1 -- 172.21.15.89:0/1648261217 >> v1:172.21.15.154:6789/0 conn(0x7f058c06bd10 legacy=0x7f058c0c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:51.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.615+0000 7f058a7fc640 1 -- 172.21.15.89:0/1648261217 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f058c071150 con 0x7f058c159490 2023-10-18T11:03:51.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.615+0000 7f058a7fc640 1 -- 172.21.15.89:0/1648261217 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7f057c001680 con 0x7f058c159490 2023-10-18T11:03:51.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.615+0000 7f058a7fc640 1 -- 172.21.15.89:0/1648261217 >> v1:172.21.15.89:6789/0 conn(0x7f058c159490 legacy=0x7f058c15d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:51.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.615+0000 7f058a7fc640 1 --2- 172.21.15.89:0/1648261217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0574003980 0x7f0574003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:51.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.615+0000 7f058a7fc640 1 --2- 172.21.15.89:0/1648261217 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05740047d0 0x7f0574004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:51.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.615+0000 7f058a7fc640 1 --2- 172.21.15.89:0/1648261217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0574005190 0x7f057400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:51.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.615+0000 7f058a7fc640 1 -- 172.21.15.89:0/1648261217 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058c06c1a0 con 0x7f0574005190 2023-10-18T11:03:51.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.615+0000 7f058a7fc640 1 -- 172.21.15.89:0/1648261217 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057400c130 con 0x7f0574003980 2023-10-18T11:03:51.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.615+0000 7f058a7fc640 1 -- 172.21.15.89:0/1648261217 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f057400c2f0 con 0x7f05740047d0 2023-10-18T11:03:51.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.615+0000 7f058bfff640 1 --2- 172.21.15.89:0/1648261217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0574005190 0x7f057400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:51.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.615+0000 7f058b7fe640 1 --2- 172.21.15.89:0/1648261217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0574003980 0x7f0574003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:51.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.615+0000 7f058affd640 1 --2- 172.21.15.89:0/1648261217 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05740047d0 0x7f0574004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:51.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.615+0000 7f058bfff640 1 -- 172.21.15.89:0/1648261217 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05740047d0 msgr2=0x7f0574004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:51.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.615+0000 7f058bfff640 1 --2- 172.21.15.89:0/1648261217 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05740047d0 0x7f0574004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.615+0000 7f058bfff640 1 -- 172.21.15.89:0/1648261217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0574003980 msgr2=0x7f0574003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:51.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.615+0000 7f058bfff640 1 --2- 172.21.15.89:0/1648261217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0574003980 0x7f0574003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.615+0000 7f058bfff640 1 -- 172.21.15.89:0/1648261217 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0574003530 con 0x7f0574005190 2023-10-18T11:03:51.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.615+0000 7f058bfff640 1 -- 172.21.15.89:0/1648261217 reap_dead start 2023-10-18T11:03:51.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.615+0000 7f058bfff640 1 --2- 172.21.15.89:0/1648261217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0574005190 0x7f057400b990 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f0584001e70 tx=0x7f058401c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f58e2608b9db72 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:51.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.615+0000 7f058a7fc640 1 -- 172.21.15.89:0/1648261217 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0584003b80 con 0x7f0574005190 2023-10-18T11:03:51.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.615+0000 7f058a7fc640 1 -- 172.21.15.89:0/1648261217 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0584003d20 con 0x7f0574005190 2023-10-18T11:03:51.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.615+0000 7f058a7fc640 1 -- 172.21.15.89:0/1648261217 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0584005310 con 0x7f0574005190 2023-10-18T11:03:51.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.619+0000 7f05921fa640 1 -- 172.21.15.89:0/1648261217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0574005190 msgr2=0x7f057400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:51.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.619+0000 7f05921fa640 1 --2- 172.21.15.89:0/1648261217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0574005190 0x7f057400b990 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f0584001e70 tx=0x7f058401c760 comp rx=0 tx=0).stop 2023-10-18T11:03:51.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.619+0000 7f05921fa640 1 -- 172.21.15.89:0/1648261217 shutdown_connections 2023-10-18T11:03:51.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.619+0000 7f05921fa640 1 --2- 172.21.15.89:0/1648261217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0574005190 0x7f057400b990 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.619+0000 7f05921fa640 1 -- 172.21.15.89:0/1648261217 >> 172.21.15.89:0/1648261217 conn(0x7f058c069f00 msgr2=0x7f058c0cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:51.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.619+0000 7f05921fa640 1 -- 172.21.15.89:0/1648261217 shutdown_connections 2023-10-18T11:03:51.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.619+0000 7f05921fa640 1 -- 172.21.15.89:0/1648261217 wait complete. 2023-10-18T11:03:51.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.619+0000 7f05921fa640 1 Processor -- start 2023-10-18T11:03:51.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.619+0000 7f05921fa640 1 -- start start 2023-10-18T11:03:51.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.619+0000 7f05921fa640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0574005190 0x7f058c1583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:51.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.619+0000 7f05921fa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f058c1f5ca0 0x7f058c0b1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:51.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.619+0000 7f05921fa640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f058c15aa80 0x7f058c15af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:51.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.619+0000 7f05921fa640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058c15b440 con 0x7f058c15aa80 2023-10-18T11:03:51.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.619+0000 7f05921fa640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058c0b2a10 con 0x7f0574005190 2023-10-18T11:03:51.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.619+0000 7f05921fa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f058c0abb90 con 0x7f058c1f5ca0 2023-10-18T11:03:51.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.619+0000 7f058bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f058c15aa80 0x7f058c15af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:51.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.619+0000 7f058bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f058c15aa80 0x7f058c15af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55800/0 (socket says 172.21.15.89:55800) 2023-10-18T11:03:51.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.619+0000 7f058bfff640 1 -- 172.21.15.89:0/905490643 learned_addr learned my addr 172.21.15.89:0/905490643 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:51.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.619+0000 7f058affd640 1 --2- 172.21.15.89:0/905490643 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f058c1f5ca0 0x7f058c0b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:51.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.619+0000 7f058b7fe640 1 --2- 172.21.15.89:0/905490643 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0574005190 0x7f058c1583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:51.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.623+0000 7f058bfff640 1 -- 172.21.15.89:0/905490643 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f058c1f5ca0 msgr2=0x7f058c0b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:51.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.623+0000 7f058bfff640 1 --2- 172.21.15.89:0/905490643 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f058c1f5ca0 0x7f058c0b1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.623+0000 7f058bfff640 1 -- 172.21.15.89:0/905490643 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0574005190 msgr2=0x7f058c1583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:51.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.623+0000 7f058bfff640 1 --2- 172.21.15.89:0/905490643 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0574005190 0x7f058c1583f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.623+0000 7f058bfff640 1 -- 172.21.15.89:0/905490643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f058c0b2170 con 0x7f058c15aa80 2023-10-18T11:03:51.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.623+0000 7f058bfff640 1 --2- 172.21.15.89:0/905490643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f058c15aa80 0x7f058c15af00 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f058401c590 tx=0x7f0574005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:51.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.623+0000 7f0573fff640 1 -- 172.21.15.89:0/905490643 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0584003b80 con 0x7f058c15aa80 2023-10-18T11:03:51.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.623+0000 7f0573fff640 1 -- 172.21.15.89:0/905490643 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0584003e90 con 0x7f058c15aa80 2023-10-18T11:03:51.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.623+0000 7f0573fff640 1 -- 172.21.15.89:0/905490643 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0584006d60 con 0x7f058c15aa80 2023-10-18T11:03:51.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.623+0000 7f05921fa640 1 -- 172.21.15.89:0/905490643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f058c0b2450 con 0x7f058c15aa80 2023-10-18T11:03:51.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.623+0000 7f05921fa640 1 -- 172.21.15.89:0/905490643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f058c0abde0 con 0x7f058c15aa80 2023-10-18T11:03:51.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.623+0000 7f05921fa640 1 -- 172.21.15.89:0/905490643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f058c04dbe0 con 0x7f058c15aa80 2023-10-18T11:03:51.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.623+0000 7f0573fff640 1 -- 172.21.15.89:0/905490643 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f05840a9460 con 0x7f058c15aa80 2023-10-18T11:03:51.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.623+0000 7f0573fff640 1 --2- 172.21.15.89:0/905490643 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0578065680 0x7f0578067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:51.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.623+0000 7f0573fff640 1 -- 172.21.15.89:0/905490643 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f05840b0070 con 0x7f058c15aa80 2023-10-18T11:03:51.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.627+0000 7f058b7fe640 1 --2- 172.21.15.89:0/905490643 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0578065680 0x7f0578067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:51.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.627+0000 7f058b7fe640 1 --2- 172.21.15.89:0/905490643 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0578065680 0x7f0578067b40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f057c000c10 tx=0x7f057c000ee0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:51.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.627+0000 7f0573fff640 1 -- 172.21.15.89:0/905490643 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f05840e9990 con 0x7f058c15aa80 2023-10-18T11:03:51.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.775+0000 7f05921fa640 1 -- 172.21.15.89:0/905490643 --> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f058c071710 con 0x7f0578065680 2023-10-18T11:03:51.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.779+0000 7f0573fff640 1 -- 172.21.15.89:0/905490643 <== mgr.4099 v2:172.21.15.154:6832/1086608968 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f057c001560 con 0x7f0578065680 2023-10-18T11:03:51.778 INFO:teuthology.orchestra.run.smithi089.stderr:instructed osd(s) 1 to deep-scrub 2023-10-18T11:03:51.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.779+0000 7f05921fa640 1 -- 172.21.15.89:0/905490643 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0578065680 msgr2=0x7f0578067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:51.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.779+0000 7f05921fa640 1 --2- 172.21.15.89:0/905490643 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0578065680 0x7f0578067b40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f057c000c10 tx=0x7f057c000ee0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.779+0000 7f05921fa640 1 -- 172.21.15.89:0/905490643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f058c15aa80 msgr2=0x7f058c15af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:51.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.779+0000 7f05921fa640 1 --2- 172.21.15.89:0/905490643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f058c15aa80 0x7f058c15af00 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f058401c590 tx=0x7f0574005800 comp rx=0 tx=0).stop 2023-10-18T11:03:51.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.779+0000 7f05921fa640 1 -- 172.21.15.89:0/905490643 shutdown_connections 2023-10-18T11:03:51.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.779+0000 7f05921fa640 1 --2- 172.21.15.89:0/905490643 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f0578065680 0x7f0578067b40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.779+0000 7f05921fa640 1 --2- 172.21.15.89:0/905490643 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f058c1f5ca0 0x7f058c0b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.779+0000 7f05921fa640 1 --2- 172.21.15.89:0/905490643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f058c15aa80 0x7f058c15af00 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.779+0000 7f05921fa640 1 --2- 172.21.15.89:0/905490643 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0574005190 0x7f058c1583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.779+0000 7f05921fa640 1 -- 172.21.15.89:0/905490643 >> 172.21.15.89:0/905490643 conn(0x7f058c069f00 msgr2=0x7f058c0b0630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:51.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.779+0000 7f05921fa640 1 -- 172.21.15.89:0/905490643 shutdown_connections 2023-10-18T11:03:51.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.779+0000 7f05921fa640 1 -- 172.21.15.89:0/905490643 wait complete. 2023-10-18T11:03:51.798 INFO:tasks.ceph:Scrubbing osd.2 2023-10-18T11:03:51.799 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-18T11:03:51.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.939+0000 7f5316340640 1 Processor -- start 2023-10-18T11:03:51.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.939+0000 7f5316340640 1 -- start start 2023-10-18T11:03:51.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.943+0000 7f5316340640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f531015a5e0 0x7f531006c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:51.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.943+0000 7f5316340640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f53100afd40 0x7f53100b1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:51.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.943+0000 7f5316340640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f531006f8c0 0x7f531006fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:51.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.943+0000 7f5316340640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5310070270 con 0x7f531015a5e0 2023-10-18T11:03:51.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.943+0000 7f5316340640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5310071c00 con 0x7f53100afd40 2023-10-18T11:03:51.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.943+0000 7f5316340640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f531015a9c0 con 0x7f531006f8c0 2023-10-18T11:03:51.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.943+0000 7f530ffff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f531015a5e0 0x7f531006c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:51.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.943+0000 7f530ffff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f531015a5e0 0x7f531006c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55812/0 (socket says 172.21.15.89:55812) 2023-10-18T11:03:51.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.943+0000 7f530ffff640 1 -- 172.21.15.89:0/1059893664 learned_addr learned my addr 172.21.15.89:0/1059893664 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:51.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.943+0000 7f53148dd640 1 --2- 172.21.15.89:0/1059893664 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f531006f8c0 0x7f531006fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:51.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.943+0000 7f530f7fe640 1 --2- 172.21.15.89:0/1059893664 >> v2:172.21.15.154:3300/0 conn(0x7f53100afd40 0x7f53100b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:51.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.943+0000 7f530ffff640 1 -- 172.21.15.89:0/1059893664 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f531006f8c0 msgr2=0x7f531006fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:51.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.943+0000 7f530ffff640 1 --2- 172.21.15.89:0/1059893664 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f531006f8c0 0x7f531006fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.943+0000 7f530ffff640 1 -- 172.21.15.89:0/1059893664 >> v2:172.21.15.154:3300/0 conn(0x7f53100afd40 msgr2=0x7f53100b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:51.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.943+0000 7f530ffff640 1 --2- 172.21.15.89:0/1059893664 >> v2:172.21.15.154:3300/0 conn(0x7f53100afd40 0x7f53100b1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.943+0000 7f530ffff640 1 -- 172.21.15.89:0/1059893664 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f531006c720 con 0x7f531015a5e0 2023-10-18T11:03:51.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.947+0000 7f530ffff640 1 --2- 172.21.15.89:0/1059893664 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531015a5e0 0x7f531006c1e0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f531006aba0 tx=0x7f530001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c76fb231ceb357 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:51.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.947+0000 7f530effd640 1 -- 172.21.15.89:0/1059893664 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53000aa5f0 con 0x7f531015a5e0 2023-10-18T11:03:51.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.947+0000 7f530effd640 1 -- 172.21.15.89:0/1059893664 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53000aa750 con 0x7f531015a5e0 2023-10-18T11:03:51.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.947+0000 7f530effd640 1 -- 172.21.15.89:0/1059893664 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53000aaae0 con 0x7f531015a5e0 2023-10-18T11:03:51.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.947+0000 7f5316340640 1 -- 172.21.15.89:0/1059893664 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531015a5e0 msgr2=0x7f531006c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:51.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.947+0000 7f5316340640 1 --2- 172.21.15.89:0/1059893664 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531015a5e0 0x7f531006c1e0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f531006aba0 tx=0x7f530001c4d0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.947+0000 7f5316340640 1 -- 172.21.15.89:0/1059893664 shutdown_connections 2023-10-18T11:03:51.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.947+0000 7f5316340640 1 --2- 172.21.15.89:0/1059893664 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f531006f8c0 0x7f531006fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.947+0000 7f5316340640 1 --2- 172.21.15.89:0/1059893664 >> v2:172.21.15.154:3300/0 conn(0x7f53100afd40 0x7f53100b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.947+0000 7f5316340640 1 --2- 172.21.15.89:0/1059893664 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531015a5e0 0x7f531006c1e0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.947+0000 7f5316340640 1 -- 172.21.15.89:0/1059893664 >> 172.21.15.89:0/1059893664 conn(0x7f53100cad40 msgr2=0x7f53100cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:51.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.947+0000 7f5316340640 1 -- 172.21.15.89:0/1059893664 shutdown_connections 2023-10-18T11:03:51.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.947+0000 7f5316340640 1 -- 172.21.15.89:0/1059893664 wait complete. 2023-10-18T11:03:51.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.947+0000 7f5316340640 1 Processor -- start 2023-10-18T11:03:51.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.947+0000 7f5316340640 1 -- start start 2023-10-18T11:03:51.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.947+0000 7f5316340640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531006f8c0 0x7f531014ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:51.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.951+0000 7f5316340640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53100afd40 0x7f53100c7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:51.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.951+0000 7f5316340640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f531015a5e0 0x7f5310155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:51.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.951+0000 7f5316340640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53100c71c0 con 0x7f531006f8c0 2023-10-18T11:03:51.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.951+0000 7f5316340640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531014cf90 con 0x7f53100afd40 2023-10-18T11:03:51.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.951+0000 7f5316340640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f531014d4b0 con 0x7f531015a5e0 2023-10-18T11:03:51.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.951+0000 7f530ffff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531006f8c0 0x7f531014ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:51.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.951+0000 7f530ffff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531006f8c0 0x7f531014ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55818/0 (socket says 172.21.15.89:55818) 2023-10-18T11:03:51.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.951+0000 7f530ffff640 1 -- 172.21.15.89:0/1457596775 learned_addr learned my addr 172.21.15.89:0/1457596775 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:51.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.951+0000 7f530f7fe640 1 --2- 172.21.15.89:0/1457596775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53100afd40 0x7f53100c7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:51.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.951+0000 7f53148dd640 1 --2- 172.21.15.89:0/1457596775 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f531015a5e0 0x7f5310155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:51.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.951+0000 7f530ffff640 1 -- 172.21.15.89:0/1457596775 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f531015a5e0 msgr2=0x7f5310155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:51.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.951+0000 7f530ffff640 1 --2- 172.21.15.89:0/1457596775 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f531015a5e0 0x7f5310155c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.951+0000 7f530ffff640 1 -- 172.21.15.89:0/1457596775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53100afd40 msgr2=0x7f53100c7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:51.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.951+0000 7f530ffff640 1 --2- 172.21.15.89:0/1457596775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53100afd40 0x7f53100c7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.951+0000 7f530ffff640 1 -- 172.21.15.89:0/1457596775 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53000094d0 con 0x7f531006f8c0 2023-10-18T11:03:51.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.951+0000 7f530ffff640 1 --2- 172.21.15.89:0/1457596775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531006f8c0 0x7f531014ca50 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f530001ab80 tx=0x7f530001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:51.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.951+0000 7f530cff9640 1 -- 172.21.15.89:0/1457596775 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53000aa5f0 con 0x7f531006f8c0 2023-10-18T11:03:51.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.951+0000 7f530cff9640 1 -- 172.21.15.89:0/1457596775 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53000b3800 con 0x7f531006f8c0 2023-10-18T11:03:51.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.951+0000 7f5316340640 1 -- 172.21.15.89:0/1457596775 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53100c6540 con 0x7f531006f8c0 2023-10-18T11:03:51.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.951+0000 7f530cff9640 1 -- 172.21.15.89:0/1457596775 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53000b3b90 con 0x7f531006f8c0 2023-10-18T11:03:51.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.951+0000 7f5316340640 1 -- 172.21.15.89:0/1457596775 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53100c6b00 con 0x7f531006f8c0 2023-10-18T11:03:51.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.951+0000 7f5316340640 1 -- 172.21.15.89:0/1457596775 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f53100c89b0 con 0x7f531006f8c0 2023-10-18T11:03:51.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.951+0000 7f530cff9640 1 -- 172.21.15.89:0/1457596775 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f53000b3cf0 con 0x7f531006f8c0 2023-10-18T11:03:51.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.955+0000 7f530cff9640 1 --2- 172.21.15.89:0/1457596775 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f52fc0656b0 0x7f52fc067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:51.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.955+0000 7f530cff9640 1 -- 172.21.15.89:0/1457596775 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f53000a7070 con 0x7f531006f8c0 2023-10-18T11:03:51.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.955+0000 7f530cff9640 1 --2- 172.21.15.89:0/1457596775 >> [v2:172.21.15.89:6808/4229386532,v1:172.21.15.89:6809/4229386532] conn(0x7f52fc06c020 0x7f52fc06e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:51.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.955+0000 7f530cff9640 1 -- 172.21.15.89:0/1457596775 --> [v2:172.21.15.89:6808/4229386532,v1:172.21.15.89:6809/4229386532] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f52fc06eb30 con 0x7f52fc06c020 2023-10-18T11:03:51.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.955+0000 7f530cff9640 1 -- 172.21.15.89:0/1457596775 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f530011fa70 con 0x7f531006f8c0 2023-10-18T11:03:51.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.955+0000 7f530f7fe640 1 --2- 172.21.15.89:0/1457596775 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f52fc0656b0 0x7f52fc067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:51.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.955+0000 7f53148dd640 1 --2- 172.21.15.89:0/1457596775 >> [v2:172.21.15.89:6808/4229386532,v1:172.21.15.89:6809/4229386532] conn(0x7f52fc06c020 0x7f52fc06e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:51.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.955+0000 7f53148dd640 1 --2- 172.21.15.89:0/1457596775 >> [v2:172.21.15.89:6808/4229386532,v1:172.21.15.89:6809/4229386532] conn(0x7f52fc06c020 0x7f52fc06e460 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:51.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.955+0000 7f530f7fe640 1 --2- 172.21.15.89:0/1457596775 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f52fc0656b0 0x7f52fc067b70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f531006bf80 tx=0x7f5304008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:51.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.955+0000 7f530cff9640 1 -- 172.21.15.89:0/1457596775 <== osd.2 v2:172.21.15.89:6808/4229386532 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f52fc06eb30 con 0x7f52fc06c020 2023-10-18T11:03:51.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.967+0000 7f5316340640 1 -- 172.21.15.89:0/1457596775 --> [v2:172.21.15.89:6808/4229386532,v1:172.21.15.89:6809/4229386532] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f53100b07a0 con 0x7f52fc06c020 2023-10-18T11:03:51.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.967+0000 7f530cff9640 1 -- 172.21.15.89:0/1457596775 <== osd.2 v2:172.21.15.89:6808/4229386532 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f5308009ad0 con 0x7f52fc06c020 2023-10-18T11:03:51.968 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-18T11:03:51.968 INFO:teuthology.orchestra.run.smithi089.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-18T11:03:51.969 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-18T11:03:51.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.967+0000 7f5316340640 1 -- 172.21.15.89:0/1457596775 >> [v2:172.21.15.89:6808/4229386532,v1:172.21.15.89:6809/4229386532] conn(0x7f52fc06c020 msgr2=0x7f52fc06e460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:51.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.967+0000 7f5316340640 1 --2- 172.21.15.89:0/1457596775 >> [v2:172.21.15.89:6808/4229386532,v1:172.21.15.89:6809/4229386532] conn(0x7f52fc06c020 0x7f52fc06e460 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:51.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.967+0000 7f5316340640 1 -- 172.21.15.89:0/1457596775 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f52fc0656b0 msgr2=0x7f52fc067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:51.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.967+0000 7f5316340640 1 --2- 172.21.15.89:0/1457596775 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f52fc0656b0 0x7f52fc067b70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f531006bf80 tx=0x7f5304008040 comp rx=0 tx=0).stop 2023-10-18T11:03:51.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.967+0000 7f5316340640 1 -- 172.21.15.89:0/1457596775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531006f8c0 msgr2=0x7f531014ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:51.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.967+0000 7f5316340640 1 --2- 172.21.15.89:0/1457596775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531006f8c0 0x7f531014ca50 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f530001ab80 tx=0x7f530001ad10 comp rx=0 tx=0).stop 2023-10-18T11:03:51.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.967+0000 7f53148dd640 1 -- 172.21.15.89:0/1457596775 reap_dead start 2023-10-18T11:03:51.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.967+0000 7f5316340640 1 -- 172.21.15.89:0/1457596775 shutdown_connections 2023-10-18T11:03:51.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.967+0000 7f5316340640 1 -- 172.21.15.89:0/1457596775 >> 172.21.15.89:0/1457596775 conn(0x7f53100cad40 msgr2=0x7f53100c99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:51.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.967+0000 7f5316340640 1 -- 172.21.15.89:0/1457596775 shutdown_connections 2023-10-18T11:03:51.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:51.967+0000 7f5316340640 1 -- 172.21.15.89:0/1457596775 wait complete. 2023-10-18T11:03:51.986 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-18T11:03:52.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.123+0000 7f1e1973a640 1 Processor -- start 2023-10-18T11:03:52.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.123+0000 7f1e1973a640 1 -- start start 2023-10-18T11:03:52.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.127+0000 7f1e1973a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1e1415c940 0x7f1e140d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:52.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.127+0000 7f1e1973a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e140d2f40 0x7f1e140d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:52.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.127+0000 7f1e1973a640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e1415da50 con 0x7f1e140d0ca0 2023-10-18T11:03:52.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.127+0000 7f1e1973a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e140d1130 con 0x7f1e1415c940 2023-10-18T11:03:52.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.127+0000 7f1e1973a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1415ce40 con 0x7f1e140d2f40 2023-10-18T11:03:52.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.127+0000 7f1e137fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1e140d0ca0 0x7f1e140cf060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48194/0 (socket says 172.21.15.89:48194) 2023-10-18T11:03:52.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.127+0000 7f1e137fe640 1 -- 172.21.15.89:0/3160296944 learned_addr learned my addr 172.21.15.89:0/3160296944 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:52.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.127+0000 7f1e12ffd640 1 --2- 172.21.15.89:0/3160296944 >> v2:172.21.15.154:3300/0 conn(0x7f1e1415c940 0x7f1e140d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:52.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.127+0000 7f1e127fc640 1 --2- 172.21.15.89:0/3160296944 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e140d2f40 0x7f1e140d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:52.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.127+0000 7f1e11ffb640 1 -- 172.21.15.89:0/3160296944 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 743909716 0 0) 0x7f1e1415da50 con 0x7f1e140d0ca0 2023-10-18T11:03:52.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.127+0000 7f1e11ffb640 1 -- 172.21.15.89:0/3160296944 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e00003530 con 0x7f1e140d0ca0 2023-10-18T11:03:52.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.127+0000 7f1e11ffb640 1 -- 172.21.15.89:0/3160296944 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1791146946 0 0) 0x7f1e00003530 con 0x7f1e140d0ca0 2023-10-18T11:03:52.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.127+0000 7f1e11ffb640 1 -- 172.21.15.89:0/3160296944 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e140d2f40 msgr2=0x7f1e140d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:52.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.127+0000 7f1e11ffb640 1 --2- 172.21.15.89:0/3160296944 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e140d2f40 0x7f1e140d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:52.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.127+0000 7f1e11ffb640 1 -- 172.21.15.89:0/3160296944 >> v2:172.21.15.154:3300/0 conn(0x7f1e1415c940 msgr2=0x7f1e140d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:52.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.127+0000 7f1e11ffb640 1 --2- 172.21.15.89:0/3160296944 >> v2:172.21.15.154:3300/0 conn(0x7f1e1415c940 0x7f1e140d2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:52.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.127+0000 7f1e11ffb640 1 -- 172.21.15.89:0/3160296944 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1415d120 con 0x7f1e140d0ca0 2023-10-18T11:03:52.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.127+0000 7f1e11ffb640 1 -- 172.21.15.89:0/3160296944 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7f1e0c001680 con 0x7f1e140d0ca0 2023-10-18T11:03:52.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.127+0000 7f1e11ffb640 1 -- 172.21.15.89:0/3160296944 >> v1:172.21.15.89:6789/0 conn(0x7f1e140d0ca0 legacy=0x7f1e140cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:52.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.127+0000 7f1e11ffb640 1 --2- 172.21.15.89:0/3160296944 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e00003980 0x7f1e00003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:52.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.127+0000 7f1e11ffb640 1 --2- 172.21.15.89:0/3160296944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e000047d0 0x7f1e00004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:52.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.127+0000 7f1e11ffb640 1 --2- 172.21.15.89:0/3160296944 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e00005190 0x7f1e0000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:52.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.127+0000 7f1e11ffb640 1 -- 172.21.15.89:0/3160296944 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e140d1130 con 0x7f1e000047d0 2023-10-18T11:03:52.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.127+0000 7f1e11ffb640 1 -- 172.21.15.89:0/3160296944 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1415ce40 con 0x7f1e00003980 2023-10-18T11:03:52.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.127+0000 7f1e11ffb640 1 -- 172.21.15.89:0/3160296944 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e00004570 con 0x7f1e00005190 2023-10-18T11:03:52.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.127+0000 7f1e127fc640 1 --2- 172.21.15.89:0/3160296944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e000047d0 0x7f1e00004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:52.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.127+0000 7f1e12ffd640 1 --2- 172.21.15.89:0/3160296944 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e00003980 0x7f1e00003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:52.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.127+0000 7f1e137fe640 1 --2- 172.21.15.89:0/3160296944 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e00005190 0x7f1e0000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:52.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.127+0000 7f1e127fc640 1 -- 172.21.15.89:0/3160296944 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e00005190 msgr2=0x7f1e0000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:52.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.127+0000 7f1e127fc640 1 --2- 172.21.15.89:0/3160296944 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e00005190 0x7f1e0000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:52.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.127+0000 7f1e127fc640 1 -- 172.21.15.89:0/3160296944 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e00003980 msgr2=0x7f1e00003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:52.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.127+0000 7f1e127fc640 1 --2- 172.21.15.89:0/3160296944 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e00003980 0x7f1e00003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:52.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.127+0000 7f1e127fc640 1 -- 172.21.15.89:0/3160296944 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e00003530 con 0x7f1e000047d0 2023-10-18T11:03:52.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.127+0000 7f1e137fe640 1 --2- 172.21.15.89:0/3160296944 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e00005190 0x7f1e0000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:03:52.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.131+0000 7f1e127fc640 1 --2- 172.21.15.89:0/3160296944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e000047d0 0x7f1e00004c50 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc009ae0 tx=0x7f1dfc01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a8b3fca4e09c6a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:52.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.131+0000 7f1e11ffb640 1 -- 172.21.15.89:0/3160296944 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc0a5640 con 0x7f1e000047d0 2023-10-18T11:03:52.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.131+0000 7f1e11ffb640 1 -- 172.21.15.89:0/3160296944 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dfc0a57a0 con 0x7f1e000047d0 2023-10-18T11:03:52.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.131+0000 7f1e11ffb640 1 -- 172.21.15.89:0/3160296944 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc0a5b50 con 0x7f1e000047d0 2023-10-18T11:03:52.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.131+0000 7f1e1973a640 1 -- 172.21.15.89:0/3160296944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e000047d0 msgr2=0x7f1e00004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:52.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.131+0000 7f1e1973a640 1 --2- 172.21.15.89:0/3160296944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e000047d0 0x7f1e00004c50 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc009ae0 tx=0x7f1dfc01ca50 comp rx=0 tx=0).stop 2023-10-18T11:03:52.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.131+0000 7f1e137fe640 1 -- 172.21.15.89:0/3160296944 reap_dead start 2023-10-18T11:03:52.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.131+0000 7f1e1973a640 1 -- 172.21.15.89:0/3160296944 shutdown_connections 2023-10-18T11:03:52.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.131+0000 7f1e1973a640 1 -- 172.21.15.89:0/3160296944 >> 172.21.15.89:0/3160296944 conn(0x7f1e14089fd0 msgr2=0x7f1e1409b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:52.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.131+0000 7f1e1973a640 1 -- 172.21.15.89:0/3160296944 shutdown_connections 2023-10-18T11:03:52.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.131+0000 7f1e1973a640 1 -- 172.21.15.89:0/3160296944 wait complete. 2023-10-18T11:03:52.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.135+0000 7f1e1973a640 1 Processor -- start 2023-10-18T11:03:52.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.135+0000 7f1e1973a640 1 -- start start 2023-10-18T11:03:52.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.135+0000 7f1e1973a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e000047d0 0x7f1e140b7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:52.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.135+0000 7f1e1973a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e1415ad70 0x7f1e140b6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:52.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.135+0000 7f1e1973a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e1415ba80 0x7f1e1415bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:52.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.135+0000 7f1e1973a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1415c440 con 0x7f1e1415ad70 2023-10-18T11:03:52.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.135+0000 7f1e1973a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e140b7450 con 0x7f1e1415ba80 2023-10-18T11:03:52.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.135+0000 7f1e1973a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e140ca640 con 0x7f1e000047d0 2023-10-18T11:03:52.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.135+0000 7f1e127fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e1415ad70 0x7f1e140b6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:52.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.135+0000 7f1e127fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e1415ad70 0x7f1e140b6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55846/0 (socket says 172.21.15.89:55846) 2023-10-18T11:03:52.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.135+0000 7f1e127fc640 1 -- 172.21.15.89:0/722162062 learned_addr learned my addr 172.21.15.89:0/722162062 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:52.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.135+0000 7f1e137fe640 1 --2- 172.21.15.89:0/722162062 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e1415ba80 0x7f1e1415bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:52.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.135+0000 7f1e12ffd640 1 --2- 172.21.15.89:0/722162062 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e000047d0 0x7f1e140b7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:52.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.135+0000 7f1e127fc640 1 -- 172.21.15.89:0/722162062 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e000047d0 msgr2=0x7f1e140b7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:52.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.135+0000 7f1e127fc640 1 --2- 172.21.15.89:0/722162062 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e000047d0 0x7f1e140b7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:52.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.135+0000 7f1e127fc640 1 -- 172.21.15.89:0/722162062 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e1415ba80 msgr2=0x7f1e1415bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:52.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.135+0000 7f1e127fc640 1 --2- 172.21.15.89:0/722162062 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e1415ba80 0x7f1e1415bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:52.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.135+0000 7f1e127fc640 1 -- 172.21.15.89:0/722162062 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e140c98c0 con 0x7f1e1415ad70 2023-10-18T11:03:52.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.135+0000 7f1e127fc640 1 --2- 172.21.15.89:0/722162062 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e1415ad70 0x7f1e140b6c00 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc0b0040 tx=0x7f1e00005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:52.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.135+0000 7f1e0b7fe640 1 -- 172.21.15.89:0/722162062 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc0be070 con 0x7f1e1415ad70 2023-10-18T11:03:52.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.135+0000 7f1e0b7fe640 1 -- 172.21.15.89:0/722162062 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dfc004020 con 0x7f1e1415ad70 2023-10-18T11:03:52.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.135+0000 7f1e1973a640 1 -- 172.21.15.89:0/722162062 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e140c9b60 con 0x7f1e1415ad70 2023-10-18T11:03:52.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.135+0000 7f1e0b7fe640 1 -- 172.21.15.89:0/722162062 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc0043d0 con 0x7f1e1415ad70 2023-10-18T11:03:52.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.135+0000 7f1e1973a640 1 -- 172.21.15.89:0/722162062 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e140ca120 con 0x7f1e1415ad70 2023-10-18T11:03:52.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.135+0000 7f1e1973a640 1 -- 172.21.15.89:0/722162062 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e1404dbe0 con 0x7f1e1415ad70 2023-10-18T11:03:52.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.139+0000 7f1e0b7fe640 1 -- 172.21.15.89:0/722162062 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f1dfc0b4040 con 0x7f1e1415ad70 2023-10-18T11:03:52.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.139+0000 7f1e0b7fe640 1 --2- 172.21.15.89:0/722162062 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1df40656b0 0x7f1df4067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:52.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.139+0000 7f1e0b7fe640 1 -- 172.21.15.89:0/722162062 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f1dfc11c450 con 0x7f1e1415ad70 2023-10-18T11:03:52.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.139+0000 7f1e12ffd640 1 --2- 172.21.15.89:0/722162062 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1df40656b0 0x7f1df4067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:52.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.139+0000 7f1e12ffd640 1 --2- 172.21.15.89:0/722162062 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1df40656b0 0x7f1df4067b70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1e04007520 tx=0x7f1e0400b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:52.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.143+0000 7f1e0b7fe640 1 -- 172.21.15.89:0/722162062 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f1dfc0ea130 con 0x7f1e1415ad70 2023-10-18T11:03:52.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.291+0000 7f1e1973a640 1 -- 172.21.15.89:0/722162062 --> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e1415d6e0 con 0x7f1df40656b0 2023-10-18T11:03:52.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.291+0000 7f1e0b7fe640 1 -- 172.21.15.89:0/722162062 <== mgr.4099 v2:172.21.15.154:6832/1086608968 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1e0400f040 con 0x7f1df40656b0 2023-10-18T11:03:52.291 INFO:teuthology.orchestra.run.smithi089.stderr:instructed osd(s) 2 to deep-scrub 2023-10-18T11:03:52.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.291+0000 7f1e1973a640 1 -- 172.21.15.89:0/722162062 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1df40656b0 msgr2=0x7f1df4067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:52.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.291+0000 7f1e1973a640 1 --2- 172.21.15.89:0/722162062 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1df40656b0 0x7f1df4067b70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1e04007520 tx=0x7f1e0400b040 comp rx=0 tx=0).stop 2023-10-18T11:03:52.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.291+0000 7f1e1973a640 1 -- 172.21.15.89:0/722162062 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e1415ad70 msgr2=0x7f1e140b6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:52.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.291+0000 7f1e1973a640 1 --2- 172.21.15.89:0/722162062 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e1415ad70 0x7f1e140b6c00 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc0b0040 tx=0x7f1e00005800 comp rx=0 tx=0).stop 2023-10-18T11:03:52.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.291+0000 7f1e1973a640 1 -- 172.21.15.89:0/722162062 shutdown_connections 2023-10-18T11:03:52.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.291+0000 7f1e1973a640 1 --2- 172.21.15.89:0/722162062 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f1df40656b0 0x7f1df4067b70 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:52.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.291+0000 7f1e1973a640 1 --2- 172.21.15.89:0/722162062 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e1415ba80 0x7f1e1415bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:52.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.291+0000 7f1e1973a640 1 --2- 172.21.15.89:0/722162062 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e1415ad70 0x7f1e140b6c00 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:52.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.291+0000 7f1e1973a640 1 --2- 172.21.15.89:0/722162062 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e000047d0 0x7f1e140b7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:52.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.291+0000 7f1e1973a640 1 -- 172.21.15.89:0/722162062 >> 172.21.15.89:0/722162062 conn(0x7f1e14089fd0 msgr2=0x7f1e141600d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:52.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.291+0000 7f1e1973a640 1 -- 172.21.15.89:0/722162062 shutdown_connections 2023-10-18T11:03:52.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.291+0000 7f1e1973a640 1 -- 172.21.15.89:0/722162062 wait complete. 2023-10-18T11:03:52.311 INFO:tasks.ceph:Scrubbing osd.3 2023-10-18T11:03:52.312 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-18T11:03:52.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.455+0000 7faa6fc6a640 1 Processor -- start 2023-10-18T11:03:52.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.455+0000 7faa6fc6a640 1 -- start start 2023-10-18T11:03:52.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.455+0000 7faa6fc6a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7faa681600b0 0x7faa680b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:52.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.455+0000 7faa6fc6a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa68069900 0x7faa68069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:52.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.455+0000 7faa6fc6a640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa6806a4c0 con 0x7faa680d0c70 2023-10-18T11:03:52.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.455+0000 7faa6fc6a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa680b1a80 con 0x7faa681600b0 2023-10-18T11:03:52.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.455+0000 7faa6fc6a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa6806a2d0 con 0x7faa68069900 2023-10-18T11:03:52.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.455+0000 7faa6d205640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7faa680d0c70 0x7faa6806a950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:46398/0 (socket says 172.21.15.89:46398) 2023-10-18T11:03:52.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.455+0000 7faa6d205640 1 -- 172.21.15.89:0/2786264198 learned_addr learned my addr 172.21.15.89:0/2786264198 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:52.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.455+0000 7faa6e207640 1 --2- 172.21.15.89:0/2786264198 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa68069900 0x7faa68069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:52.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.455+0000 7faa6da06640 1 --2- 172.21.15.89:0/2786264198 >> v2:172.21.15.154:3300/0 conn(0x7faa681600b0 0x7faa680b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:52.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.455+0000 7faa6ca04640 1 -- 172.21.15.89:0/2786264198 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1795262981 0 0) 0x7faa6806a4c0 con 0x7faa680d0c70 2023-10-18T11:03:52.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.455+0000 7faa6ca04640 1 -- 172.21.15.89:0/2786264198 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa54003530 con 0x7faa680d0c70 2023-10-18T11:03:52.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.455+0000 7faa6e207640 1 -- 172.21.15.89:0/2786264198 >> v2:172.21.15.154:3300/0 conn(0x7faa681600b0 msgr2=0x7faa680b12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:52.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.455+0000 7faa6e207640 1 --2- 172.21.15.89:0/2786264198 >> v2:172.21.15.154:3300/0 conn(0x7faa681600b0 0x7faa680b12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:52.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.455+0000 7faa6e207640 1 -- 172.21.15.89:0/2786264198 >> v1:172.21.15.89:6789/0 conn(0x7faa680d0c70 legacy=0x7faa6806a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:52.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.455+0000 7faa6e207640 1 -- 172.21.15.89:0/2786264198 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa68160490 con 0x7faa68069900 2023-10-18T11:03:52.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.459+0000 7faa6e207640 1 --2- 172.21.15.89:0/2786264198 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa68069900 0x7faa68069d90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7faa60009d90 tx=0x7faa6001c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a770f29932cd242b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:52.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.459+0000 7faa6ca04640 1 -- 172.21.15.89:0/2786264198 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa600a5a10 con 0x7faa68069900 2023-10-18T11:03:52.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.459+0000 7faa6ca04640 1 -- 172.21.15.89:0/2786264198 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa600a5b70 con 0x7faa68069900 2023-10-18T11:03:52.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.459+0000 7faa6ca04640 1 -- 172.21.15.89:0/2786264198 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa600ad440 con 0x7faa68069900 2023-10-18T11:03:52.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.459+0000 7faa6fc6a640 1 -- 172.21.15.89:0/2786264198 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa68069900 msgr2=0x7faa68069d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:52.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.459+0000 7faa6fc6a640 1 --2- 172.21.15.89:0/2786264198 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa68069900 0x7faa68069d90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7faa60009d90 tx=0x7faa6001c950 comp rx=0 tx=0).stop 2023-10-18T11:03:52.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.459+0000 7faa6fc6a640 1 -- 172.21.15.89:0/2786264198 shutdown_connections 2023-10-18T11:03:52.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.459+0000 7faa6fc6a640 1 --2- 172.21.15.89:0/2786264198 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa68069900 0x7faa68069d90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:52.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.459+0000 7faa6fc6a640 1 --2- 172.21.15.89:0/2786264198 >> v2:172.21.15.154:3300/0 conn(0x7faa681600b0 0x7faa680b12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:52.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.459+0000 7faa6fc6a640 1 -- 172.21.15.89:0/2786264198 >> 172.21.15.89:0/2786264198 conn(0x7faa6809ba00 msgr2=0x7faa680d2690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:52.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.459+0000 7faa6fc6a640 1 -- 172.21.15.89:0/2786264198 shutdown_connections 2023-10-18T11:03:52.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.459+0000 7faa6fc6a640 1 -- 172.21.15.89:0/2786264198 wait complete. 2023-10-18T11:03:52.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.463+0000 7faa6fc6a640 1 Processor -- start 2023-10-18T11:03:52.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.463+0000 7faa6fc6a640 1 -- start start 2023-10-18T11:03:52.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.463+0000 7faa6fc6a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa68069900 0x7faa6814d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:52.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.463+0000 7faa6fc6a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa680d0c70 0x7faa680c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:52.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.463+0000 7faa6fc6a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa681600b0 0x7faa680ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:52.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.463+0000 7faa6fc6a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa68158790 con 0x7faa68069900 2023-10-18T11:03:52.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.463+0000 7faa6fc6a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa68155850 con 0x7faa680d0c70 2023-10-18T11:03:52.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.463+0000 7faa6fc6a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa68157b10 con 0x7faa681600b0 2023-10-18T11:03:52.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.463+0000 7faa6da06640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa68069900 0x7faa6814d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:52.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.463+0000 7faa6da06640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa68069900 0x7faa6814d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51492/0 (socket says 172.21.15.89:51492) 2023-10-18T11:03:52.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.463+0000 7faa6da06640 1 -- 172.21.15.89:0/3685502898 learned_addr learned my addr 172.21.15.89:0/3685502898 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:52.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.463+0000 7faa6d205640 1 --2- 172.21.15.89:0/3685502898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa680d0c70 0x7faa680c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:52.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.463+0000 7faa6e207640 1 --2- 172.21.15.89:0/3685502898 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa681600b0 0x7faa680ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:52.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.463+0000 7faa6da06640 1 -- 172.21.15.89:0/3685502898 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa681600b0 msgr2=0x7faa680ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:52.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.463+0000 7faa6da06640 1 --2- 172.21.15.89:0/3685502898 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa681600b0 0x7faa680ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:52.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.463+0000 7faa6da06640 1 -- 172.21.15.89:0/3685502898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa680d0c70 msgr2=0x7faa680c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:52.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.463+0000 7faa6da06640 1 --2- 172.21.15.89:0/3685502898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa680d0c70 0x7faa680c9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:52.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.463+0000 7faa6da06640 1 -- 172.21.15.89:0/3685502898 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa60009a70 con 0x7faa68069900 2023-10-18T11:03:52.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.463+0000 7faa6da06640 1 --2- 172.21.15.89:0/3685502898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa68069900 0x7faa6814d660 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7faa6400e870 tx=0x7faa6400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:52.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.463+0000 7faa5e7fc640 1 -- 172.21.15.89:0/3685502898 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa64009510 con 0x7faa68069900 2023-10-18T11:03:52.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.467+0000 7faa5e7fc640 1 -- 172.21.15.89:0/3685502898 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa6400eec0 con 0x7faa68069900 2023-10-18T11:03:52.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.467+0000 7faa6fc6a640 1 -- 172.21.15.89:0/3685502898 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa68157df0 con 0x7faa68069900 2023-10-18T11:03:52.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.467+0000 7faa5e7fc640 1 -- 172.21.15.89:0/3685502898 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6400cb40 con 0x7faa68069900 2023-10-18T11:03:52.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.467+0000 7faa6fc6a640 1 -- 172.21.15.89:0/3685502898 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa681583b0 con 0x7faa68069900 2023-10-18T11:03:52.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.467+0000 7faa6fc6a640 1 -- 172.21.15.89:0/3685502898 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faa6806b7e0 con 0x7faa68069900 2023-10-18T11:03:52.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.467+0000 7faa5e7fc640 1 -- 172.21.15.89:0/3685502898 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7faa6400cca0 con 0x7faa68069900 2023-10-18T11:03:52.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.467+0000 7faa5e7fc640 1 --2- 172.21.15.89:0/3685502898 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7faa4c065680 0x7faa4c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:52.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.467+0000 7faa5e7fc640 1 -- 172.21.15.89:0/3685502898 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5659+0+0 (secure 0 0 0) 0x7faa64082ff0 con 0x7faa68069900 2023-10-18T11:03:52.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.467+0000 7faa5e7fc640 1 --2- 172.21.15.89:0/3685502898 >> [v2:172.21.15.89:6822/2771730391,v1:172.21.15.89:6824/2771730391] conn(0x7faa4c06bff0 0x7faa4c06e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:52.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.467+0000 7faa5e7fc640 1 -- 172.21.15.89:0/3685502898 --> [v2:172.21.15.89:6822/2771730391,v1:172.21.15.89:6824/2771730391] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faa4c06eb00 con 0x7faa4c06bff0 2023-10-18T11:03:52.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.467+0000 7faa5e7fc640 1 -- 172.21.15.89:0/3685502898 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa64085650 con 0x7faa68069900 2023-10-18T11:03:52.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.467+0000 7faa6e207640 1 --2- 172.21.15.89:0/3685502898 >> [v2:172.21.15.89:6822/2771730391,v1:172.21.15.89:6824/2771730391] conn(0x7faa4c06bff0 0x7faa4c06e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:52.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.467+0000 7faa6d205640 1 --2- 172.21.15.89:0/3685502898 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7faa4c065680 0x7faa4c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:52.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.467+0000 7faa6e207640 1 --2- 172.21.15.89:0/3685502898 >> [v2:172.21.15.89:6822/2771730391,v1:172.21.15.89:6824/2771730391] conn(0x7faa4c06bff0 0x7faa4c06e430 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:52.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.471+0000 7faa6d205640 1 --2- 172.21.15.89:0/3685502898 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7faa4c065680 0x7faa4c067b40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7faa68154220 tx=0x7faa58007d40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:52.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.471+0000 7faa5e7fc640 1 -- 172.21.15.89:0/3685502898 <== osd.3 v2:172.21.15.89:6822/2771730391 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7faa4c06eb00 con 0x7faa4c06bff0 2023-10-18T11:03:52.480 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:03:52.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.483+0000 7faa6fc6a640 1 -- 172.21.15.89:0/3685502898 --> [v2:172.21.15.89:6822/2771730391,v1:172.21.15.89:6824/2771730391] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7faa6806b500 con 0x7faa4c06bff0 2023-10-18T11:03:52.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.483+0000 7faa5e7fc640 1 -- 172.21.15.89:0/3685502898 <== osd.3 v2:172.21.15.89:6822/2771730391 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7faa60009450 con 0x7faa4c06bff0 2023-10-18T11:03:52.483 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-18T11:03:52.483 INFO:teuthology.orchestra.run.smithi089.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-18T11:03:52.483 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-18T11:03:52.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.483+0000 7faa6fc6a640 1 -- 172.21.15.89:0/3685502898 >> [v2:172.21.15.89:6822/2771730391,v1:172.21.15.89:6824/2771730391] conn(0x7faa4c06bff0 msgr2=0x7faa4c06e430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:52.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.483+0000 7faa6fc6a640 1 --2- 172.21.15.89:0/3685502898 >> [v2:172.21.15.89:6822/2771730391,v1:172.21.15.89:6824/2771730391] conn(0x7faa4c06bff0 0x7faa4c06e430 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:52.485 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:03:52.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.483+0000 7faa6fc6a640 1 -- 172.21.15.89:0/3685502898 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7faa4c065680 msgr2=0x7faa4c067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:52.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.483+0000 7faa6fc6a640 1 --2- 172.21.15.89:0/3685502898 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7faa4c065680 0x7faa4c067b40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7faa68154220 tx=0x7faa58007d40 comp rx=0 tx=0).stop 2023-10-18T11:03:52.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.483+0000 7faa6fc6a640 1 -- 172.21.15.89:0/3685502898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa68069900 msgr2=0x7faa6814d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:52.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.483+0000 7faa6fc6a640 1 --2- 172.21.15.89:0/3685502898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa68069900 0x7faa6814d660 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7faa6400e870 tx=0x7faa6400ec70 comp rx=0 tx=0).stop 2023-10-18T11:03:52.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.483+0000 7faa6e207640 1 -- 172.21.15.89:0/3685502898 reap_dead start 2023-10-18T11:03:52.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.483+0000 7faa6fc6a640 1 -- 172.21.15.89:0/3685502898 shutdown_connections 2023-10-18T11:03:52.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.483+0000 7faa6fc6a640 1 -- 172.21.15.89:0/3685502898 >> 172.21.15.89:0/3685502898 conn(0x7faa6809ba00 msgr2=0x7faa6815ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:52.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.483+0000 7faa6fc6a640 1 -- 172.21.15.89:0/3685502898 shutdown_connections 2023-10-18T11:03:52.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.483+0000 7faa6fc6a640 1 -- 172.21.15.89:0/3685502898 wait complete. 2023-10-18T11:03:52.501 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-18T11:03:52.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.719+0000 7efcce574640 1 Processor -- start 2023-10-18T11:03:52.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.719+0000 7efcce574640 1 -- start start 2023-10-18T11:03:52.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.719+0000 7efcce574640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efcc8069900 0x7efcc8069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:52.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.719+0000 7efcce574640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcc815c8f0 0x7efcc806c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:52.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.719+0000 7efcce574640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efcc806cd60 0x7efcc806da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:52.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.719+0000 7efcce574640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcc806a220 con 0x7efcc806cd60 2023-10-18T11:03:52.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.719+0000 7efcce574640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcc806d210 con 0x7efcc8069900 2023-10-18T11:03:52.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.719+0000 7efcce574640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcc815cdf0 con 0x7efcc815c8f0 2023-10-18T11:03:52.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.719+0000 7efcccb11640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efcc806cd60 0x7efcc806da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:52.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.719+0000 7efcccb11640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efcc806cd60 0x7efcc806da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51502/0 (socket says 172.21.15.89:51502) 2023-10-18T11:03:52.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.719+0000 7efcccb11640 1 -- 172.21.15.89:0/3279434209 learned_addr learned my addr 172.21.15.89:0/3279434209 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:52.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.719+0000 7efcc77fe640 1 --2- 172.21.15.89:0/3279434209 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcc815c8f0 0x7efcc806c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:52.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.719+0000 7efcc7fff640 1 --2- 172.21.15.89:0/3279434209 >> v2:172.21.15.154:3300/0 conn(0x7efcc8069900 0x7efcc8069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:52.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.719+0000 7efcccb11640 1 -- 172.21.15.89:0/3279434209 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcc815c8f0 msgr2=0x7efcc806c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:52.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.719+0000 7efcccb11640 1 --2- 172.21.15.89:0/3279434209 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcc815c8f0 0x7efcc806c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:52.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.719+0000 7efcccb11640 1 -- 172.21.15.89:0/3279434209 >> v2:172.21.15.154:3300/0 conn(0x7efcc8069900 msgr2=0x7efcc8069ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:52.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.719+0000 7efcccb11640 1 --2- 172.21.15.89:0/3279434209 >> v2:172.21.15.154:3300/0 conn(0x7efcc8069900 0x7efcc8069ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:52.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.719+0000 7efcccb11640 1 -- 172.21.15.89:0/3279434209 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc815d0d0 con 0x7efcc806cd60 2023-10-18T11:03:52.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.719+0000 7efcc77fe640 1 --2- 172.21.15.89:0/3279434209 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcc815c8f0 0x7efcc806c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:03:52.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.719+0000 7efcc7fff640 1 --2- 172.21.15.89:0/3279434209 >> v2:172.21.15.154:3300/0 conn(0x7efcc8069900 0x7efcc8069ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:03:52.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.723+0000 7efcccb11640 1 --2- 172.21.15.89:0/3279434209 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcc806cd60 0x7efcc806da00 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7efcc80b16e0 tx=0x7efcc001c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9370f1196a0820a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:52.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.723+0000 7efcc6ffd640 1 -- 172.21.15.89:0/3279434209 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc00a5560 con 0x7efcc806cd60 2023-10-18T11:03:52.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.723+0000 7efcc6ffd640 1 -- 172.21.15.89:0/3279434209 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcc00a56c0 con 0x7efcc806cd60 2023-10-18T11:03:52.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.723+0000 7efcc6ffd640 1 -- 172.21.15.89:0/3279434209 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc00a5a50 con 0x7efcc806cd60 2023-10-18T11:03:52.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.723+0000 7efcce574640 1 -- 172.21.15.89:0/3279434209 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcc806cd60 msgr2=0x7efcc806da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:52.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.723+0000 7efcce574640 1 --2- 172.21.15.89:0/3279434209 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcc806cd60 0x7efcc806da00 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7efcc80b16e0 tx=0x7efcc001c6e0 comp rx=0 tx=0).stop 2023-10-18T11:03:52.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.723+0000 7efcce574640 1 -- 172.21.15.89:0/3279434209 shutdown_connections 2023-10-18T11:03:52.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.723+0000 7efcce574640 1 --2- 172.21.15.89:0/3279434209 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcc806cd60 0x7efcc806da00 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:52.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.723+0000 7efcce574640 1 --2- 172.21.15.89:0/3279434209 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcc815c8f0 0x7efcc806c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:52.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.723+0000 7efcce574640 1 --2- 172.21.15.89:0/3279434209 >> v2:172.21.15.154:3300/0 conn(0x7efcc8069900 0x7efcc8069ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:52.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.723+0000 7efcce574640 1 -- 172.21.15.89:0/3279434209 >> 172.21.15.89:0/3279434209 conn(0x7efcc80cb5a0 msgr2=0x7efcc80cb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:52.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.723+0000 7efcce574640 1 -- 172.21.15.89:0/3279434209 shutdown_connections 2023-10-18T11:03:52.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.723+0000 7efcce574640 1 -- 172.21.15.89:0/3279434209 wait complete. 2023-10-18T11:03:52.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.723+0000 7efcce574640 1 Processor -- start 2023-10-18T11:03:52.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.727+0000 7efcce574640 1 -- start start 2023-10-18T11:03:52.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.727+0000 7efcce574640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcc8069900 0x7efcc81500c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:52.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.727+0000 7efcce574640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcc806cd60 0x7efcc81534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:52.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.727+0000 7efcce574640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcc815c8f0 0x7efcc8152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:52.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.727+0000 7efcce574640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc8150600 con 0x7efcc806cd60 2023-10-18T11:03:52.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.727+0000 7efcce574640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc80cb3c0 con 0x7efcc815c8f0 2023-10-18T11:03:52.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.727+0000 7efcce574640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcc81523d0 con 0x7efcc8069900 2023-10-18T11:03:52.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.727+0000 7efcc77fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcc806cd60 0x7efcc81534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:52.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.727+0000 7efcc77fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcc806cd60 0x7efcc81534d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51504/0 (socket says 172.21.15.89:51504) 2023-10-18T11:03:52.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.727+0000 7efcc77fe640 1 -- 172.21.15.89:0/3811309107 learned_addr learned my addr 172.21.15.89:0/3811309107 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:52.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.727+0000 7efcc7fff640 1 --2- 172.21.15.89:0/3811309107 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcc8069900 0x7efcc81500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:52.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.727+0000 7efcccb11640 1 --2- 172.21.15.89:0/3811309107 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcc815c8f0 0x7efcc8152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:52.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.727+0000 7efcc77fe640 1 -- 172.21.15.89:0/3811309107 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcc8069900 msgr2=0x7efcc81500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:52.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.727+0000 7efcc77fe640 1 --2- 172.21.15.89:0/3811309107 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcc8069900 0x7efcc81500c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:52.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.727+0000 7efcc77fe640 1 -- 172.21.15.89:0/3811309107 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcc815c8f0 msgr2=0x7efcc8152860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:52.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.727+0000 7efcc77fe640 1 --2- 172.21.15.89:0/3811309107 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcc815c8f0 0x7efcc8152860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:52.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.727+0000 7efcc77fe640 1 -- 172.21.15.89:0/3811309107 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc0009b30 con 0x7efcc806cd60 2023-10-18T11:03:52.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.727+0000 7efcc7fff640 1 --2- 172.21.15.89:0/3811309107 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcc8069900 0x7efcc81500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:03:52.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.727+0000 7efcc77fe640 1 --2- 172.21.15.89:0/3811309107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcc806cd60 0x7efcc81534d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7efcbc00c7f0 tx=0x7efcbc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:52.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.727+0000 7efcc4ff9640 1 -- 172.21.15.89:0/3811309107 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc007820 con 0x7efcc806cd60 2023-10-18T11:03:52.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.727+0000 7efcc4ff9640 1 -- 172.21.15.89:0/3811309107 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcbc007980 con 0x7efcc806cd60 2023-10-18T11:03:52.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.727+0000 7efcc4ff9640 1 -- 172.21.15.89:0/3811309107 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc007d30 con 0x7efcc806cd60 2023-10-18T11:03:52.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.727+0000 7efcce574640 1 -- 172.21.15.89:0/3811309107 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcc80cb0e0 con 0x7efcc806cd60 2023-10-18T11:03:52.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.727+0000 7efcce574640 1 -- 172.21.15.89:0/3811309107 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc8152060 con 0x7efcc806cd60 2023-10-18T11:03:52.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.727+0000 7efcce574640 1 -- 172.21.15.89:0/3811309107 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcc8152580 con 0x7efcc806cd60 2023-10-18T11:03:52.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.731+0000 7efcc4ff9640 1 -- 172.21.15.89:0/3811309107 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7efcbc0113f0 con 0x7efcc806cd60 2023-10-18T11:03:52.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.731+0000 7efcc4ff9640 1 --2- 172.21.15.89:0/3811309107 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efcb4065680 0x7efcb4067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:52.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.731+0000 7efcc4ff9640 1 -- 172.21.15.89:0/3811309107 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5659+0+0 (secure 0 0 0) 0x7efcbc082ed0 con 0x7efcc806cd60 2023-10-18T11:03:52.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.731+0000 7efcc7fff640 1 --2- 172.21.15.89:0/3811309107 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efcb4065680 0x7efcb4067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:52.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.731+0000 7efcc7fff640 1 --2- 172.21.15.89:0/3811309107 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efcb4065680 0x7efcb4067b40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7efcb8006fd0 tx=0x7efcb8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:52.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.735+0000 7efcc4ff9640 1 -- 172.21.15.89:0/3811309107 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7efcc8152580 con 0x7efcc806cd60 2023-10-18T11:03:52.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.887+0000 7efcce574640 1 -- 172.21.15.89:0/3811309107 --> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7efcc815d970 con 0x7efcb4065680 2023-10-18T11:03:52.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.887+0000 7efcc4ff9640 1 -- 172.21.15.89:0/3811309107 <== mgr.4099 v2:172.21.15.154:6832/1086608968 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7efcb800f860 con 0x7efcb4065680 2023-10-18T11:03:52.886 INFO:teuthology.orchestra.run.smithi089.stderr:instructed osd(s) 3 to deep-scrub 2023-10-18T11:03:52.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.887+0000 7efcce574640 1 -- 172.21.15.89:0/3811309107 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efcb4065680 msgr2=0x7efcb4067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:52.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.887+0000 7efcce574640 1 --2- 172.21.15.89:0/3811309107 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efcb4065680 0x7efcb4067b40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7efcb8006fd0 tx=0x7efcb8008040 comp rx=0 tx=0).stop 2023-10-18T11:03:52.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.887+0000 7efcce574640 1 -- 172.21.15.89:0/3811309107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcc806cd60 msgr2=0x7efcc81534d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:52.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.887+0000 7efcce574640 1 --2- 172.21.15.89:0/3811309107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcc806cd60 0x7efcc81534d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7efcbc00c7f0 tx=0x7efcbc00ccb0 comp rx=0 tx=0).stop 2023-10-18T11:03:52.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.887+0000 7efcce574640 1 -- 172.21.15.89:0/3811309107 shutdown_connections 2023-10-18T11:03:52.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.887+0000 7efcce574640 1 --2- 172.21.15.89:0/3811309107 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7efcb4065680 0x7efcb4067b40 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:52.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.887+0000 7efcce574640 1 --2- 172.21.15.89:0/3811309107 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcc815c8f0 0x7efcc8152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:52.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.887+0000 7efcce574640 1 --2- 172.21.15.89:0/3811309107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcc806cd60 0x7efcc81534d0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:52.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.887+0000 7efcce574640 1 --2- 172.21.15.89:0/3811309107 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcc8069900 0x7efcc81500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:52.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.887+0000 7efcce574640 1 -- 172.21.15.89:0/3811309107 >> 172.21.15.89:0/3811309107 conn(0x7efcc80cb5a0 msgr2=0x7efcc80afe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:52.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.887+0000 7efcce574640 1 -- 172.21.15.89:0/3811309107 shutdown_connections 2023-10-18T11:03:52.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:52.887+0000 7efcce574640 1 -- 172.21.15.89:0/3811309107 wait complete. 2023-10-18T11:03:52.907 INFO:tasks.ceph:Scrubbing osd.4 2023-10-18T11:03:52.907 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-18T11:03:53.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.047+0000 7f2ef6d8f640 1 Processor -- start 2023-10-18T11:03:53.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.047+0000 7f2ef6d8f640 1 -- start start 2023-10-18T11:03:53.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.051+0000 7f2ef6d8f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2ef015c950 0x7f2ef006a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:53.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.051+0000 7f2ef6d8f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ef00d2600 0x7f2ef00d2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:53.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.051+0000 7f2ef6d8f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2ef0160460 0x7f2ef01608d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:53.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.051+0000 7f2ef6d8f640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ef00d0c60 con 0x7f2ef015c950 2023-10-18T11:03:53.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.051+0000 7f2ef6d8f640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ef015cd30 con 0x7f2ef0160460 2023-10-18T11:03:53.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.051+0000 7f2ef6d8f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef006a850 con 0x7f2ef00d2600 2023-10-18T11:03:53.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.051+0000 7f2ef4b2b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2ef015c950 0x7f2ef006a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:53.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.051+0000 7f2ef4b2b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2ef015c950 0x7f2ef006a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51520/0 (socket says 172.21.15.89:51520) 2023-10-18T11:03:53.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.051+0000 7f2ef4b2b640 1 -- 172.21.15.89:0/821073297 learned_addr learned my addr 172.21.15.89:0/821073297 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:53.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.051+0000 7f2eeffff640 1 --2- 172.21.15.89:0/821073297 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ef00d2600 0x7f2ef00d2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:53.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.051+0000 7f2ef532c640 1 --2- 172.21.15.89:0/821073297 >> v2:172.21.15.154:3300/0 conn(0x7f2ef0160460 0x7f2ef01608d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:53.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.051+0000 7f2ef4b2b640 1 -- 172.21.15.89:0/821073297 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ef00d2600 msgr2=0x7f2ef00d2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:53.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.051+0000 7f2ef4b2b640 1 --2- 172.21.15.89:0/821073297 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ef00d2600 0x7f2ef00d2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:53.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.051+0000 7f2ef4b2b640 1 -- 172.21.15.89:0/821073297 >> v2:172.21.15.154:3300/0 conn(0x7f2ef0160460 msgr2=0x7f2ef01608d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:53.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.051+0000 7f2ef4b2b640 1 --2- 172.21.15.89:0/821073297 >> v2:172.21.15.154:3300/0 conn(0x7f2ef0160460 0x7f2ef01608d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:53.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.051+0000 7f2ef4b2b640 1 -- 172.21.15.89:0/821073297 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef00d2f80 con 0x7f2ef015c950 2023-10-18T11:03:53.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.055+0000 7f2ef4b2b640 1 --2- 172.21.15.89:0/821073297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ef015c950 0x7f2ef006a310 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0161500 tx=0x7f2ed801c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69caf717711e280e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:53.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.055+0000 7f2eef7fe640 1 -- 172.21.15.89:0/821073297 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed80ab600 con 0x7f2ef015c950 2023-10-18T11:03:53.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.055+0000 7f2eef7fe640 1 -- 172.21.15.89:0/821073297 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed80a7050 con 0x7f2ef015c950 2023-10-18T11:03:53.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.055+0000 7f2eef7fe640 1 -- 172.21.15.89:0/821073297 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed80ab980 con 0x7f2ef015c950 2023-10-18T11:03:53.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.055+0000 7f2ef6d8f640 1 -- 172.21.15.89:0/821073297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ef015c950 msgr2=0x7f2ef006a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:53.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.055+0000 7f2ef6d8f640 1 --2- 172.21.15.89:0/821073297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ef015c950 0x7f2ef006a310 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0161500 tx=0x7f2ed801c3b0 comp rx=0 tx=0).stop 2023-10-18T11:03:53.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.055+0000 7f2ef6d8f640 1 -- 172.21.15.89:0/821073297 shutdown_connections 2023-10-18T11:03:53.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.055+0000 7f2ef6d8f640 1 --2- 172.21.15.89:0/821073297 >> v2:172.21.15.154:3300/0 conn(0x7f2ef0160460 0x7f2ef01608d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:53.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.055+0000 7f2ef6d8f640 1 --2- 172.21.15.89:0/821073297 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ef00d2600 0x7f2ef00d2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:53.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.055+0000 7f2ef6d8f640 1 --2- 172.21.15.89:0/821073297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ef015c950 0x7f2ef006a310 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:53.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.055+0000 7f2ef6d8f640 1 -- 172.21.15.89:0/821073297 >> 172.21.15.89:0/821073297 conn(0x7f2ef008c0b0 msgr2=0x7f2ef009b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:53.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.055+0000 7f2ef6d8f640 1 -- 172.21.15.89:0/821073297 shutdown_connections 2023-10-18T11:03:53.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.055+0000 7f2ef6d8f640 1 -- 172.21.15.89:0/821073297 wait complete. 2023-10-18T11:03:53.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.055+0000 7f2ef6d8f640 1 Processor -- start 2023-10-18T11:03:53.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.055+0000 7f2ef6d8f640 1 -- start start 2023-10-18T11:03:53.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.059+0000 7f2ef6d8f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ef00d2600 0x7f2ef0155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:53.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.059+0000 7f2ef6d8f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ef015c950 0x7f2ef00c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:53.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.059+0000 7f2ef6d8f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ef0160460 0x7f2ef00c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:53.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.059+0000 7f2ef6d8f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef00cb570 con 0x7f2ef00d2600 2023-10-18T11:03:53.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.059+0000 7f2ef6d8f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef00c7b70 con 0x7f2ef0160460 2023-10-18T11:03:53.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.059+0000 7f2ef6d8f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef00c6db0 con 0x7f2ef015c950 2023-10-18T11:03:53.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.059+0000 7f2ef4b2b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ef00d2600 0x7f2ef0155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:53.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.059+0000 7f2ef4b2b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ef00d2600 0x7f2ef0155a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51536/0 (socket says 172.21.15.89:51536) 2023-10-18T11:03:53.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.059+0000 7f2ef4b2b640 1 -- 172.21.15.89:0/746028524 learned_addr learned my addr 172.21.15.89:0/746028524 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:53.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.059+0000 7f2eeffff640 1 --2- 172.21.15.89:0/746028524 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ef015c950 0x7f2ef00c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:53.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.059+0000 7f2ef532c640 1 --2- 172.21.15.89:0/746028524 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ef0160460 0x7f2ef00c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:53.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.059+0000 7f2ef4b2b640 1 -- 172.21.15.89:0/746028524 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ef015c950 msgr2=0x7f2ef00c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:53.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.059+0000 7f2ef4b2b640 1 --2- 172.21.15.89:0/746028524 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ef015c950 0x7f2ef00c7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:53.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.059+0000 7f2ef4b2b640 1 -- 172.21.15.89:0/746028524 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ef0160460 msgr2=0x7f2ef00c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:53.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.059+0000 7f2ef4b2b640 1 --2- 172.21.15.89:0/746028524 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ef0160460 0x7f2ef00c8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:53.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.059+0000 7f2ef4b2b640 1 -- 172.21.15.89:0/746028524 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed8009480 con 0x7f2ef00d2600 2023-10-18T11:03:53.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.059+0000 7f2ef4b2b640 1 --2- 172.21.15.89:0/746028524 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ef00d2600 0x7f2ef0155a70 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2ed801cc50 tx=0x7f2ed8019150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:53.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.059+0000 7f2eed7fa640 1 -- 172.21.15.89:0/746028524 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8005ba0 con 0x7f2ef00d2600 2023-10-18T11:03:53.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.059+0000 7f2eed7fa640 1 -- 172.21.15.89:0/746028524 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed8005d00 con 0x7f2ef00d2600 2023-10-18T11:03:53.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.059+0000 7f2ef6d8f640 1 -- 172.21.15.89:0/746028524 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef00c7090 con 0x7f2ef00d2600 2023-10-18T11:03:53.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.059+0000 7f2eed7fa640 1 -- 172.21.15.89:0/746028524 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8003ab0 con 0x7f2ef00d2600 2023-10-18T11:03:53.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.059+0000 7f2ef6d8f640 1 -- 172.21.15.89:0/746028524 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef00c7650 con 0x7f2ef00d2600 2023-10-18T11:03:53.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.059+0000 7f2ef6d8f640 1 -- 172.21.15.89:0/746028524 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2ef00d0eb0 con 0x7f2ef00d2600 2023-10-18T11:03:53.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.059+0000 7f2eed7fa640 1 -- 172.21.15.89:0/746028524 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f2ed80b2070 con 0x7f2ef00d2600 2023-10-18T11:03:53.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.063+0000 7f2eed7fa640 1 --2- 172.21.15.89:0/746028524 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2ed0065680 0x7f2ed0067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:53.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.063+0000 7f2eed7fa640 1 -- 172.21.15.89:0/746028524 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f2ed811d490 con 0x7f2ef00d2600 2023-10-18T11:03:53.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.063+0000 7f2eed7fa640 1 --2- 172.21.15.89:0/746028524 >> [v2:172.21.15.154:6800/2025560293,v1:172.21.15.154:6801/2025560293] conn(0x7f2ed006bff0 0x7f2ed006e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:53.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.063+0000 7f2eed7fa640 1 -- 172.21.15.89:0/746028524 --> [v2:172.21.15.154:6800/2025560293,v1:172.21.15.154:6801/2025560293] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ed006eb00 con 0x7f2ed006bff0 2023-10-18T11:03:53.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.063+0000 7f2eed7fa640 1 -- 172.21.15.89:0/746028524 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ed811d800 con 0x7f2ef00d2600 2023-10-18T11:03:53.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.063+0000 7f2eeffff640 1 --2- 172.21.15.89:0/746028524 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2ed0065680 0x7f2ed0067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:53.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.063+0000 7f2ef532c640 1 --2- 172.21.15.89:0/746028524 >> [v2:172.21.15.154:6800/2025560293,v1:172.21.15.154:6801/2025560293] conn(0x7f2ed006bff0 0x7f2ed006e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:53.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.063+0000 7f2eeffff640 1 --2- 172.21.15.89:0/746028524 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2ed0065680 0x7f2ed0067b40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0167760 tx=0x7f2ee0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:53.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.063+0000 7f2ef532c640 1 --2- 172.21.15.89:0/746028524 >> [v2:172.21.15.154:6800/2025560293,v1:172.21.15.154:6801/2025560293] conn(0x7f2ed006bff0 0x7f2ed006e430 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:53.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.063+0000 7f2eed7fa640 1 -- 172.21.15.89:0/746028524 <== osd.4 v2:172.21.15.154:6800/2025560293 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f2ed006eb00 con 0x7f2ed006bff0 2023-10-18T11:03:53.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.075+0000 7f2ef6d8f640 1 -- 172.21.15.89:0/746028524 --> [v2:172.21.15.154:6800/2025560293,v1:172.21.15.154:6801/2025560293] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2ef00d0880 con 0x7f2ed006bff0 2023-10-18T11:03:53.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.075+0000 7f2eed7fa640 1 -- 172.21.15.89:0/746028524 <== osd.4 v2:172.21.15.154:6800/2025560293 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f2ee8009ad0 con 0x7f2ed006bff0 2023-10-18T11:03:53.077 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-18T11:03:53.077 INFO:teuthology.orchestra.run.smithi089.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-18T11:03:53.077 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-18T11:03:53.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.079+0000 7f2ef6d8f640 1 -- 172.21.15.89:0/746028524 >> [v2:172.21.15.154:6800/2025560293,v1:172.21.15.154:6801/2025560293] conn(0x7f2ed006bff0 msgr2=0x7f2ed006e430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:53.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.079+0000 7f2ef6d8f640 1 --2- 172.21.15.89:0/746028524 >> [v2:172.21.15.154:6800/2025560293,v1:172.21.15.154:6801/2025560293] conn(0x7f2ed006bff0 0x7f2ed006e430 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:53.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.079+0000 7f2ef6d8f640 1 -- 172.21.15.89:0/746028524 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2ed0065680 msgr2=0x7f2ed0067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:53.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.079+0000 7f2ef6d8f640 1 --2- 172.21.15.89:0/746028524 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2ed0065680 0x7f2ed0067b40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0167760 tx=0x7f2ee0008040 comp rx=0 tx=0).stop 2023-10-18T11:03:53.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.079+0000 7f2ef6d8f640 1 -- 172.21.15.89:0/746028524 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ef00d2600 msgr2=0x7f2ef0155a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:53.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.079+0000 7f2ef6d8f640 1 --2- 172.21.15.89:0/746028524 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ef00d2600 0x7f2ef0155a70 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2ed801cc50 tx=0x7f2ed8019150 comp rx=0 tx=0).stop 2023-10-18T11:03:53.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.079+0000 7f2ef532c640 1 -- 172.21.15.89:0/746028524 reap_dead start 2023-10-18T11:03:53.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.079+0000 7f2ef6d8f640 1 -- 172.21.15.89:0/746028524 shutdown_connections 2023-10-18T11:03:53.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.079+0000 7f2ef6d8f640 1 -- 172.21.15.89:0/746028524 >> 172.21.15.89:0/746028524 conn(0x7f2ef008c0b0 msgr2=0x7f2ef00b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:53.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.079+0000 7f2ef6d8f640 1 -- 172.21.15.89:0/746028524 shutdown_connections 2023-10-18T11:03:53.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.079+0000 7f2ef6d8f640 1 -- 172.21.15.89:0/746028524 wait complete. 2023-10-18T11:03:53.095 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-18T11:03:53.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.235+0000 7fca0e967640 1 Processor -- start 2023-10-18T11:03:53.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.235+0000 7fca0e967640 1 -- start start 2023-10-18T11:03:53.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.235+0000 7fca0e967640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca08070600 0x7fca0806b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:53.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.235+0000 7fca0e967640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fca0806bd10 0x7fca080c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:53.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.235+0000 7fca0e967640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca08070b90 con 0x7fca08159490 2023-10-18T11:03:53.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.235+0000 7fca0e967640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca0806c1a0 con 0x7fca0806bd10 2023-10-18T11:03:53.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.235+0000 7fca0e967640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca08070e70 con 0x7fca08070600 2023-10-18T11:03:53.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.235+0000 7fca0cf04640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fca0806bd10 0x7fca080c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:53.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.239+0000 7fca0cf04640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fca0806bd10 0x7fca080c8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51544/0 (socket says 172.21.15.89:51544) 2023-10-18T11:03:53.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.239+0000 7fca0cf04640 1 -- 172.21.15.89:0/3391398741 learned_addr learned my addr 172.21.15.89:0/3391398741 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:53.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.239+0000 7fca077fe640 1 --2- 172.21.15.89:0/3391398741 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca08070600 0x7fca0806b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:53.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.239+0000 7fca0cf04640 1 -- 172.21.15.89:0/3391398741 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca08070600 msgr2=0x7fca0806b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:53.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.239+0000 7fca0cf04640 1 --2- 172.21.15.89:0/3391398741 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca08070600 0x7fca0806b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:53.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.239+0000 7fca0cf04640 1 -- 172.21.15.89:0/3391398741 >> v1:172.21.15.154:6789/0 conn(0x7fca08159490 legacy=0x7fca0815d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:53.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.239+0000 7fca0cf04640 1 -- 172.21.15.89:0/3391398741 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca08071150 con 0x7fca0806bd10 2023-10-18T11:03:53.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.239+0000 7fca0cf04640 1 --2- 172.21.15.89:0/3391398741 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca0806bd10 0x7fca080c8960 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fca00009e50 tx=0x7fca0001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a71a1e7cab6baa server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:53.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.239+0000 7fca06ffd640 1 -- 172.21.15.89:0/3391398741 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca000a5810 con 0x7fca0806bd10 2023-10-18T11:03:53.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.239+0000 7fca06ffd640 1 -- 172.21.15.89:0/3391398741 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca000a5970 con 0x7fca0806bd10 2023-10-18T11:03:53.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.239+0000 7fca06ffd640 1 -- 172.21.15.89:0/3391398741 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca000a5d00 con 0x7fca0806bd10 2023-10-18T11:03:53.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.239+0000 7fca0e967640 1 -- 172.21.15.89:0/3391398741 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca0806bd10 msgr2=0x7fca080c8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:53.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.239+0000 7fca0e967640 1 --2- 172.21.15.89:0/3391398741 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca0806bd10 0x7fca080c8960 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fca00009e50 tx=0x7fca0001c950 comp rx=0 tx=0).stop 2023-10-18T11:03:53.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.243+0000 7fca0e967640 1 -- 172.21.15.89:0/3391398741 shutdown_connections 2023-10-18T11:03:53.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.243+0000 7fca0e967640 1 --2- 172.21.15.89:0/3391398741 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca0806bd10 0x7fca080c8960 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:53.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.243+0000 7fca0e967640 1 --2- 172.21.15.89:0/3391398741 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca08070600 0x7fca0806b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:53.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.243+0000 7fca0e967640 1 -- 172.21.15.89:0/3391398741 >> 172.21.15.89:0/3391398741 conn(0x7fca08069f00 msgr2=0x7fca080cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:53.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.243+0000 7fca0e967640 1 -- 172.21.15.89:0/3391398741 shutdown_connections 2023-10-18T11:03:53.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.243+0000 7fca0e967640 1 -- 172.21.15.89:0/3391398741 wait complete. 2023-10-18T11:03:53.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.243+0000 7fca0e967640 1 Processor -- start 2023-10-18T11:03:53.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.243+0000 7fca0e967640 1 -- start start 2023-10-18T11:03:53.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.243+0000 7fca0e967640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca0806bd10 0x7fca080c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:53.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.243+0000 7fca0e967640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca08070600 0x7fca080b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:53.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.243+0000 7fca0e967640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca08159490 0x7fca08154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:53.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.243+0000 7fca0e967640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca080c38c0 con 0x7fca08070600 2023-10-18T11:03:53.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.243+0000 7fca0e967640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca080c6b20 con 0x7fca0806bd10 2023-10-18T11:03:53.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.243+0000 7fca0e967640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca080c82d0 con 0x7fca08159490 2023-10-18T11:03:53.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.243+0000 7fca077fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca08070600 0x7fca080b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:53.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.243+0000 7fca077fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca08070600 0x7fca080b0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51558/0 (socket says 172.21.15.89:51558) 2023-10-18T11:03:53.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.243+0000 7fca077fe640 1 -- 172.21.15.89:0/2353422580 learned_addr learned my addr 172.21.15.89:0/2353422580 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:53.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.243+0000 7fca07fff640 1 --2- 172.21.15.89:0/2353422580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca0806bd10 0x7fca080c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:53.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.243+0000 7fca0cf04640 1 --2- 172.21.15.89:0/2353422580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca08159490 0x7fca08154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:53.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.243+0000 7fca077fe640 1 -- 172.21.15.89:0/2353422580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca08159490 msgr2=0x7fca08154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:53.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.243+0000 7fca077fe640 1 --2- 172.21.15.89:0/2353422580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca08159490 0x7fca08154680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:53.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.243+0000 7fca077fe640 1 -- 172.21.15.89:0/2353422580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca0806bd10 msgr2=0x7fca080c65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:53.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.243+0000 7fca077fe640 1 --2- 172.21.15.89:0/2353422580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca0806bd10 0x7fca080c65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:53.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.243+0000 7fca077fe640 1 -- 172.21.15.89:0/2353422580 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca00009b30 con 0x7fca08070600 2023-10-18T11:03:53.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.243+0000 7fca0cf04640 1 --2- 172.21.15.89:0/2353422580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca08159490 0x7fca08154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:03:53.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.247+0000 7fca077fe640 1 --2- 172.21.15.89:0/2353422580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca08070600 0x7fca080b0060 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc00d800 tx=0x7fc9fc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:53.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.247+0000 7fca04ff9640 1 -- 172.21.15.89:0/2353422580 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc00b6c0 con 0x7fca08070600 2023-10-18T11:03:53.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.247+0000 7fca04ff9640 1 -- 172.21.15.89:0/2353422580 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9fc00b820 con 0x7fca08070600 2023-10-18T11:03:53.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.247+0000 7fca04ff9640 1 -- 172.21.15.89:0/2353422580 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc00bb90 con 0x7fca08070600 2023-10-18T11:03:53.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.247+0000 7fca0e967640 1 -- 172.21.15.89:0/2353422580 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca080c85b0 con 0x7fca08070600 2023-10-18T11:03:53.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.247+0000 7fca0e967640 1 -- 172.21.15.89:0/2353422580 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca080c34e0 con 0x7fca08070600 2023-10-18T11:03:53.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.247+0000 7fca0e967640 1 -- 172.21.15.89:0/2353422580 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca0815d6c0 con 0x7fca08070600 2023-10-18T11:03:53.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.247+0000 7fca04ff9640 1 -- 172.21.15.89:0/2353422580 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fc9fc00bcf0 con 0x7fca08070600 2023-10-18T11:03:53.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.247+0000 7fca04ff9640 1 --2- 172.21.15.89:0/2353422580 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc9f4065680 0x7fc9f4067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:53.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.247+0000 7fca04ff9640 1 -- 172.21.15.89:0/2353422580 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fc9fc083150 con 0x7fca08070600 2023-10-18T11:03:53.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.247+0000 7fca07fff640 1 --2- 172.21.15.89:0/2353422580 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc9f4065680 0x7fc9f4067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:53.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.251+0000 7fca07fff640 1 --2- 172.21.15.89:0/2353422580 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc9f4065680 0x7fc9f4067b40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8007fd0 tx=0x7fc9f8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:53.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.251+0000 7fca04ff9640 1 -- 172.21.15.89:0/2353422580 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fc9fc054820 con 0x7fca08070600 2023-10-18T11:03:53.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.403+0000 7fca0e967640 1 -- 172.21.15.89:0/2353422580 --> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fca080cda60 con 0x7fc9f4065680 2023-10-18T11:03:53.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.403+0000 7fca04ff9640 1 -- 172.21.15.89:0/2353422580 <== mgr.4099 v2:172.21.15.154:6832/1086608968 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc9f8001450 con 0x7fc9f4065680 2023-10-18T11:03:53.402 INFO:teuthology.orchestra.run.smithi089.stderr:instructed osd(s) 4 to deep-scrub 2023-10-18T11:03:53.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.403+0000 7fca0e967640 1 -- 172.21.15.89:0/2353422580 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc9f4065680 msgr2=0x7fc9f4067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:53.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.403+0000 7fca0e967640 1 --2- 172.21.15.89:0/2353422580 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc9f4065680 0x7fc9f4067b40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8007fd0 tx=0x7fc9f8009040 comp rx=0 tx=0).stop 2023-10-18T11:03:53.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.403+0000 7fca0e967640 1 -- 172.21.15.89:0/2353422580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca08070600 msgr2=0x7fca080b0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:53.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.403+0000 7fca0e967640 1 --2- 172.21.15.89:0/2353422580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca08070600 0x7fca080b0060 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc00d800 tx=0x7fc9fc00dcc0 comp rx=0 tx=0).stop 2023-10-18T11:03:53.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.403+0000 7fca0e967640 1 -- 172.21.15.89:0/2353422580 shutdown_connections 2023-10-18T11:03:53.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.403+0000 7fca0e967640 1 --2- 172.21.15.89:0/2353422580 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc9f4065680 0x7fc9f4067b40 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:53.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.403+0000 7fca0e967640 1 --2- 172.21.15.89:0/2353422580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fca08159490 0x7fca08154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:53.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.403+0000 7fca0e967640 1 --2- 172.21.15.89:0/2353422580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca08070600 0x7fca080b0060 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:53.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.403+0000 7fca0e967640 1 --2- 172.21.15.89:0/2353422580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca0806bd10 0x7fca080c65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:53.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.403+0000 7fca0e967640 1 -- 172.21.15.89:0/2353422580 >> 172.21.15.89:0/2353422580 conn(0x7fca08069f00 msgr2=0x7fca080aec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:53.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.403+0000 7fca0e967640 1 -- 172.21.15.89:0/2353422580 shutdown_connections 2023-10-18T11:03:53.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.403+0000 7fca0e967640 1 -- 172.21.15.89:0/2353422580 wait complete. 2023-10-18T11:03:53.423 INFO:tasks.ceph:Scrubbing osd.5 2023-10-18T11:03:53.424 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-18T11:03:53.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.567+0000 7f469aee6640 1 Processor -- start 2023-10-18T11:03:53.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.567+0000 7f469aee6640 1 -- start start 2023-10-18T11:03:53.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.567+0000 7f469aee6640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f469415c950 0x7f469406a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:53.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.567+0000 7f469aee6640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46940af2b0 0x7f46940af700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:53.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.567+0000 7f469aee6640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46940d0c60 con 0x7f46940d2600 2023-10-18T11:03:53.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.567+0000 7f469aee6640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f469415cd30 con 0x7f469415c950 2023-10-18T11:03:53.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.567+0000 7f469aee6640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f469406a850 con 0x7f46940af2b0 2023-10-18T11:03:53.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.567+0000 7f4698c82640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f469415c950 0x7f469406a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:53.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.567+0000 7f4698c82640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f469415c950 0x7f469406a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51568/0 (socket says 172.21.15.89:51568) 2023-10-18T11:03:53.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.567+0000 7f4698c82640 1 -- 172.21.15.89:0/3363539823 learned_addr learned my addr 172.21.15.89:0/3363539823 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:53.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.567+0000 7f4698c82640 1 -- 172.21.15.89:0/3363539823 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46940af2b0 msgr2=0x7f46940af700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:53.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.567+0000 7f4698c82640 1 --2- 172.21.15.89:0/3363539823 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46940af2b0 0x7f46940af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:53.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.567+0000 7f4698c82640 1 -- 172.21.15.89:0/3363539823 >> v1:172.21.15.154:6789/0 conn(0x7f46940d2600 legacy=0x7f46940d2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:53.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.567+0000 7f4698c82640 1 -- 172.21.15.89:0/3363539823 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46940d02c0 con 0x7f469415c950 2023-10-18T11:03:53.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.571+0000 7f4698c82640 1 --2- 172.21.15.89:0/3363539823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f469415c950 0x7f469406a310 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f4684006fd0 tx=0x7f468401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb77e17aa90c3e1b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:53.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.571+0000 7f46937fe640 1 -- 172.21.15.89:0/3363539823 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4684004230 con 0x7f469415c950 2023-10-18T11:03:53.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.571+0000 7f46937fe640 1 -- 172.21.15.89:0/3363539823 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46840a7050 con 0x7f469415c950 2023-10-18T11:03:53.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.571+0000 7f46937fe640 1 -- 172.21.15.89:0/3363539823 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4684003620 con 0x7f469415c950 2023-10-18T11:03:53.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.571+0000 7f469aee6640 1 -- 172.21.15.89:0/3363539823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f469415c950 msgr2=0x7f469406a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:53.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.571+0000 7f469aee6640 1 --2- 172.21.15.89:0/3363539823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f469415c950 0x7f469406a310 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f4684006fd0 tx=0x7f468401c390 comp rx=0 tx=0).stop 2023-10-18T11:03:53.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.571+0000 7f469aee6640 1 -- 172.21.15.89:0/3363539823 shutdown_connections 2023-10-18T11:03:53.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.571+0000 7f469aee6640 1 --2- 172.21.15.89:0/3363539823 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46940af2b0 0x7f46940af700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:53.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.571+0000 7f469aee6640 1 --2- 172.21.15.89:0/3363539823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f469415c950 0x7f469406a310 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:53.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.571+0000 7f469aee6640 1 -- 172.21.15.89:0/3363539823 >> 172.21.15.89:0/3363539823 conn(0x7f469408c0b0 msgr2=0x7f469409b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:53.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.571+0000 7f469aee6640 1 -- 172.21.15.89:0/3363539823 shutdown_connections 2023-10-18T11:03:53.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.571+0000 7f469aee6640 1 -- 172.21.15.89:0/3363539823 wait complete. 2023-10-18T11:03:53.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.575+0000 7f469aee6640 1 Processor -- start 2023-10-18T11:03:53.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.575+0000 7f469aee6640 1 -- start start 2023-10-18T11:03:53.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.575+0000 7f469aee6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46940af2b0 0x7f46940c72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:53.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.575+0000 7f469aee6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46940d2600 0x7f46940c6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:53.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.575+0000 7f469aee6640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f469415c950 0x7f46940cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:53.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.575+0000 7f469aee6640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46940cf440 con 0x7f46940af2b0 2023-10-18T11:03:53.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.575+0000 7f469aee6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46940cee30 con 0x7f46940d2600 2023-10-18T11:03:53.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.575+0000 7f469aee6640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46940ce800 con 0x7f469415c950 2023-10-18T11:03:53.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.575+0000 7f4698c82640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46940af2b0 0x7f46940c72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:53.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.575+0000 7f4698c82640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46940af2b0 0x7f46940c72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51582/0 (socket says 172.21.15.89:51582) 2023-10-18T11:03:53.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.575+0000 7f4698c82640 1 -- 172.21.15.89:0/349029204 learned_addr learned my addr 172.21.15.89:0/349029204 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:53.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.575+0000 7f4699483640 1 --2- 172.21.15.89:0/349029204 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f469415c950 0x7f46940cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:53.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.575+0000 7f4693fff640 1 --2- 172.21.15.89:0/349029204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46940d2600 0x7f46940c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:53.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.575+0000 7f4698c82640 1 -- 172.21.15.89:0/349029204 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f469415c950 msgr2=0x7f46940cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:53.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.575+0000 7f4698c82640 1 --2- 172.21.15.89:0/349029204 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f469415c950 0x7f46940cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:53.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.575+0000 7f4698c82640 1 -- 172.21.15.89:0/349029204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46940d2600 msgr2=0x7f46940c6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:53.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.575+0000 7f4698c82640 1 --2- 172.21.15.89:0/349029204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46940d2600 0x7f46940c6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:53.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.575+0000 7f4698c82640 1 -- 172.21.15.89:0/349029204 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4684009480 con 0x7f46940af2b0 2023-10-18T11:03:53.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.575+0000 7f4698c82640 1 --2- 172.21.15.89:0/349029204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46940af2b0 0x7f46940c72d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4684006fd0 tx=0x7f4684019130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:53.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.575+0000 7f46917fa640 1 -- 172.21.15.89:0/349029204 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4684003c70 con 0x7f46940af2b0 2023-10-18T11:03:53.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.575+0000 7f46917fa640 1 -- 172.21.15.89:0/349029204 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4684003dd0 con 0x7f46940af2b0 2023-10-18T11:03:53.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.575+0000 7f46917fa640 1 -- 172.21.15.89:0/349029204 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4684005be0 con 0x7f46940af2b0 2023-10-18T11:03:53.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.575+0000 7f469aee6640 1 -- 172.21.15.89:0/349029204 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46940cf0d0 con 0x7f46940af2b0 2023-10-18T11:03:53.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.575+0000 7f469aee6640 1 -- 172.21.15.89:0/349029204 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46940c7810 con 0x7f46940af2b0 2023-10-18T11:03:53.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.575+0000 7f469aee6640 1 -- 172.21.15.89:0/349029204 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f46940cecb0 con 0x7f46940af2b0 2023-10-18T11:03:53.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.579+0000 7f46917fa640 1 -- 172.21.15.89:0/349029204 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f4684005d40 con 0x7f46940af2b0 2023-10-18T11:03:53.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.579+0000 7f46917fa640 1 --2- 172.21.15.89:0/349029204 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4680065680 0x7f4680067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:53.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.579+0000 7f46917fa640 1 -- 172.21.15.89:0/349029204 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f468411d140 con 0x7f46940af2b0 2023-10-18T11:03:53.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.579+0000 7f46917fa640 1 --2- 172.21.15.89:0/349029204 >> [v2:172.21.15.154:6808/357281174,v1:172.21.15.154:6809/357281174] conn(0x7f468006bff0 0x7f468006e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:53.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.579+0000 7f46917fa640 1 -- 172.21.15.89:0/349029204 --> [v2:172.21.15.154:6808/357281174,v1:172.21.15.154:6809/357281174] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f468006eb00 con 0x7f468006bff0 2023-10-18T11:03:53.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.579+0000 7f46917fa640 1 -- 172.21.15.89:0/349029204 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f468411f720 con 0x7f46940af2b0 2023-10-18T11:03:53.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.579+0000 7f4693fff640 1 --2- 172.21.15.89:0/349029204 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4680065680 0x7f4680067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:53.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.579+0000 7f4699483640 1 --2- 172.21.15.89:0/349029204 >> [v2:172.21.15.154:6808/357281174,v1:172.21.15.154:6809/357281174] conn(0x7f468006bff0 0x7f468006e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:53.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.579+0000 7f4693fff640 1 --2- 172.21.15.89:0/349029204 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4680065680 0x7f4680067b40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f46941f8310 tx=0x7f4688009060 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:53.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.579+0000 7f4699483640 1 --2- 172.21.15.89:0/349029204 >> [v2:172.21.15.154:6808/357281174,v1:172.21.15.154:6809/357281174] conn(0x7f468006bff0 0x7f468006e430 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:53.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.579+0000 7f46917fa640 1 -- 172.21.15.89:0/349029204 <== osd.5 v2:172.21.15.154:6808/357281174 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f468006eb00 con 0x7f468006bff0 2023-10-18T11:03:53.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.595+0000 7f469aee6640 1 -- 172.21.15.89:0/349029204 --> [v2:172.21.15.154:6808/357281174,v1:172.21.15.154:6809/357281174] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f46940d1190 con 0x7f468006bff0 2023-10-18T11:03:53.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.595+0000 7f46917fa640 1 -- 172.21.15.89:0/349029204 <== osd.5 v2:172.21.15.154:6808/357281174 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f468c00a4e0 con 0x7f468006bff0 2023-10-18T11:03:53.594 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-18T11:03:53.594 INFO:teuthology.orchestra.run.smithi089.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-18T11:03:53.594 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-18T11:03:53.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.595+0000 7f469aee6640 1 -- 172.21.15.89:0/349029204 >> [v2:172.21.15.154:6808/357281174,v1:172.21.15.154:6809/357281174] conn(0x7f468006bff0 msgr2=0x7f468006e430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:53.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.595+0000 7f469aee6640 1 --2- 172.21.15.89:0/349029204 >> [v2:172.21.15.154:6808/357281174,v1:172.21.15.154:6809/357281174] conn(0x7f468006bff0 0x7f468006e430 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:53.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.595+0000 7f469aee6640 1 -- 172.21.15.89:0/349029204 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4680065680 msgr2=0x7f4680067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:53.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.595+0000 7f469aee6640 1 --2- 172.21.15.89:0/349029204 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4680065680 0x7f4680067b40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f46941f8310 tx=0x7f4688009060 comp rx=0 tx=0).stop 2023-10-18T11:03:53.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.595+0000 7f469aee6640 1 -- 172.21.15.89:0/349029204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46940af2b0 msgr2=0x7f46940c72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:53.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.595+0000 7f469aee6640 1 --2- 172.21.15.89:0/349029204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46940af2b0 0x7f46940c72d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4684006fd0 tx=0x7f4684019130 comp rx=0 tx=0).stop 2023-10-18T11:03:53.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.595+0000 7f4699483640 1 -- 172.21.15.89:0/349029204 reap_dead start 2023-10-18T11:03:53.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.595+0000 7f469aee6640 1 -- 172.21.15.89:0/349029204 shutdown_connections 2023-10-18T11:03:53.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.595+0000 7f469aee6640 1 -- 172.21.15.89:0/349029204 >> 172.21.15.89:0/349029204 conn(0x7f469408c0b0 msgr2=0x7f4694069900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:53.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.595+0000 7f469aee6640 1 -- 172.21.15.89:0/349029204 shutdown_connections 2023-10-18T11:03:53.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.595+0000 7f469aee6640 1 -- 172.21.15.89:0/349029204 wait complete. 2023-10-18T11:03:53.612 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-18T11:03:53.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.751+0000 7fc1a57e1640 1 Processor -- start 2023-10-18T11:03:53.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.751+0000 7fc1a57e1640 1 -- start start 2023-10-18T11:03:53.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.751+0000 7fc1a57e1640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc1a015c940 0x7fc1a00d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:53.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.751+0000 7fc1a57e1640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc1a00d0e70 0x7fc1a00cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:53.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.751+0000 7fc1a57e1640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1a015da50 con 0x7fc1a00d2f40 2023-10-18T11:03:53.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.751+0000 7fc1a57e1640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1a015ce40 con 0x7fc1a00d0e70 2023-10-18T11:03:53.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.751+0000 7fc1a57e1640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1a015d120 con 0x7fc1a015c940 2023-10-18T11:03:53.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.751+0000 7fc1967fc640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc1a00d2f40 0x7fc1a00d0760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:46414/0 (socket says 172.21.15.89:46414) 2023-10-18T11:03:53.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.751+0000 7fc1967fc640 1 -- 172.21.15.89:0/1117454958 learned_addr learned my addr 172.21.15.89:0/1117454958 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:53.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.751+0000 7fc19f7fe640 1 --2- 172.21.15.89:0/1117454958 >> v2:172.21.15.89:3300/0 conn(0x7fc1a00d0e70 0x7fc1a00cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:53.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.751+0000 7fc19effd640 1 --2- 172.21.15.89:0/1117454958 >> v2:172.21.15.154:3300/0 conn(0x7fc1a015c940 0x7fc1a00d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:53.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.755+0000 7fc19e7fc640 1 -- 172.21.15.89:0/1117454958 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 313538024 0 0) 0x7fc1a015da50 con 0x7fc1a00d2f40 2023-10-18T11:03:53.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.755+0000 7fc19e7fc640 1 -- 172.21.15.89:0/1117454958 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc18c003530 con 0x7fc1a00d2f40 2023-10-18T11:03:53.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.755+0000 7fc19f7fe640 1 -- 172.21.15.89:0/1117454958 >> v2:172.21.15.154:3300/0 conn(0x7fc1a015c940 msgr2=0x7fc1a00d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:53.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.755+0000 7fc19f7fe640 1 --2- 172.21.15.89:0/1117454958 >> v2:172.21.15.154:3300/0 conn(0x7fc1a015c940 0x7fc1a00d2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:53.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.755+0000 7fc19f7fe640 1 -- 172.21.15.89:0/1117454958 >> v1:172.21.15.89:6789/0 conn(0x7fc1a00d2f40 legacy=0x7fc1a00d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:53.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.755+0000 7fc19f7fe640 1 -- 172.21.15.89:0/1117454958 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a015d400 con 0x7fc1a00d0e70 2023-10-18T11:03:53.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.755+0000 7fc19f7fe640 1 --2- 172.21.15.89:0/1117454958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1a00d0e70 0x7fc1a00cd160 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fc1a00725b0 tx=0x7fc19801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d80d028d1db5ce server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:53.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.755+0000 7fc19e7fc640 1 -- 172.21.15.89:0/1117454958 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1980a5940 con 0x7fc1a00d0e70 2023-10-18T11:03:53.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.755+0000 7fc19e7fc640 1 -- 172.21.15.89:0/1117454958 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1980a5aa0 con 0x7fc1a00d0e70 2023-10-18T11:03:53.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.755+0000 7fc19e7fc640 1 -- 172.21.15.89:0/1117454958 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1980a5e30 con 0x7fc1a00d0e70 2023-10-18T11:03:53.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.755+0000 7fc1a57e1640 1 -- 172.21.15.89:0/1117454958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1a00d0e70 msgr2=0x7fc1a00cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:53.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.755+0000 7fc1a57e1640 1 --2- 172.21.15.89:0/1117454958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1a00d0e70 0x7fc1a00cd160 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fc1a00725b0 tx=0x7fc19801c8e0 comp rx=0 tx=0).stop 2023-10-18T11:03:53.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.759+0000 7fc1a57e1640 1 -- 172.21.15.89:0/1117454958 shutdown_connections 2023-10-18T11:03:53.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.759+0000 7fc1a57e1640 1 --2- 172.21.15.89:0/1117454958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1a00d0e70 0x7fc1a00cd160 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:53.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.759+0000 7fc1a57e1640 1 --2- 172.21.15.89:0/1117454958 >> v2:172.21.15.154:3300/0 conn(0x7fc1a015c940 0x7fc1a00d2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:53.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.759+0000 7fc1a57e1640 1 -- 172.21.15.89:0/1117454958 >> 172.21.15.89:0/1117454958 conn(0x7fc1a0089fd0 msgr2=0x7fc1a009b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:53.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.759+0000 7fc1a57e1640 1 -- 172.21.15.89:0/1117454958 shutdown_connections 2023-10-18T11:03:53.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.759+0000 7fc1a57e1640 1 -- 172.21.15.89:0/1117454958 wait complete. 2023-10-18T11:03:53.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.759+0000 7fc1a57e1640 1 Processor -- start 2023-10-18T11:03:53.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.759+0000 7fc1a57e1640 1 -- start start 2023-10-18T11:03:53.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.759+0000 7fc1a57e1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1a00d0e70 0x7fc1a00c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:53.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.759+0000 7fc1a57e1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1a00c8a90 0x7fc1a0155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:53.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.759+0000 7fc1a57e1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1a015f840 0x7fc1a015fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:53.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.759+0000 7fc1a57e1640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a0160200 con 0x7fc1a00c8a90 2023-10-18T11:03:53.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.759+0000 7fc1a57e1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a00c70d0 con 0x7fc1a015f840 2023-10-18T11:03:53.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.759+0000 7fc1a57e1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a00c7710 con 0x7fc1a00d0e70 2023-10-18T11:03:53.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.759+0000 7fc1967fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1a00c8a90 0x7fc1a0155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:53.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.759+0000 7fc1967fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1a00c8a90 0x7fc1a0155510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51594/0 (socket says 172.21.15.89:51594) 2023-10-18T11:03:53.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.759+0000 7fc1967fc640 1 -- 172.21.15.89:0/95338375 learned_addr learned my addr 172.21.15.89:0/95338375 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:53.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.759+0000 7fc19effd640 1 --2- 172.21.15.89:0/95338375 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1a00d0e70 0x7fc1a00c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:53.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.759+0000 7fc19f7fe640 1 --2- 172.21.15.89:0/95338375 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1a015f840 0x7fc1a015fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:53.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.759+0000 7fc1967fc640 1 -- 172.21.15.89:0/95338375 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1a00d0e70 msgr2=0x7fc1a00c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:53.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.759+0000 7fc1967fc640 1 --2- 172.21.15.89:0/95338375 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1a00d0e70 0x7fc1a00c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:53.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.759+0000 7fc1967fc640 1 -- 172.21.15.89:0/95338375 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1a015f840 msgr2=0x7fc1a015fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:53.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.759+0000 7fc1967fc640 1 --2- 172.21.15.89:0/95338375 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1a015f840 0x7fc1a015fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:53.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.759+0000 7fc1967fc640 1 -- 172.21.15.89:0/95338375 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc198009a70 con 0x7fc1a00c8a90 2023-10-18T11:03:53.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.759+0000 7fc19f7fe640 1 --2- 172.21.15.89:0/95338375 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1a015f840 0x7fc1a015fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:03:53.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.759+0000 7fc19effd640 1 --2- 172.21.15.89:0/95338375 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1a00d0e70 0x7fc1a00c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:03:53.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.759+0000 7fc1967fc640 1 --2- 172.21.15.89:0/95338375 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1a00c8a90 0x7fc1a0155510 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fc18800c4d0 tx=0x7fc18800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:53.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.763+0000 7fc197fff640 1 -- 172.21.15.89:0/95338375 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc188005cf0 con 0x7fc1a00c8a90 2023-10-18T11:03:53.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.763+0000 7fc197fff640 1 -- 172.21.15.89:0/95338375 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc188005e50 con 0x7fc1a00c8a90 2023-10-18T11:03:53.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.763+0000 7fc197fff640 1 -- 172.21.15.89:0/95338375 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc188007270 con 0x7fc1a00c8a90 2023-10-18T11:03:53.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.763+0000 7fc1a57e1640 1 -- 172.21.15.89:0/95338375 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1a00c79f0 con 0x7fc1a00c8a90 2023-10-18T11:03:53.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.763+0000 7fc1a57e1640 1 -- 172.21.15.89:0/95338375 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a00c7fb0 con 0x7fc1a00c8a90 2023-10-18T11:03:53.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.763+0000 7fc1a57e1640 1 -- 172.21.15.89:0/95338375 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1a004dbe0 con 0x7fc1a00c8a90 2023-10-18T11:03:53.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.763+0000 7fc197fff640 1 -- 172.21.15.89:0/95338375 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fc1880073d0 con 0x7fc1a00c8a90 2023-10-18T11:03:53.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.763+0000 7fc197fff640 1 --2- 172.21.15.89:0/95338375 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc180065680 0x7fc180067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:53.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.763+0000 7fc197fff640 1 -- 172.21.15.89:0/95338375 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fc1880821e0 con 0x7fc1a00c8a90 2023-10-18T11:03:53.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.763+0000 7fc19effd640 1 --2- 172.21.15.89:0/95338375 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc180065680 0x7fc180067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:53.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.767+0000 7fc19effd640 1 --2- 172.21.15.89:0/95338375 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc180065680 0x7fc180067b40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc190009a40 tx=0x7fc190008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:53.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.767+0000 7fc197fff640 1 -- 172.21.15.89:0/95338375 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fc18804fe40 con 0x7fc1a00c8a90 2023-10-18T11:03:53.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.915+0000 7fc1a57e1640 1 -- 172.21.15.89:0/95338375 --> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1a00b85d0 con 0x7fc180065680 2023-10-18T11:03:53.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.919+0000 7fc197fff640 1 -- 172.21.15.89:0/95338375 <== mgr.4099 v2:172.21.15.154:6832/1086608968 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc1a00b85d0 con 0x7fc180065680 2023-10-18T11:03:53.917 INFO:teuthology.orchestra.run.smithi089.stderr:instructed osd(s) 5 to deep-scrub 2023-10-18T11:03:53.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.919+0000 7fc1a57e1640 1 -- 172.21.15.89:0/95338375 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc180065680 msgr2=0x7fc180067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:53.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.919+0000 7fc1a57e1640 1 --2- 172.21.15.89:0/95338375 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc180065680 0x7fc180067b40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc190009a40 tx=0x7fc190008040 comp rx=0 tx=0).stop 2023-10-18T11:03:53.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.919+0000 7fc1a57e1640 1 -- 172.21.15.89:0/95338375 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1a00c8a90 msgr2=0x7fc1a0155510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:53.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.919+0000 7fc1a57e1640 1 --2- 172.21.15.89:0/95338375 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1a00c8a90 0x7fc1a0155510 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fc18800c4d0 tx=0x7fc18800c990 comp rx=0 tx=0).stop 2023-10-18T11:03:53.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.919+0000 7fc1a57e1640 1 -- 172.21.15.89:0/95338375 shutdown_connections 2023-10-18T11:03:53.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.919+0000 7fc1a57e1640 1 --2- 172.21.15.89:0/95338375 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc180065680 0x7fc180067b40 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:53.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.919+0000 7fc1a57e1640 1 --2- 172.21.15.89:0/95338375 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1a015f840 0x7fc1a015fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:53.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.919+0000 7fc1a57e1640 1 --2- 172.21.15.89:0/95338375 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1a00c8a90 0x7fc1a0155510 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:53.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.919+0000 7fc1a57e1640 1 --2- 172.21.15.89:0/95338375 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc1a00d0e70 0x7fc1a00c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:53.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.919+0000 7fc1a57e1640 1 -- 172.21.15.89:0/95338375 >> 172.21.15.89:0/95338375 conn(0x7fc1a0089fd0 msgr2=0x7fc1a015bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:53.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.919+0000 7fc1a57e1640 1 -- 172.21.15.89:0/95338375 shutdown_connections 2023-10-18T11:03:53.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:53.919+0000 7fc1a57e1640 1 -- 172.21.15.89:0/95338375 wait complete. 2023-10-18T11:03:53.938 INFO:tasks.ceph:Scrubbing osd.6 2023-10-18T11:03:53.938 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-18T11:03:54.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.079+0000 7f20741a3640 1 Processor -- start 2023-10-18T11:03:54.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.079+0000 7f20741a3640 1 -- start start 2023-10-18T11:03:54.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.079+0000 7f20741a3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f206c161ba0 0x7f206c0c8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:54.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.079+0000 7f20741a3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f206c0afd40 0x7f206c0b01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:54.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.079+0000 7f20741a3640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f206c162010 con 0x7f206c164b80 2023-10-18T11:03:54.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.079+0000 7f20741a3640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f206c1617a0 con 0x7f206c161ba0 2023-10-18T11:03:54.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.079+0000 7f20741a3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f206c0b0710 con 0x7f206c0afd40 2023-10-18T11:03:54.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.079+0000 7f2071f3f640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f206c164b80 0x7f206c164f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:46416/0 (socket says 172.21.15.89:46416) 2023-10-18T11:03:54.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.079+0000 7f2071f3f640 1 -- 172.21.15.89:0/1999039622 learned_addr learned my addr 172.21.15.89:0/1999039622 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:54.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.079+0000 7f2072740640 1 --2- 172.21.15.89:0/1999039622 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f206c0afd40 0x7f206c0b01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:54.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.079+0000 7f207173e640 1 --2- 172.21.15.89:0/1999039622 >> v2:172.21.15.154:3300/0 conn(0x7f206c161ba0 0x7f206c0c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:54.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.083+0000 7f2070f3d640 1 -- 172.21.15.89:0/1999039622 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2289926124 0 0) 0x7f206c162010 con 0x7f206c164b80 2023-10-18T11:03:54.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.083+0000 7f2070f3d640 1 -- 172.21.15.89:0/1999039622 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2058003530 con 0x7f206c164b80 2023-10-18T11:03:54.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.083+0000 7f2072740640 1 -- 172.21.15.89:0/1999039622 >> v2:172.21.15.154:3300/0 conn(0x7f206c161ba0 msgr2=0x7f206c0c8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:54.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.083+0000 7f2072740640 1 --2- 172.21.15.89:0/1999039622 >> v2:172.21.15.154:3300/0 conn(0x7f206c161ba0 0x7f206c0c8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:54.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.083+0000 7f2072740640 1 -- 172.21.15.89:0/1999039622 >> v1:172.21.15.89:6789/0 conn(0x7f206c164b80 legacy=0x7f206c164f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:54.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.083+0000 7f2072740640 1 -- 172.21.15.89:0/1999039622 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f206c161100 con 0x7f206c0afd40 2023-10-18T11:03:54.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.083+0000 7f2072740640 1 --2- 172.21.15.89:0/1999039622 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f206c0afd40 0x7f206c0b01d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f20640097c0 tx=0x7f206401c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d057b0d398d8bdd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:54.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.083+0000 7f2070f3d640 1 -- 172.21.15.89:0/1999039622 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20640ab610 con 0x7f206c0afd40 2023-10-18T11:03:54.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.083+0000 7f2070f3d640 1 -- 172.21.15.89:0/1999039622 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20640ab770 con 0x7f206c0afd40 2023-10-18T11:03:54.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.087+0000 7f2070f3d640 1 -- 172.21.15.89:0/1999039622 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20640abae0 con 0x7f206c0afd40 2023-10-18T11:03:54.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.087+0000 7f20741a3640 1 -- 172.21.15.89:0/1999039622 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f206c0afd40 msgr2=0x7f206c0b01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:54.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.087+0000 7f20741a3640 1 --2- 172.21.15.89:0/1999039622 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f206c0afd40 0x7f206c0b01d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f20640097c0 tx=0x7f206401c3f0 comp rx=0 tx=0).stop 2023-10-18T11:03:54.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.087+0000 7f20741a3640 1 -- 172.21.15.89:0/1999039622 shutdown_connections 2023-10-18T11:03:54.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.087+0000 7f20741a3640 1 --2- 172.21.15.89:0/1999039622 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f206c0afd40 0x7f206c0b01d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:54.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.087+0000 7f20741a3640 1 --2- 172.21.15.89:0/1999039622 >> v2:172.21.15.154:3300/0 conn(0x7f206c161ba0 0x7f206c0c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:54.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.087+0000 7f20741a3640 1 -- 172.21.15.89:0/1999039622 >> 172.21.15.89:0/1999039622 conn(0x7f206c0cad40 msgr2=0x7f206c0cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:54.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.087+0000 7f20741a3640 1 -- 172.21.15.89:0/1999039622 shutdown_connections 2023-10-18T11:03:54.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.087+0000 7f20741a3640 1 -- 172.21.15.89:0/1999039622 wait complete. 2023-10-18T11:03:54.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.087+0000 7f20741a3640 1 Processor -- start 2023-10-18T11:03:54.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.087+0000 7f20741a3640 1 -- start start 2023-10-18T11:03:54.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.087+0000 7f20741a3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f206c0afd40 0x7f206c14e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:54.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.087+0000 7f20741a3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f206c161ba0 0x7f206c157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:54.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.087+0000 7f20741a3640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f206c164b80 0x7f206c1579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:54.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.087+0000 7f20741a3640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206c1f9d80 con 0x7f206c164b80 2023-10-18T11:03:54.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.087+0000 7f20741a3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206c1f9100 con 0x7f206c161ba0 2023-10-18T11:03:54.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.087+0000 7f20741a3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f206c1f93e0 con 0x7f206c0afd40 2023-10-18T11:03:54.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.087+0000 7f2072740640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f206c164b80 0x7f206c1579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:54.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.087+0000 7f2072740640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f206c164b80 0x7f206c1579c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51596/0 (socket says 172.21.15.89:51596) 2023-10-18T11:03:54.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.087+0000 7f2072740640 1 -- 172.21.15.89:0/227406087 learned_addr learned my addr 172.21.15.89:0/227406087 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:54.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.087+0000 7f2071f3f640 1 --2- 172.21.15.89:0/227406087 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f206c0afd40 0x7f206c14e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:54.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.087+0000 7f207173e640 1 --2- 172.21.15.89:0/227406087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f206c161ba0 0x7f206c157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:54.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.091+0000 7f2072740640 1 -- 172.21.15.89:0/227406087 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f206c0afd40 msgr2=0x7f206c14e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:54.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.091+0000 7f2072740640 1 --2- 172.21.15.89:0/227406087 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f206c0afd40 0x7f206c14e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:54.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.091+0000 7f2072740640 1 -- 172.21.15.89:0/227406087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f206c161ba0 msgr2=0x7f206c157480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:54.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.091+0000 7f2072740640 1 --2- 172.21.15.89:0/227406087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f206c161ba0 0x7f206c157480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:54.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.091+0000 7f2072740640 1 -- 172.21.15.89:0/227406087 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2064009420 con 0x7f206c164b80 2023-10-18T11:03:54.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.091+0000 7f2072740640 1 --2- 172.21.15.89:0/227406087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f206c164b80 0x7f206c1579c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f20640ad040 tx=0x7f206401ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:54.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.091+0000 7f20627fc640 1 -- 172.21.15.89:0/227406087 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20640bf610 con 0x7f206c164b80 2023-10-18T11:03:54.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.091+0000 7f20627fc640 1 -- 172.21.15.89:0/227406087 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20640bf770 con 0x7f206c164b80 2023-10-18T11:03:54.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.091+0000 7f20741a3640 1 -- 172.21.15.89:0/227406087 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f206c1f96c0 con 0x7f206c164b80 2023-10-18T11:03:54.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.091+0000 7f20627fc640 1 -- 172.21.15.89:0/227406087 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20640bfae0 con 0x7f206c164b80 2023-10-18T11:03:54.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.091+0000 7f20741a3640 1 -- 172.21.15.89:0/227406087 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f206c1f8de0 con 0x7f206c164b80 2023-10-18T11:03:54.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.091+0000 7f20741a3640 1 -- 172.21.15.89:0/227406087 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f206c1613c0 con 0x7f206c164b80 2023-10-18T11:03:54.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.091+0000 7f20627fc640 1 -- 172.21.15.89:0/227406087 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f20640b5400 con 0x7f206c164b80 2023-10-18T11:03:54.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.091+0000 7f20627fc640 1 --2- 172.21.15.89:0/227406087 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2050065680 0x7f2050067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:54.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.091+0000 7f20627fc640 1 -- 172.21.15.89:0/227406087 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f206411d770 con 0x7f206c164b80 2023-10-18T11:03:54.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.091+0000 7f20627fc640 1 --2- 172.21.15.89:0/227406087 >> [v2:172.21.15.154:6822/1133012599,v1:172.21.15.154:6824/1133012599] conn(0x7f205006bff0 0x7f205006e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:54.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.091+0000 7f20627fc640 1 -- 172.21.15.89:0/227406087 --> [v2:172.21.15.154:6822/1133012599,v1:172.21.15.154:6824/1133012599] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f205006eb00 con 0x7f205006bff0 2023-10-18T11:03:54.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.091+0000 7f20627fc640 1 -- 172.21.15.89:0/227406087 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f206411dae0 con 0x7f206c164b80 2023-10-18T11:03:54.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.095+0000 7f2071f3f640 1 --2- 172.21.15.89:0/227406087 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2050065680 0x7f2050067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:54.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.095+0000 7f207173e640 1 --2- 172.21.15.89:0/227406087 >> [v2:172.21.15.154:6822/1133012599,v1:172.21.15.154:6824/1133012599] conn(0x7f205006bff0 0x7f205006e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:54.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.095+0000 7f2071f3f640 1 --2- 172.21.15.89:0/227406087 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2050065680 0x7f2050067b40 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2068005c10 tx=0x7f2068009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:54.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.095+0000 7f207173e640 1 --2- 172.21.15.89:0/227406087 >> [v2:172.21.15.154:6822/1133012599,v1:172.21.15.154:6824/1133012599] conn(0x7f205006bff0 0x7f205006e430 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:54.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.095+0000 7f20627fc640 1 -- 172.21.15.89:0/227406087 <== osd.6 v2:172.21.15.154:6822/1133012599 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f205006eb00 con 0x7f205006bff0 2023-10-18T11:03:54.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.107+0000 7f20741a3640 1 -- 172.21.15.89:0/227406087 --> [v2:172.21.15.154:6822/1133012599,v1:172.21.15.154:6824/1133012599] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f206c1f9ec0 con 0x7f205006bff0 2023-10-18T11:03:54.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.107+0000 7f20627fc640 1 -- 172.21.15.89:0/227406087 <== osd.6 v2:172.21.15.154:6822/1133012599 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+757 (crc 0 0 0) 0x7f206c1f9ec0 con 0x7f205006bff0 2023-10-18T11:03:54.107 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-18T11:03:54.108 INFO:teuthology.orchestra.run.smithi089.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_pg_delete_cost = '15728640' (not observed, change may require restart) osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2023-10-18T11:03:54.108 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-18T11:03:54.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.107+0000 7f20741a3640 1 -- 172.21.15.89:0/227406087 >> [v2:172.21.15.154:6822/1133012599,v1:172.21.15.154:6824/1133012599] conn(0x7f205006bff0 msgr2=0x7f205006e430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:54.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.107+0000 7f20741a3640 1 --2- 172.21.15.89:0/227406087 >> [v2:172.21.15.154:6822/1133012599,v1:172.21.15.154:6824/1133012599] conn(0x7f205006bff0 0x7f205006e430 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:54.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.107+0000 7f20741a3640 1 -- 172.21.15.89:0/227406087 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2050065680 msgr2=0x7f2050067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:54.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.107+0000 7f20741a3640 1 --2- 172.21.15.89:0/227406087 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f2050065680 0x7f2050067b40 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2068005c10 tx=0x7f2068009040 comp rx=0 tx=0).stop 2023-10-18T11:03:54.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.107+0000 7f20741a3640 1 -- 172.21.15.89:0/227406087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f206c164b80 msgr2=0x7f206c1579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:54.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.107+0000 7f20741a3640 1 --2- 172.21.15.89:0/227406087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f206c164b80 0x7f206c1579c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f20640ad040 tx=0x7f206401ac30 comp rx=0 tx=0).stop 2023-10-18T11:03:54.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.107+0000 7f2072740640 1 -- 172.21.15.89:0/227406087 reap_dead start 2023-10-18T11:03:54.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.107+0000 7f20741a3640 1 -- 172.21.15.89:0/227406087 shutdown_connections 2023-10-18T11:03:54.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.107+0000 7f20741a3640 1 -- 172.21.15.89:0/227406087 >> 172.21.15.89:0/227406087 conn(0x7f206c0cad40 msgr2=0x7f206c0ce250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:54.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.107+0000 7f20741a3640 1 -- 172.21.15.89:0/227406087 shutdown_connections 2023-10-18T11:03:54.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.107+0000 7f20741a3640 1 -- 172.21.15.89:0/227406087 wait complete. 2023-10-18T11:03:54.125 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-18T11:03:54.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.263+0000 7fec527ba640 1 Processor -- start 2023-10-18T11:03:54.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec527ba640 1 -- start start 2023-10-18T11:03:54.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec527ba640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fec4c159490 0x7fec4c15d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:54.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec527ba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec4c06bee0 0x7fec4c0c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:54.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec527ba640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec4c15ddf0 con 0x7fec4c070600 2023-10-18T11:03:54.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec527ba640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec4c070b90 con 0x7fec4c159490 2023-10-18T11:03:54.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec527ba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c070e70 con 0x7fec4c06bee0 2023-10-18T11:03:54.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec4b7fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fec4c070600 0x7fec4c06b7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:46424/0 (socket says 172.21.15.89:46424) 2023-10-18T11:03:54.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec4b7fe640 1 -- 172.21.15.89:0/3016794407 learned_addr learned my addr 172.21.15.89:0/3016794407 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:54.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec4bfff640 1 --2- 172.21.15.89:0/3016794407 >> v2:172.21.15.154:3300/0 conn(0x7fec4c159490 0x7fec4c15d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:54.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec50d57640 1 --2- 172.21.15.89:0/3016794407 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec4c06bee0 0x7fec4c0c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:54.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec4affd640 1 -- 172.21.15.89:0/3016794407 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1413312554 0 0) 0x7fec4c15ddf0 con 0x7fec4c070600 2023-10-18T11:03:54.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec4affd640 1 -- 172.21.15.89:0/3016794407 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec34003530 con 0x7fec4c070600 2023-10-18T11:03:54.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec4affd640 1 -- 172.21.15.89:0/3016794407 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1372622738 0 0) 0x7fec34003530 con 0x7fec4c070600 2023-10-18T11:03:54.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec4affd640 1 -- 172.21.15.89:0/3016794407 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec4c06bee0 msgr2=0x7fec4c0c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:54.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec4affd640 1 --2- 172.21.15.89:0/3016794407 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec4c06bee0 0x7fec4c0c8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:54.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec4affd640 1 -- 172.21.15.89:0/3016794407 >> v2:172.21.15.154:3300/0 conn(0x7fec4c159490 msgr2=0x7fec4c15d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:54.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec4affd640 1 --2- 172.21.15.89:0/3016794407 >> v2:172.21.15.154:3300/0 conn(0x7fec4c159490 0x7fec4c15d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:54.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec4affd640 1 -- 172.21.15.89:0/3016794407 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec4c071150 con 0x7fec4c070600 2023-10-18T11:03:54.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec4affd640 1 -- 172.21.15.89:0/3016794407 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7fec40001680 con 0x7fec4c070600 2023-10-18T11:03:54.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec4affd640 1 -- 172.21.15.89:0/3016794407 >> v1:172.21.15.89:6789/0 conn(0x7fec4c070600 legacy=0x7fec4c06b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:54.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec4affd640 1 --2- 172.21.15.89:0/3016794407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec34003980 0x7fec34003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:54.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec4affd640 1 --2- 172.21.15.89:0/3016794407 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec4c06bee0 0x7fec340047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:54.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec4affd640 1 --2- 172.21.15.89:0/3016794407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec34004d10 0x7fec34005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:54.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec4affd640 1 -- 172.21.15.89:0/3016794407 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c070b90 con 0x7fec34003980 2023-10-18T11:03:54.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec4affd640 1 -- 172.21.15.89:0/3016794407 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c070e70 con 0x7fec34004d10 2023-10-18T11:03:54.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec4affd640 1 -- 172.21.15.89:0/3016794407 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec3400b9c0 con 0x7fec4c06bee0 2023-10-18T11:03:54.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec4bfff640 1 --2- 172.21.15.89:0/3016794407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec34003980 0x7fec34003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:54.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec4b7fe640 1 --2- 172.21.15.89:0/3016794407 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec4c06bee0 0x7fec340047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:54.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec50d57640 1 --2- 172.21.15.89:0/3016794407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec34004d10 0x7fec34005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:54.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec4bfff640 1 -- 172.21.15.89:0/3016794407 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec4c06bee0 msgr2=0x7fec340047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:54.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec4bfff640 1 --2- 172.21.15.89:0/3016794407 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec4c06bee0 0x7fec340047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:54.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec4bfff640 1 -- 172.21.15.89:0/3016794407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec34004d10 msgr2=0x7fec34005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:54.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec4bfff640 1 --2- 172.21.15.89:0/3016794407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec34004d10 0x7fec34005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:54.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec4bfff640 1 -- 172.21.15.89:0/3016794407 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec34003530 con 0x7fec34003980 2023-10-18T11:03:54.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec50d57640 1 --2- 172.21.15.89:0/3016794407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec34004d10 0x7fec34005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:03:54.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.267+0000 7fec4b7fe640 1 --2- 172.21.15.89:0/3016794407 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec4c06bee0 0x7fec340047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:03:54.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.271+0000 7fec4bfff640 1 --2- 172.21.15.89:0/3016794407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec34003980 0x7fec34003da0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009ae0 tx=0x7fec3c01ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0d89d89f73dea7b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:54.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.271+0000 7fec4affd640 1 -- 172.21.15.89:0/3016794407 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c0a5630 con 0x7fec34003980 2023-10-18T11:03:54.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.271+0000 7fec4affd640 1 -- 172.21.15.89:0/3016794407 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3c0a5790 con 0x7fec34003980 2023-10-18T11:03:54.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.271+0000 7fec4affd640 1 -- 172.21.15.89:0/3016794407 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c0a5b40 con 0x7fec34003980 2023-10-18T11:03:54.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.271+0000 7fec527ba640 1 -- 172.21.15.89:0/3016794407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec34003980 msgr2=0x7fec34003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:54.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.271+0000 7fec527ba640 1 --2- 172.21.15.89:0/3016794407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec34003980 0x7fec34003da0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009ae0 tx=0x7fec3c01ca60 comp rx=0 tx=0).stop 2023-10-18T11:03:54.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.271+0000 7fec50d57640 1 -- 172.21.15.89:0/3016794407 reap_dead start 2023-10-18T11:03:54.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.271+0000 7fec527ba640 1 -- 172.21.15.89:0/3016794407 shutdown_connections 2023-10-18T11:03:54.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.271+0000 7fec527ba640 1 -- 172.21.15.89:0/3016794407 >> 172.21.15.89:0/3016794407 conn(0x7fec4c069f00 msgr2=0x7fec4c0cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:54.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.271+0000 7fec527ba640 1 -- 172.21.15.89:0/3016794407 shutdown_connections 2023-10-18T11:03:54.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.271+0000 7fec527ba640 1 -- 172.21.15.89:0/3016794407 wait complete. 2023-10-18T11:03:54.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.275+0000 7fec527ba640 1 Processor -- start 2023-10-18T11:03:54.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.275+0000 7fec527ba640 1 -- start start 2023-10-18T11:03:54.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.275+0000 7fec527ba640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec34003980 0x7fec4c154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:54.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.275+0000 7fec527ba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec4c0c1e40 0x7fec4c0c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:54.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.275+0000 7fec527ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec4c0c22c0 0x7fec4c0c4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:54.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.275+0000 7fec527ba640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c0c2e50 con 0x7fec34003980 2023-10-18T11:03:54.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.275+0000 7fec527ba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c0c3a60 con 0x7fec4c0c22c0 2023-10-18T11:03:54.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.275+0000 7fec527ba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c0c2890 con 0x7fec4c0c1e40 2023-10-18T11:03:54.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.275+0000 7fec4bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec34003980 0x7fec4c154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:54.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.275+0000 7fec4bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec34003980 0x7fec4c154990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51606/0 (socket says 172.21.15.89:51606) 2023-10-18T11:03:54.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.275+0000 7fec4bfff640 1 -- 172.21.15.89:0/3357224263 learned_addr learned my addr 172.21.15.89:0/3357224263 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:54.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.275+0000 7fec4b7fe640 1 --2- 172.21.15.89:0/3357224263 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec4c0c1e40 0x7fec4c0c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:54.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.275+0000 7fec50d57640 1 --2- 172.21.15.89:0/3357224263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec4c0c22c0 0x7fec4c0c4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:54.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.275+0000 7fec4bfff640 1 -- 172.21.15.89:0/3357224263 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec4c0c1e40 msgr2=0x7fec4c0c3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:54.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.275+0000 7fec4bfff640 1 --2- 172.21.15.89:0/3357224263 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec4c0c1e40 0x7fec4c0c3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:54.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.275+0000 7fec4bfff640 1 -- 172.21.15.89:0/3357224263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec4c0c22c0 msgr2=0x7fec4c0c4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:54.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.275+0000 7fec4bfff640 1 --2- 172.21.15.89:0/3357224263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec4c0c22c0 0x7fec4c0c4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:54.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.275+0000 7fec4bfff640 1 -- 172.21.15.89:0/3357224263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec4c0c2b70 con 0x7fec34003980 2023-10-18T11:03:54.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.275+0000 7fec4b7fe640 1 --2- 172.21.15.89:0/3357224263 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec4c0c1e40 0x7fec4c0c3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:03:54.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.275+0000 7fec50d57640 1 --2- 172.21.15.89:0/3357224263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec4c0c22c0 0x7fec4c0c4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:03:54.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.275+0000 7fec4bfff640 1 --2- 172.21.15.89:0/3357224263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec34003980 0x7fec4c154990 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009ae0 tx=0x7fec34005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:54.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.275+0000 7fec48ff9640 1 -- 172.21.15.89:0/3357224263 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c0be070 con 0x7fec34003980 2023-10-18T11:03:54.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.275+0000 7fec48ff9640 1 -- 172.21.15.89:0/3357224263 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3c00f000 con 0x7fec34003980 2023-10-18T11:03:54.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.275+0000 7fec527ba640 1 -- 172.21.15.89:0/3357224263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec4c0c36f0 con 0x7fec34003980 2023-10-18T11:03:54.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.275+0000 7fec48ff9640 1 -- 172.21.15.89:0/3357224263 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c0a5630 con 0x7fec34003980 2023-10-18T11:03:54.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.275+0000 7fec527ba640 1 -- 172.21.15.89:0/3357224263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec4c0c3390 con 0x7fec34003980 2023-10-18T11:03:54.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.279+0000 7fec527ba640 1 -- 172.21.15.89:0/3357224263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec4c0c3550 con 0x7fec34003980 2023-10-18T11:03:54.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.279+0000 7fec48ff9640 1 -- 172.21.15.89:0/3357224263 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fec3c0b4040 con 0x7fec34003980 2023-10-18T11:03:54.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.279+0000 7fec48ff9640 1 --2- 172.21.15.89:0/3357224263 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fec380656a0 0x7fec38067b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:54.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.279+0000 7fec48ff9640 1 -- 172.21.15.89:0/3357224263 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fec3c11c8a0 con 0x7fec34003980 2023-10-18T11:03:54.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.279+0000 7fec4b7fe640 1 --2- 172.21.15.89:0/3357224263 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fec380656a0 0x7fec38067b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:54.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.279+0000 7fec4b7fe640 1 --2- 172.21.15.89:0/3357224263 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fec380656a0 0x7fec38067b60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fec40001020 tx=0x7fec40009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:54.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.283+0000 7fec48ff9640 1 -- 172.21.15.89:0/3357224263 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fec4c0c3550 con 0x7fec34003980 2023-10-18T11:03:54.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.431+0000 7fec527ba640 1 -- 172.21.15.89:0/3357224263 --> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fec4c0cda60 con 0x7fec380656a0 2023-10-18T11:03:54.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.435+0000 7fec48ff9640 1 -- 172.21.15.89:0/3357224263 <== mgr.4099 v2:172.21.15.154:6832/1086608968 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fec400109a0 con 0x7fec380656a0 2023-10-18T11:03:54.433 INFO:teuthology.orchestra.run.smithi089.stderr:instructed osd(s) 6 to deep-scrub 2023-10-18T11:03:54.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.435+0000 7fec527ba640 1 -- 172.21.15.89:0/3357224263 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fec380656a0 msgr2=0x7fec38067b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:54.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.435+0000 7fec527ba640 1 --2- 172.21.15.89:0/3357224263 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fec380656a0 0x7fec38067b60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fec40001020 tx=0x7fec40009040 comp rx=0 tx=0).stop 2023-10-18T11:03:54.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.435+0000 7fec527ba640 1 -- 172.21.15.89:0/3357224263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec34003980 msgr2=0x7fec4c154990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:54.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.435+0000 7fec527ba640 1 --2- 172.21.15.89:0/3357224263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec34003980 0x7fec4c154990 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009ae0 tx=0x7fec34005800 comp rx=0 tx=0).stop 2023-10-18T11:03:54.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.435+0000 7fec527ba640 1 -- 172.21.15.89:0/3357224263 shutdown_connections 2023-10-18T11:03:54.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.435+0000 7fec527ba640 1 --2- 172.21.15.89:0/3357224263 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fec380656a0 0x7fec38067b60 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:54.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.435+0000 7fec527ba640 1 --2- 172.21.15.89:0/3357224263 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec4c0c1e40 0x7fec4c0c3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:54.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.435+0000 7fec527ba640 1 --2- 172.21.15.89:0/3357224263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec4c0c22c0 0x7fec4c0c4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:54.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.435+0000 7fec527ba640 1 --2- 172.21.15.89:0/3357224263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec34003980 0x7fec4c154990 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:54.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.435+0000 7fec527ba640 1 -- 172.21.15.89:0/3357224263 >> 172.21.15.89:0/3357224263 conn(0x7fec4c069f00 msgr2=0x7fec4c06f750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:54.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.435+0000 7fec527ba640 1 -- 172.21.15.89:0/3357224263 shutdown_connections 2023-10-18T11:03:54.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.435+0000 7fec527ba640 1 -- 172.21.15.89:0/3357224263 wait complete. 2023-10-18T11:03:54.455 INFO:tasks.ceph:Scrubbing osd.7 2023-10-18T11:03:54.456 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-18T11:03:54.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.599+0000 7ff8d7bfa640 1 Processor -- start 2023-10-18T11:03:54.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.599+0000 7ff8d7bfa640 1 -- start start 2023-10-18T11:03:54.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.599+0000 7ff8d7bfa640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff8d015a5e0 0x7ff8d006c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:54.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.599+0000 7ff8d7bfa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8d006f8c0 0x7ff8d006fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:54.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.599+0000 7ff8d7bfa640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8d0071c00 con 0x7ff8d00afd40 2023-10-18T11:03:54.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.599+0000 7ff8d7bfa640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8d0070270 con 0x7ff8d015a5e0 2023-10-18T11:03:54.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.599+0000 7ff8d7bfa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d015a9c0 con 0x7ff8d006f8c0 2023-10-18T11:03:54.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.599+0000 7ff8d5195640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff8d00afd40 0x7ff8d00b1110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:46430/0 (socket says 172.21.15.89:46430) 2023-10-18T11:03:54.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.599+0000 7ff8d5195640 1 -- 172.21.15.89:0/3998959416 learned_addr learned my addr 172.21.15.89:0/3998959416 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:54.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.599+0000 7ff8d5996640 1 --2- 172.21.15.89:0/3998959416 >> v2:172.21.15.154:3300/0 conn(0x7ff8d015a5e0 0x7ff8d006c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:54.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.599+0000 7ff8d6197640 1 --2- 172.21.15.89:0/3998959416 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8d006f8c0 0x7ff8d006fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:54.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.599+0000 7ff8d4994640 1 -- 172.21.15.89:0/3998959416 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2036735988 0 0) 0x7ff8c0001120 con 0x7ff8d00afd40 2023-10-18T11:03:54.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.599+0000 7ff8d4994640 1 -- 172.21.15.89:0/3998959416 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8bc003530 con 0x7ff8d00afd40 2023-10-18T11:03:54.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.599+0000 7ff8d6197640 1 -- 172.21.15.89:0/3998959416 >> v2:172.21.15.154:3300/0 conn(0x7ff8d015a5e0 msgr2=0x7ff8d006c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:54.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.599+0000 7ff8d6197640 1 --2- 172.21.15.89:0/3998959416 >> v2:172.21.15.154:3300/0 conn(0x7ff8d015a5e0 0x7ff8d006c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:54.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.599+0000 7ff8d6197640 1 -- 172.21.15.89:0/3998959416 >> v1:172.21.15.89:6789/0 conn(0x7ff8d00afd40 legacy=0x7ff8d00b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:54.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.599+0000 7ff8d6197640 1 -- 172.21.15.89:0/3998959416 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d006c720 con 0x7ff8d006f8c0 2023-10-18T11:03:54.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.603+0000 7ff8d6197640 1 --2- 172.21.15.89:0/3998959416 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8d006f8c0 0x7ff8d006fd30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8009d90 tx=0x7ff8c801c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4b041898c7fa87d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:54.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.603+0000 7ff8d4994640 1 -- 172.21.15.89:0/3998959416 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c80a59e0 con 0x7ff8d006f8c0 2023-10-18T11:03:54.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.603+0000 7ff8d4994640 1 -- 172.21.15.89:0/3998959416 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c80a5b40 con 0x7ff8d006f8c0 2023-10-18T11:03:54.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.603+0000 7ff8d4994640 1 -- 172.21.15.89:0/3998959416 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c80ad440 con 0x7ff8d006f8c0 2023-10-18T11:03:54.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.603+0000 7ff8d7bfa640 1 -- 172.21.15.89:0/3998959416 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8d006f8c0 msgr2=0x7ff8d006fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:54.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.603+0000 7ff8d7bfa640 1 --2- 172.21.15.89:0/3998959416 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8d006f8c0 0x7ff8d006fd30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8009d90 tx=0x7ff8c801c920 comp rx=0 tx=0).stop 2023-10-18T11:03:54.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.603+0000 7ff8d7bfa640 1 -- 172.21.15.89:0/3998959416 shutdown_connections 2023-10-18T11:03:54.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.603+0000 7ff8d7bfa640 1 --2- 172.21.15.89:0/3998959416 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8d006f8c0 0x7ff8d006fd30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:54.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.603+0000 7ff8d7bfa640 1 --2- 172.21.15.89:0/3998959416 >> v2:172.21.15.154:3300/0 conn(0x7ff8d015a5e0 0x7ff8d006c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:54.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.603+0000 7ff8d7bfa640 1 -- 172.21.15.89:0/3998959416 >> 172.21.15.89:0/3998959416 conn(0x7ff8d00cad40 msgr2=0x7ff8d00cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:54.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.603+0000 7ff8d7bfa640 1 -- 172.21.15.89:0/3998959416 shutdown_connections 2023-10-18T11:03:54.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.603+0000 7ff8d7bfa640 1 -- 172.21.15.89:0/3998959416 wait complete. 2023-10-18T11:03:54.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.607+0000 7ff8d7bfa640 1 Processor -- start 2023-10-18T11:03:54.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.607+0000 7ff8d7bfa640 1 -- start start 2023-10-18T11:03:54.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.607+0000 7ff8d7bfa640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8d006f8c0 0x7ff8d01515c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:54.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.607+0000 7ff8d7bfa640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8d00afd40 0x7ff8d01577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:54.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.607+0000 7ff8d7bfa640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8d015a5e0 0x7ff8d01551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:54.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.607+0000 7ff8d7bfa640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d0157d10 con 0x7ff8d015a5e0 2023-10-18T11:03:54.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.607+0000 7ff8d7bfa640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d0157eb0 con 0x7ff8d006f8c0 2023-10-18T11:03:54.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.607+0000 7ff8d7bfa640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d0151c10 con 0x7ff8d00afd40 2023-10-18T11:03:54.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.607+0000 7ff8d6197640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8d015a5e0 0x7ff8d01551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:54.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.607+0000 7ff8d6197640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8d015a5e0 0x7ff8d01551e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51614/0 (socket says 172.21.15.89:51614) 2023-10-18T11:03:54.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.607+0000 7ff8d6197640 1 -- 172.21.15.89:0/1990071677 learned_addr learned my addr 172.21.15.89:0/1990071677 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:54.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.607+0000 7ff8d5996640 1 --2- 172.21.15.89:0/1990071677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8d006f8c0 0x7ff8d01515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:54.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.607+0000 7ff8d5195640 1 --2- 172.21.15.89:0/1990071677 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8d00afd40 0x7ff8d01577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:54.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.607+0000 7ff8d6197640 1 -- 172.21.15.89:0/1990071677 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8d00afd40 msgr2=0x7ff8d01577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:54.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.607+0000 7ff8d6197640 1 --2- 172.21.15.89:0/1990071677 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8d00afd40 0x7ff8d01577d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:54.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.607+0000 7ff8d6197640 1 -- 172.21.15.89:0/1990071677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8d006f8c0 msgr2=0x7ff8d01515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:54.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.607+0000 7ff8d6197640 1 --2- 172.21.15.89:0/1990071677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8d006f8c0 0x7ff8d01515c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:54.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.607+0000 7ff8d6197640 1 -- 172.21.15.89:0/1990071677 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c8009a70 con 0x7ff8d015a5e0 2023-10-18T11:03:54.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.607+0000 7ff8d6197640 1 --2- 172.21.15.89:0/1990071677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8d015a5e0 0x7ff8d01551e0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff8c80b0040 tx=0x7ff8c80196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:54.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.607+0000 7ff8c67fc640 1 -- 172.21.15.89:0/1990071677 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c80adce0 con 0x7ff8d015a5e0 2023-10-18T11:03:54.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.607+0000 7ff8c67fc640 1 -- 172.21.15.89:0/1990071677 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c80ade40 con 0x7ff8d015a5e0 2023-10-18T11:03:54.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.607+0000 7ff8c67fc640 1 -- 172.21.15.89:0/1990071677 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c80b9690 con 0x7ff8d015a5e0 2023-10-18T11:03:54.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.607+0000 7ff8d7bfa640 1 -- 172.21.15.89:0/1990071677 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8d0151ef0 con 0x7ff8d015a5e0 2023-10-18T11:03:54.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.611+0000 7ff8d7bfa640 1 -- 172.21.15.89:0/1990071677 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8d01583e0 con 0x7ff8d015a5e0 2023-10-18T11:03:54.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.611+0000 7ff8d7bfa640 1 -- 172.21.15.89:0/1990071677 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff8d00c89b0 con 0x7ff8d015a5e0 2023-10-18T11:03:54.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.611+0000 7ff8c67fc640 1 -- 172.21.15.89:0/1990071677 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7ff8c80b97f0 con 0x7ff8d015a5e0 2023-10-18T11:03:54.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.611+0000 7ff8c67fc640 1 --2- 172.21.15.89:0/1990071677 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff8b4065680 0x7ff8b4067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:54.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.611+0000 7ff8c67fc640 1 -- 172.21.15.89:0/1990071677 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5659+0+0 (secure 0 0 0) 0x7ff8c811ed90 con 0x7ff8d015a5e0 2023-10-18T11:03:54.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.611+0000 7ff8c67fc640 1 --2- 172.21.15.89:0/1990071677 >> [v2:172.21.15.154:6815/4139186586,v1:172.21.15.154:6816/4139186586] conn(0x7ff8b406bff0 0x7ff8b406e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:54.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.611+0000 7ff8c67fc640 1 -- 172.21.15.89:0/1990071677 --> [v2:172.21.15.154:6815/4139186586,v1:172.21.15.154:6816/4139186586] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff8b406eb00 con 0x7ff8b406bff0 2023-10-18T11:03:54.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.611+0000 7ff8c67fc640 1 -- 172.21.15.89:0/1990071677 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8c811f100 con 0x7ff8d015a5e0 2023-10-18T11:03:54.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.611+0000 7ff8d5996640 1 --2- 172.21.15.89:0/1990071677 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff8b4065680 0x7ff8b4067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:54.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.611+0000 7ff8d5195640 1 --2- 172.21.15.89:0/1990071677 >> [v2:172.21.15.154:6815/4139186586,v1:172.21.15.154:6816/4139186586] conn(0x7ff8b406bff0 0x7ff8b406e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:54.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.611+0000 7ff8d5996640 1 --2- 172.21.15.89:0/1990071677 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff8b4065680 0x7ff8b4067b40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc009b60 tx=0x7ff8cc009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:54.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.611+0000 7ff8d5195640 1 --2- 172.21.15.89:0/1990071677 >> [v2:172.21.15.154:6815/4139186586,v1:172.21.15.154:6816/4139186586] conn(0x7ff8b406bff0 0x7ff8b406e430 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:54.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.615+0000 7ff8c67fc640 1 -- 172.21.15.89:0/1990071677 <== osd.7 v2:172.21.15.154:6815/4139186586 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7ff8b406eb00 con 0x7ff8b406bff0 2023-10-18T11:03:54.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.627+0000 7ff8d7bfa640 1 -- 172.21.15.89:0/1990071677 --> [v2:172.21.15.154:6815/4139186586,v1:172.21.15.154:6816/4139186586] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff8d00b07a0 con 0x7ff8b406bff0 2023-10-18T11:03:54.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.627+0000 7ff8c67fc640 1 -- 172.21.15.89:0/1990071677 <== osd.7 v2:172.21.15.154:6815/4139186586 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ff8c00047f0 con 0x7ff8b406bff0 2023-10-18T11:03:54.627 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-18T11:03:54.628 INFO:teuthology.orchestra.run.smithi089.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-18T11:03:54.628 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-18T11:03:54.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.627+0000 7ff8d7bfa640 1 -- 172.21.15.89:0/1990071677 >> [v2:172.21.15.154:6815/4139186586,v1:172.21.15.154:6816/4139186586] conn(0x7ff8b406bff0 msgr2=0x7ff8b406e430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:54.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.627+0000 7ff8d7bfa640 1 --2- 172.21.15.89:0/1990071677 >> [v2:172.21.15.154:6815/4139186586,v1:172.21.15.154:6816/4139186586] conn(0x7ff8b406bff0 0x7ff8b406e430 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:54.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.627+0000 7ff8d7bfa640 1 -- 172.21.15.89:0/1990071677 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff8b4065680 msgr2=0x7ff8b4067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:54.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.627+0000 7ff8d7bfa640 1 --2- 172.21.15.89:0/1990071677 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7ff8b4065680 0x7ff8b4067b40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc009b60 tx=0x7ff8cc009190 comp rx=0 tx=0).stop 2023-10-18T11:03:54.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.627+0000 7ff8d7bfa640 1 -- 172.21.15.89:0/1990071677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8d015a5e0 msgr2=0x7ff8d01551e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:54.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.627+0000 7ff8d7bfa640 1 --2- 172.21.15.89:0/1990071677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8d015a5e0 0x7ff8d01551e0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff8c80b0040 tx=0x7ff8c80196c0 comp rx=0 tx=0).stop 2023-10-18T11:03:54.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.627+0000 7ff8d6197640 1 -- 172.21.15.89:0/1990071677 reap_dead start 2023-10-18T11:03:54.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.627+0000 7ff8d7bfa640 1 -- 172.21.15.89:0/1990071677 shutdown_connections 2023-10-18T11:03:54.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.627+0000 7ff8d7bfa640 1 -- 172.21.15.89:0/1990071677 >> 172.21.15.89:0/1990071677 conn(0x7ff8d00cad40 msgr2=0x7ff8d00b1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:54.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.627+0000 7ff8d7bfa640 1 -- 172.21.15.89:0/1990071677 shutdown_connections 2023-10-18T11:03:54.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.627+0000 7ff8d7bfa640 1 -- 172.21.15.89:0/1990071677 wait complete. 2023-10-18T11:03:54.645 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-18T11:03:54.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.783+0000 7f7ed2aca640 1 Processor -- start 2023-10-18T11:03:54.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.783+0000 7f7ed2aca640 1 -- start start 2023-10-18T11:03:54.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.783+0000 7f7ed2aca640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7ecc159490 0x7f7ecc15d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:54.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.783+0000 7f7ed2aca640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ecc070600 0x7f7ecc06b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:54.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.783+0000 7f7ed2aca640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ecc15ddf0 con 0x7f7ecc06bd10 2023-10-18T11:03:54.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.783+0000 7f7ed2aca640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ecc06c1a0 con 0x7f7ecc159490 2023-10-18T11:03:54.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.783+0000 7f7ed2aca640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc070b90 con 0x7f7ecc070600 2023-10-18T11:03:54.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.783+0000 7f7ed1067640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7ecc06bd10 0x7f7ecc0cbee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:46434/0 (socket says 172.21.15.89:46434) 2023-10-18T11:03:54.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.783+0000 7f7ed1067640 1 -- 172.21.15.89:0/3800358247 learned_addr learned my addr 172.21.15.89:0/3800358247 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:03:54.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.783+0000 7f7ed0866640 1 --2- 172.21.15.89:0/3800358247 >> v2:172.21.15.89:3300/0 conn(0x7f7ecc159490 0x7f7ecc15d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:54.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.787+0000 7f7ec3fff640 1 --2- 172.21.15.89:0/3800358247 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ecc070600 0x7f7ecc06b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:54.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.787+0000 7f7ed0866640 1 -- 172.21.15.89:0/3800358247 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ecc070600 msgr2=0x7f7ecc06b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:54.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.787+0000 7f7ed0866640 1 --2- 172.21.15.89:0/3800358247 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ecc070600 0x7f7ecc06b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:54.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.787+0000 7f7ed0866640 1 -- 172.21.15.89:0/3800358247 >> v1:172.21.15.89:6789/0 conn(0x7f7ecc06bd10 legacy=0x7f7ecc0cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:54.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.787+0000 7f7ed0866640 1 -- 172.21.15.89:0/3800358247 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ecc070e70 con 0x7f7ecc159490 2023-10-18T11:03:54.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.787+0000 7f7ec37fe640 1 -- 172.21.15.89:0/3800358247 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 388477462 0 0) 0x7f7ecc15ddf0 con 0x7f7ecc06bd10 2023-10-18T11:03:54.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.787+0000 7f7ed0866640 1 --2- 172.21.15.89:0/3800358247 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ecc159490 0x7f7ecc15d8b0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f7ec80097c0 tx=0x7f7ec801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c455c2b6a3710622 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:54.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.787+0000 7f7ec37fe640 1 -- 172.21.15.89:0/3800358247 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec80ac620 con 0x7f7ecc159490 2023-10-18T11:03:54.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.787+0000 7f7ec37fe640 1 -- 172.21.15.89:0/3800358247 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec80ac780 con 0x7f7ecc159490 2023-10-18T11:03:54.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.787+0000 7f7ec37fe640 1 -- 172.21.15.89:0/3800358247 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec80acb10 con 0x7f7ecc159490 2023-10-18T11:03:54.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.791+0000 7f7ed2aca640 1 -- 172.21.15.89:0/3800358247 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ecc159490 msgr2=0x7f7ecc15d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:54.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.791+0000 7f7ed2aca640 1 --2- 172.21.15.89:0/3800358247 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ecc159490 0x7f7ecc15d8b0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f7ec80097c0 tx=0x7f7ec801c3c0 comp rx=0 tx=0).stop 2023-10-18T11:03:54.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.791+0000 7f7ed2aca640 1 -- 172.21.15.89:0/3800358247 shutdown_connections 2023-10-18T11:03:54.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.791+0000 7f7ed2aca640 1 --2- 172.21.15.89:0/3800358247 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ecc070600 0x7f7ecc06b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:54.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.791+0000 7f7ed2aca640 1 --2- 172.21.15.89:0/3800358247 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ecc159490 0x7f7ecc15d8b0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:54.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.791+0000 7f7ed2aca640 1 -- 172.21.15.89:0/3800358247 >> 172.21.15.89:0/3800358247 conn(0x7f7ecc069f00 msgr2=0x7f7ecc0cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:54.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.791+0000 7f7ed2aca640 1 -- 172.21.15.89:0/3800358247 shutdown_connections 2023-10-18T11:03:54.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.791+0000 7f7ed2aca640 1 -- 172.21.15.89:0/3800358247 wait complete. 2023-10-18T11:03:54.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.791+0000 7f7ed2aca640 1 Processor -- start 2023-10-18T11:03:54.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.791+0000 7f7ed2aca640 1 -- start start 2023-10-18T11:03:54.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.791+0000 7f7ed2aca640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ecc06bd10 0x7f7ecc1f4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:54.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.791+0000 7f7ed2aca640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ecc070600 0x7f7ecc0c5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:54.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.791+0000 7f7ed2aca640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ecc159490 0x7f7ecc155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:54.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.791+0000 7f7ed2aca640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc15acd0 con 0x7f7ecc159490 2023-10-18T11:03:54.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.791+0000 7f7ed2aca640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc15af90 con 0x7f7ecc06bd10 2023-10-18T11:03:54.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.791+0000 7f7ed2aca640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc0c6280 con 0x7f7ecc070600 2023-10-18T11:03:54.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.791+0000 7f7ed1067640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ecc159490 0x7f7ecc155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:54.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.791+0000 7f7ed1067640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ecc159490 0x7f7ecc155a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51636/0 (socket says 172.21.15.89:51636) 2023-10-18T11:03:54.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.791+0000 7f7ed1067640 1 -- 172.21.15.89:0/3561742032 learned_addr learned my addr 172.21.15.89:0/3561742032 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:54.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.791+0000 7f7ec3fff640 1 --2- 172.21.15.89:0/3561742032 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ecc070600 0x7f7ecc0c5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:54.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.791+0000 7f7ed0866640 1 --2- 172.21.15.89:0/3561742032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ecc06bd10 0x7f7ecc1f4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:54.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.795+0000 7f7ed1067640 1 -- 172.21.15.89:0/3561742032 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ecc070600 msgr2=0x7f7ecc0c5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:54.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.795+0000 7f7ed1067640 1 --2- 172.21.15.89:0/3561742032 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ecc070600 0x7f7ecc0c5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:54.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.795+0000 7f7ed1067640 1 -- 172.21.15.89:0/3561742032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ecc06bd10 msgr2=0x7f7ecc1f4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:54.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.795+0000 7f7ed1067640 1 --2- 172.21.15.89:0/3561742032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ecc06bd10 0x7f7ecc1f4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:54.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.795+0000 7f7ed1067640 1 -- 172.21.15.89:0/3561742032 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec8009420 con 0x7f7ecc159490 2023-10-18T11:03:54.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.795+0000 7f7ed1067640 1 --2- 172.21.15.89:0/3561742032 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ecc159490 0x7f7ecc155a00 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f7ec400c780 tx=0x7f7ec400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:54.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.795+0000 7f7ec17fa640 1 -- 172.21.15.89:0/3561742032 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec400ce30 con 0x7f7ecc159490 2023-10-18T11:03:54.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.795+0000 7f7ec17fa640 1 -- 172.21.15.89:0/3561742032 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec400a640 con 0x7f7ecc159490 2023-10-18T11:03:54.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.795+0000 7f7ec17fa640 1 -- 172.21.15.89:0/3561742032 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec400a9f0 con 0x7f7ecc159490 2023-10-18T11:03:54.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.795+0000 7f7ed2aca640 1 -- 172.21.15.89:0/3561742032 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ecc0c6560 con 0x7f7ecc159490 2023-10-18T11:03:54.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.795+0000 7f7ed2aca640 1 -- 172.21.15.89:0/3561742032 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ecc0c6b20 con 0x7f7ecc159490 2023-10-18T11:03:54.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.795+0000 7f7ed2aca640 1 -- 172.21.15.89:0/3561742032 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ecc15b140 con 0x7f7ecc159490 2023-10-18T11:03:54.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.795+0000 7f7ec17fa640 1 -- 172.21.15.89:0/3561742032 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f7ec400ab50 con 0x7f7ecc159490 2023-10-18T11:03:54.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.795+0000 7f7ec17fa640 1 --2- 172.21.15.89:0/3561742032 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7eb8065680 0x7f7eb8067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:54.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.795+0000 7f7ec17fa640 1 -- 172.21.15.89:0/3561742032 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f7ec4081fe0 con 0x7f7ecc159490 2023-10-18T11:03:54.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.795+0000 7f7ed0866640 1 --2- 172.21.15.89:0/3561742032 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7eb8065680 0x7f7eb8067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:54.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.799+0000 7f7ed0866640 1 --2- 172.21.15.89:0/3561742032 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7eb8065680 0x7f7eb8067b40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7ec80ae000 tx=0x7f7ec80a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:54.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.799+0000 7f7ec17fa640 1 -- 172.21.15.89:0/3561742032 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f7ecc15b140 con 0x7f7ecc159490 2023-10-18T11:03:54.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.947+0000 7f7ed2aca640 1 -- 172.21.15.89:0/3561742032 --> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ecc0cda60 con 0x7f7eb8065680 2023-10-18T11:03:54.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.947+0000 7f7ec17fa640 1 -- 172.21.15.89:0/3561742032 <== mgr.4099 v2:172.21.15.154:6832/1086608968 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7ec80bc070 con 0x7f7eb8065680 2023-10-18T11:03:54.947 INFO:teuthology.orchestra.run.smithi089.stderr:instructed osd(s) 7 to deep-scrub 2023-10-18T11:03:54.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.951+0000 7f7ed2aca640 1 -- 172.21.15.89:0/3561742032 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7eb8065680 msgr2=0x7f7eb8067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:54.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.951+0000 7f7ed2aca640 1 --2- 172.21.15.89:0/3561742032 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7eb8065680 0x7f7eb8067b40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7ec80ae000 tx=0x7f7ec80a7000 comp rx=0 tx=0).stop 2023-10-18T11:03:54.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.951+0000 7f7ed2aca640 1 -- 172.21.15.89:0/3561742032 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ecc159490 msgr2=0x7f7ecc155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:54.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.951+0000 7f7ed2aca640 1 --2- 172.21.15.89:0/3561742032 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ecc159490 0x7f7ecc155a00 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f7ec400c780 tx=0x7f7ec400cc40 comp rx=0 tx=0).stop 2023-10-18T11:03:54.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.951+0000 7f7ed2aca640 1 -- 172.21.15.89:0/3561742032 shutdown_connections 2023-10-18T11:03:54.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.951+0000 7f7ed2aca640 1 --2- 172.21.15.89:0/3561742032 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f7eb8065680 0x7f7eb8067b40 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:54.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.951+0000 7f7ed2aca640 1 --2- 172.21.15.89:0/3561742032 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ecc070600 0x7f7ecc0c5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:54.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.951+0000 7f7ed2aca640 1 --2- 172.21.15.89:0/3561742032 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ecc159490 0x7f7ecc155a00 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:54.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.951+0000 7f7ed2aca640 1 --2- 172.21.15.89:0/3561742032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ecc06bd10 0x7f7ecc1f4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:54.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.951+0000 7f7ed2aca640 1 -- 172.21.15.89:0/3561742032 >> 172.21.15.89:0/3561742032 conn(0x7f7ecc069f00 msgr2=0x7f7ecc0acf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:54.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.951+0000 7f7ed2aca640 1 -- 172.21.15.89:0/3561742032 shutdown_connections 2023-10-18T11:03:54.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:54.951+0000 7f7ed2aca640 1 -- 172.21.15.89:0/3561742032 wait complete. 2023-10-18T11:03:54.968 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T11:03:55.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.107+0000 7fc836f67640 1 Processor -- start 2023-10-18T11:03:55.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.107+0000 7fc836f67640 1 -- start start 2023-10-18T11:03:55.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.107+0000 7fc836f67640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc830145b70 0x7fc830143470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:55.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.107+0000 7fc836f67640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc8300ce070 0x7fc8300ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:55.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.107+0000 7fc836f67640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8300cd8d0 con 0x7fc8300cd4f0 2023-10-18T11:03:55.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.107+0000 7fc836f67640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc830069900 con 0x7fc830145b70 2023-10-18T11:03:55.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.107+0000 7fc836f67640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8300cea20 con 0x7fc8300ce070 2023-10-18T11:03:55.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.111+0000 7fc82ffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc830145b70 0x7fc830143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:55.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.111+0000 7fc835504640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc8300ce070 0x7fc8300ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:55.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.111+0000 7fc82ffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc830145b70 0x7fc830143470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.89:56538/0 (socket says 172.21.15.89:56538) 2023-10-18T11:03:55.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.111+0000 7fc834d03640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fc8300cd4f0 0x7fc830160cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.89:50230/0 (socket says 172.21.15.89:50230) 2023-10-18T11:03:55.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.111+0000 7fc82ffff640 1 -- 172.21.15.89:0/2835603395 learned_addr learned my addr 172.21.15.89:0/2835603395 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:55.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.111+0000 7fc835504640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc8300ce070 0x7fc8300ce4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.89:43248/0 (socket says 172.21.15.89:43248) 2023-10-18T11:03:55.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.111+0000 7fc82f7fe640 1 -- 172.21.15.89:0/2835603395 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 95790533 0 0) 0x7fc8300cd8d0 con 0x7fc8300cd4f0 2023-10-18T11:03:55.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.111+0000 7fc82ffff640 1 -- 172.21.15.89:0/2835603395 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc8300ce070 msgr2=0x7fc8300ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:03:55.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.111+0000 7fc82ffff640 1 --2- 172.21.15.89:0/2835603395 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc8300ce070 0x7fc8300ce4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:55.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.111+0000 7fc82ffff640 1 -- 172.21.15.89:0/2835603395 >> v1:172.21.15.154:6789/0 conn(0x7fc8300cd4f0 legacy=0x7fc830160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:55.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.111+0000 7fc82ffff640 1 -- 172.21.15.89:0/2835603395 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc83015cbf0 con 0x7fc830145b70 2023-10-18T11:03:55.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.111+0000 7fc835504640 1 --2- 172.21.15.89:0/2835603395 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc8300ce070 0x7fc8300ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T11:03:55.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.111+0000 7fc82ffff640 1 --2- 172.21.15.89:0/2835603395 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc830145b70 0x7fc830143470 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc830069a40 tx=0x7fc82401c4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a012c211debdbef1 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:55.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.111+0000 7fc82f7fe640 1 -- 172.21.15.89:0/2835603395 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8240aa650 con 0x7fc830145b70 2023-10-18T11:03:55.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.111+0000 7fc82f7fe640 1 -- 172.21.15.89:0/2835603395 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8240aa7b0 con 0x7fc830145b70 2023-10-18T11:03:55.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.111+0000 7fc82f7fe640 1 -- 172.21.15.89:0/2835603395 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8240aab40 con 0x7fc830145b70 2023-10-18T11:03:55.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.115+0000 7fc836f67640 1 -- 172.21.15.89:0/2835603395 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc830145b70 msgr2=0x7fc830143470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:55.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.115+0000 7fc836f67640 1 --2- 172.21.15.89:0/2835603395 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc830145b70 0x7fc830143470 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc830069a40 tx=0x7fc82401c4f0 comp rx=0 tx=0).stop 2023-10-18T11:03:55.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.115+0000 7fc836f67640 1 -- 172.21.15.89:0/2835603395 shutdown_connections 2023-10-18T11:03:55.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.115+0000 7fc836f67640 1 --2- 172.21.15.89:0/2835603395 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc8300ce070 0x7fc8300ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:55.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.115+0000 7fc836f67640 1 --2- 172.21.15.89:0/2835603395 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc830145b70 0x7fc830143470 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:55.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.115+0000 7fc836f67640 1 -- 172.21.15.89:0/2835603395 >> 172.21.15.89:0/2835603395 conn(0x7fc83006dad0 msgr2=0x7fc83006dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:55.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.115+0000 7fc836f67640 1 -- 172.21.15.89:0/2835603395 shutdown_connections 2023-10-18T11:03:55.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.115+0000 7fc836f67640 1 -- 172.21.15.89:0/2835603395 wait complete. 2023-10-18T11:03:55.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.115+0000 7fc836f67640 1 Processor -- start 2023-10-18T11:03:55.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.115+0000 7fc836f67640 1 -- start start 2023-10-18T11:03:55.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.115+0000 7fc836f67640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8300cd4f0 0x7fc83015eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:55.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.115+0000 7fc836f67640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc8300ce070 0x7fc8300c2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:55.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.115+0000 7fc836f67640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc830145b70 0x7fc8300c3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:55.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.115+0000 7fc836f67640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc830153f50 con 0x7fc8300cd4f0 2023-10-18T11:03:55.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.115+0000 7fc836f67640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83015fa30 con 0x7fc830145b70 2023-10-18T11:03:55.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.115+0000 7fc836f67640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8300c3640 con 0x7fc8300ce070 2023-10-18T11:03:55.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.115+0000 7fc834d03640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8300cd4f0 0x7fc83015eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:55.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.115+0000 7fc834d03640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8300cd4f0 0x7fc83015eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51652/0 (socket says 172.21.15.89:51652) 2023-10-18T11:03:55.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.115+0000 7fc834d03640 1 -- 172.21.15.89:0/2926318294 learned_addr learned my addr 172.21.15.89:0/2926318294 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:03:55.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.119+0000 7fc835504640 1 --2- 172.21.15.89:0/2926318294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc830145b70 0x7fc8300c3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:55.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.119+0000 7fc82ffff640 1 --2- 172.21.15.89:0/2926318294 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc8300ce070 0x7fc8300c2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:55.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.119+0000 7fc834d03640 1 -- 172.21.15.89:0/2926318294 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc8300ce070 msgr2=0x7fc8300c2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:55.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.119+0000 7fc834d03640 1 --2- 172.21.15.89:0/2926318294 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc8300ce070 0x7fc8300c2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:55.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.119+0000 7fc834d03640 1 -- 172.21.15.89:0/2926318294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc830145b70 msgr2=0x7fc8300c3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:55.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.119+0000 7fc834d03640 1 --2- 172.21.15.89:0/2926318294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc830145b70 0x7fc8300c3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:55.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.119+0000 7fc834d03640 1 -- 172.21.15.89:0/2926318294 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc828009a70 con 0x7fc8300cd4f0 2023-10-18T11:03:55.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.119+0000 7fc834d03640 1 --2- 172.21.15.89:0/2926318294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8300cd4f0 0x7fc83015eab0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fc82000c4b0 tx=0x7fc82000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:55.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.119+0000 7fc82d7fa640 1 -- 172.21.15.89:0/2926318294 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8200074e0 con 0x7fc8300cd4f0 2023-10-18T11:03:55.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.119+0000 7fc82d7fa640 1 -- 172.21.15.89:0/2926318294 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc820007660 con 0x7fc8300cd4f0 2023-10-18T11:03:55.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.119+0000 7fc836f67640 1 -- 172.21.15.89:0/2926318294 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8240094d0 con 0x7fc8300cd4f0 2023-10-18T11:03:55.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.119+0000 7fc82d7fa640 1 -- 172.21.15.89:0/2926318294 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc820004a30 con 0x7fc8300cd4f0 2023-10-18T11:03:55.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.119+0000 7fc836f67640 1 -- 172.21.15.89:0/2926318294 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc83015f470 con 0x7fc8300cd4f0 2023-10-18T11:03:55.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.119+0000 7fc836f67640 1 -- 172.21.15.89:0/2926318294 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc83004dbe0 con 0x7fc8300cd4f0 2023-10-18T11:03:55.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.119+0000 7fc82d7fa640 1 -- 172.21.15.89:0/2926318294 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7fc820004b90 con 0x7fc8300cd4f0 2023-10-18T11:03:55.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.119+0000 7fc82d7fa640 1 --2- 172.21.15.89:0/2926318294 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc81c065680 0x7fc81c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:03:55.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.119+0000 7fc82d7fa640 1 -- 172.21.15.89:0/2926318294 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fc820082900 con 0x7fc8300cd4f0 2023-10-18T11:03:55.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.123+0000 7fc82ffff640 1 --2- 172.21.15.89:0/2926318294 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc81c065680 0x7fc81c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:03:55.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.123+0000 7fc82ffff640 1 --2- 172.21.15.89:0/2926318294 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc81c065680 0x7fc81c067b40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc8240ae000 tx=0x7fc8240b5040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:03:55.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.123+0000 7fc82d7fa640 1 -- 172.21.15.89:0/2926318294 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7fc820053ff0 con 0x7fc8300cd4f0 2023-10-18T11:03:55.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.275+0000 7fc836f67640 1 -- 172.21.15.89:0/2926318294 --> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc83006a3b0 con 0x7fc81c065680 2023-10-18T11:03:55.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.275+0000 7fc82d7fa640 1 -- 172.21.15.89:0/2926318294 <== mgr.4099 v2:172.21.15.154:6832/1086608968 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72593 (secure 0 0 0) 0x7fc8240aa920 con 0x7fc81c065680 2023-10-18T11:03:55.277 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T11:03:55.277 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-18T11:03:55.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.279+0000 7fc836f67640 1 -- 172.21.15.89:0/2926318294 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc81c065680 msgr2=0x7fc81c067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:55.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.279+0000 7fc836f67640 1 --2- 172.21.15.89:0/2926318294 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc81c065680 0x7fc81c067b40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc8240ae000 tx=0x7fc8240b5040 comp rx=0 tx=0).stop 2023-10-18T11:03:55.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.279+0000 7fc836f67640 1 -- 172.21.15.89:0/2926318294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8300cd4f0 msgr2=0x7fc83015eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:03:55.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.279+0000 7fc836f67640 1 --2- 172.21.15.89:0/2926318294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8300cd4f0 0x7fc83015eab0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fc82000c4b0 tx=0x7fc82000c8b0 comp rx=0 tx=0).stop 2023-10-18T11:03:55.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.279+0000 7fc836f67640 1 -- 172.21.15.89:0/2926318294 shutdown_connections 2023-10-18T11:03:55.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.279+0000 7fc836f67640 1 --2- 172.21.15.89:0/2926318294 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7fc81c065680 0x7fc81c067b40 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:55.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.279+0000 7fc836f67640 1 --2- 172.21.15.89:0/2926318294 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc8300ce070 0x7fc8300c2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:55.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.279+0000 7fc836f67640 1 --2- 172.21.15.89:0/2926318294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc830145b70 0x7fc8300c3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:55.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.279+0000 7fc836f67640 1 --2- 172.21.15.89:0/2926318294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8300cd4f0 0x7fc83015eab0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:03:55.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.279+0000 7fc836f67640 1 -- 172.21.15.89:0/2926318294 >> 172.21.15.89:0/2926318294 conn(0x7fc83006dad0 msgr2=0x7fc830144000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:03:55.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.279+0000 7fc836f67640 1 -- 172.21.15.89:0/2926318294 shutdown_connections 2023-10-18T11:03:55.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:03:55.279+0000 7fc836f67640 1 -- 172.21.15.89:0/2926318294 wait complete. 2023-10-18T11:03:55.297 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":598,"stamp":"2023-10-18T11:03:53.358935+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":223404,"kb_used_data":6636,"kb_used_omap":18,"kb_used_meta":216621,"kb_avail":754751316,"statfs":{"total":773094113280,"available":772865347584,"internally_reserved":0,"allocated":6795264,"data_stored":3878893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19166,"internal_metadata":221820194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002212"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":90,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-18T11:03:44.227989+0000","last_change":"2023-10-18T10:55:22.931642+0000","last_active":"2023-10-18T11:03:44.227989+0000","last_peered":"2023-10-18T11:03:44.227989+0000","last_clean":"2023-10-18T11:03:44.227989+0000","last_became_active":"2023-10-18T10:55:22.931232+0000","last_became_peered":"2023-10-18T10:55:22.931232+0000","last_unstale":"2023-10-18T11:03:44.227989+0000","last_undegraded":"2023-10-18T11:03:44.227989+0000","last_fullsized":"2023-10-18T11:03:44.227989+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:49:50.637795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-18T11:03:44.242691+0000","last_change":"2023-10-18T10:55:22.928478+0000","last_active":"2023-10-18T11:03:44.242691+0000","last_peered":"2023-10-18T11:03:44.242691+0000","last_clean":"2023-10-18T11:03:44.242691+0000","last_became_active":"2023-10-18T10:55:22.928049+0000","last_became_peered":"2023-10-18T10:55:22.928049+0000","last_unstale":"2023-10-18T11:03:44.242691+0000","last_undegraded":"2023-10-18T11:03:44.242691+0000","last_fullsized":"2023-10-18T11:03:44.242691+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-18T11:03:44.566074+0000","last_change":"2023-10-18T10:45:50.585224+0000","last_active":"2023-10-18T11:03:44.566074+0000","last_peered":"2023-10-18T11:03:44.566074+0000","last_clean":"2023-10-18T11:03:44.566074+0000","last_became_active":"2023-10-18T10:45:50.584872+0000","last_became_peered":"2023-10-18T10:45:50.584872+0000","last_unstale":"2023-10-18T11:03:44.566074+0000","last_undegraded":"2023-10-18T11:03:44.566074+0000","last_fullsized":"2023-10-18T11:03:44.566074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:21:09.948010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":94,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-18T11:03:44.241863+0000","last_change":"2023-10-18T10:45:50.584946+0000","last_active":"2023-10-18T11:03:44.241863+0000","last_peered":"2023-10-18T11:03:44.241863+0000","last_clean":"2023-10-18T11:03:44.241863+0000","last_became_active":"2023-10-18T10:45:50.584638+0000","last_became_peered":"2023-10-18T10:45:50.584638+0000","last_unstale":"2023-10-18T11:03:44.241863+0000","last_undegraded":"2023-10-18T11:03:44.241863+0000","last_fullsized":"2023-10-18T11:03:44.241863+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":89,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-18T11:03:44.423992+0000","last_change":"2023-10-18T10:45:50.251505+0000","last_active":"2023-10-18T11:03:44.423992+0000","last_peered":"2023-10-18T11:03:44.423992+0000","last_clean":"2023-10-18T11:03:44.423992+0000","last_became_active":"2023-10-18T10:45:50.251111+0000","last_became_peered":"2023-10-18T10:45:50.251111+0000","last_unstale":"2023-10-18T11:03:44.423992+0000","last_undegraded":"2023-10-18T11:03:44.423992+0000","last_fullsized":"2023-10-18T11:03:44.423992+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:19:33.625865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":94,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-18T11:03:44.646406+0000","last_change":"2023-10-18T10:45:50.259019+0000","last_active":"2023-10-18T11:03:44.646406+0000","last_peered":"2023-10-18T11:03:44.646406+0000","last_clean":"2023-10-18T11:03:44.646406+0000","last_became_active":"2023-10-18T10:45:50.258607+0000","last_became_peered":"2023-10-18T10:45:50.258607+0000","last_unstale":"2023-10-18T11:03:44.646406+0000","last_undegraded":"2023-10-18T11:03:44.646406+0000","last_fullsized":"2023-10-18T11:03:44.646406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":94,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-18T11:03:44.565457+0000","last_change":"2023-10-18T10:45:50.251375+0000","last_active":"2023-10-18T11:03:44.565457+0000","last_peered":"2023-10-18T11:03:44.565457+0000","last_clean":"2023-10-18T11:03:44.565457+0000","last_became_active":"2023-10-18T10:45:50.251055+0000","last_became_peered":"2023-10-18T10:45:50.251055+0000","last_unstale":"2023-10-18T11:03:44.565457+0000","last_undegraded":"2023-10-18T11:03:44.565457+0000","last_fullsized":"2023-10-18T11:03:44.565457+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:52:57.411141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":89,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-18T11:03:44.424751+0000","last_change":"2023-10-18T10:45:50.251310+0000","last_active":"2023-10-18T11:03:44.424751+0000","last_peered":"2023-10-18T11:03:44.424751+0000","last_clean":"2023-10-18T11:03:44.424751+0000","last_became_active":"2023-10-18T10:45:50.250823+0000","last_became_peered":"2023-10-18T10:45:50.250823+0000","last_unstale":"2023-10-18T11:03:44.424751+0000","last_undegraded":"2023-10-18T11:03:44.424751+0000","last_fullsized":"2023-10-18T11:03:44.424751+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:49.227171+0000","last_clean_scrub_stamp":"2023-10-18T10:45:49.227171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:55:28.550749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":250,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-18T11:03:44.565461+0000","last_change":"2023-10-18T10:45:44.563531+0000","last_active":"2023-10-18T11:03:44.565461+0000","last_peered":"2023-10-18T11:03:44.565461+0000","last_clean":"2023-10-18T11:03:44.565461+0000","last_became_active":"2023-10-18T10:45:44.562504+0000","last_became_peered":"2023-10-18T10:45:44.562504+0000","last_unstale":"2023-10-18T11:03:44.565461+0000","last_undegraded":"2023-10-18T11:03:44.565461+0000","last_fullsized":"2023-10-18T11:03:44.565461+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:45:43.181461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:45:43.181461+0000","last_clean_scrub_stamp":"2023-10-18T10:45:43.181461+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:03:56.712280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575069,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28292,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343548,"statfs":{"total":96636764160,"available":96607793152,"internally_reserved":0,"allocated":1294336,"data_stored":927466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:03:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70399999999999996,"15min":0.70499999999999996},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.40699999999999997},"max":{"1min":0.85599999999999998,"5min":0.93799999999999994,"15min":1.097},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.74399999999999999,"15min":0.72099999999999997},"min":{"1min":0.59599999999999997,"5min":0.434,"15min":0.432},"max":{"1min":0.89300000000000002,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.47599999999999998}]},{"osd":1,"last update":"Wed Oct 18 11:03:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72299999999999998,"15min":0.71699999999999997},"min":{"1min":0.57199999999999995,"5min":0.503,"15min":0.44400000000000001},"max":{"1min":0.88800000000000001,"5min":0.93300000000000005,"15min":0.95599999999999996},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.72499999999999998,"15min":0.73199999999999998},"min":{"1min":0.54600000000000004,"5min":0.52100000000000002,"15min":0.38200000000000001},"max":{"1min":0.94699999999999995,"5min":0.95499999999999996,"15min":0.99199999999999999},"last":0.59499999999999997}]},{"osd":2,"last update":"Wed Oct 18 11:03:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.73999999999999999,"15min":0.73599999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.39700000000000002},"max":{"1min":0.85499999999999998,"5min":0.97699999999999998,"15min":1.0760000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.74299999999999999,"15min":0.73699999999999999},"min":{"1min":0.57999999999999996,"5min":0.505,"15min":0.42099999999999999},"max":{"1min":0.95799999999999996,"5min":0.97699999999999998,"15min":1.0800000000000001},"last":0.57099999999999995}]},{"osd":3,"last update":"Wed Oct 18 11:03:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.76600000000000001,"15min":0.76200000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.92400000000000004,"5min":1.004,"15min":1.032},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.79900000000000004,"15min":0.79400000000000004},"min":{"1min":0.66200000000000003,"5min":0.50700000000000001,"15min":0.5},"max":{"1min":0.998,"5min":0.998,"15min":1.1060000000000001},"last":0.70599999999999996}]},{"osd":4,"last update":"Wed Oct 18 11:03:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58099999999999996,"15min":0.59599999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.34000000000000002},"max":{"1min":0.80600000000000005,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.745},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65900000000000003,"15min":0.66800000000000004},"min":{"1min":0.42699999999999999,"5min":0.38300000000000001,"15min":0.35799999999999998},"max":{"1min":0.83799999999999997,"5min":0.92800000000000005,"15min":1.046},"last":0.72399999999999998}]},{"osd":5,"last update":"Wed Oct 18 11:03:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58599999999999997,"15min":0.59699999999999998},"min":{"1min":0.42799999999999999,"5min":0.41399999999999998,"15min":0.375},"max":{"1min":0.79300000000000004,"5min":0.86599999999999999,"15min":0.95099999999999996},"last":0.622},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60999999999999999,"15min":0.625},"min":{"1min":0.45900000000000002,"5min":0.44,"15min":0.33100000000000002},"max":{"1min":0.76100000000000001,"5min":0.94799999999999995,"15min":0.95699999999999996},"last":0.68400000000000005}]},{"osd":6,"last update":"Wed Oct 18 11:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70199999999999996,"15min":0.69499999999999995},"min":{"1min":0.53100000000000003,"5min":0.504,"15min":0.44400000000000001},"max":{"1min":0.94699999999999995,"5min":0.99299999999999999,"15min":1.004},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.73899999999999999,"15min":0.753},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.85399999999999998,"5min":1.002,"15min":1.002},"last":0.78300000000000003}]}]},{"osd":6,"up_from":55,"seq":236223201383,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28288,"kb_used_data":688,"kb_used_omap":6,"kb_used_meta":27577,"kb_avail":94343552,"statfs":{"total":96636764160,"available":96607797248,"internally_reserved":0,"allocated":704512,"data_stored":338429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6699,"internal_metadata":28239317},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.754,"15min":0.71499999999999997},"min":{"1min":0.53900000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.96899999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73299999999999998,"15min":0.70199999999999996},"min":{"1min":0.56100000000000005,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.93899999999999995,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.81499999999999995}]},{"osd":1,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.76100000000000001,"15min":0.72499999999999998},"min":{"1min":0.61299999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96999999999999997,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.78200000000000003,"15min":0.76400000000000001},"min":{"1min":0.63900000000000001,"5min":0.54600000000000004,"15min":0.51300000000000001},"max":{"1min":1.0049999999999999,"5min":1.04,"15min":1.04},"last":0.76300000000000001}]},{"osd":2,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.73499999999999999,"15min":0.72499999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.48099999999999998},"max":{"1min":0.90100000000000002,"5min":1.024,"15min":1.024},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.77600000000000002,"15min":0.76300000000000001},"min":{"1min":0.56399999999999995,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.01,"5min":1.077,"15min":1.077},"last":0.79000000000000004}]},{"osd":3,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.75900000000000001,"15min":0.73599999999999999},"min":{"1min":0.58099999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.91100000000000003,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80100000000000005,"15min":0.76900000000000002},"min":{"1min":0.66000000000000003,"5min":0.57899999999999996,"15min":0.495},"max":{"1min":0.98799999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.73599999999999999}]},{"osd":4,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58899999999999997,"15min":0.63600000000000001},"min":{"1min":0.40200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78500000000000003,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61799999999999999,"15min":0.64500000000000002},"min":{"1min":0.45700000000000002,"5min":0.42999999999999999,"15min":0.40899999999999997},"max":{"1min":0.79600000000000004,"5min":0.91300000000000003,"15min":0.95099999999999996},"last":0.94199999999999995}]},{"osd":5,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64800000000000002,"15min":0.66900000000000004},"min":{"1min":0.48499999999999999,"5min":0.443,"15min":0.436},"max":{"1min":0.84499999999999997,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69499999999999995,"15min":0.72499999999999998},"min":{"1min":0.51000000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.90800000000000003,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.91900000000000004}]},{"osd":7,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71399999999999997,"15min":0.749},"min":{"1min":0.50700000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.89200000000000002,"5min":0.98799999999999999,"15min":1.0189999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73899999999999999,"15min":0.78100000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.871,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.97199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575069,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":338429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71099999999999997,"15min":0.69699999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.40200000000000002},"max":{"1min":0.89000000000000001,"5min":0.93400000000000005,"15min":0.94499999999999995},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.76700000000000002,"15min":0.746},"min":{"1min":0.61299999999999999,"5min":0.42799999999999999,"15min":0.372},"max":{"1min":1.014,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.80400000000000005}]},{"osd":1,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.746,"15min":0.74199999999999999},"min":{"1min":0.57799999999999996,"5min":0.502,"15min":0.35599999999999998},"max":{"1min":0.95199999999999996,"5min":1.115,"15min":1.115},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72099999999999997,"15min":0.71399999999999997},"min":{"1min":0.53900000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.89600000000000002,"5min":1.038,"15min":1.038},"last":0.60599999999999998}]},{"osd":2,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.77800000000000002,"15min":0.752},"min":{"1min":0.57299999999999995,"5min":0.47999999999999998,"15min":0.40799999999999997},"max":{"1min":0.96899999999999997,"5min":1.133,"15min":1.133},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.76300000000000001,"15min":0.76200000000000001},"min":{"1min":0.53100000000000003,"5min":0.52500000000000002,"15min":0.48099999999999998},"max":{"1min":0.98199999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.745}]},{"osd":3,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.78500000000000003,"15min":0.78600000000000003},"min":{"1min":0.621,"5min":0.53200000000000003,"15min":0.52900000000000003},"max":{"1min":0.98999999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76300000000000001},"min":{"1min":0.63700000000000001,"5min":0.55600000000000005,"15min":0.44800000000000001},"max":{"1min":0.92600000000000005,"5min":1.083,"15min":1.083},"last":0.86699999999999999}]},{"osd":4,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55500000000000005,"15min":0.54600000000000004},"min":{"1min":0.36199999999999999,"5min":0.35199999999999998,"15min":0.316},"max":{"1min":0.81299999999999994,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.69099999999999995,"15min":0.68200000000000005},"min":{"1min":0.47099999999999997,"5min":0.434,"15min":0.376},"max":{"1min":0.95399999999999996,"5min":1.099,"15min":1.099},"last":0.68899999999999995}]},{"osd":6,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72499999999999998,"15min":0.71999999999999997},"min":{"1min":0.49199999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.875,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.65100000000000002,"15min":0.65600000000000003},"min":{"1min":0.497,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88200000000000001,"5min":1.004,"15min":1.004},"last":0.66000000000000003}]},{"osd":7,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.753,"15min":0.73299999999999998},"min":{"1min":0.58699999999999997,"5min":0.46600000000000003,"15min":0.41899999999999998},"max":{"1min":0.98299999999999998,"5min":1.018,"15min":1.046},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.71199999999999997,"15min":0.69599999999999995},"min":{"1min":0.51500000000000001,"5min":0.45500000000000002,"15min":0.317},"max":{"1min":0.997,"5min":1.046,"15min":1.046},"last":0.82799999999999996}]}]},{"osd":4,"up_from":13,"seq":55834575069,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27704,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":700416,"data_stored":337098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.70199999999999996,"15min":0.69899999999999995},"min":{"1min":0.496,"5min":0.47999999999999998,"15min":0.39800000000000002},"max":{"1min":0.80900000000000005,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74099999999999999,"15min":0.72099999999999997},"min":{"1min":0.57399999999999995,"5min":0.376,"15min":0.376},"max":{"1min":0.88600000000000001,"5min":0.97299999999999998,"15min":1.018},"last":0.64600000000000002}]},{"osd":1,"last update":"Wed Oct 18 11:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73399999999999999,"15min":0.754},"min":{"1min":0.52800000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.91900000000000004,"5min":0.96299999999999997,"15min":1.0449999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.80300000000000005,"15min":0.78100000000000003},"min":{"1min":0.57599999999999996,"5min":0.52900000000000003,"15min":0.40200000000000002},"max":{"1min":0.995,"5min":1.02,"15min":1.03},"last":0.72799999999999998}]},{"osd":2,"last update":"Wed Oct 18 11:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.746,"15min":0.76300000000000001},"min":{"1min":0.46200000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92900000000000005,"5min":1.022,"15min":1.022},"last":0.753},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80900000000000005,"15min":0.79000000000000004},"min":{"1min":0.624,"5min":0.57299999999999995,"15min":0.52500000000000002},"max":{"1min":0.95399999999999996,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.84699999999999998}]},{"osd":3,"last update":"Wed Oct 18 11:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.78000000000000003,"15min":0.76100000000000001},"min":{"1min":0.52800000000000002,"5min":0.48099999999999998,"15min":0.439},"max":{"1min":0.92100000000000004,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.78400000000000003,"15min":0.79800000000000004},"min":{"1min":0.61699999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.97499999999999998,"5min":1.036,"15min":1.0940000000000001},"last":0.82799999999999996}]},{"osd":5,"last update":"Wed Oct 18 11:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.63,"15min":0.626},"min":{"1min":0.38900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82299999999999995,"5min":0.96499999999999997,"15min":1.048},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.66200000000000003,"15min":0.67700000000000005},"min":{"1min":0.39700000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.0620000000000001},"last":0.77000000000000002}]},{"osd":6,"last update":"Wed Oct 18 11:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63100000000000001,"15min":0.65600000000000003},"min":{"1min":0.441,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96199999999999997,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61799999999999999,"15min":0.65500000000000003},"min":{"1min":0.46999999999999997,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94299999999999995,"5min":1.0209999999999999,"15min":1.109},"last":0.71099999999999997}]},{"osd":7,"last update":"Wed Oct 18 11:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.68999999999999995,"15min":0.67400000000000004},"min":{"1min":0.502,"5min":0.45000000000000001,"15min":0.41599999999999998},"max":{"1min":1.0449999999999999,"5min":1.0549999999999999,"15min":1.1080000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.628,"15min":0.66000000000000003},"min":{"1min":0.44800000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.0209999999999999,"5min":1.036,"15min":1.129},"last":0.78300000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575068,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":700416,"data_stored":337119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.46700000000000003,"15min":0.46000000000000002},"min":{"1min":0.29099999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.52500000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52700000000000002,"15min":0.52000000000000002},"min":{"1min":0.33700000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.65100000000000002,"5min":0.78000000000000003,"15min":0.78900000000000003},"last":0.41799999999999998}]},{"osd":1,"last update":"Wed Oct 18 11:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.48799999999999999,"15min":0.48999999999999999},"min":{"1min":0.37,"5min":0.33200000000000002,"15min":0.316},"max":{"1min":0.59999999999999998,"5min":0.76700000000000002,"15min":0.80000000000000004},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.42699999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.60299999999999998,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.39400000000000002}]},{"osd":2,"last update":"Wed Oct 18 11:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.44900000000000001,"5min":0.376,"15min":0.371},"max":{"1min":0.71099999999999997,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61399999999999999,"15min":0.60999999999999999},"min":{"1min":0.40300000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.86599999999999999,"5min":0.874,"15min":0.874},"last":0.47799999999999998}]},{"osd":4,"last update":"Wed Oct 18 11:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.754,"15min":0.77200000000000002},"min":{"1min":0.45200000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.88600000000000001,"5min":1.111,"15min":1.1919999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.78700000000000003,"15min":0.79900000000000004},"min":{"1min":0.59099999999999997,"5min":0.495,"15min":0.495},"max":{"1min":0.86599999999999999,"5min":1.137,"15min":1.246},"last":0.75700000000000001}]},{"osd":5,"last update":"Wed Oct 18 11:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.80900000000000005,"15min":0.81799999999999995},"min":{"1min":0.60299999999999998,"5min":0.59099999999999997,"15min":0.55100000000000005},"max":{"1min":0.90800000000000003,"5min":1.119,"15min":1.1859999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.78900000000000003,"15min":0.79000000000000004},"min":{"1min":0.59599999999999997,"5min":0.58499999999999996,"15min":0.49099999999999999},"max":{"1min":0.93600000000000005,"5min":1.137,"15min":1.155},"last":0.80900000000000005}]},{"osd":6,"last update":"Wed Oct 18 11:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.77100000000000002,"15min":0.83099999999999996},"min":{"1min":0.53800000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.97699999999999998,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.81699999999999995,"15min":0.85199999999999998},"min":{"1min":0.65500000000000003,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.96599999999999997,"5min":1.25,"15min":1.25},"last":0.86499999999999999}]},{"osd":7,"last update":"Wed Oct 18 11:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.78800000000000003,"15min":0.80800000000000005},"min":{"1min":0.61199999999999999,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.98599999999999999,"5min":1.1599999999999999,"15min":1.2529999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.83999999999999997,"15min":0.84599999999999997},"min":{"1min":0.65500000000000003,"5min":0.63100000000000001,"15min":0.62},"max":{"1min":0.93500000000000005,"5min":1.171,"15min":1.2410000000000001},"last":0.90600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834575070,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":696320,"data_stored":335788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.49399999999999999,"15min":0.48399999999999999},"min":{"1min":0.35399999999999998,"5min":0.313,"15min":0.27700000000000002},"max":{"1min":0.67300000000000004,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56299999999999994,"15min":0.56399999999999995},"min":{"1min":0.42499999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.73699999999999999,"5min":0.78400000000000003,"15min":0.80400000000000005},"last":0.51700000000000002}]},{"osd":1,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53500000000000003,"15min":0.53800000000000003},"min":{"1min":0.379,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.75,"5min":0.75700000000000001,"15min":0.82999999999999996},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.51600000000000001,"15min":0.51500000000000001},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.34599999999999997},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.83699999999999997},"last":0.38200000000000001}]},{"osd":3,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57999999999999996,"15min":0.58099999999999996},"min":{"1min":0.46400000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79800000000000004,"5min":0.81599999999999995,"15min":0.91400000000000003},"last":0.435},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.45900000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.73699999999999999,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.41799999999999998}]},{"osd":4,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81399999999999995,"15min":0.80600000000000005},"min":{"1min":0.60799999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0700000000000001,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.78200000000000003,"15min":0.77600000000000002},"min":{"1min":0.61799999999999999,"5min":0.42799999999999999,"15min":0.36699999999999999},"max":{"1min":0.98899999999999999,"5min":1.29,"15min":1.29},"last":0.73999999999999999}]},{"osd":5,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.80900000000000005,"15min":0.79300000000000004},"min":{"1min":0.57399999999999995,"5min":0.56999999999999995,"15min":0.51700000000000002},"max":{"1min":0.90500000000000003,"5min":1.264,"15min":1.264},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.63300000000000001,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.99099999999999999,"5min":1.163,"15min":1.3120000000000001},"last":0.754}]},{"osd":6,"last update":"Wed Oct 18 11:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.80700000000000005,"15min":0.80300000000000005},"min":{"1min":0.60699999999999998,"5min":0.48599999999999999,"15min":0.42899999999999999},"max":{"1min":0.92600000000000005,"5min":1.107,"15min":1.1339999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.82299999999999995,"15min":0.81599999999999995},"min":{"1min":0.627,"5min":0.58499999999999996,"15min":0.57699999999999996},"max":{"1min":1.054,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.71599999999999997}]},{"osd":7,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81200000000000006,"15min":0.81999999999999995},"min":{"1min":0.65100000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0660000000000001,"5min":1.1299999999999999,"15min":1.2609999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.81999999999999995,"15min":0.80200000000000005},"min":{"1min":0.60099999999999998,"5min":0.58499999999999996,"15min":0.47499999999999998},"max":{"1min":1.026,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.67300000000000004}]}]},{"osd":1,"up_from":13,"seq":55834575069,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":700416,"data_stored":337098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.443,"15min":0.45300000000000001},"min":{"1min":0.311,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.63800000000000001,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.495,"15min":0.51000000000000001},"min":{"1min":0.34899999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.752,"5min":0.752,"15min":0.92500000000000004},"last":0.30099999999999999}]},{"osd":2,"last update":"Wed Oct 18 11:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46500000000000002,"15min":0.47999999999999998},"min":{"1min":0.39500000000000002,"5min":0.32600000000000001,"15min":0.32200000000000001},"max":{"1min":0.64700000000000002,"5min":0.74199999999999999,"15min":0.85599999999999998},"last":0.498},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.52800000000000002,"15min":0.54900000000000004},"min":{"1min":0.40799999999999997,"5min":0.314,"15min":0.314},"max":{"1min":0.753,"5min":0.83499999999999996,"15min":0.97199999999999998},"last":0.39500000000000002}]},{"osd":3,"last update":"Wed Oct 18 11:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.54600000000000004,"15min":0.56599999999999995},"min":{"1min":0.44500000000000001,"5min":0.40899999999999997,"15min":0.377},"max":{"1min":0.71099999999999997,"5min":0.82899999999999996,"15min":0.90800000000000003},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.626,"5min":0.57399999999999995,"15min":0.58999999999999997},"min":{"1min":0.48299999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80500000000000005,"5min":0.91500000000000004,"15min":0.93899999999999995},"last":0.52500000000000002}]},{"osd":4,"last update":"Wed Oct 18 11:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.80600000000000005,"15min":0.79400000000000004},"min":{"1min":0.68000000000000005,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0700000000000001,"5min":1.389,"15min":1.389},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.75900000000000001,"15min":0.77500000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.50900000000000001},"max":{"1min":1.145,"5min":1.349,"15min":1.349},"last":0.67700000000000005}]},{"osd":5,"last update":"Wed Oct 18 11:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.78900000000000003,"15min":0.79900000000000004},"min":{"1min":0.65600000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.998,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.68500000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1830000000000001},"last":0.63900000000000001}]},{"osd":6,"last update":"Wed Oct 18 11:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.77700000000000002,"15min":0.83799999999999997},"min":{"1min":0.63900000000000001,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.99199999999999999,"5min":1.1339999999999999,"15min":1.157},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.81599999999999995,"15min":0.84599999999999997},"min":{"1min":0.68500000000000005,"5min":0.60699999999999998,"15min":0.56999999999999995},"max":{"1min":1.0329999999999999,"5min":1.1060000000000001,"15min":1.278},"last":0.79700000000000004}]},{"osd":7,"last update":"Wed Oct 18 11:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.81100000000000005,"15min":0.81899999999999995},"min":{"1min":0.63600000000000001,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0549999999999999,"5min":1.1899999999999999,"15min":1.2410000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.81200000000000006,"15min":0.82099999999999995},"min":{"1min":0.70599999999999996,"5min":0.58999999999999997,"15min":0.57199999999999995},"max":{"1min":1.127,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.69799999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575069,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28292,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343548,"statfs":{"total":96636764160,"available":96607793152,"internally_reserved":0,"allocated":1294336,"data_stored":927466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 11:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.45200000000000001,"15min":0.45100000000000001},"min":{"1min":0.34200000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.63800000000000001,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.502,"5min":0.504,"15min":0.51100000000000001},"min":{"1min":0.33500000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.753,"5min":0.753,"15min":0.86899999999999999},"last":0.40799999999999997}]},{"osd":2,"last update":"Wed Oct 18 11:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.50800000000000001,"15min":0.51200000000000001},"min":{"1min":0.378,"5min":0.309,"15min":0.309},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.80500000000000005},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54900000000000004,"15min":0.55100000000000005},"min":{"1min":0.42699999999999999,"5min":0.40100000000000002,"15min":0.38100000000000001},"max":{"1min":0.68100000000000005,"5min":0.80400000000000005,"15min":0.86699999999999999},"last":0.34899999999999998}]},{"osd":3,"last update":"Wed Oct 18 11:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.71199999999999997,"5min":0.77400000000000002,"15min":0.81899999999999995},"last":0.443},{"interface":"front","average":{"1min":0.622,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.45000000000000001,"5min":0.436,"15min":0.40100000000000002},"max":{"1min":0.75,"5min":0.75,"15min":0.85599999999999998},"last":0.51100000000000001}]},{"osd":4,"last update":"Wed Oct 18 11:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.81200000000000006,"15min":0.80600000000000005},"min":{"1min":0.58499999999999996,"5min":0.53000000000000003,"15min":0.48999999999999999},"max":{"1min":1.0649999999999999,"5min":1.2390000000000001,"15min":1.2470000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.81399999999999995,"15min":0.80600000000000005},"min":{"1min":0.52900000000000003,"5min":0.50900000000000001,"15min":0.45700000000000002},"max":{"1min":1.0549999999999999,"5min":1.389,"15min":1.389},"last":0.77500000000000002}]},{"osd":5,"last update":"Wed Oct 18 11:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81799999999999995,"15min":0.80700000000000005},"min":{"1min":0.61299999999999999,"5min":0.56799999999999995,"15min":0.53200000000000003},"max":{"1min":1.0920000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.82599999999999996,"15min":0.82099999999999995},"min":{"1min":0.65500000000000003,"5min":0.56299999999999994,"15min":0.50600000000000001},"max":{"1min":1.0349999999999999,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.88300000000000001}]},{"osd":6,"last update":"Wed Oct 18 11:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.81599999999999995,"15min":0.82199999999999995},"min":{"1min":0.56399999999999995,"5min":0.54900000000000004,"15min":0.54800000000000004},"max":{"1min":0.95099999999999996,"5min":1.24,"15min":1.2629999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.58499999999999996},"max":{"1min":1.008,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.79000000000000004}]},{"osd":7,"last update":"Wed Oct 18 11:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.84499999999999997,"15min":0.83799999999999997},"min":{"1min":0.65500000000000003,"5min":0.59699999999999998,"15min":0.56899999999999995},"max":{"1min":1.1259999999999999,"5min":1.409,"15min":1.409},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.82199999999999995,"15min":0.81200000000000006},"min":{"1min":0.60099999999999998,"5min":0.53700000000000003,"15min":0.503},"max":{"1min":0.98999999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.86499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T11:03:55.301 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-18T10:45:49.227171+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=10, tm_min=45, tm_sec=49, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=11, tm_min=3, tm_sec=49, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T11:03:55.302 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-18T10:45:49.227171+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=10, tm_min=45, tm_sec=49, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=11, tm_min=3, tm_sec=49, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T11:03:55.302 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-18T10:45:49.227171+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=10, tm_min=45, tm_sec=49, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=11, tm_min=3, tm_sec=49, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T11:03:55.303 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-18T10:45:49.227171+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=10, tm_min=45, tm_sec=49, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=11, tm_min=3, tm_sec=49, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T11:03:55.303 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-18T10:45:49.227171+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=10, tm_min=45, tm_sec=49, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=11, tm_min=3, tm_sec=49, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T11:03:55.304 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-18T10:45:49.227171+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=10, tm_min=45, tm_sec=49, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=11, tm_min=3, tm_sec=49, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T11:03:55.304 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-18T10:45:49.227171+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=10, tm_min=45, tm_sec=49, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=11, tm_min=3, tm_sec=49, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T11:03:55.304 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-18T10:45:49.227171+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=10, tm_min=45, tm_sec=49, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=11, tm_min=3, tm_sec=49, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T11:03:55.305 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-18T10:45:43.181461+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=10, tm_min=45, tm_sec=43, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=11, tm_min=3, tm_sec=49, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T11:03:55.305 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T11:04:15.306 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T11:04:15.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.455+0000 7f4773f8f640 1 Processor -- start 2023-10-18T11:04:15.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.455+0000 7f4773f8f640 1 -- start start 2023-10-18T11:04:15.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.459+0000 7f4773f8f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f476c06a010 0x7f476c163960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:04:15.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.459+0000 7f4773f8f640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f476c0713c0 con 0x7f476c15d040 2023-10-18T11:04:15.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.459+0000 7f4773f8f640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f476c0c9d40 con 0x7f476c158c00 2023-10-18T11:04:15.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.459+0000 7f4773f8f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f476c15d4a0 con 0x7f476c06a010 2023-10-18T11:04:15.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.459+0000 7f477152a640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f476c15d040 0x7f476c069900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41990/0 (socket says 172.21.15.89:41990) 2023-10-18T11:04:15.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.459+0000 7f477152a640 1 -- 172.21.15.89:0/3115577290 learned_addr learned my addr 172.21.15.89:0/3115577290 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:04:15.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.459+0000 7f477252c640 1 --2- 172.21.15.89:0/3115577290 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f476c06a010 0x7f476c163960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:04:15.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.459+0000 7f4770d29640 1 -- 172.21.15.89:0/3115577290 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1284759211 0 0) 0x7f476c0713c0 con 0x7f476c15d040 2023-10-18T11:04:15.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.459+0000 7f4770d29640 1 -- 172.21.15.89:0/3115577290 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4758003530 con 0x7f476c15d040 2023-10-18T11:04:15.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.459+0000 7f4770d29640 1 -- 172.21.15.89:0/3115577290 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2115340269 0 0) 0x7f4758003530 con 0x7f476c15d040 2023-10-18T11:04:15.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.459+0000 7f4770d29640 1 -- 172.21.15.89:0/3115577290 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f476c06a010 msgr2=0x7f476c163960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:04:15.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.459+0000 7f4770d29640 1 --2- 172.21.15.89:0/3115577290 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f476c06a010 0x7f476c163960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:04:15.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.459+0000 7f4770d29640 1 -- 172.21.15.89:0/3115577290 >> v1:172.21.15.154:6789/0 conn(0x7f476c158c00 legacy=0x7f476c0c9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:04:15.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.459+0000 7f4770d29640 1 -- 172.21.15.89:0/3115577290 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476c0c9f80 con 0x7f476c15d040 2023-10-18T11:04:15.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.459+0000 7f4770d29640 1 -- 172.21.15.89:0/3115577290 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2235817348 0 0) 0x7f4768001680 con 0x7f476c15d040 2023-10-18T11:04:15.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.459+0000 7f4770d29640 1 -- 172.21.15.89:0/3115577290 >> v1:172.21.15.89:6789/0 conn(0x7f476c15d040 legacy=0x7f476c069900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:04:15.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.459+0000 7f4770d29640 1 --2- 172.21.15.89:0/3115577290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47580039b0 0x7f4758003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:04:15.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.459+0000 7f4770d29640 1 --2- 172.21.15.89:0/3115577290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47580047d0 0x7f4758004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:04:15.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.459+0000 7f4770d29640 1 --2- 172.21.15.89:0/3115577290 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f476c06a010 0x7f4758005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:04:15.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.459+0000 7f4770d29640 1 -- 172.21.15.89:0/3115577290 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c15d4a0 con 0x7f47580039b0 2023-10-18T11:04:15.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.459+0000 7f4770d29640 1 -- 172.21.15.89:0/3115577290 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475800dc80 con 0x7f47580047d0 2023-10-18T11:04:15.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.459+0000 7f4770d29640 1 -- 172.21.15.89:0/3115577290 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f475800de40 con 0x7f476c06a010 2023-10-18T11:04:15.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.459+0000 7f4771d2b640 1 --2- 172.21.15.89:0/3115577290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47580039b0 0x7f4758003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:04:15.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.459+0000 7f477152a640 1 --2- 172.21.15.89:0/3115577290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47580047d0 0x7f4758004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:04:15.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.459+0000 7f477252c640 1 --2- 172.21.15.89:0/3115577290 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f476c06a010 0x7f4758005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:04:15.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.459+0000 7f4771d2b640 1 -- 172.21.15.89:0/3115577290 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f476c06a010 msgr2=0x7f4758005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:04:15.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.459+0000 7f4771d2b640 1 --2- 172.21.15.89:0/3115577290 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f476c06a010 0x7f4758005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:04:15.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.459+0000 7f4771d2b640 1 -- 172.21.15.89:0/3115577290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47580047d0 msgr2=0x7f4758004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:04:15.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.459+0000 7f4771d2b640 1 --2- 172.21.15.89:0/3115577290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47580047d0 0x7f4758004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:04:15.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.459+0000 7f4771d2b640 1 -- 172.21.15.89:0/3115577290 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4758003530 con 0x7f47580039b0 2023-10-18T11:04:15.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.459+0000 7f477252c640 1 --2- 172.21.15.89:0/3115577290 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f476c06a010 0x7f4758005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:04:15.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.463+0000 7f4771d2b640 1 --2- 172.21.15.89:0/3115577290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47580039b0 0x7f4758003dd0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f47600015e0 tx=0x7f476001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4422cf1d8244e94 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:04:15.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.463+0000 7f4770d29640 1 -- 172.21.15.89:0/3115577290 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47600052d0 con 0x7f47580039b0 2023-10-18T11:04:15.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.463+0000 7f4770d29640 1 -- 172.21.15.89:0/3115577290 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47600a7050 con 0x7f47580039b0 2023-10-18T11:04:15.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.463+0000 7f4770d29640 1 -- 172.21.15.89:0/3115577290 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4760003600 con 0x7f47580039b0 2023-10-18T11:04:15.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.463+0000 7f4773f8f640 1 -- 172.21.15.89:0/3115577290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47580039b0 msgr2=0x7f4758003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:04:15.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.463+0000 7f4773f8f640 1 --2- 172.21.15.89:0/3115577290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47580039b0 0x7f4758003dd0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f47600015e0 tx=0x7f476001c260 comp rx=0 tx=0).stop 2023-10-18T11:04:15.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.463+0000 7f477252c640 1 -- 172.21.15.89:0/3115577290 reap_dead start 2023-10-18T11:04:15.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.463+0000 7f4773f8f640 1 -- 172.21.15.89:0/3115577290 shutdown_connections 2023-10-18T11:04:15.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.463+0000 7f4773f8f640 1 -- 172.21.15.89:0/3115577290 >> 172.21.15.89:0/3115577290 conn(0x7f476c0cb5a0 msgr2=0x7f476c0cb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:04:15.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.463+0000 7f4773f8f640 1 -- 172.21.15.89:0/3115577290 shutdown_connections 2023-10-18T11:04:15.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.463+0000 7f4773f8f640 1 -- 172.21.15.89:0/3115577290 wait complete. 2023-10-18T11:04:15.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.463+0000 7f4773f8f640 1 Processor -- start 2023-10-18T11:04:15.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.467+0000 7f4773f8f640 1 -- start start 2023-10-18T11:04:15.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.467+0000 7f4773f8f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47580039b0 0x7f476c159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:04:15.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.467+0000 7f4773f8f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f476c151e30 0x7f476c14e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:04:15.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.467+0000 7f4773f8f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f476c14ec10 0x7f476c0ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:04:15.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.467+0000 7f4773f8f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c15a2f0 con 0x7f476c14ec10 2023-10-18T11:04:15.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.467+0000 7f4773f8f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c152810 con 0x7f476c151e30 2023-10-18T11:04:15.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.467+0000 7f4773f8f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f476c0ace90 con 0x7f47580039b0 2023-10-18T11:04:15.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.467+0000 7f477252c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f476c14ec10 0x7f476c0ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:04:15.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.467+0000 7f477252c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f476c14ec10 0x7f476c0ae5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56630/0 (socket says 172.21.15.89:56630) 2023-10-18T11:04:15.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.467+0000 7f477252c640 1 -- 172.21.15.89:0/3794952600 learned_addr learned my addr 172.21.15.89:0/3794952600 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:04:15.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.467+0000 7f477152a640 1 --2- 172.21.15.89:0/3794952600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f476c151e30 0x7f476c14e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:04:15.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.467+0000 7f4771d2b640 1 --2- 172.21.15.89:0/3794952600 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47580039b0 0x7f476c159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:04:15.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.467+0000 7f477252c640 1 -- 172.21.15.89:0/3794952600 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47580039b0 msgr2=0x7f476c159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:04:15.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.467+0000 7f477252c640 1 --2- 172.21.15.89:0/3794952600 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47580039b0 0x7f476c159db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:04:15.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.467+0000 7f477252c640 1 -- 172.21.15.89:0/3794952600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f476c151e30 msgr2=0x7f476c14e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:04:15.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.467+0000 7f477252c640 1 --2- 172.21.15.89:0/3794952600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f476c151e30 0x7f476c14e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:04:15.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.467+0000 7f477252c640 1 -- 172.21.15.89:0/3794952600 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476c0ad170 con 0x7f476c14ec10 2023-10-18T11:04:15.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.467+0000 7f477252c640 1 --2- 172.21.15.89:0/3794952600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f476c14ec10 0x7f476c0ae5e0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f476c06ea30 tx=0x7f47640075c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:04:15.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.467+0000 7f475e7fc640 1 -- 172.21.15.89:0/3794952600 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47640039a0 con 0x7f476c14ec10 2023-10-18T11:04:15.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.467+0000 7f475e7fc640 1 -- 172.21.15.89:0/3794952600 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4764012040 con 0x7f476c14ec10 2023-10-18T11:04:15.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.467+0000 7f475e7fc640 1 -- 172.21.15.89:0/3794952600 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4764003d40 con 0x7f476c14ec10 2023-10-18T11:04:15.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.467+0000 7f4773f8f640 1 -- 172.21.15.89:0/3794952600 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f476c0ad450 con 0x7f476c14ec10 2023-10-18T11:04:15.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.467+0000 7f4773f8f640 1 -- 172.21.15.89:0/3794952600 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f476c14f1c0 con 0x7f476c14ec10 2023-10-18T11:04:15.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.467+0000 7f4773f8f640 1 -- 172.21.15.89:0/3794952600 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f476c04dbe0 con 0x7f476c14ec10 2023-10-18T11:04:15.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.471+0000 7f475e7fc640 1 -- 172.21.15.89:0/3794952600 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f4764004030 con 0x7f476c14ec10 2023-10-18T11:04:15.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.471+0000 7f475e7fc640 1 --2- 172.21.15.89:0/3794952600 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f47500656b0 0x7f4750067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:04:15.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.471+0000 7f475e7fc640 1 -- 172.21.15.89:0/3794952600 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f4764082a10 con 0x7f476c14ec10 2023-10-18T11:04:15.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.471+0000 7f4771d2b640 1 --2- 172.21.15.89:0/3794952600 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f47500656b0 0x7f4750067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:04:15.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.471+0000 7f4771d2b640 1 --2- 172.21.15.89:0/3794952600 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f47500656b0 0x7f4750067b70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f47600015b0 tx=0x7f47600011a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:04:15.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.475+0000 7f475e7fc640 1 -- 172.21.15.89:0/3794952600 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f4764050700 con 0x7f476c14ec10 2023-10-18T11:04:15.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.667+0000 7f4773f8f640 1 -- 172.21.15.89:0/3794952600 --> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f476c0ca820 con 0x7f47500656b0 2023-10-18T11:04:15.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.671+0000 7f475e7fc640 1 -- 172.21.15.89:0/3794952600 <== mgr.4099 v2:172.21.15.154:6832/1086608968 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72772 (secure 0 0 0) 0x7f47600052f0 con 0x7f47500656b0 2023-10-18T11:04:15.675 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T11:04:15.675 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-18T11:04:15.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.675+0000 7f4773f8f640 1 -- 172.21.15.89:0/3794952600 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f47500656b0 msgr2=0x7f4750067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:04:15.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.675+0000 7f4773f8f640 1 --2- 172.21.15.89:0/3794952600 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f47500656b0 0x7f4750067b70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f47600015b0 tx=0x7f47600011a0 comp rx=0 tx=0).stop 2023-10-18T11:04:15.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.675+0000 7f4773f8f640 1 -- 172.21.15.89:0/3794952600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f476c14ec10 msgr2=0x7f476c0ae5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:04:15.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.675+0000 7f4773f8f640 1 --2- 172.21.15.89:0/3794952600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f476c14ec10 0x7f476c0ae5e0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f476c06ea30 tx=0x7f47640075c0 comp rx=0 tx=0).stop 2023-10-18T11:04:15.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.675+0000 7f4773f8f640 1 -- 172.21.15.89:0/3794952600 shutdown_connections 2023-10-18T11:04:15.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.675+0000 7f4773f8f640 1 --2- 172.21.15.89:0/3794952600 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f47500656b0 0x7f4750067b70 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:04:15.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.675+0000 7f4773f8f640 1 --2- 172.21.15.89:0/3794952600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f476c14ec10 0x7f476c0ae5e0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:04:15.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.675+0000 7f4773f8f640 1 --2- 172.21.15.89:0/3794952600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f476c151e30 0x7f476c14e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:04:15.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.675+0000 7f4773f8f640 1 --2- 172.21.15.89:0/3794952600 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47580039b0 0x7f476c159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:04:15.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.675+0000 7f4773f8f640 1 -- 172.21.15.89:0/3794952600 >> 172.21.15.89:0/3794952600 conn(0x7f476c0cb5a0 msgr2=0x7f476c0703b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:04:15.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.675+0000 7f4773f8f640 1 -- 172.21.15.89:0/3794952600 shutdown_connections 2023-10-18T11:04:15.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.675+0000 7f4773f8f640 1 -- 172.21.15.89:0/3794952600 wait complete. 2023-10-18T11:04:15.691 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":609,"stamp":"2023-10-18T11:04:15.363825+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":223352,"kb_used_data":6636,"kb_used_omap":18,"kb_used_meta":216621,"kb_avail":754751368,"statfs":{"total":773094113280,"available":772865400832,"internally_reserved":0,"allocated":6795264,"data_stored":3878893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19166,"internal_metadata":221820194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002443"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":100,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-18T11:03:58.776921+0000","last_change":"2023-10-18T11:03:58.776921+0000","last_active":"2023-10-18T11:03:58.776921+0000","last_peered":"2023-10-18T11:03:58.776921+0000","last_clean":"2023-10-18T11:03:58.776921+0000","last_became_active":"2023-10-18T10:55:22.931232+0000","last_became_peered":"2023-10-18T10:55:22.931232+0000","last_unstale":"2023-10-18T11:03:58.776921+0000","last_undegraded":"2023-10-18T11:03:58.776921+0000","last_fullsized":"2023-10-18T11:03:58.776921+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:03:58.776773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:03:58.776773+0000","last_clean_scrub_stamp":"2023-10-18T11:03:58.776773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:18:23.944251+0000","scrub_duration":0.0022213519999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":114,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-18T11:04:01.711807+0000","last_change":"2023-10-18T11:04:01.711858+0000","last_active":"2023-10-18T11:04:01.711807+0000","last_peered":"2023-10-18T11:04:01.711807+0000","last_clean":"2023-10-18T11:04:01.711807+0000","last_became_active":"2023-10-18T10:55:22.928049+0000","last_became_peered":"2023-10-18T10:55:22.928049+0000","last_unstale":"2023-10-18T11:04:01.711807+0000","last_undegraded":"2023-10-18T11:04:01.711807+0000","last_fullsized":"2023-10-18T11:04:01.711807+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:04:01.711688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:04:01.711688+0000","last_clean_scrub_stamp":"2023-10-18T11:04:01.711688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:31:16.710852+0000","scrub_duration":0.0026656969999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-18T11:04:01.313625+0000","last_change":"2023-10-18T11:04:01.313705+0000","last_active":"2023-10-18T11:04:01.313625+0000","last_peered":"2023-10-18T11:04:01.313625+0000","last_clean":"2023-10-18T11:04:01.313625+0000","last_became_active":"2023-10-18T10:45:50.584872+0000","last_became_peered":"2023-10-18T10:45:50.584872+0000","last_unstale":"2023-10-18T11:04:01.313625+0000","last_undegraded":"2023-10-18T11:04:01.313625+0000","last_fullsized":"2023-10-18T11:04:01.313625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:04:01.313515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:04:01.313515+0000","last_clean_scrub_stamp":"2023-10-18T11:04:01.313515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:44:46.849995+0000","scrub_duration":0.0029171470000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-18T11:03:58.734644+0000","last_change":"2023-10-18T11:03:58.734714+0000","last_active":"2023-10-18T11:03:58.734644+0000","last_peered":"2023-10-18T11:03:58.734644+0000","last_clean":"2023-10-18T11:03:58.734644+0000","last_became_active":"2023-10-18T10:45:50.584638+0000","last_became_peered":"2023-10-18T10:45:50.584638+0000","last_unstale":"2023-10-18T11:03:58.734644+0000","last_undegraded":"2023-10-18T11:03:58.734644+0000","last_fullsized":"2023-10-18T11:03:58.734644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:03:58.734541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:03:58.734541+0000","last_clean_scrub_stamp":"2023-10-18T11:03:58.734541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:35:28.932862+0000","scrub_duration":0.0019846360000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":98,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-18T11:03:56.479132+0000","last_change":"2023-10-18T11:03:56.479190+0000","last_active":"2023-10-18T11:03:56.479132+0000","last_peered":"2023-10-18T11:03:56.479132+0000","last_clean":"2023-10-18T11:03:56.479132+0000","last_became_active":"2023-10-18T10:45:50.251111+0000","last_became_peered":"2023-10-18T10:45:50.251111+0000","last_unstale":"2023-10-18T11:03:56.479132+0000","last_undegraded":"2023-10-18T11:03:56.479132+0000","last_fullsized":"2023-10-18T11:03:56.479132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:03:56.479034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:03:56.479034+0000","last_clean_scrub_stamp":"2023-10-18T11:03:56.479034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:26:34.683854+0000","scrub_duration":0.0026093069999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-18T11:03:53.175728+0000","last_change":"2023-10-18T11:03:53.175787+0000","last_active":"2023-10-18T11:03:53.175728+0000","last_peered":"2023-10-18T11:03:53.175728+0000","last_clean":"2023-10-18T11:03:53.175728+0000","last_became_active":"2023-10-18T10:45:50.258607+0000","last_became_peered":"2023-10-18T10:45:50.258607+0000","last_unstale":"2023-10-18T11:03:53.175728+0000","last_undegraded":"2023-10-18T11:03:53.175728+0000","last_fullsized":"2023-10-18T11:03:53.175728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:03:53.175641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:03:53.175641+0000","last_clean_scrub_stamp":"2023-10-18T11:03:53.175641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:04:23.846667+0000","scrub_duration":0.0018055289999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-18T11:03:56.191883+0000","last_change":"2023-10-18T11:03:56.191933+0000","last_active":"2023-10-18T11:03:56.191883+0000","last_peered":"2023-10-18T11:03:56.191883+0000","last_clean":"2023-10-18T11:03:56.191883+0000","last_became_active":"2023-10-18T10:45:50.251055+0000","last_became_peered":"2023-10-18T10:45:50.251055+0000","last_unstale":"2023-10-18T11:03:56.191883+0000","last_undegraded":"2023-10-18T11:03:56.191883+0000","last_fullsized":"2023-10-18T11:03:56.191883+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:03:56.191769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:03:56.191769+0000","last_clean_scrub_stamp":"2023-10-18T11:03:56.191769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:05:37.409558+0000","scrub_duration":0.0029407510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":98,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-18T11:03:57.451931+0000","last_change":"2023-10-18T11:03:57.451989+0000","last_active":"2023-10-18T11:03:57.451931+0000","last_peered":"2023-10-18T11:03:57.451931+0000","last_clean":"2023-10-18T11:03:57.451931+0000","last_became_active":"2023-10-18T10:45:50.250823+0000","last_became_peered":"2023-10-18T10:45:50.250823+0000","last_unstale":"2023-10-18T11:03:57.451931+0000","last_undegraded":"2023-10-18T11:03:57.451931+0000","last_fullsized":"2023-10-18T11:03:57.451931+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:03:57.451807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:03:57.451807+0000","last_clean_scrub_stamp":"2023-10-18T11:03:57.451807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:27:41.637233+0000","scrub_duration":0.0029221939999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"59'119","reported_seq":261,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-18T11:03:55.218820+0000","last_change":"2023-10-18T11:03:55.218878+0000","last_active":"2023-10-18T11:03:55.218820+0000","last_peered":"2023-10-18T11:03:55.218820+0000","last_clean":"2023-10-18T11:03:55.218820+0000","last_became_active":"2023-10-18T10:45:44.562504+0000","last_became_peered":"2023-10-18T10:45:44.562504+0000","last_unstale":"2023-10-18T11:03:55.218820+0000","last_undegraded":"2023-10-18T11:03:55.218820+0000","last_fullsized":"2023-10-18T11:03:55.218820+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'119","last_scrub_stamp":"2023-10-18T11:03:55.218725+0000","last_deep_scrub":"59'119","last_deep_scrub_stamp":"2023-10-18T11:03:55.218725+0000","last_clean_scrub_stamp":"2023-10-18T11:03:55.218725+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:34:52.224582+0000","scrub_duration":0.008255386,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575073,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28280,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343560,"statfs":{"total":96636764160,"available":96607805440,"internally_reserved":0,"allocated":1294336,"data_stored":927466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:03:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70399999999999996,"15min":0.70499999999999996},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.40699999999999997},"max":{"1min":0.85599999999999998,"5min":0.93799999999999994,"15min":1.097},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.74399999999999999,"15min":0.72099999999999997},"min":{"1min":0.59599999999999997,"5min":0.434,"15min":0.432},"max":{"1min":0.89300000000000002,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.75700000000000001}]},{"osd":1,"last update":"Wed Oct 18 11:03:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72299999999999998,"15min":0.71699999999999997},"min":{"1min":0.57199999999999995,"5min":0.503,"15min":0.44400000000000001},"max":{"1min":0.88800000000000001,"5min":0.93300000000000005,"15min":0.95599999999999996},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.72499999999999998,"15min":0.73199999999999998},"min":{"1min":0.54600000000000004,"5min":0.52100000000000002,"15min":0.38200000000000001},"max":{"1min":0.94699999999999995,"5min":0.95499999999999996,"15min":0.99199999999999999},"last":0.83499999999999996}]},{"osd":2,"last update":"Wed Oct 18 11:03:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.73999999999999999,"15min":0.73599999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.39700000000000002},"max":{"1min":0.85499999999999998,"5min":0.97699999999999998,"15min":1.0760000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.74299999999999999,"15min":0.73699999999999999},"min":{"1min":0.57999999999999996,"5min":0.505,"15min":0.42099999999999999},"max":{"1min":0.95799999999999996,"5min":0.97699999999999998,"15min":1.0800000000000001},"last":0.85099999999999998}]},{"osd":3,"last update":"Wed Oct 18 11:03:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.76600000000000001,"15min":0.76200000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.92400000000000004,"5min":1.004,"15min":1.032},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.79900000000000004,"15min":0.79400000000000004},"min":{"1min":0.66200000000000003,"5min":0.50700000000000001,"15min":0.5},"max":{"1min":0.998,"5min":0.998,"15min":1.1060000000000001},"last":0.88600000000000001}]},{"osd":4,"last update":"Wed Oct 18 11:03:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58099999999999996,"15min":0.59599999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.34000000000000002},"max":{"1min":0.80600000000000005,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.626},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65900000000000003,"15min":0.66800000000000004},"min":{"1min":0.42699999999999999,"5min":0.38300000000000001,"15min":0.35799999999999998},"max":{"1min":0.83799999999999997,"5min":0.92800000000000005,"15min":1.046},"last":0.64500000000000002}]},{"osd":5,"last update":"Wed Oct 18 11:03:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58599999999999997,"15min":0.59699999999999998},"min":{"1min":0.42799999999999999,"5min":0.41399999999999998,"15min":0.375},"max":{"1min":0.79300000000000004,"5min":0.86599999999999999,"15min":0.95099999999999996},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60999999999999999,"15min":0.625},"min":{"1min":0.45900000000000002,"5min":0.44,"15min":0.33100000000000002},"max":{"1min":0.76100000000000001,"5min":0.94799999999999995,"15min":0.95699999999999996},"last":0.58599999999999997}]},{"osd":6,"last update":"Wed Oct 18 11:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70199999999999996,"15min":0.69499999999999995},"min":{"1min":0.53100000000000003,"5min":0.504,"15min":0.44400000000000001},"max":{"1min":0.94699999999999995,"5min":0.99299999999999999,"15min":1.004},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.73899999999999999,"15min":0.753},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.85399999999999998,"5min":1.002,"15min":1.002},"last":0.70899999999999996}]}]},{"osd":6,"up_from":55,"seq":236223201387,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28280,"kb_used_data":688,"kb_used_omap":6,"kb_used_meta":27577,"kb_avail":94343560,"statfs":{"total":96636764160,"available":96607805440,"internally_reserved":0,"allocated":704512,"data_stored":338429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6699,"internal_metadata":28239317},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.754,"15min":0.71499999999999997},"min":{"1min":0.53900000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.96899999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73299999999999998,"15min":0.70199999999999996},"min":{"1min":0.56100000000000005,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.93899999999999995,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.80700000000000005}]},{"osd":1,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.76100000000000001,"15min":0.72499999999999998},"min":{"1min":0.61299999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96999999999999997,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.78200000000000003,"15min":0.76400000000000001},"min":{"1min":0.63900000000000001,"5min":0.54600000000000004,"15min":0.51300000000000001},"max":{"1min":1.0049999999999999,"5min":1.04,"15min":1.04},"last":0.75600000000000001}]},{"osd":2,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.73499999999999999,"15min":0.72499999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.48099999999999998},"max":{"1min":0.90100000000000002,"5min":1.024,"15min":1.024},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.77600000000000002,"15min":0.76300000000000001},"min":{"1min":0.56399999999999995,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.01,"5min":1.077,"15min":1.077},"last":0.73599999999999999}]},{"osd":3,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.75900000000000001,"15min":0.73599999999999999},"min":{"1min":0.58099999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.91100000000000003,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80100000000000005,"15min":0.76900000000000002},"min":{"1min":0.66000000000000003,"5min":0.57899999999999996,"15min":0.495},"max":{"1min":0.98799999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.68700000000000006}]},{"osd":4,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58899999999999997,"15min":0.63600000000000001},"min":{"1min":0.40200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78500000000000003,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61799999999999999,"15min":0.64500000000000002},"min":{"1min":0.45700000000000002,"5min":0.42999999999999999,"15min":0.40899999999999997},"max":{"1min":0.79600000000000004,"5min":0.91300000000000003,"15min":0.95099999999999996},"last":0.48599999999999999}]},{"osd":5,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64800000000000002,"15min":0.66900000000000004},"min":{"1min":0.48499999999999999,"5min":0.443,"15min":0.436},"max":{"1min":0.84499999999999997,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69499999999999995,"15min":0.72499999999999998},"min":{"1min":0.51000000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.90800000000000003,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.57699999999999996}]},{"osd":7,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71399999999999997,"15min":0.749},"min":{"1min":0.50700000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.89200000000000002,"5min":0.98799999999999999,"15min":1.0189999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73899999999999999,"15min":0.78100000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.871,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.60399999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575073,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":704512,"data_stored":338429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71099999999999997,"15min":0.69699999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.40200000000000002},"max":{"1min":0.89000000000000001,"5min":0.93400000000000005,"15min":0.94499999999999995},"last":0.625},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.76700000000000002,"15min":0.746},"min":{"1min":0.61299999999999999,"5min":0.42799999999999999,"15min":0.372},"max":{"1min":1.014,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.80900000000000005}]},{"osd":1,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.746,"15min":0.74199999999999999},"min":{"1min":0.57799999999999996,"5min":0.502,"15min":0.35599999999999998},"max":{"1min":0.95199999999999996,"5min":1.115,"15min":1.115},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72099999999999997,"15min":0.71399999999999997},"min":{"1min":0.53900000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.89600000000000002,"5min":1.038,"15min":1.038},"last":0.69999999999999996}]},{"osd":2,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.77800000000000002,"15min":0.752},"min":{"1min":0.57299999999999995,"5min":0.47999999999999998,"15min":0.40799999999999997},"max":{"1min":0.96899999999999997,"5min":1.133,"15min":1.133},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.76300000000000001,"15min":0.76200000000000001},"min":{"1min":0.53100000000000003,"5min":0.52500000000000002,"15min":0.48099999999999998},"max":{"1min":0.98199999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.65600000000000003}]},{"osd":3,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.78500000000000003,"15min":0.78600000000000003},"min":{"1min":0.621,"5min":0.53200000000000003,"15min":0.52900000000000003},"max":{"1min":0.98999999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76300000000000001},"min":{"1min":0.63700000000000001,"5min":0.55600000000000005,"15min":0.44800000000000001},"max":{"1min":0.92600000000000005,"5min":1.083,"15min":1.083},"last":0.74099999999999999}]},{"osd":4,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55500000000000005,"15min":0.54600000000000004},"min":{"1min":0.36199999999999999,"5min":0.35199999999999998,"15min":0.316},"max":{"1min":0.81299999999999994,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.434},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.69099999999999995,"15min":0.68200000000000005},"min":{"1min":0.47099999999999997,"5min":0.434,"15min":0.376},"max":{"1min":0.95399999999999996,"5min":1.099,"15min":1.099},"last":0.71899999999999997}]},{"osd":6,"last update":"Wed Oct 18 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72499999999999998,"15min":0.71999999999999997},"min":{"1min":0.49199999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.875,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.65100000000000002,"15min":0.65600000000000003},"min":{"1min":0.497,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88200000000000001,"5min":1.004,"15min":1.004},"last":0.52600000000000002}]},{"osd":7,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.753,"15min":0.73299999999999998},"min":{"1min":0.58699999999999997,"5min":0.46600000000000003,"15min":0.41899999999999998},"max":{"1min":0.98299999999999998,"5min":1.018,"15min":1.046},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.745,"5min":0.71199999999999997,"15min":0.69599999999999995},"min":{"1min":0.51500000000000001,"5min":0.45500000000000002,"15min":0.317},"max":{"1min":0.997,"5min":1.046,"15min":1.046},"last":0.78600000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575073,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27704,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":700416,"data_stored":337098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.70199999999999996,"15min":0.69899999999999995},"min":{"1min":0.496,"5min":0.47999999999999998,"15min":0.39800000000000002},"max":{"1min":0.80900000000000005,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74099999999999999,"15min":0.72099999999999997},"min":{"1min":0.57399999999999995,"5min":0.376,"15min":0.376},"max":{"1min":0.88600000000000001,"5min":0.97299999999999998,"15min":1.018},"last":0.79900000000000004}]},{"osd":1,"last update":"Wed Oct 18 11:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73399999999999999,"15min":0.754},"min":{"1min":0.52800000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.91900000000000004,"5min":0.96299999999999997,"15min":1.0449999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.80300000000000005,"15min":0.78100000000000003},"min":{"1min":0.57599999999999996,"5min":0.52900000000000003,"15min":0.40200000000000002},"max":{"1min":0.995,"5min":1.02,"15min":1.03},"last":0.93999999999999995}]},{"osd":2,"last update":"Wed Oct 18 11:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.746,"15min":0.76300000000000001},"min":{"1min":0.46200000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92900000000000005,"5min":1.022,"15min":1.022},"last":0.997},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80900000000000005,"15min":0.79000000000000004},"min":{"1min":0.624,"5min":0.57299999999999995,"15min":0.52500000000000002},"max":{"1min":0.95399999999999996,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.97999999999999998}]},{"osd":3,"last update":"Wed Oct 18 11:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.78000000000000003,"15min":0.76100000000000001},"min":{"1min":0.52800000000000002,"5min":0.48099999999999998,"15min":0.439},"max":{"1min":0.92100000000000004,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.78400000000000003,"15min":0.79800000000000004},"min":{"1min":0.61699999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.97499999999999998,"5min":1.036,"15min":1.0940000000000001},"last":0.92500000000000004}]},{"osd":5,"last update":"Wed Oct 18 11:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.63,"15min":0.626},"min":{"1min":0.38900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82299999999999995,"5min":0.96499999999999997,"15min":1.048},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.66200000000000003,"15min":0.67700000000000005},"min":{"1min":0.39700000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.0620000000000001},"last":0.90800000000000003}]},{"osd":6,"last update":"Wed Oct 18 11:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63100000000000001,"15min":0.65600000000000003},"min":{"1min":0.441,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96199999999999997,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61799999999999999,"15min":0.65500000000000003},"min":{"1min":0.46999999999999997,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94299999999999995,"5min":1.0209999999999999,"15min":1.109},"last":0.96799999999999997}]},{"osd":7,"last update":"Wed Oct 18 11:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.68999999999999995,"15min":0.67400000000000004},"min":{"1min":0.502,"5min":0.45000000000000001,"15min":0.41599999999999998},"max":{"1min":1.0449999999999999,"5min":1.0549999999999999,"15min":1.1080000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.628,"15min":0.66000000000000003},"min":{"1min":0.44800000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.0209999999999999,"5min":1.036,"15min":1.129},"last":0.85899999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575072,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":700416,"data_stored":337119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:04:10 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.439,"15min":0.45300000000000001},"min":{"1min":0.28999999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.48799999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.438},{"interface":"front","average":{"1min":0.442,"5min":0.50900000000000001,"15min":0.51500000000000001},"min":{"1min":0.32400000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.58099999999999996,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.53600000000000003}]},{"osd":1,"last update":"Wed Oct 18 11:04:10 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.47599999999999998,"15min":0.48499999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.316},"max":{"1min":0.55200000000000005,"5min":0.76700000000000002,"15min":0.80000000000000004},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.52000000000000002,"15min":0.54000000000000004},"min":{"1min":0.35699999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.57899999999999996,"5min":0.80200000000000005,"15min":0.81399999999999995},"last":0.57899999999999996}]},{"osd":2,"last update":"Wed Oct 18 11:04:10 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.56200000000000006,"15min":0.57899999999999996},"min":{"1min":0.38300000000000001,"5min":0.376,"15min":0.371},"max":{"1min":0.63100000000000001,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.501,"5min":0.58199999999999996,"15min":0.60299999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.63200000000000001,"5min":0.874,"15min":0.874},"last":0.63200000000000001}]},{"osd":4,"last update":"Wed Oct 18 11:04:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.72899999999999998,"15min":0.76300000000000001},"min":{"1min":0.44600000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.89700000000000002,"5min":1.091,"15min":1.1919999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.77100000000000002,"15min":0.79000000000000004},"min":{"1min":0.55400000000000005,"5min":0.495,"15min":0.495},"max":{"1min":0.88700000000000001,"5min":1.105,"15min":1.246},"last":0.73199999999999998}]},{"osd":5,"last update":"Wed Oct 18 11:04:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.78100000000000003,"15min":0.80800000000000005},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.55100000000000005},"max":{"1min":1.0009999999999999,"5min":1.119,"15min":1.1859999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.77200000000000002,"15min":0.78300000000000003},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.49099999999999999},"max":{"1min":0.89100000000000001,"5min":1.137,"15min":1.155},"last":0.75600000000000001}]},{"osd":6,"last update":"Wed Oct 18 11:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.77100000000000002,"15min":0.83099999999999996},"min":{"1min":0.53800000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.97699999999999998,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.81699999999999995,"15min":0.85199999999999998},"min":{"1min":0.65500000000000003,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.96599999999999997,"5min":1.25,"15min":1.25},"last":0.80700000000000005}]},{"osd":7,"last update":"Wed Oct 18 11:04:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.76400000000000001,"15min":0.80200000000000005},"min":{"1min":0.54900000000000004,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.91200000000000003,"5min":1.121,"15min":1.2529999999999999},"last":0.746},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.82399999999999995,"15min":0.83999999999999997},"min":{"1min":0.63900000000000001,"5min":0.63100000000000001,"15min":0.62},"max":{"1min":0.95799999999999996,"5min":1.171,"15min":1.2410000000000001},"last":0.83499999999999996}]}]},{"osd":2,"up_from":13,"seq":55834575074,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":696320,"data_stored":335788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.49399999999999999,"15min":0.48399999999999999},"min":{"1min":0.35399999999999998,"5min":0.313,"15min":0.27700000000000002},"max":{"1min":0.67300000000000004,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.495},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56299999999999994,"15min":0.56399999999999995},"min":{"1min":0.42499999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.73699999999999999,"5min":0.78400000000000003,"15min":0.80400000000000005},"last":0.46999999999999997}]},{"osd":1,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53500000000000003,"15min":0.53800000000000003},"min":{"1min":0.379,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.75,"5min":0.75700000000000001,"15min":0.82999999999999996},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.51600000000000001,"15min":0.51500000000000001},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.34599999999999997},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.83699999999999997},"last":0.375}]},{"osd":3,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57999999999999996,"15min":0.58099999999999996},"min":{"1min":0.46400000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79800000000000004,"5min":0.81599999999999995,"15min":0.91400000000000003},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.45900000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.73699999999999999,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.435}]},{"osd":4,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81399999999999995,"15min":0.80600000000000005},"min":{"1min":0.60799999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0700000000000001,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.995},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.78200000000000003,"15min":0.77600000000000002},"min":{"1min":0.61799999999999999,"5min":0.42799999999999999,"15min":0.36699999999999999},"max":{"1min":0.98899999999999999,"5min":1.29,"15min":1.29},"last":0.79200000000000004}]},{"osd":5,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.80900000000000005,"15min":0.79300000000000004},"min":{"1min":0.57399999999999995,"5min":0.56999999999999995,"15min":0.51700000000000002},"max":{"1min":0.90500000000000003,"5min":1.264,"15min":1.264},"last":0.872},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.63300000000000001,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.99099999999999999,"5min":1.163,"15min":1.3120000000000001},"last":0.81000000000000005}]},{"osd":6,"last update":"Wed Oct 18 11:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.80700000000000005,"15min":0.80300000000000005},"min":{"1min":0.60699999999999998,"5min":0.48599999999999999,"15min":0.42899999999999999},"max":{"1min":0.92600000000000005,"5min":1.107,"15min":1.1339999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.82299999999999995,"15min":0.81599999999999995},"min":{"1min":0.627,"5min":0.58499999999999996,"15min":0.57699999999999996},"max":{"1min":1.054,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.93999999999999995}]},{"osd":7,"last update":"Wed Oct 18 11:03:13 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81200000000000006,"15min":0.81999999999999995},"min":{"1min":0.65100000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0660000000000001,"5min":1.1299999999999999,"15min":1.2609999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.81999999999999995,"15min":0.80200000000000005},"min":{"1min":0.60099999999999998,"5min":0.58499999999999996,"15min":0.47499999999999998},"max":{"1min":1.026,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.97299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575073,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":700416,"data_stored":337098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.443,"15min":0.45300000000000001},"min":{"1min":0.311,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.63800000000000001,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.495,"15min":0.51000000000000001},"min":{"1min":0.34899999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.752,"5min":0.752,"15min":0.92500000000000004},"last":0.57399999999999995}]},{"osd":2,"last update":"Wed Oct 18 11:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46500000000000002,"15min":0.47999999999999998},"min":{"1min":0.39500000000000002,"5min":0.32600000000000001,"15min":0.32200000000000001},"max":{"1min":0.64700000000000002,"5min":0.74199999999999999,"15min":0.85599999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.52800000000000002,"15min":0.54900000000000004},"min":{"1min":0.40799999999999997,"5min":0.314,"15min":0.314},"max":{"1min":0.753,"5min":0.83499999999999996,"15min":0.97199999999999998},"last":0.84299999999999997}]},{"osd":3,"last update":"Wed Oct 18 11:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.54600000000000004,"15min":0.56599999999999995},"min":{"1min":0.44500000000000001,"5min":0.40899999999999997,"15min":0.377},"max":{"1min":0.71099999999999997,"5min":0.82899999999999996,"15min":0.90800000000000003},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.57399999999999995,"15min":0.58999999999999997},"min":{"1min":0.48299999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80500000000000005,"5min":0.91500000000000004,"15min":0.93899999999999995},"last":0.754}]},{"osd":4,"last update":"Wed Oct 18 11:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.80600000000000005,"15min":0.79400000000000004},"min":{"1min":0.68000000000000005,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0700000000000001,"5min":1.389,"15min":1.389},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.75900000000000001,"15min":0.77500000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.50900000000000001},"max":{"1min":1.145,"5min":1.349,"15min":1.349},"last":0.70099999999999996}]},{"osd":5,"last update":"Wed Oct 18 11:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.78900000000000003,"15min":0.79900000000000004},"min":{"1min":0.65600000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.998,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.68500000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1830000000000001},"last":0.74099999999999999}]},{"osd":6,"last update":"Wed Oct 18 11:03:40 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.77700000000000002,"15min":0.83799999999999997},"min":{"1min":0.63900000000000001,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.99199999999999999,"5min":1.1339999999999999,"15min":1.157},"last":0.877},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.81599999999999995,"15min":0.84599999999999997},"min":{"1min":0.68500000000000005,"5min":0.60699999999999998,"15min":0.56999999999999995},"max":{"1min":1.0329999999999999,"5min":1.1060000000000001,"15min":1.278},"last":0.90000000000000002}]},{"osd":7,"last update":"Wed Oct 18 11:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.81100000000000005,"15min":0.81899999999999995},"min":{"1min":0.63600000000000001,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0549999999999999,"5min":1.1899999999999999,"15min":1.2410000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.81200000000000006,"15min":0.82099999999999995},"min":{"1min":0.70599999999999996,"5min":0.58999999999999997,"15min":0.57199999999999995},"max":{"1min":1.127,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.92200000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575073,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28280,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343560,"statfs":{"total":96636764160,"available":96607805440,"internally_reserved":0,"allocated":1294336,"data_stored":927466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 11:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.45200000000000001,"15min":0.45100000000000001},"min":{"1min":0.34200000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.63800000000000001,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.379},{"interface":"front","average":{"1min":0.502,"5min":0.504,"15min":0.51100000000000001},"min":{"1min":0.33500000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.753,"5min":0.753,"15min":0.86899999999999999},"last":0.47899999999999998}]},{"osd":2,"last update":"Wed Oct 18 11:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.50800000000000001,"15min":0.51200000000000001},"min":{"1min":0.378,"5min":0.309,"15min":0.309},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.80500000000000005},"last":0.434},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54900000000000004,"15min":0.55100000000000005},"min":{"1min":0.42699999999999999,"5min":0.40100000000000002,"15min":0.38100000000000001},"max":{"1min":0.68100000000000005,"5min":0.80400000000000005,"15min":0.86699999999999999},"last":0.59299999999999997}]},{"osd":3,"last update":"Wed Oct 18 11:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.71199999999999997,"5min":0.77400000000000002,"15min":0.81899999999999995},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.45000000000000001,"5min":0.436,"15min":0.40100000000000002},"max":{"1min":0.75,"5min":0.75,"15min":0.85599999999999998},"last":0.56599999999999995}]},{"osd":4,"last update":"Wed Oct 18 11:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.81200000000000006,"15min":0.80600000000000005},"min":{"1min":0.58499999999999996,"5min":0.53000000000000003,"15min":0.48999999999999999},"max":{"1min":1.0649999999999999,"5min":1.2390000000000001,"15min":1.2470000000000001},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.81399999999999995,"15min":0.80600000000000005},"min":{"1min":0.52900000000000003,"5min":0.50900000000000001,"15min":0.45700000000000002},"max":{"1min":1.0549999999999999,"5min":1.389,"15min":1.389},"last":0.95399999999999996}]},{"osd":5,"last update":"Wed Oct 18 11:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81799999999999995,"15min":0.80700000000000005},"min":{"1min":0.61299999999999999,"5min":0.56799999999999995,"15min":0.53200000000000003},"max":{"1min":1.0920000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.82599999999999996,"15min":0.82099999999999995},"min":{"1min":0.65500000000000003,"5min":0.56299999999999994,"15min":0.50600000000000001},"max":{"1min":1.0349999999999999,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.91100000000000003}]},{"osd":6,"last update":"Wed Oct 18 11:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.81599999999999995,"15min":0.82199999999999995},"min":{"1min":0.56399999999999995,"5min":0.54900000000000004,"15min":0.54800000000000004},"max":{"1min":0.95099999999999996,"5min":1.24,"15min":1.2629999999999999},"last":0.996},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.58499999999999996},"max":{"1min":1.008,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.93200000000000005}]},{"osd":7,"last update":"Wed Oct 18 11:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.84499999999999997,"15min":0.83799999999999997},"min":{"1min":0.65500000000000003,"5min":0.59699999999999998,"15min":0.56899999999999995},"max":{"1min":1.1259999999999999,"5min":1.409,"15min":1.409},"last":0.875},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.82199999999999995,"15min":0.81200000000000006},"min":{"1min":0.60099999999999998,"5min":0.53700000000000003,"15min":0.503},"max":{"1min":0.98999999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.89100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T11:04:15.695 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-18T11:04:15.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.831+0000 7f74369e1640 1 Processor -- start 2023-10-18T11:04:15.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.831+0000 7f74369e1640 1 -- start start 2023-10-18T11:04:15.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.831+0000 7f74369e1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74300d0bc0 0x7f74300cd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:04:15.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.831+0000 7f74369e1640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f743015da50 con 0x7f74300d0760 2023-10-18T11:04:15.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.831+0000 7f74369e1640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74300d3110 con 0x7f743015c940 2023-10-18T11:04:15.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.831+0000 7f74369e1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743015ce40 con 0x7f74300d0bc0 2023-10-18T11:04:15.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.831+0000 7f742f7fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f74300d0760 0x7f74300cd160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42000/0 (socket says 172.21.15.89:42000) 2023-10-18T11:04:15.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.831+0000 7f742f7fe640 1 -- 172.21.15.89:0/971241814 learned_addr learned my addr 172.21.15.89:0/971241814 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:04:15.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.831+0000 7f7434f7e640 1 --2- 172.21.15.89:0/971241814 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74300d0bc0 0x7f74300cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:04:15.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.831+0000 7f742effd640 1 -- 172.21.15.89:0/971241814 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 176125594 0 0) 0x7f743015da50 con 0x7f74300d0760 2023-10-18T11:04:15.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.831+0000 7f742effd640 1 -- 172.21.15.89:0/971241814 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7418003530 con 0x7f74300d0760 2023-10-18T11:04:15.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.831+0000 7f742effd640 1 -- 172.21.15.89:0/971241814 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3025794860 0 0) 0x7f74300d3110 con 0x7f743015c940 2023-10-18T11:04:15.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.831+0000 7f742effd640 1 -- 172.21.15.89:0/971241814 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f743015da50 con 0x7f743015c940 2023-10-18T11:04:15.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.831+0000 7f742effd640 1 -- 172.21.15.89:0/971241814 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1164421939 0 0) 0x7f7418003530 con 0x7f74300d0760 2023-10-18T11:04:15.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.831+0000 7f7434f7e640 1 -- 172.21.15.89:0/971241814 >> v1:172.21.15.154:6789/0 conn(0x7f743015c940 legacy=0x7f74300d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:04:15.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.831+0000 7f7434f7e640 1 -- 172.21.15.89:0/971241814 >> v1:172.21.15.89:6789/0 conn(0x7f74300d0760 legacy=0x7f74300cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:04:15.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.835+0000 7f7434f7e640 1 -- 172.21.15.89:0/971241814 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743015d120 con 0x7f74300d0bc0 2023-10-18T11:04:15.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.835+0000 7f7434f7e640 1 --2- 172.21.15.89:0/971241814 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74300d0bc0 0x7f74300cd870 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7428009dc0 tx=0x7f742801c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed5960a0584bd57e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:04:15.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.835+0000 7f742effd640 1 -- 172.21.15.89:0/971241814 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7424001680 con 0x7f74300d0bc0 2023-10-18T11:04:15.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.835+0000 7f742effd640 1 -- 172.21.15.89:0/971241814 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74300ce5c0 con 0x7f74300d0bc0 2023-10-18T11:04:15.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.835+0000 7f742effd640 1 -- 172.21.15.89:0/971241814 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74280a58a0 con 0x7f74300d0bc0 2023-10-18T11:04:15.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.835+0000 7f74369e1640 1 -- 172.21.15.89:0/971241814 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74300d0bc0 msgr2=0x7f74300cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:04:15.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.835+0000 7f74369e1640 1 --2- 172.21.15.89:0/971241814 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74300d0bc0 0x7f74300cd870 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7428009dc0 tx=0x7f742801c860 comp rx=0 tx=0).stop 2023-10-18T11:04:15.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.835+0000 7f74369e1640 1 -- 172.21.15.89:0/971241814 shutdown_connections 2023-10-18T11:04:15.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.835+0000 7f74369e1640 1 --2- 172.21.15.89:0/971241814 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74300d0bc0 0x7f74300cd870 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:04:15.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.835+0000 7f74369e1640 1 -- 172.21.15.89:0/971241814 >> 172.21.15.89:0/971241814 conn(0x7f7430089fd0 msgr2=0x7f743009b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:04:15.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.835+0000 7f74369e1640 1 -- 172.21.15.89:0/971241814 shutdown_connections 2023-10-18T11:04:15.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.835+0000 7f74369e1640 1 -- 172.21.15.89:0/971241814 wait complete. 2023-10-18T11:04:15.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.839+0000 7f74369e1640 1 Processor -- start 2023-10-18T11:04:15.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.839+0000 7f74369e1640 1 -- start start 2023-10-18T11:04:15.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.839+0000 7f74369e1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74300d0760 0x7f743015e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:04:15.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.839+0000 7f74369e1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74300d0bc0 0x7f74300c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:04:15.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.839+0000 7f74369e1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74301521c0 0x7f7430152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:04:15.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.839+0000 7f74369e1640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74300c8640 con 0x7f74300d0bc0 2023-10-18T11:04:15.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.839+0000 7f74369e1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7430154130 con 0x7f74300d0760 2023-10-18T11:04:15.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.839+0000 7f74369e1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74300c75a0 con 0x7f74301521c0 2023-10-18T11:04:15.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.839+0000 7f742f7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74300d0bc0 0x7f74300c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:04:15.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.839+0000 7f742f7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74300d0bc0 0x7f74300c8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56632/0 (socket says 172.21.15.89:56632) 2023-10-18T11:04:15.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.839+0000 7f742f7fe640 1 -- 172.21.15.89:0/1948541789 learned_addr learned my addr 172.21.15.89:0/1948541789 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:04:15.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.839+0000 7f742ffff640 1 --2- 172.21.15.89:0/1948541789 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74300d0760 0x7f743015e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:04:15.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.839+0000 7f7434f7e640 1 --2- 172.21.15.89:0/1948541789 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74301521c0 0x7f7430152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:04:15.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.839+0000 7f742f7fe640 1 -- 172.21.15.89:0/1948541789 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74301521c0 msgr2=0x7f7430152650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:04:15.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.839+0000 7f742f7fe640 1 --2- 172.21.15.89:0/1948541789 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74301521c0 0x7f7430152650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:04:15.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.839+0000 7f742f7fe640 1 -- 172.21.15.89:0/1948541789 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74300d0760 msgr2=0x7f743015e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:04:15.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.839+0000 7f742f7fe640 1 --2- 172.21.15.89:0/1948541789 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74300d0760 0x7f743015e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:04:15.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.839+0000 7f742f7fe640 1 -- 172.21.15.89:0/1948541789 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7428009a70 con 0x7f74300d0bc0 2023-10-18T11:04:15.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.839+0000 7f742f7fe640 1 --2- 172.21.15.89:0/1948541789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74300d0bc0 0x7f74300c8a30 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f742400c750 tx=0x7f742400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:04:15.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.839+0000 7f742cff9640 1 -- 172.21.15.89:0/1948541789 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7424005ae0 con 0x7f74300d0bc0 2023-10-18T11:04:15.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.843+0000 7f742cff9640 1 -- 172.21.15.89:0/1948541789 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7424005c40 con 0x7f74300d0bc0 2023-10-18T11:04:15.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.843+0000 7f74369e1640 1 -- 172.21.15.89:0/1948541789 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74300c7880 con 0x7f74300d0bc0 2023-10-18T11:04:15.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.843+0000 7f742cff9640 1 -- 172.21.15.89:0/1948541789 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7424007070 con 0x7f74300d0bc0 2023-10-18T11:04:15.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.843+0000 7f74369e1640 1 -- 172.21.15.89:0/1948541789 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74300c7e40 con 0x7f74300d0bc0 2023-10-18T11:04:15.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.843+0000 7f74369e1640 1 -- 172.21.15.89:0/1948541789 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f743004dbe0 con 0x7f74300d0bc0 2023-10-18T11:04:15.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.843+0000 7f742cff9640 1 -- 172.21.15.89:0/1948541789 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f74240071d0 con 0x7f74300d0bc0 2023-10-18T11:04:15.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.843+0000 7f742cff9640 1 --2- 172.21.15.89:0/1948541789 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f741c065680 0x7f741c067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:04:15.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.843+0000 7f742cff9640 1 -- 172.21.15.89:0/1948541789 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f7424082370 con 0x7f74300d0bc0 2023-10-18T11:04:15.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.843+0000 7f742ffff640 1 --2- 172.21.15.89:0/1948541789 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f741c065680 0x7f741c067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:04:15.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.847+0000 7f742ffff640 1 --2- 172.21.15.89:0/1948541789 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f741c065680 0x7f741c067b40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f74200011f0 tx=0x7f7420009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:04:15.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.847+0000 7f742cff9640 1 -- 172.21.15.89:0/1948541789 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f742404ffd0 con 0x7f74300d0bc0 2023-10-18T11:04:15.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:15.999+0000 7f74369e1640 1 -- 172.21.15.89:0/1948541789 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f74300d3330 con 0x7f74300d0bc0 2023-10-18T11:04:16.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.003+0000 7f742cff9640 1 -- 172.21.15.89:0/1948541789 <== mon.0 v2:172.21.15.89:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f742404ffd0 con 0x7f74300d0bc0 2023-10-18T11:04:16.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.003+0000 7f742cff9640 1 -- 172.21.15.89:0/1948541789 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v17)=0 v17) v1 ==== 123+0+0 (secure 0 0 0) 0x7f74240535f0 con 0x7f74300d0bc0 2023-10-18T11:04:16.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.007+0000 7f74369e1640 1 -- 172.21.15.89:0/1948541789 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f741c065680 msgr2=0x7f741c067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:04:16.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.007+0000 7f74369e1640 1 --2- 172.21.15.89:0/1948541789 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f741c065680 0x7f741c067b40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f74200011f0 tx=0x7f7420009040 comp rx=0 tx=0).stop 2023-10-18T11:04:16.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.007+0000 7f74369e1640 1 -- 172.21.15.89:0/1948541789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74300d0bc0 msgr2=0x7f74300c8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:04:16.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.007+0000 7f74369e1640 1 --2- 172.21.15.89:0/1948541789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74300d0bc0 0x7f74300c8a30 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f742400c750 tx=0x7f742400cc10 comp rx=0 tx=0).stop 2023-10-18T11:04:16.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.007+0000 7f74369e1640 1 -- 172.21.15.89:0/1948541789 shutdown_connections 2023-10-18T11:04:16.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.007+0000 7f74369e1640 1 --2- 172.21.15.89:0/1948541789 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f741c065680 0x7f741c067b40 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:04:16.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.007+0000 7f74369e1640 1 --2- 172.21.15.89:0/1948541789 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74301521c0 0x7f7430152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:04:16.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.007+0000 7f74369e1640 1 --2- 172.21.15.89:0/1948541789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74300d0bc0 0x7f74300c8a30 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:04:16.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.007+0000 7f74369e1640 1 --2- 172.21.15.89:0/1948541789 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74300d0760 0x7f743015e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:04:16.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.007+0000 7f74369e1640 1 -- 172.21.15.89:0/1948541789 >> 172.21.15.89:0/1948541789 conn(0x7f7430089fd0 msgr2=0x7f74300c59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:04:16.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.007+0000 7f74369e1640 1 -- 172.21.15.89:0/1948541789 shutdown_connections 2023-10-18T11:04:16.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.007+0000 7f74369e1640 1 -- 172.21.15.89:0/1948541789 wait complete. 2023-10-18T11:04:16.024 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-18T11:04:16.025 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T11:04:16.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.167+0000 7f41a73ba640 1 Processor -- start 2023-10-18T11:04:16.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.167+0000 7f41a73ba640 1 -- start start 2023-10-18T11:04:16.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.167+0000 7f41a73ba640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f41a00d0760 0x7f41a00d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:04:16.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.167+0000 7f41a73ba640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41a00d1100 con 0x7f41a015c940 2023-10-18T11:04:16.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.167+0000 7f41a73ba640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41a015da50 con 0x7f41a00cd160 2023-10-18T11:04:16.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.167+0000 7f41a73ba640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41a00d3110 con 0x7f41a00d0760 2023-10-18T11:04:16.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.167+0000 7f41a5156640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f41a015c940 0x7f41a00d2a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:42010/0 (socket says 172.21.15.89:42010) 2023-10-18T11:04:16.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.167+0000 7f41a5156640 1 -- 172.21.15.89:0/3140145488 learned_addr learned my addr 172.21.15.89:0/3140145488 (peer_addr_for_me v1:172.21.15.89:0/0) 2023-10-18T11:04:16.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.167+0000 7f41a4955640 1 --2- 172.21.15.89:0/3140145488 >> v2:172.21.15.154:3300/0 conn(0x7f41a00d0760 0x7f41a00d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:04:16.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.167+0000 7f4197fff640 1 -- 172.21.15.89:0/3140145488 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2744115855 0 0) 0x7f41a00d1100 con 0x7f41a015c940 2023-10-18T11:04:16.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.167+0000 7f4197fff640 1 -- 172.21.15.89:0/3140145488 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f418c003530 con 0x7f41a015c940 2023-10-18T11:04:16.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.167+0000 7f4197fff640 1 -- 172.21.15.89:0/3140145488 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1913065541 0 0) 0x7f418c003530 con 0x7f41a015c940 2023-10-18T11:04:16.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.167+0000 7f41a4955640 1 -- 172.21.15.89:0/3140145488 >> v1:172.21.15.154:6789/0 conn(0x7f41a00cd160 legacy=0x7f41a00cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:04:16.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.167+0000 7f41a4955640 1 -- 172.21.15.89:0/3140145488 >> v1:172.21.15.89:6789/0 conn(0x7f41a015c940 legacy=0x7f41a00d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:04:16.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.167+0000 7f41a4955640 1 -- 172.21.15.89:0/3140145488 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a015ce40 con 0x7f41a00d0760 2023-10-18T11:04:16.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.171+0000 7f41a4955640 1 --2- 172.21.15.89:0/3140145488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41a00d0760 0x7f41a00d0bc0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f41900097c0 tx=0x7f419001c220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32e2a4fc2ff15c56 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:04:16.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.171+0000 7f4197fff640 1 -- 172.21.15.89:0/3140145488 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c001a00 con 0x7f41a00d0760 2023-10-18T11:04:16.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.171+0000 7f4197fff640 1 -- 172.21.15.89:0/3140145488 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f41900a7050 con 0x7f41a00d0760 2023-10-18T11:04:16.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.171+0000 7f4197fff640 1 -- 172.21.15.89:0/3140145488 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41900ab640 con 0x7f41a00d0760 2023-10-18T11:04:16.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.171+0000 7f41a73ba640 1 -- 172.21.15.89:0/3140145488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41a00d0760 msgr2=0x7f41a00d0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:04:16.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.171+0000 7f41a73ba640 1 --2- 172.21.15.89:0/3140145488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41a00d0760 0x7f41a00d0bc0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f41900097c0 tx=0x7f419001c220 comp rx=0 tx=0).stop 2023-10-18T11:04:16.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.171+0000 7f41a73ba640 1 -- 172.21.15.89:0/3140145488 shutdown_connections 2023-10-18T11:04:16.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.171+0000 7f41a73ba640 1 --2- 172.21.15.89:0/3140145488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41a00d0760 0x7f41a00d0bc0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:04:16.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.171+0000 7f41a73ba640 1 -- 172.21.15.89:0/3140145488 >> 172.21.15.89:0/3140145488 conn(0x7f41a0089fd0 msgr2=0x7f41a009b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:04:16.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.171+0000 7f41a73ba640 1 -- 172.21.15.89:0/3140145488 shutdown_connections 2023-10-18T11:04:16.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.171+0000 7f41a73ba640 1 -- 172.21.15.89:0/3140145488 wait complete. 2023-10-18T11:04:16.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.175+0000 7f41a73ba640 1 Processor -- start 2023-10-18T11:04:16.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.175+0000 7f41a73ba640 1 -- start start 2023-10-18T11:04:16.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.175+0000 7f41a73ba640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41a00cd160 0x7f41a00cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:04:16.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.175+0000 7f41a73ba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f41a00d0760 0x7f41a01532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:04:16.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.175+0000 7f41a73ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41a00b7a30 0x7f41a006ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:04:16.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.175+0000 7f41a73ba640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a00cbe20 con 0x7f41a00cd160 2023-10-18T11:04:16.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.175+0000 7f41a73ba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a0153de0 con 0x7f41a00b7a30 2023-10-18T11:04:16.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.175+0000 7f41a73ba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a00c97c0 con 0x7f41a00d0760 2023-10-18T11:04:16.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.175+0000 7f41a5156640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41a00cd160 0x7f41a00cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:04:16.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.175+0000 7f41a5156640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41a00cd160 0x7f41a00cb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56644/0 (socket says 172.21.15.89:56644) 2023-10-18T11:04:16.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.175+0000 7f41a5156640 1 -- 172.21.15.89:0/2383351848 learned_addr learned my addr 172.21.15.89:0/2383351848 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-18T11:04:16.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.175+0000 7f41a5156640 1 -- 172.21.15.89:0/2383351848 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f41a00d0760 msgr2=0x7f41a01532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:04:16.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.175+0000 7f41a5156640 1 --2- 172.21.15.89:0/2383351848 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f41a00d0760 0x7f41a01532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:04:16.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.175+0000 7f41a5156640 1 -- 172.21.15.89:0/2383351848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41a00b7a30 msgr2=0x7f41a006ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:04:16.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.175+0000 7f41a5156640 1 --2- 172.21.15.89:0/2383351848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41a00b7a30 0x7f41a006ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:04:16.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.175+0000 7f41a5156640 1 -- 172.21.15.89:0/2383351848 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4190009420 con 0x7f41a00cd160 2023-10-18T11:04:16.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.175+0000 7f41a5156640 1 --2- 172.21.15.89:0/2383351848 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41a00cd160 0x7f41a00cb8e0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f419c00caf0 tx=0x7f419c002090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:04:16.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.175+0000 7f4195ffb640 1 -- 172.21.15.89:0/2383351848 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c00a700 con 0x7f41a00cd160 2023-10-18T11:04:16.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.175+0000 7f4195ffb640 1 -- 172.21.15.89:0/2383351848 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f419c00a880 con 0x7f41a00cd160 2023-10-18T11:04:16.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.175+0000 7f4195ffb640 1 -- 172.21.15.89:0/2383351848 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c00ac50 con 0x7f41a00cd160 2023-10-18T11:04:16.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.175+0000 7f41a73ba640 1 -- 172.21.15.89:0/2383351848 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41a00c9aa0 con 0x7f41a00cd160 2023-10-18T11:04:16.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.175+0000 7f41a73ba640 1 -- 172.21.15.89:0/2383351848 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a0153810 con 0x7f41a00cd160 2023-10-18T11:04:16.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.179+0000 7f41a73ba640 1 -- 172.21.15.89:0/2383351848 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41a00c9fc0 con 0x7f41a00cd160 2023-10-18T11:04:16.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.179+0000 7f4195ffb640 1 -- 172.21.15.89:0/2383351848 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) v1 ==== 84962+0+0 (secure 0 0 0) 0x7f419c0070d0 con 0x7f41a00cd160 2023-10-18T11:04:16.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.179+0000 7f4195ffb640 1 --2- 172.21.15.89:0/2383351848 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4184065680 0x7f4184067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:04:16.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.179+0000 7f4195ffb640 1 -- 172.21.15.89:0/2383351848 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f419c082440 con 0x7f41a00cd160 2023-10-18T11:04:16.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.179+0000 7f41a4955640 1 --2- 172.21.15.89:0/2383351848 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4184065680 0x7f4184067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:04:16.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.179+0000 7f41a4955640 1 --2- 172.21.15.89:0/2383351848 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4184065680 0x7f4184067b40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f41900ad000 tx=0x7f4190004740 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:04:16.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.183+0000 7f4195ffb640 1 -- 172.21.15.89:0/2383351848 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171304 (secure 0 0 0) 0x7f41a00c9fc0 con 0x7f41a00cd160 2023-10-18T11:04:16.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.363+0000 7f41a73ba640 1 -- 172.21.15.89:0/2383351848 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41a00c9fc0 con 0x7f41a00cd160 2023-10-18T11:04:16.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.363+0000 7f4195ffb640 1 -- 172.21.15.89:0/2383351848 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+3699 (secure 0 0 0) 0x7f419c018090 con 0x7f41a00cd160 2023-10-18T11:04:16.364 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-18T11:04:16.365 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":22,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5045,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/654729641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":654729641},{"type":"v1","addr":"172.21.15.154:6836","nonce":654729641}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/964566480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":964566480},{"type":"v1","addr":"172.21.15.154:6837","nonce":964566480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2261878538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2261878538},{"type":"v1","addr":"172.21.15.89:6833","nonce":2261878538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":6195,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2580347375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2580347375},{"type":"v1","addr":"172.21.15.89:6835","nonce":2580347375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[]} 2023-10-18T11:04:16.366 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 22 2023-10-18T11:04:16.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.367+0000 7f41a73ba640 1 -- 172.21.15.89:0/2383351848 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4184065680 msgr2=0x7f4184067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:04:16.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.367+0000 7f41a73ba640 1 --2- 172.21.15.89:0/2383351848 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4184065680 0x7f4184067b40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f41900ad000 tx=0x7f4190004740 comp rx=0 tx=0).stop 2023-10-18T11:04:16.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.367+0000 7f41a73ba640 1 -- 172.21.15.89:0/2383351848 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41a00cd160 msgr2=0x7f41a00cb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:04:16.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.367+0000 7f41a73ba640 1 --2- 172.21.15.89:0/2383351848 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41a00cd160 0x7f41a00cb8e0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f419c00caf0 tx=0x7f419c002090 comp rx=0 tx=0).stop 2023-10-18T11:04:16.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.367+0000 7f41a73ba640 1 -- 172.21.15.89:0/2383351848 shutdown_connections 2023-10-18T11:04:16.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.367+0000 7f41a73ba640 1 --2- 172.21.15.89:0/2383351848 >> [v2:172.21.15.154:6832/1086608968,v1:172.21.15.154:6833/1086608968] conn(0x7f4184065680 0x7f4184067b40 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:04:16.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.367+0000 7f41a73ba640 1 --2- 172.21.15.89:0/2383351848 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f41a00d0760 0x7f41a01532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:04:16.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.367+0000 7f41a73ba640 1 --2- 172.21.15.89:0/2383351848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41a00b7a30 0x7f41a006ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:04:16.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.367+0000 7f41a73ba640 1 --2- 172.21.15.89:0/2383351848 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41a00cd160 0x7f41a00cb8e0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:04:16.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.367+0000 7f41a73ba640 1 -- 172.21.15.89:0/2383351848 >> 172.21.15.89:0/2383351848 conn(0x7f41a0089fd0 msgr2=0x7f41a015e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:04:16.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.367+0000 7f41a73ba640 1 -- 172.21.15.89:0/2383351848 shutdown_connections 2023-10-18T11:04:16.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-18T11:04:16.367+0000 7f41a73ba640 1 -- 172.21.15.89:0/2383351848 wait complete. 2023-10-18T11:04:16.384 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-18T11:04:16.385 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-18T11:04:16.385 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-18T11:04:16.386 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T11:04:16.398 INFO:tasks.ceph.mds.a:Stopped 2023-10-18T11:04:16.398 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-18T11:04:16.398 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T11:04:16.455 INFO:tasks.ceph.mds.c:Stopped 2023-10-18T11:04:16.455 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-18T11:04:16.456 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T11:04:16.470 INFO:tasks.ceph.mds.b:Stopped 2023-10-18T11:04:16.471 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-18T11:04:16.471 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T11:04:16.525 INFO:tasks.ceph.mds.d:Stopped 2023-10-18T11:04:16.526 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-18T11:04:16.526 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-18T11:04:16.527 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T11:04:16.596 INFO:tasks.ceph.osd.0:Stopped 2023-10-18T11:04:16.596 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-18T11:04:16.597 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T11:04:16.687 INFO:tasks.ceph.osd.1:Stopped 2023-10-18T11:04:16.688 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-18T11:04:16.688 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T11:04:16.786 INFO:tasks.ceph.osd.2:Stopped 2023-10-18T11:04:16.787 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-18T11:04:16.787 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T11:04:16.843 INFO:tasks.ceph.osd.3:Stopped 2023-10-18T11:04:16.843 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-18T11:04:16.844 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T11:04:16.927 INFO:tasks.ceph.osd.4:Stopped 2023-10-18T11:04:16.928 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-18T11:04:16.928 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T11:04:17.022 INFO:tasks.ceph.osd.5:Stopped 2023-10-18T11:04:17.023 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-18T11:04:17.023 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T11:04:17.113 INFO:tasks.ceph.osd.6:Stopped 2023-10-18T11:04:17.113 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-18T11:04:17.114 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T11:04:22.537 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:04:22.541 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:04:22.859 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2023-10-18T11:04:23.216 INFO:tasks.ceph.osd.7:Stopped 2023-10-18T11:04:23.216 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-18T11:04:23.217 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-18T11:04:23.217 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T11:04:23.270 INFO:tasks.ceph.mgr.y:Stopped 2023-10-18T11:04:23.271 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-18T11:04:23.271 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T11:04:23.330 INFO:tasks.ceph.mgr.x:Stopped 2023-10-18T11:04:23.330 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-18T11:04:23.331 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-18T11:04:23.331 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T11:04:23.351 INFO:tasks.ceph.mon.a:Stopped 2023-10-18T11:04:23.352 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-18T11:04:23.352 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T11:04:23.368 INFO:tasks.ceph.mon.b:Stopped 2023-10-18T11:04:23.369 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-18T11:04:23.369 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T11:04:23.432 INFO:tasks.ceph.mon.c:Stopped 2023-10-18T11:04:23.433 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-18T11:04:23.434 DEBUG:teuthology.orchestra.run.smithi089:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'evicting unresponsive client' | egrep -v 'but it is still running' | egrep -v 'slow request' | egrep -v MDS_CLIENT_LATE_RELEASE | egrep -v 't responding to mclientcaps' | egrep -v 'file system flag refuse_client_session is set' | egrep -v 'Degraded data redundancy' | egrep -v MDS_CLIENTS_LAGGY | egrep -v 'Reduced data availability' | head -n 1 2023-10-18T11:04:23.470 INFO:teuthology.orchestra.run.smithi089.stdout:2023-10-18T10:45:55.739499+0000 mon.a (mon.0) 142 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-18T11:04:23.472 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-10-18T11:04:23.472 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi089.front.sepia.ceph.com 2023-10-18T11:04:23.473 DEBUG:teuthology.orchestra.run.smithi089:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-18T11:04:23.826 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi089.front.sepia.ceph.com 2023-10-18T11:04:23.827 DEBUG:teuthology.orchestra.run.smithi089:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-18T11:04:24.074 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi089.front.sepia.ceph.com 2023-10-18T11:04:24.074 DEBUG:teuthology.orchestra.run.smithi089:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-18T11:04:24.255 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi089.front.sepia.ceph.com 2023-10-18T11:04:24.256 DEBUG:teuthology.orchestra.run.smithi089:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-18T11:04:24.463 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi154.front.sepia.ceph.com 2023-10-18T11:04:24.464 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-18T11:04:24.699 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi154.front.sepia.ceph.com 2023-10-18T11:04:24.700 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-18T11:04:24.910 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi154.front.sepia.ceph.com 2023-10-18T11:04:24.910 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-18T11:04:25.092 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi154.front.sepia.ceph.com 2023-10-18T11:04:25.092 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-18T11:04:25.268 INFO:tasks.ceph:Archiving mon data... 2023-10-18T11:04:25.269 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/dparmar-2023-10-18_10:19:04-fs:functional-wip-63105-distro-default-smithi/7431296/data/mon.a.tgz 2023-10-18T11:04:25.270 DEBUG:teuthology.orchestra.run.smithi089:> mktemp 2023-10-18T11:04:25.279 INFO:teuthology.orchestra.run.smithi089.stdout:/tmp/tmp.bRmRG9dweS 2023-10-18T11:04:25.280 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.bRmRG9dweS 2023-10-18T11:04:25.397 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0666 /tmp/tmp.bRmRG9dweS 2023-10-18T11:04:25.477 DEBUG:teuthology.orchestra.remote:smithi089:/tmp/tmp.bRmRG9dweS is 279KB 2023-10-18T11:04:25.543 DEBUG:teuthology.orchestra.run.smithi089:> rm -fr /tmp/tmp.bRmRG9dweS 2023-10-18T11:04:25.552 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/dparmar-2023-10-18_10:19:04-fs:functional-wip-63105-distro-default-smithi/7431296/data/mon.b.tgz 2023-10-18T11:04:25.552 DEBUG:teuthology.orchestra.run.smithi154:> mktemp 2023-10-18T11:04:25.562 INFO:teuthology.orchestra.run.smithi154.stdout:/tmp/tmp.FVtxY0rj5C 2023-10-18T11:04:25.562 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.FVtxY0rj5C 2023-10-18T11:04:25.687 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0666 /tmp/tmp.FVtxY0rj5C 2023-10-18T11:04:25.763 DEBUG:teuthology.orchestra.remote:smithi154:/tmp/tmp.FVtxY0rj5C is 288KB 2023-10-18T11:04:25.829 DEBUG:teuthology.orchestra.run.smithi154:> rm -fr /tmp/tmp.FVtxY0rj5C 2023-10-18T11:04:25.838 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/dparmar-2023-10-18_10:19:04-fs:functional-wip-63105-distro-default-smithi/7431296/data/mon.c.tgz 2023-10-18T11:04:25.839 DEBUG:teuthology.orchestra.run.smithi154:> mktemp 2023-10-18T11:04:25.891 INFO:teuthology.orchestra.run.smithi154.stdout:/tmp/tmp.RRJiH0XOKv 2023-10-18T11:04:25.891 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.RRJiH0XOKv 2023-10-18T11:04:26.015 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0666 /tmp/tmp.RRJiH0XOKv 2023-10-18T11:04:26.095 DEBUG:teuthology.orchestra.remote:smithi154:/tmp/tmp.RRJiH0XOKv is 288KB 2023-10-18T11:04:26.158 DEBUG:teuthology.orchestra.run.smithi154:> rm -fr /tmp/tmp.RRJiH0XOKv 2023-10-18T11:04:26.167 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-18T11:04:26.168 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-18T11:04:26.172 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-18T11:04:26.231 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-18T11:04:26.232 DEBUG:teuthology.orchestra.run.smithi089:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-18T11:04:26.235 DEBUG:teuthology.orchestra.run.smithi154:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-18T11:04:26.267 INFO:teuthology.orchestra.run.smithi089.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-18T11:04:26.305 INFO:teuthology.orchestra.run.smithi154.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-18T11:04:26.312 INFO:tasks.ceph:Archiving crash dumps... 2023-10-18T11:04:26.314 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/lib/ceph/crash to /home/teuthworker/archive/dparmar-2023-10-18_10:19:04-fs:functional-wip-63105-distro-default-smithi/7431296/remote/smithi089/crash 2023-10-18T11:04:26.315 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-18T11:04:26.337 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/lib/ceph/crash to /home/teuthworker/archive/dparmar-2023-10-18_10:19:04-fs:functional-wip-63105-distro-default-smithi/7431296/remote/smithi154/crash 2023-10-18T11:04:26.338 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-18T11:04:26.374 INFO:tasks.ceph:Shutting down logrotate 2023-10-18T11:04:26.374 DEBUG:teuthology.orchestra.run.smithi089:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:04:26.383 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:04:26.482 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-18T11:04:26.503 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-18T11:04:26.523 INFO:tasks.ceph:Compressing logs... 2023-10-18T11:04:26.523 DEBUG:teuthology.orchestra.run.smithi089:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-18T11:04:26.547 DEBUG:teuthology.orchestra.run.smithi154:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-18T11:04:26.591 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38324.log 2023-10-18T11:04:26.592 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44467.log 2023-10-18T11:04:26.593 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.38324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.44350.log 2023-10-18T11:04:26.593 INFO:teuthology.orchestra.run.smithi154.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38324.log.gz 2023-10-18T11:04:26.594 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50962.log 2023-10-18T11:04:26.595 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.1.44467.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44467.log.gz 2023-10-18T11:04:26.595 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-10-18T11:04:26.596 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.3.44350.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.44350.log.gz 2023-10-18T11:04:26.596 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-10-18T11:04:26.601 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38180.log 2023-10-18T11:04:26.601 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44274.log 2023-10-18T11:04:26.602 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.38180.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38180.log.gz 2023-10-18T11:04:26.602 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38228.log 2023-10-18T11:04:26.602 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.1.44274.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44274.log.gz 2023-10-18T11:04:26.603 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-10-18T11:04:26.603 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.38228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38156.log 2023-10-18T11:04:26.603 INFO:teuthology.orchestra.run.smithi154.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38228.log.gz 2023-10-18T11:04:26.603 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44472.log 2023-10-18T11:04:26.605 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51884.log 2023-10-18T11:04:26.606 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.50962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45633.log 2023-10-18T11:04:26.606 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50962.log.gz 2023-10-18T11:04:26.606 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.51884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52165.log 2023-10-18T11:04:26.607 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51884.log.gz 2023-10-18T11:04:26.607 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47856.log 2023-10-18T11:04:26.607 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45633.log.gz 2023-10-18T11:04:26.607 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.52165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49563.log 2023-10-18T11:04:26.607 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52165.log.gz 2023-10-18T11:04:26.608 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46649.log 2023-10-18T11:04:26.608 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47856.log.gz 2023-10-18T11:04:26.608 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.49563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45034.log 2023-10-18T11:04:26.608 INFO:teuthology.orchestra.run.smithi089.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49563.log.gz 2023-10-18T11:04:26.609 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.46649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45425.log 2023-10-18T11:04:26.609 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46649.log.gz 2023-10-18T11:04:26.609 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48902.log 2023-10-18T11:04:26.610 INFO:teuthology.orchestra.run.smithi089.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45034.log.gz 2023-10-18T11:04:26.610 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49589.log 2023-10-18T11:04:26.610 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45425.log.gz 2023-10-18T11:04:26.610 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43673.log 2023-10-18T11:04:26.611 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48902.log.gz 2023-10-18T11:04:26.611 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.49589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50846.log 2023-10-18T11:04:26.611 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49589.log.gz 2023-10-18T11:04:26.613 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.38156.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38156.log.gz 2023-10-18T11:04:26.613 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-10-18T11:04:26.614 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.1.44472.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-18T11:04:26.614 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44472.log.gz 2023-10-18T11:04:26.614 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38252.log 2023-10-18T11:04:26.615 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-10-18T11:04:26.615 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.38252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38204.log 2023-10-18T11:04:26.615 INFO:teuthology.orchestra.run.smithi154.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38252.log.gz 2023-10-18T11:04:26.615 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-18T11:04:26.616 INFO:teuthology.orchestra.run.smithi154.stderr: 93.7% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-18T11:04:26.616 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.38204.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38204.log.gz 2023-10-18T11:04:26.616 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38276.log 2023-10-18T11:04:26.620 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-10-18T11:04:26.620 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.38276.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38276.log.gz 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-18T11:04:26.621 INFO:teuthology.orchestra.run.smithi154.stderr: 2023-10-18T11:04:26.621 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38300.log 2023-10-18T11:04:26.623 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.43673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48311.log 2023-10-18T11:04:26.624 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43673.log.gz 2023-10-18T11:04:26.624 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.50846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43901.log 2023-10-18T11:04:26.624 INFO:teuthology.orchestra.run.smithi089.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.50846.log.gz 2023-10-18T11:04:26.625 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49104.log 2023-10-18T11:04:26.625 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48311.log.gz 2023-10-18T11:04:26.625 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.43901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50064.log 2023-10-18T11:04:26.625 INFO:teuthology.orchestra.run.smithi089.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.43901.log.gz 2023-10-18T11:04:26.626 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.49104.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-10-18T11:04:26.626 INFO:teuthology.orchestra.run.smithi089.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49104.log.gz 2023-10-18T11:04:26.626 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.50064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45164.log 2023-10-18T11:04:26.626 INFO:teuthology.orchestra.run.smithi089.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.50064.log.gz 2023-10-18T11:04:26.627 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46050.log 2023-10-18T11:04:26.627 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45164.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45164.log.gz -5 2023-10-18T11:04:26.627 INFO:teuthology.orchestra.run.smithi089.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43927.log 2023-10-18T11:04:26.627 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.46050.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46050.log.gz 2023-10-18T11:04:26.628 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46102.log 2023-10-18T11:04:26.628 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.43927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51254.log 2023-10-18T11:04:26.628 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43927.log.gz 2023-10-18T11:04:26.629 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.46102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49130.log 2023-10-18T11:04:26.629 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46102.log.gz 2023-10-18T11:04:26.629 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.51254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50326.log 2023-10-18T11:04:26.629 INFO:teuthology.orchestra.run.smithi089.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51254.log.gz 2023-10-18T11:04:26.630 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.49130.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49130.log.gz 2023-10-18T11:04:26.630 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46857.log 2023-10-18T11:04:26.630 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.50326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49314.log 2023-10-18T11:04:26.630 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50326.log.gz 2023-10-18T11:04:26.631 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.46857.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46857.log.gz 2023-10-18T11:04:26.631 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51358.log 2023-10-18T11:04:26.631 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.49314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43847.log 2023-10-18T11:04:26.631 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49314.log.gz 2023-10-18T11:04:26.632 INFO:teuthology.orchestra.run.smithi089.stderr: 94.8% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-10-18T11:04:26.632 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.51358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49511.log 2023-10-18T11:04:26.632 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51358.log.gz 2023-10-18T11:04:26.632 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.43847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51492.log 2023-10-18T11:04:26.633 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43847.log.gz 2023-10-18T11:04:26.636 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.44355.log 2023-10-18T11:04:26.636 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.38300.log: 94.0% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-10-18T11:04:26.636 INFO:teuthology.orchestra.run.smithi154.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38300.log.gz 2023-10-18T11:04:26.637 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-10-18T11:04:26.637 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.3.44355.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.44355.log.gz 2023-10-18T11:04:26.637 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-10-18T11:04:26.637 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-10-18T11:04:26.638 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-osd.4.log: 94.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-10-18T11:04:26.638 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44265.log 2023-10-18T11:04:26.638 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.1.44265.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44265.log.gz 2023-10-18T11:04:26.643 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.49511.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-10-18T11:04:26.643 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49511.log.gz 2023-10-18T11:04:26.644 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.51492.log: gzip -5 89.4% --verbose -- replaced with /var/log/ceph/ceph-client.admin.51492.log.gz -- 2023-10-18T11:04:26.644 INFO:teuthology.orchestra.run.smithi089.stderr: /var/log/ceph/ceph-client.admin.50731.log 2023-10-18T11:04:26.644 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51832.log 2023-10-18T11:04:26.644 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.50731.log: gzip -5 87.7% --verbose -- replaced with /var/log/ceph/ceph-client.admin.50731.log.gz -- 2023-10-18T11:04:26.645 INFO:teuthology.orchestra.run.smithi089.stderr: /var/log/ceph/ceph-client.admin.44638.log 2023-10-18T11:04:26.645 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.51832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52269.log 2023-10-18T11:04:26.645 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51832.log.gz 2023-10-18T11:04:26.645 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.44638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51045.log 2023-10-18T11:04:26.646 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44638.log.gz 2023-10-18T11:04:26.646 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.52269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50123.log 2023-10-18T11:04:26.646 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52269.log.gz 2023-10-18T11:04:26.647 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.51045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49182.log 87.6% 2023-10-18T11:04:26.647 INFO:teuthology.orchestra.run.smithi089.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51045.log.gz 2023-10-18T11:04:26.647 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.50123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50244.log 2023-10-18T11:04:26.647 INFO:teuthology.orchestra.run.smithi089.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.50123.log.gz 2023-10-18T11:04:26.648 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.49182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50384.log 2023-10-18T11:04:26.648 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49182.log.gz 2023-10-18T11:04:26.648 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.50244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46024.log 2023-10-18T11:04:26.648 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50244.log.gz 2023-10-18T11:04:26.649 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.50384.log: 87.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.50384.log.gz 2023-10-18T11:04:26.649 INFO:teuthology.orchestra.run.smithi089.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46623.log 2023-10-18T11:04:26.649 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.46024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44769.log 2023-10-18T11:04:26.649 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46024.log.gz 2023-10-18T11:04:26.650 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.46623.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46623.log.gz 2023-10-18T11:04:26.650 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44534.log 2023-10-18T11:04:26.650 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.44769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47638.log 2023-10-18T11:04:26.650 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44769.log.gz 2023-10-18T11:04:26.651 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.44534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49342.log 2023-10-18T11:04:26.651 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44534.log.gz 2023-10-18T11:04:26.651 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52008.log 2023-10-18T11:04:26.652 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47638.log.gz 2023-10-18T11:04:26.652 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.49342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50557.log 2023-10-18T11:04:26.652 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49342.log.gz 2023-10-18T11:04:26.652 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.52008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44743.log 2023-10-18T11:04:26.653 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52008.log.gz 2023-10-18T11:04:26.653 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.50557.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50557.log.gz 2023-10-18T11:04:26.653 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45407.log 2023-10-18T11:04:26.659 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.44743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46701.log 2023-10-18T11:04:26.660 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44743.log.gz 2023-10-18T11:04:26.660 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45407.log: gzip 89.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.45407.log.gz --verbose 2023-10-18T11:04:26.660 INFO:teuthology.orchestra.run.smithi089.stderr: -- /var/log/ceph/ceph-client.admin.45199.log 2023-10-18T11:04:26.661 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.46701.log: gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46701.log.gz -5 2023-10-18T11:04:26.661 INFO:teuthology.orchestra.run.smithi089.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49937.log 2023-10-18T11:04:26.661 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47352.log 2023-10-18T11:04:26.662 INFO:teuthology.orchestra.run.smithi089.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45199.log.gz 2023-10-18T11:04:26.662 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.49937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51623.log 2023-10-18T11:04:26.662 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49937.log.gz 2023-10-18T11:04:26.663 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47586.log 2023-10-18T11:04:26.663 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47352.log.gz 2023-10-18T11:04:26.663 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.51623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45920.log 2023-10-18T11:04:26.663 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51623.log.gz 2023-10-18T11:04:26.664 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48649.log 2023-10-18T11:04:26.664 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47586.log.gz 2023-10-18T11:04:26.664 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43590.log 2023-10-18T11:04:26.665 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45920.log.gz 2023-10-18T11:04:26.665 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48649.log: gzip -5 --verbose -- 88.9% /var/log/ceph/ceph-client.admin.45210.log -- replaced with /var/log/ceph/ceph-client.admin.48649.log.gz 2023-10-18T11:04:26.665 INFO:teuthology.orchestra.run.smithi089.stderr: 2023-10-18T11:04:26.665 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.43590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44229.log 2023-10-18T11:04:26.666 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43590.log.gz 2023-10-18T11:04:26.666 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45210.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45210.log.gz 2023-10-18T11:04:26.666 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48779.log 2023-10-18T11:04:26.667 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.44229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51097.log 2023-10-18T11:04:26.667 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44229.log.gz 2023-10-18T11:04:26.667 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51858.log 2023-10-18T11:04:26.667 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48779.log.gz 2023-10-18T11:04:26.668 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.51097.log: gzip 87.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.51097.log.gz --verbose 2023-10-18T11:04:26.668 INFO:teuthology.orchestra.run.smithi089.stderr: -- /var/log/ceph/ceph-client.admin.52321.log 2023-10-18T11:04:26.668 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.51858.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51858.log.gz 2023-10-18T11:04:26.669 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46753.log 2023-10-18T11:04:26.669 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.52321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49402.log 2023-10-18T11:04:26.669 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52321.log.gz 2023-10-18T11:04:26.674 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.46753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49078.log 2023-10-18T11:04:26.674 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46753.log.gz 2023-10-18T11:04:26.675 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.49402.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49402.log.gz 2023-10-18T11:04:26.675 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43875.log 2023-10-18T11:04:26.675 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.49078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45608.log 2023-10-18T11:04:26.676 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49078.log.gz 2023-10-18T11:04:26.676 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.43875.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43875.log.gz 2023-10-18T11:04:26.676 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-10-18T11:04:26.677 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48850.log 2023-10-18T11:04:26.677 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45608.log.gz 2023-10-18T11:04:26.677 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47939.log 2023-10-18T11:04:26.677 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49691.log 2023-10-18T11:04:26.678 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48850.log.gz 2023-10-18T11:04:26.678 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47939.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47939.log.gz 2023-10-18T11:04:26.678 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46232.log 2023-10-18T11:04:26.679 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.49691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48202.log 2023-10-18T11:04:26.679 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49691.log.gz 2023-10-18T11:04:26.679 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.46232.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46232.log.gz 2023-10-18T11:04:26.679 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49428.log 2023-10-18T11:04:26.680 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48202.log: gzip -5 --verbose 88.5% -- -- replaced with /var/log/ceph/ceph-client.admin.48202.log.gz /var/log/ceph/ceph-client.admin.51702.log 2023-10-18T11:04:26.680 INFO:teuthology.orchestra.run.smithi089.stderr: 2023-10-18T11:04:26.680 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.49428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51226.log 2023-10-18T11:04:26.680 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49428.log.gz 2023-10-18T11:04:26.681 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.51702.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51702.log.gz 2023-10-18T11:04:26.681 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-10-18T11:04:26.681 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.0.51226.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51226.log.gz 2023-10-18T11:04:26.685 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50184.log 2023-10-18T11:04:26.685 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44904.log 2023-10-18T11:04:26.685 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.50184.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50184.log.gz 2023-10-18T11:04:26.686 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48228.log 2023-10-18T11:04:26.686 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.44904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48545.log 2023-10-18T11:04:26.686 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44904.log.gz 2023-10-18T11:04:26.686 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48228.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48228.log.gz 2023-10-18T11:04:26.687 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48441.log 2023-10-18T11:04:26.687 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52453.log 2023-10-18T11:04:26.687 INFO:teuthology.orchestra.run.smithi089.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48545.log.gz 2023-10-18T11:04:26.688 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48441.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48441.log.gz 2023-10-18T11:04:26.688 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47196.log 2023-10-18T11:04:26.688 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.52453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45138.log 2023-10-18T11:04:26.688 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52453.log.gz 2023-10-18T11:04:26.691 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47196.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47196.log.gz 2023-10-18T11:04:26.691 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38157.log 2023-10-18T11:04:26.691 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45138.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45138.log.gz 2023-10-18T11:04:26.691 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50270.log 2023-10-18T11:04:26.692 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.38157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44930.log 2023-10-18T11:04:26.692 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38157.log.gz 2023-10-18T11:04:26.692 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.50270.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50270.log.gz 2023-10-18T11:04:26.693 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51466.log 2023-10-18T11:04:26.693 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.44930.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44930.log.gz 2023-10-18T11:04:26.694 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47404.log 2023-10-18T11:04:26.695 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.51466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50616.log 2023-10-18T11:04:26.695 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51466.log.gz 2023-10-18T11:04:26.695 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47404.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47404.log.gz 2023-10-18T11:04:26.696 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45212.log 2023-10-18T11:04:26.697 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.50616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44612.log 2023-10-18T11:04:26.697 INFO:teuthology.orchestra.run.smithi089.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50616.log.gz 2023-10-18T11:04:26.698 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45212.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45212.log.gz 2023-10-18T11:04:26.698 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46206.log 2023-10-18T11:04:26.699 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.44612.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44612.log.gz 2023-10-18T11:04:26.699 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43953.log 2023-10-18T11:04:26.699 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.46206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38047.log 2023-10-18T11:04:26.700 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46206.log.gz 2023-10-18T11:04:26.701 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.43953.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43953.log.gz 2023-10-18T11:04:26.701 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50327.log 2023-10-18T11:04:26.702 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.38047.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.38047.log.gz 2023-10-18T11:04:26.702 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49737.log 2023-10-18T11:04:26.703 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.50327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44982.log 2023-10-18T11:04:26.704 INFO:teuthology.orchestra.run.smithi089.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.50327.log.gz 2023-10-18T11:04:26.704 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.0.49737.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49737.log.gz 2023-10-18T11:04:26.705 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46727.log 2023-10-18T11:04:26.706 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.44982.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44982.log.gz 2023-10-18T11:04:26.706 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44795.log 2023-10-18T11:04:26.707 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.46727.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46727.log.gz 2023-10-18T11:04:26.708 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49641.log 2023-10-18T11:04:26.709 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.44795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48623.log 2023-10-18T11:04:26.709 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44795.log.gz 2023-10-18T11:04:26.710 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.49641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49050.log 2023-10-18T11:04:26.711 INFO:teuthology.orchestra.run.smithi089.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.49641.log.gz 2023-10-18T11:04:26.711 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48623.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48623.log.gz 2023-10-18T11:04:26.712 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44035.log 2023-10-18T11:04:26.713 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.49050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43648.log 2023-10-18T11:04:26.713 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49050.log.gz 2023-10-18T11:04:26.714 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.44035.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44035.log.gz 2023-10-18T11:04:26.715 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52401.log 2023-10-18T11:04:26.715 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.43648.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43648.log.gz 2023-10-18T11:04:26.716 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43821.log 2023-10-18T11:04:26.717 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.52401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44203.log 2023-10-18T11:04:26.717 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52401.log.gz 2023-10-18T11:04:26.718 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.43821.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43821.log.gz 2023-10-18T11:04:26.719 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51180.log 2023-10-18T11:04:26.720 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.44203.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44203.log.gz 2023-10-18T11:04:26.720 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46516.log 2023-10-18T11:04:26.721 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.51180.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51180.log.gz 2023-10-18T11:04:26.722 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48043.log 2023-10-18T11:04:26.722 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.46516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48285.log 2023-10-18T11:04:26.723 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46516.log.gz 2023-10-18T11:04:26.724 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48043.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48043.log.gz 2023-10-18T11:04:26.724 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49838.log 2023-10-18T11:04:26.725 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48285.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48285.log.gz 2023-10-18T11:04:26.726 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47882.log 2023-10-18T11:04:26.727 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.49838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48124.log 2023-10-18T11:04:26.727 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49838.log.gz 2023-10-18T11:04:26.728 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47882.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47882.log.gz 2023-10-18T11:04:26.728 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47804.log 2023-10-18T11:04:26.729 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48124.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48124.log.gz 2023-10-18T11:04:26.730 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50185.log 2023-10-18T11:04:26.734 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47804.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47804.log.gz 2023-10-18T11:04:26.734 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48934.log 2023-10-18T11:04:26.735 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.50185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49024.log 2023-10-18T11:04:26.735 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50185.log.gz 2023-10-18T11:04:26.735 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48934.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48934.log.gz 2023-10-18T11:04:26.735 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47752.log 2023-10-18T11:04:26.737 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.49024.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49024.log.gz 2023-10-18T11:04:26.737 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45086.log 2023-10-18T11:04:26.737 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47752.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47752.log.gz 2023-10-18T11:04:26.738 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50499.log 2023-10-18T11:04:26.739 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45086.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45086.log.gz 2023-10-18T11:04:26.739 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50787.log 2023-10-18T11:04:26.740 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.50499.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50499.log.gz 2023-10-18T11:04:26.740 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50122.log 2023-10-18T11:04:26.741 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.50787.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50787.log.gz 2023-10-18T11:04:26.742 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52635.log 2023-10-18T11:04:26.743 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.50122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49907.log 2023-10-18T11:04:26.743 INFO:teuthology.orchestra.run.smithi089.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50122.log.gz 2023-10-18T11:04:26.744 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.52635.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52635.log.gz 2023-10-18T11:04:26.744 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-10-18T11:04:26.746 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.49907.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49907.log.gz 2023-10-18T11:04:26.746 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45112.log 2023-10-18T11:04:26.747 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45414.log 2023-10-18T11:04:26.749 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52479.log 2023-10-18T11:04:26.749 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45112.log.gz 2023-10-18T11:04:26.750 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45414.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.33235.log 2023-10-18T11:04:26.750 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45414.log.gz 2023-10-18T11:04:26.752 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.52479.log: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.52479.log.gz -5 2023-10-18T11:04:26.752 INFO:teuthology.orchestra.run.smithi089.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48150.log 2023-10-18T11:04:26.753 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph.tmp-client.admin.33235.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.33235.log.gz 2023-10-18T11:04:26.753 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47430.log 2023-10-18T11:04:26.755 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49864.log 2023-10-18T11:04:26.755 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48150.log.gz 2023-10-18T11:04:26.756 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47430.log: gzip -5 88.1% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.47430.log.gz /var/log/ceph/ceph-client.admin.44069.log 2023-10-18T11:04:26.756 INFO:teuthology.orchestra.run.smithi089.stderr: 2023-10-18T11:04:26.757 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.49864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45208.log 2023-10-18T11:04:26.758 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49864.log.gz 2023-10-18T11:04:26.759 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.44069.log: gzip -5 --verbose -- 88.6% /var/log/ceph/ceph-client.admin.44255.log 2023-10-18T11:04:26.759 INFO:teuthology.orchestra.run.smithi089.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44069.log.gz 2023-10-18T11:04:26.760 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45842.log 2023-10-18T11:04:26.760 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45208.log.gz 2023-10-18T11:04:26.762 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.44255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47300.log 2023-10-18T11:04:26.762 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44255.log.gz 2023-10-18T11:04:26.763 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44125.log 2023-10-18T11:04:26.764 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45842.log.gz 2023-10-18T11:04:26.764 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44873.log 2023-10-18T11:04:26.765 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47300.log.gz 2023-10-18T11:04:26.766 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.44125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46779.log 2023-10-18T11:04:26.766 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44125.log.gz 2023-10-18T11:04:26.767 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.44873.log: gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44873.log.gz -5 2023-10-18T11:04:26.768 INFO:teuthology.orchestra.run.smithi089.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51597.log 2023-10-18T11:04:26.769 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.46779.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46779.log.gz 2023-10-18T11:04:26.769 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47170.log 2023-10-18T11:04:26.771 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.51597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50498.log 2023-10-18T11:04:26.771 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51597.log.gz 2023-10-18T11:04:26.772 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47692.log 2023-10-18T11:04:26.772 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47170.log.gz 2023-10-18T11:04:26.774 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.50498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46935.log 2023-10-18T11:04:26.774 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50498.log.gz 2023-10-18T11:04:26.775 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47692.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47692.log.gz 2023-10-18T11:04:26.775 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44151.log 2023-10-18T11:04:26.776 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.46935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51518.log 2023-10-18T11:04:26.777 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46935.log.gz 2023-10-18T11:04:26.777 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.44151.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44151.log.gz 2023-10-18T11:04:26.778 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51231.log 2023-10-18T11:04:26.779 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.51518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48176.log 2023-10-18T11:04:26.780 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51518.log.gz 2023-10-18T11:04:26.780 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.0.51231.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51231.log.gz 2023-10-18T11:04:26.781 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52609.log 2023-10-18T11:04:26.782 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48176.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48176.log.gz 2023-10-18T11:04:26.783 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50961.log 2023-10-18T11:04:26.784 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.52609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44690.log 2023-10-18T11:04:26.784 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52609.log.gz 2023-10-18T11:04:26.785 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.50961.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-10-18T11:04:26.786 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50961.log.gz 2023-10-18T11:04:26.787 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.44690.log: 89.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.44690.log.gz 2023-10-18T11:04:26.787 INFO:teuthology.orchestra.run.smithi089.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49484.log 2023-10-18T11:04:26.788 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44560.log 2023-10-18T11:04:26.790 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.49484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50063.log 2023-10-18T11:04:26.790 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49484.log.gz 2023-10-18T11:04:26.791 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.44560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.46603.log 2023-10-18T11:04:26.791 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44560.log.gz 2023-10-18T11:04:26.792 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.50063.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50063.log.gz 2023-10-18T11:04:26.793 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46496.log 2023-10-18T11:04:26.793 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.2.46603.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.46603.log.gz 2023-10-18T11:04:26.795 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.0.46496.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46496.log.gz 2023-10-18T11:04:26.796 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52427.log 2023-10-18T11:04:26.797 INFO:teuthology.orchestra.run.smithi154.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-10-18T11:04:26.797 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45424.log 2023-10-18T11:04:26.798 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.52427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44095.log 2023-10-18T11:04:26.799 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52427.log.gz 2023-10-18T11:04:26.800 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49537.log 2023-10-18T11:04:26.800 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45424.log.gz 2023-10-18T11:04:26.801 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.44095.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44095.log.gz 2023-10-18T11:04:26.802 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.49537.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49537.log.gz 2023-10-18T11:04:26.803 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48069.log 2023-10-18T11:04:26.804 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49615.log 2023-10-18T11:04:26.806 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48337.log 2023-10-18T11:04:26.806 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48069.log.gz 2023-10-18T11:04:26.807 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.49615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52685.log 2023-10-18T11:04:26.808 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49615.log.gz 2023-10-18T11:04:26.809 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51019.log 2023-10-18T11:04:26.809 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48337.log.gz 2023-10-18T11:04:26.811 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.52685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46258.log 2023-10-18T11:04:26.811 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52685.log.gz 2023-10-18T11:04:26.812 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.51019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48098.log 2023-10-18T11:04:26.812 INFO:teuthology.orchestra.run.smithi089.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.51019.log.gz 2023-10-18T11:04:26.813 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.46258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47666.log 2023-10-18T11:04:26.814 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46258.log.gz 2023-10-18T11:04:26.815 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51780.log 2023-10-18T11:04:26.815 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48098.log.gz 2023-10-18T11:04:26.816 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50298.log 2023-10-18T11:04:26.816 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47666.log.gz 2023-10-18T11:04:26.818 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.51780.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51780.log.gz 2023-10-18T11:04:26.818 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51071.log 2023-10-18T11:04:26.819 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.50298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50556.log 2023-10-18T11:04:26.819 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50298.log.gz 2023-10-18T11:04:26.821 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.51071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50005.log 2023-10-18T11:04:26.821 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51071.log.gz 2023-10-18T11:04:26.822 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.50556.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50556.log.gz 2023-10-18T11:04:26.822 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44717.log 2023-10-18T11:04:26.824 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.50005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45216.log 2023-10-18T11:04:26.824 INFO:teuthology.orchestra.run.smithi089.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.50005.log.gz 2023-10-18T11:04:26.825 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.44717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44847.log 2023-10-18T11:04:26.825 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44717.log.gz 2023-10-18T11:04:26.826 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45216.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45216.log.gz 2023-10-18T11:04:26.828 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.44847.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44847.log.gz 2023-10-18T11:04:26.828 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52217.log 2023-10-18T11:04:26.829 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51332.log 2023-10-18T11:04:26.831 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.52217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50902.log 2023-10-18T11:04:26.832 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52217.log.gz 2023-10-18T11:04:26.833 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.51332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47014.log 2023-10-18T11:04:26.833 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51332.log.gz 2023-10-18T11:04:26.834 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.50902.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50902.log.gz 2023-10-18T11:04:26.835 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47014.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47014.log.gz 2023-10-18T11:04:26.836 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46076.log 2023-10-18T11:04:26.837 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46284.log 2023-10-18T11:04:26.839 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.46076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51955.log 2023-10-18T11:04:26.839 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46076.log.gz 2023-10-18T11:04:26.840 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.46284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46675.log 2023-10-18T11:04:26.841 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46284.log.gz 2023-10-18T11:04:26.842 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.51955.log: gzip -5 --verbose 89.3% -- -- replaced with /var/log/ceph/ceph-client.admin.51955.log.gz /var/log/ceph/ceph-client.admin.44956.log 2023-10-18T11:04:26.842 INFO:teuthology.orchestra.run.smithi089.stderr: 2023-10-18T11:04:26.843 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.46675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49286.log 2023-10-18T11:04:26.844 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46675.log.gz 2023-10-18T11:04:26.845 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.44956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50730.log 2023-10-18T11:04:26.845 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44956.log.gz 2023-10-18T11:04:26.846 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.49286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44307.log 2023-10-18T11:04:26.846 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49286.log.gz 2023-10-18T11:04:26.847 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.50730.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-18T11:04:26.848 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50730.log.gz 2023-10-18T11:04:26.849 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.44307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45619.log 2023-10-18T11:04:26.850 INFO:teuthology.orchestra.run.smithi089.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.44307.log.gz 2023-10-18T11:04:26.852 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.45619.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45619.log.gz 2023-10-18T11:04:26.853 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47274.log 2023-10-18T11:04:26.853 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-18T11:04:26.855 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47274.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47274.log.gz 2023-10-18T11:04:26.856 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48254.log 2023-10-18T11:04:26.857 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46154.log 2023-10-18T11:04:26.859 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44007.log 2023-10-18T11:04:26.859 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48254.log.gz 2023-10-18T11:04:26.860 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.46154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47248.log 2023-10-18T11:04:26.860 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46154.log.gz 2023-10-18T11:04:26.861 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.44007.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44007.log.gz 2023-10-18T11:04:26.862 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52138.log 2023-10-18T11:04:26.863 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47248.log: gzip 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47248.log.gz -5 2023-10-18T11:04:26.864 INFO:teuthology.orchestra.run.smithi089.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48805.log 2023-10-18T11:04:26.865 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.52138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45946.log 2023-10-18T11:04:26.865 INFO:teuthology.orchestra.run.smithi089.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52138.log.gz 2023-10-18T11:04:26.866 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48805.log: gzip 87.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.48805.log.gz --verbose 2023-10-18T11:04:26.866 INFO:teuthology.orchestra.run.smithi089.stderr: -- /var/log/ceph/ceph-client.admin.48998.log 2023-10-18T11:04:26.867 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47092.log 2023-10-18T11:04:26.868 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45946.log.gz 2023-10-18T11:04:26.869 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48998.log: gzip 88.9% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.48998.log.gz -- 2023-10-18T11:04:26.869 INFO:teuthology.orchestra.run.smithi089.stderr: /var/log/ceph/ceph-client.admin.50441.log 2023-10-18T11:04:26.870 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45409.log 2023-10-18T11:04:26.871 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47092.log.gz 2023-10-18T11:04:26.872 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.50441.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.50441.log.gz -5 2023-10-18T11:04:26.872 INFO:teuthology.orchestra.run.smithi089.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45218.log 2023-10-18T11:04:26.874 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45409.log: gzip -5 88.1% --verbose -- replaced with /var/log/ceph/ceph-client.admin.45409.log.gz -- 2023-10-18T11:04:26.874 INFO:teuthology.orchestra.run.smithi089.stderr: /var/log/ceph/ceph-client.admin.46909.log 2023-10-18T11:04:26.875 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44821.log 2023-10-18T11:04:26.875 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45218.log.gz 2023-10-18T11:04:26.876 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.46909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49376.log 2023-10-18T11:04:26.877 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46909.log.gz 2023-10-18T11:04:26.878 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.44821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52243.log 2023-10-18T11:04:26.878 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44821.log.gz 2023-10-18T11:04:26.879 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.49376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48467.log 2023-10-18T11:04:26.879 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49376.log.gz 2023-10-18T11:04:26.880 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.52243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49812.log 2023-10-18T11:04:26.881 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52243.log.gz 2023-10-18T11:04:26.882 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52661.log 2023-10-18T11:04:26.882 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48467.log.gz 2023-10-18T11:04:26.884 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.49812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45636.log 2023-10-18T11:04:26.884 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49812.log.gz 2023-10-18T11:04:26.885 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.52661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51306.log 2023-10-18T11:04:26.885 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52661.log.gz 2023-10-18T11:04:26.886 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45636.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45636.log.gz 2023-10-18T11:04:26.888 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.51306.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51306.log.gz 2023-10-18T11:04:26.888 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47726.log 2023-10-18T11:04:26.890 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46310.log 2023-10-18T11:04:26.891 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45972.log 2023-10-18T11:04:26.892 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47726.log.gz 2023-10-18T11:04:26.893 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.46310.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.46310.log.gz 2023-10-18T11:04:26.894 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45972.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45972.log.gz 2023-10-18T11:04:26.894 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52505.log 2023-10-18T11:04:26.896 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51754.log 2023-10-18T11:04:26.897 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.52505.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52505.log.gz 2023-10-18T11:04:26.899 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.51754.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51754.log.gz 2023-10-18T11:04:26.899 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48017.log 2023-10-18T11:04:26.901 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48675.log 2023-10-18T11:04:26.902 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48017.log: gzip 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48017.log.gz -5 2023-10-18T11:04:26.903 INFO:teuthology.orchestra.run.smithi089.stderr: --verbose -- /var/log/ceph/ceph-client.admin.46128.log 2023-10-18T11:04:26.903 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48597.log 2023-10-18T11:04:26.904 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48675.log.gz 2023-10-18T11:04:26.905 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.46128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51929.log 2023-10-18T11:04:26.905 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46128.log.gz 2023-10-18T11:04:26.906 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48597.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48597.log.gz 2023-10-18T11:04:26.907 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49156.log 2023-10-18T11:04:26.908 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.51929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51571.log 2023-10-18T11:04:26.908 INFO:teuthology.orchestra.run.smithi089.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51929.log.gz 2023-10-18T11:04:26.909 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.49156.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49156.log.gz 2023-10-18T11:04:26.911 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.51571.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51571.log.gz 2023-10-18T11:04:26.911 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49756.log 2023-10-18T11:04:26.913 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52711.log 2023-10-18T11:04:26.914 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.49756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44664.log 2023-10-18T11:04:26.915 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49756.log.gz 2023-10-18T11:04:26.916 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.52711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45628.log 2023-10-18T11:04:26.916 INFO:teuthology.orchestra.run.smithi089.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52711.log.gz 2023-10-18T11:04:26.917 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.44664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51675.log 2023-10-18T11:04:26.917 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44664.log.gz 2023-10-18T11:04:26.918 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45628.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45628.log.gz 2023-10-18T11:04:26.919 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45998.log 2023-10-18T11:04:26.920 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.51675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44177.log 2023-10-18T11:04:26.921 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51675.log.gz 2023-10-18T11:04:26.921 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45998.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45998.log.gz 2023-10-18T11:04:26.923 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.44177.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44177.log.gz 2023-10-18T11:04:26.923 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52557.log 2023-10-18T11:04:26.925 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48415.log 2023-10-18T11:04:26.926 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.52557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47534.log 2023-10-18T11:04:26.927 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52557.log.gz 2023-10-18T11:04:26.927 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48415.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48415.log.gz 2023-10-18T11:04:26.929 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47534.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47534.log.gz 2023-10-18T11:04:26.930 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52583.log 2023-10-18T11:04:26.931 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44586.log 2023-10-18T11:04:26.932 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.52583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48519.log 2023-10-18T11:04:26.933 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52583.log.gz 2023-10-18T11:04:26.933 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.44586.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44586.log.gz 2023-10-18T11:04:26.934 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47482.log 2023-10-18T11:04:26.935 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48519.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-18T11:04:26.936 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48519.log.gz 2023-10-18T11:04:26.937 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47482.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47482.log.gz 2023-10-18T11:04:26.940 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph.audit.log: 88.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-18T11:04:26.940 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-10-18T11:04:26.942 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48389.log 2023-10-18T11:04:26.943 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46883.log 2023-10-18T11:04:26.945 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48389.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48389.log.gz 2023-10-18T11:04:26.946 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.46883.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46883.log.gz 2023-10-18T11:04:26.947 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47991.log 2023-10-18T11:04:26.948 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52191.log 2023-10-18T11:04:26.950 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47991.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47991.log.gz 2023-10-18T11:04:26.951 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.52191.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52191.log.gz 2023-10-18T11:04:26.951 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47508.log 2023-10-18T11:04:26.953 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48571.log 2023-10-18T11:04:26.955 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47508.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47508.log.gz 2023-10-18T11:04:26.956 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48571.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48571.log.gz 2023-10-18T11:04:26.957 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-10-18T11:04:26.958 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38266.log 2023-10-18T11:04:26.961 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.38266.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38266.log.gz 2023-10-18T11:04:26.962 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50788.log 2023-10-18T11:04:26.965 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.50788.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50788.log.gz 2023-10-18T11:04:26.965 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43979.log 2023-10-18T11:04:26.968 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.43979.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43979.log.gz 2023-10-18T11:04:26.968 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49234.log 2023-10-18T11:04:26.971 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.49234.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49234.log.gz 2023-10-18T11:04:26.972 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51806.log 2023-10-18T11:04:26.975 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.51806.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51806.log.gz 2023-10-18T11:04:26.975 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45816.log 2023-10-18T11:04:26.975 INFO:teuthology.orchestra.run.smithi154.stderr: 92.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-10-18T11:04:26.978 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45816.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45816.log.gz 2023-10-18T11:04:26.978 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50615.log 2023-10-18T11:04:26.981 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45635.log 2023-10-18T11:04:26.981 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.50615.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50615.log.gz 2023-10-18T11:04:26.984 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50383.log 2023-10-18T11:04:26.984 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45635.log.gz 2023-10-18T11:04:26.989 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52112.log 2023-10-18T11:04:26.990 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.50383.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50383.log.gz 2023-10-18T11:04:26.992 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.52112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47066.log 2023-10-18T11:04:26.993 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52112.log.gz 2023-10-18T11:04:26.994 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50004.log 2023-10-18T11:04:27.000 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47066.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47066.log.gz 2023-10-18T11:04:27.001 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.50004.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50004.log.gz 2023-10-18T11:04:27.001 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46805.log 2023-10-18T11:04:27.001 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.46805.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46805.log.gz 2023-10-18T11:04:27.001 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51649.log 2023-10-18T11:04:27.002 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.51649.log: 92.6% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-10-18T11:04:27.004 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51649.log.gz 2023-10-18T11:04:27.004 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45201.log 2023-10-18T11:04:27.005 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46487.log 2023-10-18T11:04:27.007 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45201.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45201.log.gz 2023-10-18T11:04:27.008 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.0.46487.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46487.log.gz 2023-10-18T11:04:27.008 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46180.log 2023-10-18T11:04:27.010 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45418.log 2023-10-18T11:04:27.011 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.46180.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46180.log.gz 2023-10-18T11:04:27.013 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45418.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45418.log.gz 2023-10-18T11:04:27.013 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47913.log 2023-10-18T11:04:27.014 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49208.log 2023-10-18T11:04:27.016 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47913.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47913.log.gz 2023-10-18T11:04:27.017 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.49208.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49208.log.gz 2023-10-18T11:04:27.018 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50674.log 2023-10-18T11:04:27.019 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51123.log 2023-10-18T11:04:27.021 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.50674.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.50674.log.gz 2023-10-18T11:04:27.022 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.51123.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51123.log.gz 2023-10-18T11:04:27.023 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47326.log 2023-10-18T11:04:27.024 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51440.log 2023-10-18T11:04:27.025 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47326.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47326.log.gz 2023-10-18T11:04:27.026 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.51440.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51440.log.gz 2023-10-18T11:04:27.027 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47118.log 2023-10-18T11:04:27.029 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47222.log 2023-10-18T11:04:27.030 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47118.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47118.log.gz 2023-10-18T11:04:27.031 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47222.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47222.log.gz 2023-10-18T11:04:27.032 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47560.log 2023-10-18T11:04:27.034 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48701.log 2023-10-18T11:04:27.035 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47560.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47560.log.gz 2023-10-18T11:04:27.036 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48701.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48701.log.gz 2023-10-18T11:04:27.037 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50440.log 2023-10-18T11:04:27.038 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52034.log 2023-10-18T11:04:27.039 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.50440.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50440.log.gz 2023-10-18T11:04:27.041 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.52034.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52034.log.gz 2023-10-18T11:04:27.041 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49260.log 2023-10-18T11:04:27.043 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48727.log 2023-10-18T11:04:27.045 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.49260.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49260.log.gz 2023-10-18T11:04:27.045 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45008.log 2023-10-18T11:04:27.047 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48727.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48727.log.gz 2023-10-18T11:04:27.047 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46376.log 2023-10-18T11:04:27.048 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45008.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45008.log.gz 2023-10-18T11:04:27.050 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.46376.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46376.log.gz 2023-10-18T11:04:27.050 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52531.log 2023-10-18T11:04:27.052 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47612.log 2023-10-18T11:04:27.054 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.52531.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52531.log.gz 2023-10-18T11:04:27.054 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47612.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47612.log.gz 2023-10-18T11:04:27.055 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45060.log 2023-10-18T11:04:27.057 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44480.log 2023-10-18T11:04:27.058 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45060.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45060.log.gz 2023-10-18T11:04:27.059 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.44480.log: 94.2% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-10-18T11:04:27.061 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49782.log 2023-10-18T11:04:27.061 INFO:teuthology.orchestra.run.smithi089.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.44480.log.gz 2023-10-18T11:04:27.062 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48363.log 2023-10-18T11:04:27.064 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.49782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45630.log 2023-10-18T11:04:27.064 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49782.log.gz 2023-10-18T11:04:27.065 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48363.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48363.log.gz 2023-10-18T11:04:27.065 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51414.log 2023-10-18T11:04:27.067 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44281.log 2023-10-18T11:04:27.068 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45630.log.gz 2023-10-18T11:04:27.069 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.51414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47378.log 2023-10-18T11:04:27.069 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51414.log.gz 2023-10-18T11:04:27.070 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.44281.log: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45419.log 2023-10-18T11:04:27.071 INFO:teuthology.orchestra.run.smithi089.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44281.log.gz 2023-10-18T11:04:27.071 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47378.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47378.log.gz 2023-10-18T11:04:27.072 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49742.log 2023-10-18T11:04:27.073 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45419.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45419.log.gz 2023-10-18T11:04:27.074 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48876.log 2023-10-18T11:04:27.075 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.0.49742.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49742.log.gz 2023-10-18T11:04:27.075 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45894.log 2023-10-18T11:04:27.076 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47144.log 2023-10-18T11:04:27.077 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48876.log.gz 2023-10-18T11:04:27.078 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51384.log 2023-10-18T11:04:27.078 INFO:teuthology.orchestra.run.smithi089.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45894.log.gz 2023-10-18T11:04:27.079 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52295.log 2023-10-18T11:04:27.080 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47144.log.gz 2023-10-18T11:04:27.081 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.51384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43795.log 2023-10-18T11:04:27.081 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51384.log.gz 2023-10-18T11:04:27.082 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.52295.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52295.log.gz 2023-10-18T11:04:27.083 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.43795.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43795.log.gz 2023-10-18T11:04:27.084 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47830.log 2023-10-18T11:04:27.086 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49972.log 2023-10-18T11:04:27.087 INFO:teuthology.orchestra.run.smithi154.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-10-18T11:04:27.088 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52373.log 2023-10-18T11:04:27.088 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47830.log.gz 2023-10-18T11:04:27.088 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.49972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45426.log 2023-10-18T11:04:27.089 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49972.log.gz 2023-10-18T11:04:27.090 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.52373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45624.log 87.8% 2023-10-18T11:04:27.090 INFO:teuthology.orchestra.run.smithi089.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52373.log.gz 2023-10-18T11:04:27.091 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47965.log 2023-10-18T11:04:27.092 INFO:teuthology.orchestra.run.smithi089.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45426.log.gz 2023-10-18T11:04:27.093 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48493.log 2023-10-18T11:04:27.093 INFO:teuthology.orchestra.run.smithi089.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45624.log.gz 2023-10-18T11:04:27.094 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50903.log 2023-10-18T11:04:27.095 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47965.log.gz 2023-10-18T11:04:27.096 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46831.log 2023-10-18T11:04:27.096 INFO:teuthology.orchestra.run.smithi089.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48493.log.gz 2023-10-18T11:04:27.097 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.50903.log: 87.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.50903.log.gz -5 2023-10-18T11:04:27.097 INFO:teuthology.orchestra.run.smithi089.stderr: --verbose -- /var/log/ceph/ceph-client.admin.46988.log 2023-10-18T11:04:27.099 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.46831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43699.log 2023-10-18T11:04:27.099 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46831.log.gz 2023-10-18T11:04:27.100 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.46988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50845.log 2023-10-18T11:04:27.100 INFO:teuthology.orchestra.run.smithi089.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46988.log.gz 2023-10-18T11:04:27.102 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.43699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52347.log 2023-10-18T11:04:27.102 INFO:teuthology.orchestra.run.smithi089.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43699.log.gz 2023-10-18T11:04:27.103 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.50845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47040.log 2023-10-18T11:04:27.103 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50845.log.gz 2023-10-18T11:04:27.104 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.52347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47456.log 2023-10-18T11:04:27.105 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52347.log.gz 2023-10-18T11:04:27.106 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48753.log 2023-10-18T11:04:27.106 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47040.log.gz 2023-10-18T11:04:27.108 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47456.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47456.log.gz 2023-10-18T11:04:27.108 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46961.log 2023-10-18T11:04:27.109 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.48753.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48753.log.gz 2023-10-18T11:04:27.110 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.46598.log 2023-10-18T11:04:27.111 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.46961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51544.log 2023-10-18T11:04:27.112 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46961.log.gz 2023-10-18T11:04:27.112 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.2.46598.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.46598.log.gz 2023-10-18T11:04:27.113 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51728.log 2023-10-18T11:04:27.114 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.51544.log: gzip -5 88.8% --verbose -- replaced with /var/log/ceph/ceph-client.admin.51544.log.gz -- 2023-10-18T11:04:27.114 INFO:teuthology.orchestra.run.smithi089.stderr: /var/log/ceph/ceph-client.admin.47778.log 2023-10-18T11:04:27.115 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.51728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51280.log 2023-10-18T11:04:27.116 INFO:teuthology.orchestra.run.smithi089.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51728.log.gz 2023-10-18T11:04:27.117 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.47778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44357.log 2023-10-18T11:04:27.117 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47778.log.gz 2023-10-18T11:04:27.118 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.51280.log: gzip -5 --verbose -- 88.0% /var/log/ceph/ceph-client.admin.50673.log 2023-10-18T11:04:27.119 INFO:teuthology.orchestra.run.smithi089.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51280.log.gz 2023-10-18T11:04:27.120 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.44357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52060.log 2023-10-18T11:04:27.120 INFO:teuthology.orchestra.run.smithi089.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44357.log.gz 2023-10-18T11:04:27.121 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.50673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45868.log 2023-10-18T11:04:27.122 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50673.log.gz 2023-10-18T11:04:27.122 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.52060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51982.log 2023-10-18T11:04:27.123 INFO:teuthology.orchestra.run.smithi089.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52060.log.gz 2023-10-18T11:04:27.124 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43731.log 2023-10-18T11:04:27.124 INFO:teuthology.orchestra.run.smithi089.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45868.log.gz 2023-10-18T11:04:27.125 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.51982.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51982.log.gz 2023-10-18T11:04:27.126 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49456.log 2023-10-18T11:04:27.127 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.43731.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43731.log.gz 2023-10-18T11:04:27.127 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45217.log 2023-10-18T11:04:27.129 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.49456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51149.log 2023-10-18T11:04:27.130 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.45217.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45217.log.gz 2023-10-18T11:04:27.132 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.51149.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51149.log.gz 2023-10-18T11:04:27.132 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49456.log.gz 2023-10-18T11:04:27.600 INFO:teuthology.orchestra.run.smithi154.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-10-18T11:04:27.631 INFO:teuthology.orchestra.run.smithi089.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-10-18T11:04:27.672 INFO:teuthology.orchestra.run.smithi154.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-10-18T11:04:27.697 INFO:teuthology.orchestra.run.smithi089.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-10-18T11:04:27.767 INFO:teuthology.orchestra.run.smithi154.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-10-18T11:04:27.794 INFO:teuthology.orchestra.run.smithi154.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-10-18T11:04:27.797 INFO:teuthology.orchestra.run.smithi154.stderr: 2023-10-18T11:04:27.797 INFO:teuthology.orchestra.run.smithi154.stderr:real 0m1.228s 2023-10-18T11:04:27.797 INFO:teuthology.orchestra.run.smithi154.stderr:user 0m5.016s 2023-10-18T11:04:27.797 INFO:teuthology.orchestra.run.smithi154.stderr:sys 0m0.524s 2023-10-18T11:04:27.803 INFO:teuthology.orchestra.run.smithi089.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-10-18T11:04:27.860 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2023-10-18T11:04:27.874 INFO:teuthology.orchestra.run.smithi089.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-10-18T11:04:28.059 INFO:teuthology.orchestra.run.smithi089.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-10-18T11:04:28.061 INFO:teuthology.orchestra.run.smithi089.stderr: 2023-10-18T11:04:28.061 INFO:teuthology.orchestra.run.smithi089.stderr:real 0m1.510s 2023-10-18T11:04:28.061 INFO:teuthology.orchestra.run.smithi089.stderr:user 0m5.771s 2023-10-18T11:04:28.062 INFO:teuthology.orchestra.run.smithi089.stderr:sys 0m1.006s 2023-10-18T11:04:28.063 INFO:tasks.ceph:Archiving logs... 2023-10-18T11:04:28.063 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/log/ceph to /home/teuthworker/archive/dparmar-2023-10-18_10:19:04-fs:functional-wip-63105-distro-default-smithi/7431296/remote/smithi089/log 2023-10-18T11:04:28.064 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-18T11:04:28.891 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/log/ceph to /home/teuthworker/archive/dparmar-2023-10-18_10:19:04-fs:functional-wip-63105-distro-default-smithi/7431296/remote/smithi154/log 2023-10-18T11:04:28.892 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-18T11:04:29.497 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-18T11:04:29.508 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-18T11:04:29.509 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-18T11:04:29.513 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-18T11:04:29.583 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-18T11:04:29.592 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-18T11:04:29.592 DEBUG:teuthology.orchestra.run.smithi089:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-18T11:04:29.596 DEBUG:teuthology.orchestra.run.smithi154:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-18T11:04:29.609 INFO:teuthology.orchestra.run.smithi089.stdout: remote refid st t when poll reach delay offset jitter 2023-10-18T11:04:29.609 INFO:teuthology.orchestra.run.smithi089.stdout:============================================================================== 2023-10-18T11:04:29.609 INFO:teuthology.orchestra.run.smithi089.stdout:+hv01.front.sepi 67.205.162.81 3 u 29 64 377 0.089 -8.965 0.858 2023-10-18T11:04:29.610 INFO:teuthology.orchestra.run.smithi089.stdout:*hv02.front.sepi 63.231.80.2 3 u 23 64 377 0.087 -4.103 0.960 2023-10-18T11:04:29.610 INFO:teuthology.orchestra.run.smithi089.stdout:+hv03.front.sepi 74.6.168.72 3 u 33 64 377 0.100 -3.982 1.242 2023-10-18T11:04:29.610 INFO:teuthology.orchestra.run.smithi089.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2023-10-18T11:04:29.612 INFO:teuthology.orchestra.run.smithi154.stdout: remote refid st t when poll reach delay offset jitter 2023-10-18T11:04:29.612 INFO:teuthology.orchestra.run.smithi154.stdout:============================================================================== 2023-10-18T11:04:29.613 INFO:teuthology.orchestra.run.smithi154.stdout:+hv01.front.sepi 67.205.162.81 3 u 29 64 377 0.135 -12.393 0.977 2023-10-18T11:04:29.613 INFO:teuthology.orchestra.run.smithi154.stdout:*hv02.front.sepi 63.231.80.2 3 u 28 64 377 0.116 -7.692 0.824 2023-10-18T11:04:29.613 INFO:teuthology.orchestra.run.smithi154.stdout:+hv03.front.sepi 74.6.168.72 3 u 36 64 377 0.094 -6.402 0.826 2023-10-18T11:04:29.613 INFO:teuthology.orchestra.run.smithi154.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2023-10-18T11:04:29.614 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-18T11:04:29.624 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-18T11:04:29.625 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-18T11:04:29.635 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-18T11:04:29.644 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-18T11:04:29.654 INFO:teuthology.task.internal:Duration was 1867.448545 seconds 2023-10-18T11:04:29.654 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-18T11:04:29.665 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-18T11:04:29.665 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-18T11:04:29.669 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-18T11:04:29.770 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-18T11:04:29.771 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi089.front.sepia.ceph.com 2023-10-18T11:04:29.771 DEBUG:teuthology.orchestra.run.smithi089:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-18T11:04:29.794 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi154.front.sepia.ceph.com 2023-10-18T11:04:29.795 DEBUG:teuthology.orchestra.run.smithi154:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-18T11:04:29.817 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-18T11:04:29.817 DEBUG:teuthology.orchestra.run.smithi089:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-18T11:04:29.843 DEBUG:teuthology.orchestra.run.smithi154:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-18T11:04:29.896 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-18T11:04:29.896 DEBUG:teuthology.orchestra.run.smithi089:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-18T11:04:29.927 DEBUG:teuthology.orchestra.run.smithi154:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-18T11:04:30.010 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-18T11:04:30.021 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-18T11:04:30.022 DEBUG:teuthology.orchestra.run.smithi089:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-18T11:04:30.043 DEBUG:teuthology.orchestra.run.smithi154:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-18T11:04:30.064 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-18T11:04:30.075 DEBUG:teuthology.orchestra.run.smithi089:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-18T11:04:30.091 DEBUG:teuthology.orchestra.run.smithi154:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-18T11:04:30.106 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern = core 2023-10-18T11:04:30.124 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern = core 2023-10-18T11:04:30.143 DEBUG:teuthology.orchestra.run.smithi089:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-18T11:04:30.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T11:04:30.175 DEBUG:teuthology.orchestra.run.smithi154:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-18T11:04:30.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T11:04:30.194 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-18T11:04:30.204 INFO:teuthology.task.internal:Transferring archived files... 2023-10-18T11:04:30.205 DEBUG:teuthology.misc:Transferring archived files from smithi089:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/dparmar-2023-10-18_10:19:04-fs:functional-wip-63105-distro-default-smithi/7431296/remote/smithi089 2023-10-18T11:04:30.206 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-18T11:04:30.241 DEBUG:teuthology.misc:Transferring archived files from smithi154:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/dparmar-2023-10-18_10:19:04-fs:functional-wip-63105-distro-default-smithi/7431296/remote/smithi154 2023-10-18T11:04:30.241 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-18T11:04:30.265 INFO:teuthology.task.internal:Removing archive directory... 2023-10-18T11:04:30.266 DEBUG:teuthology.orchestra.run.smithi089:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-18T11:04:30.287 DEBUG:teuthology.orchestra.run.smithi154:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-18T11:04:30.320 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-18T11:04:30.330 INFO:teuthology.task.internal:Not uploading archives. 2023-10-18T11:04:30.331 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-18T11:04:30.343 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-18T11:04:30.343 DEBUG:teuthology.orchestra.run.smithi089:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-18T11:04:30.346 DEBUG:teuthology.orchestra.run.smithi154:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-18T11:04:30.352 INFO:teuthology.orchestra.run.smithi089.stdout: 657971 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 18 11:04 /home/ubuntu/cephtest 2023-10-18T11:04:30.371 INFO:teuthology.orchestra.run.smithi154.stdout: 657968 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 18 11:04 /home/ubuntu/cephtest 2023-10-18T11:04:30.373 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-18T11:04:30.385 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-18T11:04:30.416 INFO:teuthology.nuke:Checking targets against current locks 2023-10-18T11:04:30.443 DEBUG:teuthology.nuke:shortname: smithi089 2023-10-18T11:04:30.443 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-18T11:04:30.465 DEBUG:teuthology.nuke:shortname: smithi154 2023-10-18T11:04:30.466 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-18T11:04:30.469 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi089.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2023-10-18_10:19:04-fs:functional-wip-63105-distro-default-smithi/7431296', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-18 10:22:32.967283', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK1OYlDgi4O1/6ng/uXhZ8MUwlMXFIH1rfYalkqOVAKHxilpUJbwZU6muYWI8DFrjmTYl+hk901BCq77/ZnsJAE='} 2023-10-18T11:04:30.491 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2023-10-18_10:19:04-fs:functional-wip-63105-distro-default-smithi/7431296', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-18 10:22:32.969279', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD206tYTbUO+0hlBk/fd4mL1UBAKK4y+NSWI2uFM6AlUPiOt/tJXuysZRQtUD6n/+FEL5LTD7erHsfj2X4n3pZ4='} 2023-10-18T11:04:30.518 INFO:teuthology.orchestra.console.smithi089:Power off 2023-10-18T11:04:30.519 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-18T11:04:30.543 DEBUG:teuthology.orchestra.console.smithi089:power off output: Chassis Power Control: Down/Off 2023-10-18T11:04:30.543 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T11:04:30.560 INFO:teuthology.orchestra.console.smithi154:Power off 2023-10-18T11:04:30.561 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-18T11:04:30.570 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is on 2023-10-18T11:04:30.582 DEBUG:teuthology.orchestra.console.smithi154:power off output: Chassis Power Control: Down/Off 2023-10-18T11:04:30.583 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T11:04:30.604 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is on 2023-10-18T11:04:34.572 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T11:04:34.605 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T11:04:34.683 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is on 2023-10-18T11:04:34.714 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is on 2023-10-18T11:04:38.685 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T11:04:38.715 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T11:04:38.795 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is on 2023-10-18T11:04:38.824 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is off 2023-10-18T11:04:38.925 INFO:teuthology.orchestra.console.smithi154:Power off completed 2023-10-18T11:04:42.802 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T11:04:42.912 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is off 2023-10-18T11:04:43.013 INFO:teuthology.orchestra.console.smithi089:Power off completed 2023-10-18T11:04:43.118 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 47672119149b103ec7cec2cc65c063ccbb001ac7 client.1-kernel-sha1: 47672119149b103ec7cec2cc65c063ccbb001ac7 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-recovery} duration: 1867.4485445022583 failure_reason: 'Test failure: test_client_eviction_if_config_is_set (tasks.cephfs.test_client_recovery.TestClientOnLaggyOSD)' flavor: default owner: scheduled_dparmar@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=2373597022de49a9bdc383235bb1fedd status: fail success: false 2023-10-18T11:04:43.119 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-18T11:04:43.210 INFO:teuthology.run:FAIL